Optional argument CVmix_conv_params_user used in subroutine.

[mark-petersen]

This line:

CVMix-src/src/shared/cvmix_convection.F90

Line 227 in 52aac95

CVmix_conv_params_user)

uses an optional argument, and produces a run-time error when compiled in debug if that optional argument is not provided. I think the intention was

227                            CVmix_conv_params_in)

Similarly, this

CVMix-src/src/shared/cvmix_convection.F90

Line 404 in 52aac95

CVmix_conv_params_user)

should be

 404                           CVmix_conv_params_out)

[mnlevy1981]

Thanks for reporting this, and apologies for taking so long to get back to you. What version of the compiler is complaining in debug mode? I'm a little confused about what it is balking at, since CVmix_conv_params_user is being passed to another optional argument:

CVMix-src/src/shared/cvmix_convection.F90

Line 263 in 52aac95

type (cvmix_conv_params_type), optional, target, intent(in) :: &

I wonder if a better fix would be to explicitly call cvmix_coeffs_conv_low() instead of relying on the cvmix_coeffs_conv() interface in the current implementation. I'm happy to play around with it once I know the compiler / flags that lead to the error.

[mark-petersen]

Thanks @mnlevy1981. The failure is with intel 19.0.4 using debug flags, specifically:

mpiifort -g -convert big_endian -free -CU -CB -check all -fpe0 -traceback

I'm guessing that -check all is the critical one. It compiles fine, but get the runtime error:

Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.

Backtrace for this error:
#2  0xe411bb in __cvmix_convection_MOD_cvmix_coeffs_conv_wrap
	at /usr/projects/climate/mpeterse/repos/compass/pr/E3SM-Project/components/mpas-ocean/src/cvmix/src/shared/cvmix_convection.F90:227

222     call cvmix_coeffs_conv(new_Mdiff, new_Tdiff,                              &
223                            CVmix_vars%SqrBuoyancyFreq_iface,                  &
224                            CVmix_vars%WaterDensity_cntr,                      &
225                            CVmix_vars%AdiabWaterDensity_cntr,                 &
226                            nlev, max_nlev, nint(CVMix_vars%kOBL_depth)+1,     &
227                            CVmix_conv_params_user)

and confirmed that this change:

components/mpas-ocean/src/cvmix/src/shared/cvmix_convection.F90:227
227                            CVmix_conv_params_in)

fixes the problem with my set-up.

This was originally posted here:
MPAS-Dev/compass#249 (comment)

[mnlevy1981]

I have created a branch at https://github.com/mnlevy1981/CVMix-src/tree/add_convective_test that does two things:

1. Includes a call to cvmix_coeffs_conv from the KPP test directory, and
2. Updates the compiler flags to build with the flags you mentioned when using the intel compiler:

ifort -g -convert big_endian -free -CU -CB -check all -fpe0 -traceback -module /home/mlevy/codes/CVMix/bld/obj -cpp -nogen-interface -I/home/mlevy/codes/CVMix/include -c /home/mlevy/codes/CVMix/src/shared/cvmix_convection.F90 -o /home/mlevy/codes/CVMix/bld/obj/cvmix_convection.o

If you could test this out on your machine, that would be great. The steps to take:

$ git clone -b add_convective_test https://github.com/mnlevy1981/CVMix-src.git CVMix_test
$ cd CVMix_test/reg_tests/kpp
$ ./kpp-test.sh

That script should prompt for information about your compiler and netcdf path.

(cvmix_setup): This utility needs to be run before the included Makefile will
(cvmix_setup): successfully compile CVMix. It collects information about your compiler
(cvmix_setup): and associated libraries, and saves it in the .CVMix_env file. Note that
(cvmix_setup): it only needs to be run once, though you should run it again if anything
(cvmix_setup): changes.

Fortran compiler (mpi not necessary):
Directory containing netcdf configuration tool nc-config (or enter "no-nc-config" to enter location of netcdf include and netcdf lib directories):

Use ifort for your fortran compiler, and the netcdf stuff isn't important because by default these tests write binary output instead of netcdf... so you can enter no-nc-config and then just hit enter again when prompted for

Directory containing netcdf (enter "need_both" to enter netcdf include directory and netcdf lib directory separately):

That will launch the build, and also run the test case (assuming a successful build). The tail of the output that I see is

 Test 7: Convective mixing
 ----------
 Successfully called cvmix_coeffs_conv()

but I think this should mimic your setup sufficiently to trigger the runtime error with 19.0.4? I've built this with 19.0.2 and 19.0.5, though, and both of those executables run successfully... so either the problem you are seeing is limited to 19.0.4 (and maybe 19.0.3), or something in how I set up the test is not quite the same as whatever is causing your error. (If it's the latter, we can certainly keep iterating until we can reproduce the issue!)

And one comment about the stand-alone test suite; if you want the test suite to rebuild the model, you need to run make allclean from the ${CVMIX}/src directory; make distclean will also delete the stored responses from the script that prompts you for compiler information (so the next time you run make, you will be asked what compiler / netcdf library you are using).

edit: realized that I phrased the last sentence poorly; the script that gathers compiler information doesn't run during make distclean but will be run during the next make

[mark-petersen]

I tested as you suggested above and it ended with:

 Test 7: Convective mixing
 ----------
 Successfully called cvmix_coeffs_conv()

so this all works. Thanks for your help.

For my own reference, this is on LANL badger with modules loaded from compass master abb60740 and

source load_dev_compass_1.0.0_badger_intel_impi.sh

I provided the ' ./kpp-test.sh' with the ifort compiler.

[mnlevy1981]

I tested as you suggested above and it ended with:

 Test 7: Convective mixing
 ----------
 Successfully called cvmix_coeffs_conv()

so this all works. Thanks for your help.

Hmm,that's interesting... I was hoping the test would fail with the same error you were seeing out of the MPAS runs. I wonder if I missed a flag when I updated bld/CompileFlags.mak