forrtl: error (65): floating invalid / pow_ct.c

[Christian.Koch23@web.de]

Hello Mohiders,
I hope all of you had a good start this year. I wish you all the best for 2013.
I changed the exe from MohidWater_ForIntel_DoublePrecision to MOHIDWater_x64_release_double_openmp.exe. with this change a problem ocured and I would like to understand the problem to find a proper solution. I cant really place the error messages (and the end of this post) that Mohid provide and hope someone can comment on it. Here are some information on the model:
- Purpose: water quality in a reservoir
- so far I use the following water properties: salinity, temperature, oxygen, carbon dioxide (to test the degassing of CO2), alkalinity, phytoplankton, cohesive sediment
- horizontal grid 75 x 75 m,
- Vertical grid TYPE : HARMONIC, LAYERS : 26, LAYERTHICKNESS : 5 5 5 5 3 3 3 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
- 3 boxes defined for discharge to account for groundwater inflow and surface water inflow using DISCHARGE_UNIFORM : 1
- Model grid with water depths ranging from 0.5 m to 47.8 m
I assume there is a problem with instabilities. Unfortunately I cant find the error in the input files. Maybe someone can help.
Here are the things I have tried so far:
- Filtering the bathymetry using MohidGIS
- Set the minimum layer thickness to 0.2 m
- Changed the bathymetry values in a way that is not possible to have very thin layers (all values are multiples of 0.2 m e.g. 1.78 became 1.
- using different vertical grids (sigma and harmonic) and chose harmonic in the end
- I tired to run with varying initial elevations.
Below you find the error messages I get using the MOHIDWater_x64_release_double_openmp.exe. I added the chosen DT and the Courant numbers provided by Mohid. In the end I had to reduce the Courant number to 1.15 to get the model running. The total calculation time in that case is about 12 days using the 4 intel CPUs on my computer. That is long considering that I planned 4 scenarios. Further I included the hydrodynamic.dat, turbulence.dat and waterproperties.dat in this post.
Thank you very much in advance.
Cheers,
Christian

----------------------

DT = 10 s
Courant Number is : 5.78660522240804

forrtl: error (65): floating invalid
Image PC Routine Line Source
MOHIDWater_x64_re 0000000140592A68 pow.L 1932 pow_ct.c
MOHIDWater_x64_re 00000001403ABCA9 MODULELIGHTEXTINC 1321 ModuleLightExtinction.F90
MOHIDWater_x64_re 00000001403AA239 MODULELIGHTEXTINC 1050 ModuleLightExtinction.F90
MOHIDWater_x64_re 00000001403ACB78 MODULELIGHTEXTINC 1005 ModuleLightExtinction.F90
MOHIDWater_x64_re 0000000140202A61 MODULEWATERPROPER 14360 ModuleWaterProperties.F90
MOHIDWater_x64_re 0000000140201534 MODULEWATERPROPER 14220 ModuleWaterProperties.F90
MOHIDWater_x64_re 00000001401F43FC MODULEWATERPROPER 8037 ModuleWaterProperties.F90
MOHIDWater_x64_re 00000001402F3B11 MODULEMODEL_mp_RU 1826 ModuleModel.F90
MOHIDWater_x64_re 00000001402F3576 MODULEMODEL_mp_RU 1613 ModuleModel.F90
MOHIDWater_x64_re 00000001402F5787 MOHIDWATER_ip_DOO 1319 Main.F90
MOHIDWater_x64_re 00000001402F5D2D MOHIDWATER_ip_MOD 1199 Main.F90
MOHIDWater_x64_re 00000001402F513D MAIN__ 220 Main.F90
MOHIDWater_x64_re 000000014074601C Unknown Unknown Unknown
MOHIDWater_x64_re 00000001405A1CEF Unknown Unknown Unknown
kernel32.dll 0000000076EF652D Unknown Unknown Unknown
ntdll.dll 000000007718C521 Unknown Unknown Unknown


----------------------

DT = 5 s
Courant Number is : 2.89330261120402

forrtl: error (72): floating overflow
Image PC Routine Line Source
MOHIDWater_x64_re 00000001402027BC MODULEWATERPROPER 14254 ModuleWaterProperties.F90
libiomp5md.dll 000000001004ECAC Unknown Unknown Unknown
libiomp5md.dll 0000000010031622 Unknown Unknown Unknown
libiomp5md.dll 0000000010034220 Unknown Unknown Unknown
libiomp5md.dll 0000000010016251 Unknown Unknown Unknown
MOHIDWater_x64_re 00000001402016F0 MODULEWATERPROPER 14237 ModuleWaterProperties.F90
MOHIDWater_x64_re 00000001401F43FC MODULEWATERPROPER 8037 ModuleWaterProperties.F90
MOHIDWater_x64_re 00000001402F3B11 MODULEMODEL_mp_RU 1826 ModuleModel.F90
MOHIDWater_x64_re 00000001402F3576 MODULEMODEL_mp_RU 1613 ModuleModel.F90
MOHIDWater_x64_re 00000001402F5787 MOHIDWATER_ip_DOO 1319 Main.F90
MOHIDWater_x64_re 00000001402F5D2D MOHIDWATER_ip_MOD 1199 Main.F90
MOHIDWater_x64_re 00000001402F513D MAIN__ 220 Main.F90
MOHIDWater_x64_re 000000014074601C Unknown Unknown Unknown
MOHIDWater_x64_re 00000001405A1CEF Unknown Unknown Unknown
kernel32.dll 0000000076EF652D Unknown Unknown Unknown
ntdll.dll 000000007718C521 Unknown Unknown Unknown


----------------------

DT 3 s
Courant Number is : 1.73598156672241

forrtl: error (65): floating invalid
Image PC Routine Line Source
MOHIDWater_x64_re 0000000140592A68 pow.L 1932 pow_ct.c
MOHIDWater_x64_re 00000001403ABCA9 MODULELIGHTEXTINC 1321 ModuleLightExtinction.F90
MOHIDWater_x64_re 00000001403AA239 MODULELIGHTEXTINC 1050 ModuleLightExtinction.F90
MOHIDWater_x64_re 00000001403ACB78 MODULELIGHTEXTINC 1005 ModuleLightExtinction.F90
MOHIDWater_x64_re 0000000140202A61 MODULEWATERPROPER 14360 ModuleWaterProperties.F90
MOHIDWater_x64_re 0000000140201534 MODULEWATERPROPER 14220 ModuleWaterProperties.F90
MOHIDWater_x64_re 00000001401F43FC MODULEWATERPROPER 8037 ModuleWaterProperties.F90
MOHIDWater_x64_re 00000001402F3B11 MODULEMODEL_mp_RU 1826 ModuleModel.F90
MOHIDWater_x64_re 00000001402F3576 MODULEMODEL_mp_RU 1613 ModuleModel.F90
MOHIDWater_x64_re 00000001402F5787 MOHIDWATER_ip_DOO 1319 Main.F90
MOHIDWater_x64_re 00000001402F5D2D MOHIDWATER_ip_MOD 1199 Main.F90
MOHIDWater_x64_re 00000001402F513D MAIN__ 220 Main.F90
MOHIDWater_x64_re 000000014074601C Unknown Unknown Unknown
MOHIDWater_x64_re 00000001405A1CEF Unknown Unknown Unknown
kernel32.dll 0000000076EF652D Unknown Unknown Unknown
ntdll.dll 000000007718C521 Unknown Unknown Unknown

----------------------

DT = 2 s
Courant Number is : 1.15732104448161

hydrodynamic_1.dat
-----------------------------------------
BAROCLINIC : 1
CONTINUOUS : 0
TIDE : 0
ENTERING_WAVE : 0
RADIATION : 2
MIN_VELOCITY : 1e-6
MIN_COMPONENT : 1e-3
WATER_DISCHARGES : 1
RESIDUAL : 1
ENERGY : 1
UP_CENTER : 1
IMPLICIT_VERTADVECTION : 1
IMPLICIT_VERTDIFFUSION : 1
DISCRETIZATION : 2
UPSTREAM : Upwind
TIME_SERIE : 0
EVOLUTION : Solve_Equations
CORIOLIS : 1
VOLUMEVARIATION : 1
HORIZONTALADVECTION : 1
HORIZONTALDIFFUSION : 1
INITIAL_ELEVATION : 1
INITIAL_ELEVATION_VALUE : 0.2
INITIAL_VEL_U : 0.0
INITIAL_VEL_V : 0.0
BOUNDARYBAROCLINIC : 1
RAMP : 1
INERTIAL_PERIODS : 6
VERTICALADVECTION : 1
VERTICALDIFFUSION : 1
VELTANGENTIALBOUNDARY : 2
VELNORMALBOUNDARY : 2
NULL_BOUND_HORADV : 0
TIDEPOTENTIAL : 0
WIND : 2
WIND_SMOOTH_PERIOD : 86400.
ATM_PRESSURE : 1
ATM_PERIOD : 86400.
SURFACEWATERFLUX : 1
HMIN_CHEZY : 0.10
VMIN_CHEZY : 0.10
DATA_ASSIMILATION : 0
IMPLICIT_HORADVECTION : 0
RECORDING : 0
MOMENTUM_DISCHARGE : 0
SLOWSTART : 0.0
HMIN_ADVECTION : 0.5
LOCAL_DENSITY : 1
CYCLIC_BOUNDARY : 0
CONSERVATIVE_HOR_DIF : 0
BIHARMONIC : 1
BIHARMONIC_COEF : 8e5
SUBMODEL : 0
MISSING_NULL : 1
DEADZONE : 0
BAROCLINIC_RADIATION : 2
LOCAL_SOLUTION : 1
BOTTOMWATERFLUX : 0
SLIPPING_CONDITION : 1
NONHYDROSTATIC : 0
STATISTICS : 0
OUTPUT_TIME : 0 86400.
BRFORCE : 0
RESTART_FILE_OUTPUT_TIME : 0. 86400.
RESTART_FILE_OVERWRITE : 1

turbulenc.dat
-------------------------------------------
OUTPUT_TIME : 0. 86400.
RESTART_FILE_OUTPUT_TIME : 0. 86400.
RESTART_FILE_OVERWRITE : 1
MODVISH : smagorinsky
HORCON : 0.2
VISH_REF : 1.000
MODTURB : turbulence_equation
MLD_Method : 2
TKE_MLD : 1e-5
RICH_MLD : 0.5
OUTPUT_HDF : 1

waterproperties.dat
------------------------------------------
OUTPUT_TIME : 0. 86400.
!ALTITUDE : 0.1

RESTART_FILE_OUTPUT_TIME : 0. 86400.
RESTART_FILE_OVERWRITE : 1

!REFERENCE_DENSITY : 1001.2
DENSITY_METHOD : 2

SW_EXTINCTION_TYPE : 4 !2chl_a 3sediments 4combi chl_a u sed 6multi 1const
LW_PERCENTAGE : 0.4
SW_PERCENTAGE : 0.6
LW_EXTINCTION_COEF : 0.67
SW_EXTINCTION_COEF : 0.3


<beginproperty>
NAME : salinity
UNITS : psu
DESCRIPTION : salinity
DEFAULTVALUE : 0.7
ADVECTION_DIFFUSION : 1
REMAIN_CONSTANT : 0
DISCHARGES : 1
OUTPUT_HDF : 1
<endproperty>

<beginproperty>
NAME : temperature
UNITS : °C
DESCRIPTION : Temp
INITIALIZATION_METHOD : CONSTANT
DEFAULTVALUE : 2.1
SURFACE_FLUXES : 1
ADVECTION_DIFFUSION : 1
REMAIN_CONSTANT : 0
OUTPUT_HDF : 1
OLD : 0
DISCHARGES : 1
CEQUALW2 : 0
<endproperty>

<beginproperty>
NAME : oxygen
UNITS : mg/l
DESCRIPTION : oxygen
INITIALIZATION_METHOD : CONSTANT
DEFAULTVALUE : 10
REMAIN_CONSTANT : 0
ADVECTION_DIFFUSION : 1
BOUNDARY_CONDITION : 1 !1-MassConservation 3-VerticalDiffusion
ADV_DIF_NUM_STABILITY : 1
SCHMIDT_NUMBER_H : 1
SCHMIDT_COEF_V : 1
SCHMIDT_BACKGROUND_V : 1E-08
ADVECTION_V_IMP_EXP : 0
ADVECTION_H_IMP_EXP : 1
DIFFUSION_V_IMP_EXP : 0
ADVECTION_UP_DC : 1
SURFACE_FLUXES : 1
DOSAT_TYPE : 2 !3-Mortimer 2-Henry 1-Apha
BOTTOM_FLUXES : 0
WATER_QUALITY : 0
OLD : 0
OUTPUT_HDF : 1
CEQUALW2 : 0
DISCHARGES : 0
<endproperty>

<beginproperty>
NAME : carbon dioxide
UNITS : mg/l
DESCRIPTION : CO2
INITIALIZATION_METHOD : CONSTANT
DEFAULTVALUE : 4
REMAIN_CONSTANT : 0
SURFACE_FLUXES : 1
OUTPUT_HDF : 1
<endproperty>

<beginproperty>
NAME : alkalinity
UNITS : mg/l
DESCRIPTION : Alk
IS_COEF : 1
INITIALIZATION_METHOD : CONSTANT
DEFAULTVALUE : 0
REMAIN_CONSTANT : 0
ADVECTION_DIFFUSION : 1
BOUNDARY_CONDITION : 1
ADVECTION_UP_DC : 1
SURFACE_FLUXES : 0
BOTTOM_FLUXES : 0
WATER_QUALITY : 0
CEQUALW2 : 0
DISCHARGES : 1
OUTPUT_HDF : 1
<endproperty>

<beginproperty>
NAME : phytoplankton
UNITS : mg/l
DESCRIPTION : phytoplankton concentration
IS_COEF : 0.001
INITIALIZATION_METHOD : CONSTANT
DEFAULTVALUE : 1
REMAIN_CONSTANT : 1
ADVECTION_DIFFUSION : 1
BOUNDARY_CONDITION : 1
ADV_DIF_NUM_STABILITY : 1
SCHMIDT_NUMBER_H : 1
SCHMIDT_COEF_V : 1
SCHMIDT_BACKGROUND_V : 1E-08
ADVECTION_V_IMP_EXP : 0
ADVECTION_H_IMP_EXP : 1
DIFFUSION_V_IMP_EXP : 0
ADVECTION_UP_DC : 1
WATER_QUALITY : 0
CEQUALW2 : 0
DISCHARGES : 0
OUTPUT_HDF : 1
<endproperty>

<beginproperty>
NAME : cohesive sediment
UNITS : mg/l
DESCRIPTION : cohesive sediment
IS_COEF : 0.001
PARTICULATE : 1
SPM : 0
OLD : 0
INITIALIZATION_METHOD : CONSTANT
REMAIN_CONSTANT : 1
DEFAULTVALUE : 10.0
DEFAULTBOUNDARY : 10.0
ADVECTION_DIFFUSION : 1
ADV_DIF_NUM_STABILITY : 0
BOUNDARY_CONDITION : 1
ADVECTION_UP_DC : 1
ADVECTION_H_IMP_EXP : 0
ADVECTION_V_IMP_EXP : 0
DIFFUSION_V_IMP_EXP : 0
NULLDIF : 0
DECAY_TIME : 0
DISCHARGES : 0
DISCHARGES_TRACKING : 0
VERTICAL_MOVEMENT : 0
SURFACE_FLUXES : 0
BOTTOM_FLUXES : 0
DATA_ASSIMILATION : 0
SED_INTERFACE : 0
PARTITION : 0
WATER_QUALITY : 0
TIME_SERIE : 0
BOX_TIME_SERIE : 0
OUTPUT_HDF : 1
CEQUALW2 : 0
<endproperty>

[frank.maretec@ist.utl.pt]

Dear Christian,

your setup is to complex to give a straight answer. It seems indeed to be a numerical issue.

I recommend that you switch of all processes (wind, discharges, baroclinic) and start to switch them on one by one, so you can get a better idea where the problem comes from.

When the model crashes, run a very very short period with a high frequency of outputs, so you can see, in the HDF files, if the error comes from one single hotspot (error in the bathymetry / vertical descritization) or if it is general (instability everywhere).

Also, where did you get your compiled executable from? Are they recent (e.g. Source Code / MOHID Studio)?

Hope this helps,
Frank

[Christian.Koch23@web.de]

Dear Frank,
thank you very much for your reply and the provided information. I tried the steps you mentioned and came to the conclusion that the error is either related to the vertical grid or is more general (as you said: instability everywhere). That was not conclusive, at least to me. Finally I remembered a hint Guilherme gave me and I changed the bihamonic coefficient, which I calculated from average_velocity / 16 * DX^3 = 0,5 [m/s] / 16 * 75[m]^3 = 13200 (in the hydrodynamic file I provided in my question, I set the value to 8e5). Finally the model started to run. I was able to increase the value of DT to 10 s (Courant number of 5.78). So far the model is running stable. I checked some velocities and temperature profiles, that were calculated by Mohid. The results seem to be sensible.
Thanks for your help.
Christian