Enable package logic code to be automatically generated through new XML attribute, active_when #1321

mgduda · May 22, 2025

This PR enables the package logic code that determines when a package is active
to be automatically generated by the registry through the use of a new XML attribute
for <package> elements, active_when.

Prior to the changes in this PR, defining a new package in the Registry with
a <package> element required code to be written (typically in a core's
setup_packages routine) to determine whether that package was active. For simple
conditions -- e.g., a package being active if and only if a logical namelist
option is true -- the need to write boilerplate-type Fortran code to retrieve
package and namelist pointers, check that those pointers are valid, and to
finally set the package variable, was unnecessarily cumbersome.

This PR introduces a new XML attribute, active_when, that may be added to
<package> elements in the Registry, along with new functionality in the registry
processor to automatically generate package-logic code (the equivalent to what
would otherwise need to be written by hand in a core's setup_packages routine)
at compile time.

If provided, the active_when attribute contains a Fortran expression that
evaluates to a logical, and that may use any literal constants and namelist
options (which are assumed to begin with the string 'config'). The expression
from the active_when attribute is essentially used verbatim in Fortran code to
set the package variable.

For example, consider a package named foo that is active if and only if the
namelist option config_use_physics is true. This package might have the
following definition in the Registry:

  <package name="foo" active_when="config_do_physics"/>

Logically, the generated code is equivalent to:

  fooActive = ( config_do_physics )

As another example, consider the following package definition in the Registry:

  <package name="bar" active_when="config_nsteps > 0 .and. .not. config_dryrun"/>

The generated Fortran logic would be:

  barActive = ( config_nsteps > 0 .and. .not. config_dryrun )

Note that in both examples, the package's active_when logic is a valid Fortran
logical expression that makes uses of namelist options and literal constants.

In order to make use of this functionality introduced in this PR, a core
must make two changes in its mpas_<CORE>_core_interface module:

The module should include through a pre-processor directive the
registry-generated package-logic code, which resides in a file named
setup_packages.inc in the core's inc/ directory; e.g.,

   #include "setup_packages.inc"

The setup_packages routine in the module should make a call to the generated
<CORE>_setup_packages_when routine, which is defined in the
setup_packages.inc file; e.g.,

   call atm_setup_packages_when(configs, packages)

The registry-generated package-logic code for packages that employ an
active_when attribute also contains calls to mpas_log_write to record
information about the activity of packages to a core's log file. For example,
the following messages are now written to the log file if two packages, foo and
bar, were active and inactive, respectively:

 Configuring registry-specified packages...
   foo : T
   bar : F
 ----- done configuring registry-specified packages -----

mattldawson

Thanks for including me in the review @mgduda! Looks great to me, although I will admit I don't have a lot of experience writing code generators, so my suggestions are pretty trivial.

One corner case I was wondering about, is how a user would get this type of expression to set the active flag:

fooActive = ( trim( string_might_be_foo ) == "FOO" )

which contains double quotes? Are there escape characters, or would you use single quotes to enclose the conditional logic in the XML file? (full disclosure, I'm also not that familiar with XML)

Anyway, looks like a useful feature to me!

src/tools/registry/gen_inc.c

mgduda · May 23, 2025

Thanks for including me in the review @mgduda! Looks great to me, although I will admit I don't have a lot of experience writing code generators, so my suggestions are pretty trivial.

One corner case I was wondering about, is how a user would get this type of expression to set the active flag:
fooActive = ( trim( string_might_be_foo ) == "FOO" )
which contains double quotes? Are there escape characters, or would you use single quotes to enclose the conditional logic in the XML file? (full disclosure, I'm also not that familiar with XML)

Anyway, looks like a useful feature to me!

I should add this to the documentation or commit message somewhere, but for string literals, you'll need to use single quotes; e.g.,

<package name="blaz" active_when="trim(config_physics_suite) == 'mesoscale'"/>

abishekg7 · May 23, 2025

I think the code changes look good overall, and I've done some testing - seems fine. I can approve when this PR is finalized.

…he Registry Prior to the changes in this commit, defining a new package in the Registry with a <package> element required code to be written (typically in a core's setup_packages routine) to determine whether that package was active. For simple conditions -- e.g., a package being active if and only if a logical namelist option is true -- the need to write boilerplate-type Fortran code to retrieve package and namelist pointers, check that those pointers are valid, and to finally set the package variable, was unnecessarily cumbersome. This commit introduces a new XML attribute, active_when, that may be added to <package> elements in the Registry, along with new functionality in the registry processor to automatically generate package-logic code (the equivalent to what would otherwise need to be written by hand in a core's setup_packages routine) at compile time. If provided, the active_when attribute contains a Fortran expression that evaluates to a logical, and that may use any literal constants and namelist options (which are assumed to begin with the string 'config'). The expression from the active_when attribute is essentially used verbatim in Fortran code to set the package variable. For example, consider a package named 'foo' that is active if and only if the namelist option 'config_use_physics' is true. This package might have the following definition in the Registry: <package name="foo" active_when="config_do_physics"/> Logically, the generated code is equivalent to: fooActive = ( config_do_physics ) As another example, consider the following package definition in the Registry: <package name="bar" active_when="config_nsteps > 0 .and. .not. config_dryrun"/> The generated Fortran logic would be: barActive = ( config_nsteps > 0 .and. .not. config_dryrun ) When the package's logic involves string literal values, those values must use single-quote characters in their Registry definition, rather than double-quotes; for example: <package name="blaz" active_when="trim(config_physics_suite) == 'mesoscale'"/> Note that in these examples, the package's active_when logic is a valid Fortran logical expression that makes uses of namelist options and literal constants. In order to make use of this functionality introduced in this commit, a core must make two changes in its mpas_<CORE>_core_interface module: 1) The module should include through a pre-processor directive the registry-generated package-logic code, which resides in a file named 'setup_packages.inc' in the core's inc/ directory; e.g., 2) The setup_packages routine in the module should make a call to the generated <CORE>_setup_packages_when routine, which is defined in the 'setup_packages.inc' file; e.g., call atm_setup_packages_when(configs, packages)

Although the return error codes from calls to various functions in parse_reg_xml were saved to a local variable, the parse_reg_xml function always returned 0. As a result, any errors were effectively masked, and the registry 'parse' program always appeared to complete successfully. This commit modifies the Registry parse_reg_xml so that the return error codes from functions are checked, and if non-zero, cause the parse_reg_xml function to return a non-zero error code to the main program.

The registry-generated package-logic code for packages that employ an active_when attribute now contains calls to mpas_log_write to record information about the activity of packages to a core's log file. For example, the following messages are now written to the log file if two packages, foo and bar, were active and inactive, respectively: Configuring registry-specified packages... foo : T bar : F ----- done configuring registry-specified packages -----

…DEBUG=true To aid in debugging packages that employ the active_when attribute in the Registry, this commit adds log messages to the registry-generated package-logic code that are only written to a core's log file when MPAS is compiled with DEBUG=true. For example, if the following two packages were defined: <package name="foo" active_when="config_do_physics"/> <package name="bar" active_when="config_nsteps > 0 .and. .not. config_dryrun"/> compiling with DEBUG=true would result in log messages that would look like the following (assuming the indicated values for namelist options config_do_physics, config_nsteps, and config_dryrun): Configuring registry-specified packages... foo is active when (config_do_physics) namelist settings: ------------------ config_do_physics = T foo : T bar is active when (config_nsteps > 0 .and. .not. config_dryrun) namelist settings: ------------------ config_nsteps = 100 config_dryrun = T bar : F ----- done configuring registry-specified packages -----

In order for the init_atmosphere core to make use of the ability for package logic to be generated by the registry through the active_when XML attribute, this commit includes the generated code in 'setup_packages.inc' in the init_atm_core_interface module, and it also adds a call to the generated init_atm_setup_packages_when routine from the init_atm_setup_packages routine. Note that the call to the generated init_atm_setup_packages_when routine occurs before any of the hand-written logic in the init_atm_setup_packages routine. As a consequence, settings of packages in the generated logic will be superseded by settings from hand-written logic for those packages.

In order for the atmosphere core to make use of the ability for package logic to be generated by the registry through the active_when XML attribute, this commit includes the generated code in 'setup_packages.inc' in the atm_core_interface module, and it also adds a call to the generated atm_setup_packages_when routine from the atm_setup_packages routine. Note that the call to the generated atm_setup_packages_when routine occurs before any of the hand-written logic in the atm_setup_packages routine. As a consequence, settings of packages in the generated logic will be superseded by settings from hand-written logic for those packages.

mgduda · May 27, 2025

I think the code changes look good overall, and I've done some testing - seems fine. I can approve when this PR is finalized.

Ok, this PR should be ready for a final review now.

abishekg7

Ran more checks and everything looks good.

MPAS Version 8.3.0 This release of MPAS introduces new capabilities and improvements in the MPAS-Atmosphere model and its supporting software infrastructure. Notable changes are listed below. Initialization: * Addition of support for 30" BNU soil category dataset. The 30" BNU soil category dataset can be selected by setting the new namelist option config_soilcat_data to 'BNU' in the &data_sources namelist group. Use of this dataset requires a separate static dataset download. (PR #1322) * Addition of support for 15" MODIS land use dataset. The 15" MODIS land use dataset may be selected by setting the existing namelist option config_landuse_data to 'MODIFIED_IGBP_MODIS_NOAH_15s' in the &data_sources namelist group. Use of this dataset requires a separate static dataset download. (PR #1322) * Introduction of a new namelist option, config_lu_supersample_factor, to control the super-sampling of land use data, which may now be on either a 30" or a 15" grid, depending on the choice of dataset. The existing namelist option config_30s_supersample_factor now controls the super-sampling for 30" terrain, soil category, and MODIS FPAR monthly vegetation fraction data only. (PR #1322) * A change in the horizontal interpolation from a four-point bilinear interpolation to a sixteen-point overlapping parabolic interpolation for both initial conditions and lateral boundary conditions. (PR #1303) * Ability to use ICON soil moisture and soil temperature fields. (PR #1298) * Addition of an option to skip processing of Noah-MP-only static fields in the init_atmosphere core. Setting the new config_noahmp_static namelist option to false in the &data_sources namelist group prevents the Noah-MP static fields from being processed when config_static_interp = true in the namelist.init_atmosphere file; this also permits existing static files that lack the Noah-MP fields 'soilcomp', 'soilcl1', 'soilcl2', 'soilcl3', and 'soilcl4' to be used by the init_atmosphere_model program. (PR #1239) * Memory scaling improvements to the gravity wave drag (GWD) static field processing in the init_atmosphere core (when 'config_native_gwd_static = true') to reduce memory usage when multiple MPI ranks are used. In many cases, these changes eliminate the need to undersubscribe computing resources, which was previously required in order to work around lack of memory scaling in the GWD static field processing. (PR #1235) Physics: * Update of the RRTMG LW and SW schemes, most notably with the addition of the exponential and exponential_random cloud overlap assumptions. The cloud overlap assumption and decorrelation length are now available as namelist options (config_radt_cld_overlap and config_radt_cld_dcorrlen, respectively). (PR #1296 and PR #1297) * The incorporation of NOAA's Unified Forecast System (UFS) Unified Gravity Wave Physics (UGWP) suite of physics parameterizations. This physics package is the "NOAA/GSL" orographic gravity wave drag (GWD) suite introduced in WRF Version 4.3 (activated by WRF namelist option 'gwd_opt=3'), but with the addition of a non-stationary GWD parameterization that represents gravity wave sources such as deep convection and frontal instability. The use of the UGWP suite requires additional static field downloads. (PR #1276) Dynamics: * Complete port of all routines in the dynamical core to GPUs using OpenACC directives, including routines used by limited-area simulations. Not included in this release, though, is the optimization of data movement between the CPU and GPU memory, and the profiling and optimization of the computational kernels. * A change in the zero-gradient LBC for w to a constant value of w=0 in the specified zone. For limited-area configurations, the change from a zero-gradient boundary condition for the vertical velocity, w, to a setting of the vertical velocity to zero in the specified region alleviates spurious streamers and instabilities that appeared near the boundaries in regions of strong inflow. (PR #1304) Infrastructure: * Implementation of a new capability to automatically generate package logic code, which determines when a package is active. This package logic is generated by the registry at build time through the use of a new XML attribute, active_when, for <package> elements. (PR #1321) Other: * Addition of a new Python script for setting up MPAS-Atmosphere run directories. (PR #1326) * Addition of 3-d 10 cm radar reflectivity (refl10cm) to the 'da_state' stream, useful for radar DA and radar obs comparison purposes. (PR #1323)

MPAS Version 8.3.0 This release of MPAS introduces new capabilities and improvements in the MPAS-Atmosphere model and its supporting software infrastructure. Notable changes are listed below. Initialization: * Addition of support for 30" BNU soil category dataset. The 30" BNU soil category dataset can be selected by setting the new namelist option config_soilcat_data to 'BNU' in the &data_sources namelist group. Use of this dataset requires a separate static dataset download. (PR MPAS-Dev#1322) * Addition of support for 15" MODIS land use dataset. The 15" MODIS land use dataset may be selected by setting the existing namelist option config_landuse_data to 'MODIFIED_IGBP_MODIS_NOAH_15s' in the &data_sources namelist group. Use of this dataset requires a separate static dataset download. (PR MPAS-Dev#1322) * Introduction of a new namelist option, config_lu_supersample_factor, to control the super-sampling of land use data, which may now be on either a 30" or a 15" grid, depending on the choice of dataset. The existing namelist option config_30s_supersample_factor now controls the super-sampling for 30" terrain, soil category, and MODIS FPAR monthly vegetation fraction data only. (PR MPAS-Dev#1322) * A change in the horizontal interpolation from a four-point bilinear interpolation to a sixteen-point overlapping parabolic interpolation for both initial conditions and lateral boundary conditions. (PR MPAS-Dev#1303) * Ability to use ICON soil moisture and soil temperature fields. (PR MPAS-Dev#1298) * Addition of an option to skip processing of Noah-MP-only static fields in the init_atmosphere core. Setting the new config_noahmp_static namelist option to false in the &data_sources namelist group prevents the Noah-MP static fields from being processed when config_static_interp = true in the namelist.init_atmosphere file; this also permits existing static files that lack the Noah-MP fields 'soilcomp', 'soilcl1', 'soilcl2', 'soilcl3', and 'soilcl4' to be used by the init_atmosphere_model program. (PR MPAS-Dev#1239) * Memory scaling improvements to the gravity wave drag (GWD) static field processing in the init_atmosphere core (when 'config_native_gwd_static = true') to reduce memory usage when multiple MPI ranks are used. In many cases, these changes eliminate the need to undersubscribe computing resources, which was previously required in order to work around lack of memory scaling in the GWD static field processing. (PR MPAS-Dev#1235) Physics: * Update of the RRTMG LW and SW schemes, most notably with the addition of the exponential and exponential_random cloud overlap assumptions. The cloud overlap assumption and decorrelation length are now available as namelist options (config_radt_cld_overlap and config_radt_cld_dcorrlen, respectively). (PR MPAS-Dev#1296 and PR MPAS-Dev#1297) * The incorporation of NOAA's Unified Forecast System (UFS) Unified Gravity Wave Physics (UGWP) suite of physics parameterizations. This physics package is the "NOAA/GSL" orographic gravity wave drag (GWD) suite introduced in WRF Version 4.3 (activated by WRF namelist option 'gwd_opt=3'), but with the addition of a non-stationary GWD parameterization that represents gravity wave sources such as deep convection and frontal instability. The use of the UGWP suite requires additional static field downloads. (PR MPAS-Dev#1276) Dynamics: * Complete port of all routines in the dynamical core to GPUs using OpenACC directives, including routines used by limited-area simulations. Not included in this release, though, is the optimization of data movement between the CPU and GPU memory, and the profiling and optimization of the computational kernels. * A change in the zero-gradient LBC for w to a constant value of w=0 in the specified zone. For limited-area configurations, the change from a zero-gradient boundary condition for the vertical velocity, w, to a setting of the vertical velocity to zero in the specified region alleviates spurious streamers and instabilities that appeared near the boundaries in regions of strong inflow. (PR MPAS-Dev#1304) Infrastructure: * Implementation of a new capability to automatically generate package logic code, which determines when a package is active. This package logic is generated by the registry at build time through the use of a new XML attribute, active_when, for <package> elements. (PR MPAS-Dev#1321) Other: * Addition of a new Python script for setting up MPAS-Atmosphere run directories. (PR MPAS-Dev#1326) * Addition of 3-d 10 cm radar reflectivity (refl10cm) to the 'da_state' stream, useful for radar DA and radar obs comparison purposes. (PR MPAS-Dev#1323)

mgduda added the Framework label May 22, 2025

mgduda requested review from abishekg7 and mattldawson May 22, 2025 18:48

mattldawson approved these changes May 23, 2025

View reviewed changes

src/tools/registry/gen_inc.c Outdated Show resolved Hide resolved

src/tools/registry/gen_inc.c Outdated Show resolved Hide resolved

mgduda force-pushed the framework/generated_package_logic branch from 545bace to 1797ea1 Compare May 27, 2025 19:55

mgduda added 5 commits May 27, 2025 13:56

mgduda force-pushed the framework/generated_package_logic branch from 1797ea1 to a62065e Compare May 27, 2025 19:59

mgduda marked this pull request as ready for review May 27, 2025 19:59

abishekg7 approved these changes May 27, 2025

View reviewed changes

mgduda merged commit 736ce99 into MPAS-Dev:develop May 27, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Enable package logic code to be automatically generated through new XML attribute, active_when #1321

Enable package logic code to be automatically generated through new XML attribute, active_when #1321

Uh oh!

mgduda commented May 22, 2025

Uh oh!

mattldawson left a comment

Uh oh!

Uh oh!

Uh oh!

mgduda commented May 23, 2025

Uh oh!

abishekg7 commented May 23, 2025

Uh oh!

mgduda commented May 27, 2025

Uh oh!

abishekg7 left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Search code, repositories, users, issues, pull requests...

Enable package logic code to be automatically generated through new XML attribute, active_when #1321

Enable package logic code to be automatically generated through new XML attribute, active_when #1321

Uh oh!

Conversation

mgduda commented May 22, 2025

Uh oh!

mattldawson left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mgduda commented May 23, 2025

Uh oh!

abishekg7 commented May 23, 2025

Uh oh!

mgduda commented May 27, 2025

Uh oh!

abishekg7 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants