Extend initial conditions from 1D to 2D and 2D to 3D #844

DimAdam-01 · May 26, 2025

Description

Please include a summary of the changes and the related issue(s) if they exist.
Please also include relevant motivation and context.

his pull request adds the ability to generate initial conditions by mapping a 1D simulation’s results onto 2D and 2D into 3D grids. It also introduces support for importing 1D profiles (for example, a premixed‐flame solution from Cantera) and using them to initialize and run a 1D simulation.
Fixes #(issue) [optional]

Type of change

Please delete options that are not relevant.

New feature (non-breaking change which adds functionality)

Scope

This PR comprises a set of related changes with a common goal

If you cannot check the above box, please split your PR into multiple PRs that each have a common goal.

How Has This Been Tested?

Please describe the tests that you ran to verify your changes.
Provide instructions so we can reproduce.
Please also list any relevant details for your test configuration

Test A: 1D → 2D Mapping (Uniform Domain)

Use a 2D domain whose x-length exactly matches the 1D simulation.
Distribute the grid across multiple MPI ranks.
Verify that the reconstructed 2D initial field matches the original 1D data.

-[x] Test 2: 1D → 2D Mapping (Partial Domain + Patch)

Extend only a portion of the 2D domain with the 1D profile, fill the remainder via a prescribed patch in case.py.
Run across multiple ranks with varying grid decompositions (different rank counts and grid-size ratios).
Test different horizontal offsets (e.g.\ placing the 1D profile at the left edge, centered, etc.) and confirm correct placement and values.

The same two tests were then carried over to 2D → 3D extensions, exercising uniform-domain mapping and partial-domain + patch scenarios in three dimensions.

Test Configuration:

What computers and compilers did you use to test this:

My Laptop and Delta (A100)

Checklist

I have added comments for the new code
I added Doxygen docstrings to the new code
I have made corresponding changes to the documentation (docs/)
I have added regression tests to the test suite so that people can verify in the future that the feature is behaving as expected
I have added example cases in examples/ that demonstrate my new feature performing as expected.
They run to completion and demonstrate "interesting physics"
I ran ./mfc.sh format before committing my code
New and existing tests pass locally with my changes, including with GPU capability enabled (both NVIDIA hardware with NVHPC compilers and AMD hardware with CRAY compilers) and disabled
This PR does not introduce any repeated code (it follows the DRY principle)
I cannot think of a way to condense this code and reduce any introduced additional line count

If your code changes any code source files (anything in `src/simulation`)

To make sure the code is performing as expected on GPU devices, I have:

Checked that the code compiles using NVHPC compilers
Checked that the code compiles using CRAY compilers
Ran the code on either V100, A100, or H100 GPUs and ensured the new feature performed as expected (the GPU results match the CPU results)
Ran the code on MI200+ GPUs and ensure the new features performed as expected (the GPU results match the CPU results)
Enclosed the new feature via nvtx ranges so that they can be identified in profiles
Ran a Nsight Systems profile using ./mfc.sh run XXXX --gpu -t simulation --nsys, and have attached the output file (.nsys-rep) and plain text results to this PR
Ran a Rocprof Systems profile using ./mfc.sh run XXXX --gpu -t simulation --rsys --hip-trace, and have attached the output file and plain text results to this PR.
Ran my code using various numbers of different GPUs (1, 2, and 8, for example) in parallel and made sure that the results scale similarly to what happens if you run without the new code/feature

codecov · May 26, 2025

Codecov Report

Attention: Patch coverage is 33.33333% with 4 lines in your changes missing coverage. Please review.

Project coverage is 43.46%. Comparing base (2f8eef1) to head (fd4aca9).

Files with missing lines	Patch %	Lines
src/pre_process/m_patches.fpp	33.33%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #844      +/-   ##
==========================================
- Coverage   43.47%   43.46%   -0.02%     
==========================================
  Files          68       68              
  Lines       19766    19772       +6     
  Branches     2375     2375              
==========================================
  Hits         8593     8593              
- Misses       9726     9732       +6     
  Partials     1447     1447

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

sbryngelson · May 26, 2025

Put an example and add to the documentation how to use this.

src/pre_process/include/1dHardcodedIC.fpp

sbryngelson · May 26, 2025

src/pre_process/include/1dHardcodedIC.fpp

+                    read (unit, *, iostat=ios) x_coords(iter), stored_values(iter, f)
+                    if (ios /= 0) then
+                        print *, "Error reading file ", trim(fileNames(f)), " at row ", iter
+                        exit  ! Exit loop on error


don't use print like this, it will print 1000 lines of same line if you use 1000 MPI ranks.

don't use exit or stop. use s_mpi_abort('Reason for exit')

src/pre_process/include/1dHardcodedIC.fpp

src/pre_process/include/2dHardcodedIC.fpp

sbryngelson · May 26, 2025

src/pre_process/include/2dHardcodedIC.fpp

+            write (file_num_str, '(I1)') f  ! Single digit
+        else
+            write (file_num_str, '(I2)') f  ! Double digit
+            ! For more than 99 files, you might need to adjust this format


what are these files you speak of

They are produced if we set paraller io = F in the case file, the files names are for example prim.1.00.00000.dat

what if the user uses parallel_IO=T? also you only get prim.1.00.00000.dat if you have prim_var_wrt true.

This is noted in the 1D_hardcoded description: the files are generated when parallel I/O is disabled in case.py, I will add it in the other hardcoded.fpp files.

does this now apply to all hardcoded patches, then? because obviously some of them we don't want to do this to...

I moved the reading of the files to the specific case (for example case(270) )

src/pre_process/include/2dHardcodedIC.fpp

src/pre_process/include/3dHardcodedIC.fpp

sbryngelson · May 26, 2025

src/pre_process/include/1dHardcodedIC.fpp

I'm not sure if this works with other hardcoded ICs? ask @wilfonba

sbryngelson · May 27, 2025

examples/2D_Detonation/case.py

how does this case work? can i just run it and it will run the 1d case and then restart in 2D? if not, then we need two case files and an explanation (readme?) describing how to use them. it also looks like you are disabling chemistry and enabling scaling?

You need to first run the 1D case, for example, the 1D_reactive_shocktube, and then run the 2D case (2D_Detonation).

that's not obvious at all

sbryngelson · May 27, 2025

src/pre_process/include/1dHardcodedIC.fpp

@@ -1,5 +1,53 @@
 #:def Hardcoded1DVariables()
+
+!> @file 1DHardcodedIC.fpp


confused on how exactly this works... it looks like it will do this for every hardcoded 1d case? we don't want that

and obviously not every hardcoded case is supposed to work like this (with chemistry and so on). maybe i'm just misunderstanding how this file works.

Currently with the last changes, it is only necessary to specify the init_dir and the number of rows (nRows). These parameters could be moved to the individual case file for better modularity. While specifying nRows is not strictly required, omitting it will result in significantly higher memory usage, since the parameter m (the actual grid size) can be used instead of nRows to determine the required allocation.

im not sure i understand what you mean

sbryngelson · May 27, 2025

src/pre_process/include/1dHardcodedIC.fpp

+    real(wp), dimension(nRows) :: x_coords
+    logical :: files_loaded = .false.
+    real(wp) :: domain_start, domain_end
+    character(len=*), parameter :: init_dir = "/home/YourDirectory" ! For example /home/MFC/examples/1D_Shock/D/


if we had hardcoded cases in the test suite, this wouldn't work, right? and we might want that in the future

I am pretty confident that it wouldn't cause an error, I will talk with @wilfonba and try to test it.

sbryngelson · May 27, 2025

src/pre_process/include/2dHardcodedIC.fpp

+    real(wp), dimension(nRows) :: x_coords
+    logical :: files_loaded = .false.
+    real(wp) :: domain_start, domain_end
+    character(len=*), parameter :: init_dir = "/home/pain/MFC-Adam/examples/1D_reactive_shocktube/D/"


you can't leave it like this i don't think... also since it's a parameter the user can't change it without recompiling, right?

sbryngelson · May 27, 2025

src/pre_process/include/3dHardcodedIC.fpp

    real(wp) :: rhoH, rhoL, pRef, pInt, h, lam, wl, amp, intH, alph

    real(wp) :: eps

+    integer, parameter :: nFiles = 15   ! Number of files (variables)
+    integer, parameter :: xRows = 401   ! Number of points in x


can't you make this part of the case dictionary? it looks like everytime someone wants to do this they have to modify the source code

maybe @wilfonba has an opinion on these hard coded patches because I don't use them much and am not super familiar with how one usually uses them

It was more practical for me to define the parameters directly in the hardcoded file, as most of the modifications in the cases I implemented, such as adding perturbations were concentrated there, rather than in case.py.

it seems like these parameters can be inferred from the input file. right now you have too many things you're hoping the user to do themself.

I suppose I could allocate memory, read the file to determine the number of grid points, and then resize the memory accordingly. The only thing the user would need to specify is the directory containing the files, along with the filename pattern (e.g., prim.xx.yy.zzzzz.dat).

can't you declare the variable as allocatable and then allocate it once you know how big the file is?

sbryngelson · May 29, 2025

src/pre_process/include/1dHardcodedIC.fpp

    ! Place any declaration of intermediate variables here
+    integer, parameter :: nRows = 512   ! Number of grid points


per MPI rank or global? what if you run with multiple ranks for the 1D case?

I mentioned in the Doxygen comments that it represents the total number of grid points from the 1D file, but I forgot to include this description when declaring the variable.

you can't read that info from the file itself?

sbryngelson · May 29, 2025

docs/documentation/case.md

+specific timestep to be extended uniformly in the transverse direction and used as a full 2D initial condition.
+
+case(370) in 3dHardcodedIC.fpp: used to build 3D domains by extruding 2D data along the third dimension.
+


Are these the only ones provided?

Yes, for three different dimensions. I usually add perturbations directly in the hardcoded case file rather than in the case.py file.

I'm confused why these are the only hardcoded ICs in the entire documentation or code. It seems like we should have many (and several examples). What's going on here? @wilfonba should know - or you should figure out from him

…atches, update docs and remove 2D_Detonation/case.py

DimAdam-01 and others added 18 commits May 6, 2025 14:42

Chemistry Bug Fixes

bcaaa35

Chemistry Bug Fixes

4833d7b

Format

c8e7f9f

Testing-Changes-In-CodeGenerator-For-Pyrometheus

652d0fa

Merge branch 'MFlowCode:master' into my_new_branch

e63683a

Extend Frontier Run Time

cc20ee9

Debugggg

b720ecc

Small Changes

1182c2c

Lodi for Multicomponent

24a324c

Merge

b02547e

Merge_2

308d919

Small stuff

b58a29f

Small stuff

1ba9082

lkele

415598a

Small Changes

8a04716

Small Changes v2

de149bd

Small Stuff

7739f59

Formatting

7e61509

DimAdam-01 requested a review from a team as a code owner May 26, 2025 03:15

DimAdam-01 changed the title ~~2/3 d extend ic~~ Extend initial conditions from 1D to 2D and 2D to 3D May 26, 2025

sbryngelson reviewed May 26, 2025

View reviewed changes

src/pre_process/include/1dHardcodedIC.fpp Outdated Show resolved Hide resolved

sbryngelson reviewed May 26, 2025

View reviewed changes

src/pre_process/include/1dHardcodedIC.fpp Outdated Show resolved Hide resolved

sbryngelson reviewed May 26, 2025

View reviewed changes

src/pre_process/include/2dHardcodedIC.fpp Outdated Show resolved Hide resolved

sbryngelson reviewed May 26, 2025

View reviewed changes

src/pre_process/include/2dHardcodedIC.fpp Outdated Show resolved Hide resolved

sbryngelson reviewed May 26, 2025

View reviewed changes

src/pre_process/include/3dHardcodedIC.fpp Outdated Show resolved Hide resolved

sbryngelson reviewed May 26, 2025

View reviewed changes

src/pre_process/include/3dHardcodedIC.fpp Outdated Show resolved Hide resolved

sbryngelson reviewed May 26, 2025

View reviewed changes

src/pre_process/include/1dHardcodedIC.fpp

Copy link

Member

sbryngelson May 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this works with other hardcoded ICs? ask @wilfonba

DimAdam-01 added 2 commits May 27, 2025 14:39

Improvements

d55b19e

Minor Changes

ec14e77

DimAdam-01 requested a review from a team as a code owner May 27, 2025 18:42

DimAdam-01 added 2 commits May 27, 2025 14:45

Formatting

d14900c

Changes in hardcoded fpp description

0342538

sbryngelson reviewed May 27, 2025

View reviewed changes

DimAdam-01 added 2 commits May 28, 2025 01:06

Changes in Hardcoded implemention

31822b8

...

f7d279b

sbryngelson reviewed May 29, 2025

View reviewed changes

sbryngelson and others added 6 commits May 29, 2025 15:49

Merge branch 'master' into 2/3D_Extend_IC

fe78533

Merge branch 'MFlowCode:master' into 2/3D_Extend_IC

651ddf9

Update HardcodedIC includes, add ExtrusionHardcodedIC.fpp, modify m_p…

f5dad3c

…atches, update docs and remove 2D_Detonation/case.py

Resolve merge conflict in m_patches.fpp

bf9ea7d

Hardcoded Extrusion

5b30ddf

Small Bugs

fd4aca9

		! Place any declaration of intermediate variables here
		integer, parameter :: nRows = 512 ! Number of grid points

		specific timestep to be extended uniformly in the transverse direction and used as a full 2D initial condition.

		case(370) in 3dHardcodedIC.fpp: used to build 3D domains by extruding 2D data along the third dimension.

Search code, repositories, users, issues, pull requests...

Extend initial conditions from 1D to 2D and 2D to 3D #844

Are you sure you want to change the base?

Extend initial conditions from 1D to 2D and 2D to 3D #844

Uh oh!

Conversation

DimAdam-01 commented May 26, 2025

Description

Type of change

Scope

How Has This Been Tested?

Checklist

If your code changes any code source files (anything in src/simulation)

Uh oh!

codecov bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sbryngelson commented May 26, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DimAdam-01 May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

If your code changes any code source files (anything in `src/simulation`)

codecov bot commented May 26, 2025 •

edited

Loading

DimAdam-01 May 28, 2025 •

edited

Loading