Add hurricane cases from legacy compass #390
Conversation
|
@xylar - This is still a work in progress, but I started the PR in case you want to keep tabs and point out any issues along the way. |
fb1cba7 to
b8d9709
Compare
306b33d to
199e954
Compare
|
@xylar, I still need to do the documentation on this, but I think it's ready for you to look at when you have time. |
3e6e394 to
cdb5ebf
Compare
xylar
left a comment
There was a problem hiding this comment.
This is superb work! Very exciting to see this come together so nicely.
When I try to set up the 3 tests on Anvil, I'm seeing:
$ compass setup -n 211 212 213 -p E3SM-Project/components/mpas-ocean -w /lcrc/group/e3sm/ac.xylar/compass_1.1/anvil/test_20220607/hurricane
Setting up test cases:
ocean/hurricane/DEQU120at30cr10rr2/mesh
ocean/hurricane/DEQU120at30cr10rr2/init
ocean/hurricane/DEQU120at30cr10rr2/sandy
Traceback (most recent call last):
File "/home/ac.xylar/anvil/miniconda3/envs/compass_hurricane/bin/compass", line 33, in <module>
sys.exit(load_entry_point('compass', 'console_scripts', 'compass')())
File "/gpfs/fs1/home/ac.xylar/mpas-work/compass/hurricane/compass/__main__.py", line 63, in main
commands[args.command]()
File "/gpfs/fs1/home/ac.xylar/mpas-work/compass/hurricane/compass/setup.py", line 372, in main
setup_cases(tests=tests, numbers=args.case_num,
File "/gpfs/fs1/home/ac.xylar/mpas-work/compass/hurricane/compass/setup.py", line 146, in setup_cases
setup_case(path, test_case, config_file, machine, work_dir,
File "/gpfs/fs1/home/ac.xylar/mpas-work/compass/hurricane/compass/setup.py", line 289, in setup_case
step.setup()
File "/gpfs/fs1/home/ac.xylar/mpas-work/compass/hurricane/compass/ocean/tests/hurricane/analysis/__init__.py", line 72, in setup
with open(path) as stations_file:
TypeError: expected str, bytes or os.PathLike object, not _GeneratorContextManager
I'll see if I can understand what might be going wrong there.
I'm running some checks to make sure the nightly suite still works as expected as well.
compass/ocean/tests/hurricane/mesh/dequ120at30cr10rr2/dequ120at30cr10rr2.cfg
Outdated
Show resolved
Hide resolved
compass/ocean/tests/hurricane/mesh/dequ120at30cr10rr2/dequ120at30cr10rr2.cfg
Outdated
Show resolved
Hide resolved
|
Nightly test suite looks good. I didn't try a BFB comparison yet. |
- Add do_inject_bathymetry, preserve_floodplain, and floodplain_elevation to compass.ocean.tests.global_ocean.mesh.MeshStep
xylar
left a comment
There was a problem hiding this comment.
Coming along! But i'm still running into an issue, see below.
|
@sbrus89, it doesn't look like you pushed the changes that address my comments yet. Happy to re-re-review when that happens. |
xylar
left a comment
There was a problem hiding this comment.
I got a bit further in my testing but it seems like we'll need to work on this still because of memory constraints.
When running on Anvil, I'm seeing:
File "/gpfs/fs1/home/ac.xylar/mpas-work/compass/hurricane/compass/run/serial.py", line 130, in run_tests
test_case.run()
File "/gpfs/fs1/home/ac.xylar/mpas-work/compass/hurricane/compass/ocean/tests/hurricane/init/__init__.py", line 61, in run
super().run()
File "/gpfs/fs1/home/ac.xylar/mpas-work/compass/hurricane/compass/testcase.py", line 166, in run
self._run_step(step, self.new_step_log_file)
File "/gpfs/fs1/home/ac.xylar/mpas-work/compass/hurricane/compass/testcase.py", line 282, in _run_step
step.run()
File "/gpfs/fs1/home/ac.xylar/mpas-work/compass/hurricane/compass/ocean/tests/hurricane/init/interpolate_atm_forcing.py", line 251, in run
np.square(v_data[2][:]))
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 16.9 GiB for an array with shape (1465, 881, 1761) and data type float64
| # number of threads | ||
| forward_threads = 1 | ||
| .. _global_ocean_mesh: |
There was a problem hiding this comment.
| .. _global_ocean_mesh: | |
| .. _hurricane_mesh: |
| vel_interp = u_interp | ||
| vel_interp[2][:] = np.sqrt(np.square(u_interp[2][:]) + | ||
| np.square(v_interp[2][:])) | ||
| vel_data = u_data | ||
| vel_data[2][:] = np.sqrt(np.square(u_data[2][:]) + | ||
| np.square(v_data[2][:])) | ||
| self.plot_interp_data(vel_data, vel_interp, | ||
| 'velocity magnitude', 'vel', xtime) |
There was a problem hiding this comment.
It might be necessary to iterate over one of the 3 dimensions here (probably time) and compute the velocity magnitude separately for each time index. Slower but much less memory intensive.
If you were to do this with xarray instead, you could use its chunking in time to handle this.
|
@sbrus89, this is the only other PR currently slated for inclusion in v1.1.0. But, as we discussed on Wednesday, that's up to you. If you want to fix the memory issue sometime between now and July 4th, I'll include it. If you don't have time, I'll move it to v1.2.0. |
|
@xylar, sounds good. I'm working on testing to see if I was able to fix the memory issues on Anvil right now. |
|
@xylar, it looks like this is working for me on Anvil now. |
|
Thanks @xylar ! |
No description provided.