JOSS paper submission #124
Conversation
|
We were calling astropy every time we wanted the age of a stellar particle. I've move to using a lookup table. I verified it should be accurate using the following script |
|
I changed the documentation so that properties now link to any footnotes they have |
|
I removed a redundant sort from SO_properties which I had introduced for the concentration calculation. |
|
I've switched off iterative calculation of inertia tensors for the time being, although we may instead use a subsample of the particles to determine the inertia tensor. |
|
I have merged in #126, which allows for SOAP to restart. I have updated the property table and parameter files so that filters (e.g. basic, general, stars) are now specified in the parameter file. I'm removed some leftover VR FOFSubhalo code |
|
I'm now sorting halos on each chunk so that the largest objects are processed first |
|
For projected apertures and exclusive spheres we were often recomputing properties using the same particles each time. This occurred for small subhalos, where their EncloseRadius (the distance from the halo centre to the furthest particle) was smaller than multiple apertures (e.g. if EncloseRadius=30kpc, then the 50kpc & 100kpc ExlcusiveSpheres would be the same). In these cases we now just copy across the values from the previous aperture, as they would be the same. There is one wrinkle, which is that the area of the circle we use for calculating the COLIBRE has quite a large number of high z snapshots. This means we were spending a lot of time on InclusiveSpheres. I've added a parameter that allows halo to be skipped if the InclusiveSphere is larger than the EncloseRadius. This parameter defaults to False, but we will set it to True for COLIBRE. |
* Add shorthand to obtain grik luminosities. * Add basic functions to measure disc to total luminosity in different bands. * Make new properties not be computed by default * Add call of unimplemented function that computes light weighted L quantities. * Define function for luminosity weighted L quantities. Not implemented for now. * Add docs on parameters for get_angular_momentum_and_kappa_corot_luminosity_weighted * Make kappa_corot and counterotating mass give a value per GAMMA band * Add counterrotating luminosity array * Handle multiple luminosity bands when computing average angular momentum * Compute projected angular momentum if we do_counterrot_luminosity * We now compute if stars are counterrotating or not for each luminosity band. Expanded description of unyt array shapes to keep track in each step. * fix typo * Basic implementation done. * Add correct return and remove NotImplemented error * Change variables assigned to subhaloes to prevent overwritting mass-weighted calculations * Only return a single property, which includes all GAMMA bands. * Return a mass and light-based D/T. * Add kappa_corot for luminosity bands * change variable name * Add property to save all luminosity-weighted angular momentum * Add property definition in the Aperture properties * Added new properties to the property table * Add properties to the subhalo properties parameter file of COLIBRE * Remove lazy properties that were not needed anymore * Update docstring * Add luminosity-weighted angular momenta to bound subhalo properties * Add properties to the ApertureProperties parameter file section * Change output expected type * Change condition to execute new properties. Mainly because of an annoying inconsistent between how starless subhaloes are handled. Mstar = 0 but Luminosities = None. * Fix some indicies and if condition * Add important clarification * Fix bug in kappa_corot definition. * Formatting alignment * Add luminosity-weighted quantities to SO apertures * Clarification comment * Individual angular momentum for particles is now correct for kinetic energy calculation * Fix GAMA typo * Remove mass term form average angular momentum we keep it in the individual particle angular momentum to obtain the kinetic energy in rotation. * Create copy of inertia tensor to implement luminosity weighting * Update description and inputs of inertia tensor * Mask luminosities if we mask other properties * Add comment... * Create correct shape for eigenval and eigenvec matricies * Shapes for each luminosity band * Add stopping criteria on a per band basis * Define the luminosity-weighted inertia tensor for BoundSubhalos * Compute particle positions for each band. Perhaps unneccessary since they will have the same positions in every band... Check how to improve later on. * Update break condition to be done on a luminosity band basis * Assign luminosity-based weights * is_converged is now a boolean array * Remove comment that was already addressed * Tensor for each band done. Still need to check if it gives me the correct element ordering relative to the default calculation of the inertia tensor. * Re-add removed calculation of inertia tensor eigenvalue/vectors * Create variable to hold the number of provided luminosity bands * Return flattened matricies for each band * Remove bug in stopping criteria for iterative tensor * Added missing np.abs() call It was causing the new function to stop iterating early * Add import of new function * Fix to prevent overflows into negative values for eigenvectors of inertia tensor * Defined luminosity weighted projected inertia tensor * Add call to luminosity-weighted function (undefined yet) * Define luminosity weighted function and pass luminosity arrays. * Correct typo from main branch * Finish implementation of get_projected_inertia_tensor_luminosity_weighted. Directly taken from 3D inertia calculations. Will likely need some restructuring. * Fix incorrect comment statement * Bug: using 3D positions instead of projected... * Change location of axis ratio transpose * Handle q = 0 values when testing convergence. * Handle 0 / 0 division. * Make q be an array from the get go * Only compute tensors for non converged bands * Do not rely on disabling warnings to handle q = 0 cases. * Make q = 0 tensors to have all matrix entries to be equal to 0. * Formatting * Update variable names and parameter files * Make get_inertia_tensor_luminosity_weighted only do computations for non-converged bands More similar to get_projected_inertia_tensor_luminosity_weighted * Fix issue resulting from merging * Make 3D inertia tensors general * Fix typo en error message * Make mass-weighted inertia tensor more self-evident * Make projected inertia tensors general * Two bugs: incorrect function call and units in reduced tensors * Move inertia tensor functions into their own file * Fix unyt array initialisation to specify unit registry * Generalise kappa_corot * Shorten the description of helper functions * Update colibre test * Reformat * Add note on single particle * Remove questions --------- Co-authored-by: robjmcgibbon <robjmcgibbon@gmail.com>
* Add more EAGLE snapshot properties * Add script to approximate H fractions * Use proper index for SOAP EAGLE * Update scripts * Squash bugs
* Add note on timing flags * Allow for property defined apertures * Add error messages if required radii are disabled * Missing method * Update pdf documentation * Add massive halos for testing * Format
* Merge fof position test * Update docs * Add project scripts & fix tests * Format * Add venv script for strw
* Backbone broken * Fix test in calculate_fof_pos * Run matching * Match to SOAP idx * Check matches are consistent * Output to file * Squash bug when saving * Update docs * Centrals only * Update README * Add __main__ * Run formatter
|
I've installed this branch to play around a bit. I have a little feedback on the packaging - as currently set up you can If the idea is for these directories to only have "scripts" to be run from the command line... well... I have a use case for the virtual snapshot writer function ;) |
|
Thanks @kyleaoman, I can change it to |
* Set lower limit * Format
* Add sbatch script for FOF radii * Skip assert for ranks with no halos * Nicer check
* Add format workflow * Add --check * Run formatter
* Exclude COLIBRE core * HalfMassRadii H
* Read binding energy from HBT * Output TotalBindingEnergy * Vmax for aperture properties * Squash bug and format * Stop ranks hanging when reading binding energy * Read SnapshotIndexOfLastIsolation * Read in potential energy instead of binding * Remove duplicate line * Use vCoM from all particles * Move subhalo_properties check for Ntot * Update COLIBRE test * Remove Ntot warning * Add documentation of SpecificPotentialEnergies * Add parameter file option to read energy * Review changes
* Editorial changes. * Update paper.bib * A couple of more changes * Update title capitalization
* Add sbatch script for FOF radii * Skip assert for ranks with no halos * Nicer check * More logging * Add option to copy datasets * Add node requirements
SOAP is taking a long time to run on COLIBRE. This PR adds some recording of how long we are spending processing each subhalo, and which properties are dominating the runtime. I will try and add comments for any speedups I add.
For satellites (for which we skip SO properties) we were still checking if the load region around them dropped below the target density of the lowest density SO. This had a significant effect on runtime, especially for high resolution COLIBRE runs where we identify lots of substructure.
EDIT There's a bunch of other changes here too, but they're all described by the comments. I'd like this branch to be the one that we use when we publish the SOAP paper. TODO before then: