APE26: Removing data storage (representations) from coordinate frames #112
APE26: Removing data storage (representations) from coordinate frames #112kbwestfall merged 50 commits into
Conversation
|
Pinging @eerovaher @eteq @StuartLittlefair @Cadair @ayshih @taldcroft as other "stakeholders" who may want to take a look at this and review! |
eerovaher
left a comment
There was a problem hiding this comment.
I am convinced that there should be exactly one type for coordinate data. This would prevent users from getting confused about which type they should be using and simplifies things for both astropy and downstream developers because we'll only have to support one type of coordinate data as input for our functions.
This leaves two possibilities: the single type is a class, which is what this APE proposes, or the single type is a Protocol, which is what Implement BaseCoordinateFrame.frame property (astropy/astropy#16356) (that the APE mentions) proposed. Implementing a Protocol would be backwards-compatible and wouldn't cause disruption for downstream packages. In every other regard this APE is a better idea.
|
Yes, yes and a thousand times yes. The devil will be in the implementation details but this is a thoroughly good idea. |
|
Does anyone else have thoughts before we ask the @astropy/coordinators to review this? cc @eteq @Cadair @ayshih @taldcroft! |
|
I have been traveling the last several weeks and haven't had a chance to review this in detail, but I will try to do so this week. I am fairly sure I will have Opinions ™️ (but hopefully positive ones?) |
|
@eteq OK cool. It'd be good to get your feedback ASAP, because @jeffjennings is starting to work through some of the changes we discuss here (in draft stages, but still, it is somewhat in progress). |
|
I have two main thoughts about some effects from this change, I think they can both be managed, but probably worth having a think about.
|
Thanks for pinging me about this change. If I'm following the end result will be that coordinate frames will no longer have Reading old files with
|
eteq
left a comment
There was a problem hiding this comment.
Alright, I've left several specific inline comments. I also have several bigger picture concerns. Note that I am not dead-set against this - I get the motivation for this change and can see how it might be an improvement. But I want to be sure we address these concerns before going down this road:
- This is a major change in communicating the purpose of SkyCoord. Previously (i.e. in APE5) SkyCoord is highlighted as a "convenience class" - primarily this is 1) it has a much smarter and easier to use initializer, and 2) it carries the frame attributes along with the data, regardless of whether that frame attribute is in the current frame. This makes subtly different behavior for certain transformations depending on whether you give the to frame the same frame attributes or not. I don't really understand how this proposal will change that - i.e., if there are no frame classes with data, how do you say "do the transform that only uses the frame attributes in the to-frame, but doesn't carry them over from the from-frame"? I can thinks of ways to do this, but this was one of the prime reasons for data and frames being tied together in the first place, to keep that case simpler and not dependent on SkyCoord.
- We should get someone in Sunpy look at this before proceeding. hopefully this won't be a big change for them since I think mostly they define their own frames and transforms, which should be backwards compatible with these changes. But it may be they use the low-level classes directly more often and hence this would be a huge negative impact on them.
- Relatedly, I'm concerned about the number of examples out in the wild that use the low-level classes directly. There are various use cases where we have encouraged people to use the low-level classes with data because they are much cheaper for initialization. If we are going to remove that option they will be impacted whenever the removal finally happens.
- I think it's absolutely critical we highlight the change in philosophy from what APE5 said. This sub-package is sufficiently complex that documentation (either in APEs or otherwise) are critical to helping people understand the big-picture. This might be as simple as just making sure https://docs.astropy.org/en/stable/coordinates/index.html#astropy-coordinates-overview and related documentation spots are up-to-date, but still, that should be explicitly in the implementation plan IMHO since it's a non-trivial effort to update all the docs for this.
| different usage modes (with and without data), each exhibiting different behavior. For | ||
| instance, some methods such as ``separation`` work fine with frames with data, while | ||
| doing this on coordinate frames without data results in an error. This kind of | ||
| multi-modal structure is considered an anti-pattern because it forces any code |
There was a problem hiding this comment.
as with above I think "is considered an anti-pattern" is a bit strong. It makes code working with frame classes be in one place but explicitly have to check for the presence or absence of data. Which for someone trying to understand the code might be a benefit.
I think that this is really a tradeoff between two use cases:
- Someone trying to understand coordinates-related functionality (not just astropy maintainers - this includes e.g. sunpy folks)
- An astropy maintainer who is really groking the layout of the package, how this stuff all works
I think this change benefits 2 over 1. Which doesn't mean we shouldn't do it, but it leaves out the consequence that it makes the bigger picture harder to understand. If we want to go ahead despite that, well, we can, but we need to recognize that choice is being made and be explicit about it here.
(I do, however, agree with the points immediately below that the end-user is the more important thing - this is more like "new maintainer" vs "old/more knowledgable maintainer")
There was a problem hiding this comment.
Thanks Erik - I've reworked the text to reflect this in 46a6d82 (including explicitly noting the design choice we propose that departs from APE 5, as you suggested).
| overlap leads to confusion where beginner users end up creating ``BaseCoordinateFrame`` | ||
| objects such as ``ICRS``, when the docs are clear that these are for more advanced users |
There was a problem hiding this comment.
I'm not sure this is an end-user gain: it's a feature not a bug that SkyCoord and ICRS both can be instantiated with data. SkyCoord is supposed to be a convenience class, with ICRS the simpler underlying layer - that's the whole principle here. I don't really understand how the proposed change makes things any clearer?
There was a problem hiding this comment.
I can see how this is a feature, but I think the improved user clarity is motivated earlier on in the text - currently, a user can initialize an ICRS object or a SkyCoord object, which "represent the same thing, but return different objects with similar but not identical APIs". The proposed change gives only way to have an object hold both data and frame information - SkyCoord - preventing user confusion about subtle differences between SkyCoord and a with-data frame like the current ICRS (including which one to use, and why their API differs in perhaps unexpected ways).
There was a problem hiding this comment.
Ah, I see that... But I'm a bit concerned that this still doesn't address the fact that there was a reason for this (i.e., they are intentionally different APIs because the objects behave differently when transformed and instantiated)
| 2. Switching ``SkyCoord`` to use the data-less frame classes, and enabling automatic | ||
| conversion of the with-data frames into ``SkyCoord`` objects. |
There was a problem hiding this comment.
I'm confused how this doesn't create a huge mess: does this mean all of the coordinate conversion process during the transition phase is duplicated between the "with data" classes and the "new SkyCoord" classes?
There was a problem hiding this comment.
There wouldn't be duplication because the coordination conversion machinery would be in the new (data-less) frame classes, and the legacy (with-data) frame classes would subclass the new (data-less) classes.
Name changes
Co-authored-by: Adrian Price-Whelan <adrianmpw@gmail.com>
Co-authored-by: Adrian Price-Whelan <adrianmpw@gmail.com>
Co-authored-by: Nathaniel Starkman <nstarman@users.noreply.github.com>
|
Hi @ayshih and @Cadair, we're starting to finalize this APE and are wondering if you anticipate any major concerns with respect to SunPy (as well as any suggestions you have, e.g. for addressing Stuart's point 2 above). The first phase of the APE wouldn't introduce any breaking changes, the second phase (timeline flexible) would add deprecation warnings, and the final phase (several major astropy versions away) would require changes to SunPy. Thanks! |
Co-authored-by: Eero Vaher <eero.vaher@gmail.com>
In the light of this development, CoCo will hold off on further reviews for now. Thank you. |
|
Possible updates from Astropy Coordination Meeting 2025:
|
Our proposed deprecation stages for
Keeping
|
|
Hi all, we now have a prototype and demo available for review. Please provide any comments within 2 weeks, by November 14. We recommend reading through the demo notebook and then taking a look at the prototype implementation as needed (see the APE's "Implementation" section for links). Note that the implementation is not exhaustive, e.g., Signed, |
eerovaher
left a comment
There was a problem hiding this comment.
-
If the
Coordinateclass is inherently a good idea then it shouldn't need the backing of an APE to get it implemented. If it exists mostly as a replacement forBaseCoordinateFrameinstances with data then I'm not fully convinced it is required at all. Either way, it seems to me the existence ofCoordinateshould not be dictated by the APE. If there is some good reason to include it in the APE then the APE should justify it better. -
The design proposed here is the second option from the APE 5 "Alternatives" section. APE 5 seems to reject that option because of a claim that it is impossible to implement the "high-level" classes as generics (although APE 5 does not use the term "generics"). To me it is obvious that the claim is false, which is why it is surprising to me that no-one seems to have called it out when APE 5 was discussed. Nonetheless, because APE 5 does make that claim then this APE should have a paragraph that comments on that. Additionally, the demonstration notebook should contain examples of both spherical representations that do not use RA and Dec, and also non-spherical representations, because that is what APE 5 claims will be difficult to implement.
| represent the same thing, but return different objects with similar | ||
| but not identical APIs: |
There was a problem hiding this comment.
Provide examples of the differences.
| also now see it from the perspective of an anti-pattern. It | ||
| forces any code interacting with the frame classes to handle both | ||
| cases (checking ``frame.has_data``), complicating the codebase. |
There was a problem hiding this comment.
I suspect that in practice it is more common to avoid code complications by simply assuming that all frames have data, or that none of the frames have data.
There was a problem hiding this comment.
Yes although both cases still have to be supported in the current framework, while the proposed changes would avoid the issue both in practice and conceptually.
| objects). We also introduce a new class, ``Coordinate``, which | ||
| is akin to ``SkyCoord`` (containing both frame and data), but | ||
| without extra features like keeping frame attributes not | ||
| associated with the current frame, caching and flexible input | ||
| parsing. In this way ``Coordinate`` operates very similarly to | ||
| the current ``BaseCoordinateFrame`` objects when they have | ||
| data, and is meant to be their direct replacement in the new |
There was a problem hiding this comment.
Is the purpose of Coordinate to be a replacement for current frame classes with data? Would it be sensible to define both SkyCoord and Coordinate if you were designing an API from scratch without any historical baggage?
There was a problem hiding this comment.
I think so, they aim for slightly different things: a WYSIWYG implementation and something with a bit more user-ergonomics optimizations: more flexible init, caching, etc.
Perhaps the flexible init could be a function, e.g. make_coordinate(), but in combination with other features, like caching, IMO a class is a good choice too.
| class BaseCoordinate: | ||
| frame: BaseFrame | ||
| data: BaseRepresentation |
There was a problem hiding this comment.
The pseudocode would be much more informative with the PEP 695 Type Parameter Syntax.
There was a problem hiding this comment.
Hmm I think the pseudocode is clear in conveying inheritance, but the accompanying prototype now linked in the APE is yet another reference.
| - `Masked frames and SkyCoord #17106 | ||
| <https://github.com/astropy/astropy/pull/17016>`_ (this was later | ||
| removed and instead methods were duplicated) |
There was a problem hiding this comment.
Use an example that was merged. There should be plenty to choose from.
| - Still warn, but return a ``Coordinate``, not an instance of | ||
| its class type (by overriding ``__new__``) |
There was a problem hiding this comment.
I don't see how overriding __new__() in such a manner could possibly be a good idea. If you want to make the change more gradual than APE 2 requires then just make the deprecation period longer.
|
With the acceptance of the revisions to APE 1, we're going to start following the new APE proposal process. While all of the APE editors (for now exactly the same as the CoCo) will contribute to the process, one of the APE editors will be the point-person (chief APE editor) shepherding your team through the process. Your chief APE editor is Kyle Westfall (@kbwestfall). You can review the next steps in the APE process here, but just to briefly summarize them here:
Please let us (either your chief APE editor or the full CoCo) know if you have any questions! |
Co-authored-by: Eero Vaher <eero.vaher@gmail.com>
Co-authored-by: Eero Vaher <eero.vaher@gmail.com>
Co-authored-by: Eero Vaher <eero.vaher@gmail.com>
Co-authored-by: Eero Vaher <eero.vaher@gmail.com>
Thanks for your comments Eero. The purpose of the APE has been to propose a fairly large set of changes to the approach and implementation of a core part of
Great, I've added a note on your APE 5 point to this APE, and added examples with cylindrical coordinates to the demo. |
We prefer implementing any such changes in a single PR.
We anticipate 2 weeks will be sufficient.
Hi @astropy/ape-editor-team and @kbwestfall, thank you for all of this. I've just finalized the APE draft and left a few comments. So we're ready to move forward. |
kbwestfall
left a comment
There was a problem hiding this comment.
Here's my editorial review, which includes mostly minor wording recommendations. Thanks!
| remain essentially unchanged, except that it will operate specifically on the | ||
| low-level classes. | ||
|
|
||
| UPDATE: Portions of the implementation presented here are amended in `APE 26 |
There was a problem hiding this comment.
I'm not exactly sure of the protocol, but I expect you need to add a revised-by: section at the top of the file that briefly describes this revision. See APE 1 for an example.
You also might use an rst directive (like note or important) to highlight this.
(Although not an editing comment, this also begs the question about whether APE26 should be written to supersede APE5. I don't know APE5 well enough to comment, but it may be odd to have an APE that is only superseded in part.)
There was a problem hiding this comment.
Good point - I added a note to APE 5 in 19bfe90.
I agree that this APE only supersedes a part of APE5, and I think that trying to 'fully' supersede APE5 with APE26 might just end up being less clear than having readers of one refer to the other if needed to understand what has changed.
Thank you for such a fast review! I've incorporated all of your suggestions, but of course let me know if I can do anything else before the next step. |
Coordinate frames in
astropy.coordinatescurrently store metadata used to construct the frame (i.e. for transforming between frames) and may also store coordinate data itself. This duplicates functionality withSkyCoord, which acts as a container for both coordinate data and reference frame information. We propose to change the frame classes such that they only store metadata and never coordinate data. This would make the implementation more modular, remove ambiguity for users from having nearly duplicate functionality with slightly different APIs, and better satisfy the principle of Separation of Concerns.Authors of this APE (alphabetical): Jeff Jennings @jeffjennings, Adrian Price-Whelan @adrn, Nathaniel Starkman @nstarman, Marten van Kerkwijk @mhvk