Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
Discussion options

I just ran a very naive speed test to compare SExtractor with SourceXtractor++ on a 2230² pixels image with 2025 sources above a 10 σ background: both source finders find exactly 2025 sources, but SExtractor is more than ten times faster: 0.238s for SExtractor vs. 3.124s for SourceXtractor++ .

Is this expected or can I perhaps switch off some overhead of SourceXtractor++ to make the comparison fairer?

You must be logged in to vote

Replies: 4 comments

Comment options

Although SE++ is expected to run slower than SE2, the factor of 10 you find is quite extreme. Maybe you ran SE2 repeatedly a few times and the input data was mem-mapped by the OS?

For SE++ the performance depends strongly on which output properties you ask via --output-properties. Assuming that you use no flexible model fitting and ask for simple properties (positions, isophotal photometry) only, keep attention to use no grouping --grouping-algorithm=NONE and there is not much to speed up else in SE++. Note that you can assign more cores to SE++, but for simple measurements that does not help a lot.

My rather personal take on SE2 vs. SE++ is:

  • if processing speed is essential and SE2 does what you want, stay with it (and live heapily ever after);
  • if you want to run flexible model fitting on multi-band data, maybe coming from various telescope/experiments, use SE++ with multi-threading (since SE2 can not do the job);
You must be logged in to vote
0 replies
Comment options

Thanks for your opinion, very much appreciated.

This was with output-properties=PixelCentroid.
I guess did not cause any overhead, since it is the default value.

All values were left to default, except for detection-threshold=10 and obviously the name of the input image and the output catalogue.
And output-flush-size=0.

Setting grouping-algorithm=none did make SourceExtractor++ run slightly faster (2.7s).

You must be logged in to vote
0 replies
Comment options

To give a bit more context, SE2 uses a custom made FITS I/O library and the entire program was heavily optimized for a quick throughput of problems such as yours.

In SE++ we use CFITSIO and the optimization is for flexible model fitting (which means way more processing per source) on multi-band and multi-scale data.

Note that your image data is rather small (for modern data). It might be that SE++ needs more resources to setup the processing and things are more comparable going to larger arrays, but we really do not know.

You must be logged in to vote
0 replies
Comment options

Just to complete what Martin said. SX++ is much slower than SE when run on small images with a few sources, because there is much more going on at startup. On large images with many sources, you can speed up processing quite significantly by adding, e.g, the command line arguments --grouping-algorithm=none --tile-size=10000 --tile-memory-limit=16384 . Using a large tile size removes accelerates some of the I/O at the expense of memory usage, but turning off grouping is really what speeds up things most with current versions.
When doing model-fitting with grouping off, you may also get a significant boost by inserting set_meta_iterations(1) in the Python configuration file.
In the end, model-fitting is about 3× slower in SX++ than in SE. But this is with multithreading off, when running on many cores it runs significantly faster (and it is much more flexible!).

You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
🙏
Q&A
Labels
None yet
3 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.