From email discussion between Beth and Juan Nov 29th-

"If illumination correction and compression are optional, that definitely simplifies matters, although since right now we CAN grab the pre-computed illumination correction from CellProfiler (and in practice, probably for a while most folks will be running downstream of CellProfiler), we might as well pass that in as an optional parameter.

I think that probably therefore makes all of this pretty simple - we probably don't have to pass in much if anything extra at the config-file-level (aka Steps 4 and 5), and then it's just creating a job file (Step 6) that has whatever of these flags we want as configurable parameters (likely everything but help, logging, and cores), as well as a job parameter that has a T/F for doing an illumination correction, a job parameter for the location of the pre-computed illumination correction files (if using), a job parameter for T/F doing the histogram cropping and compression, and a job parameter that says whatever this node is supposed to be doing (aka "Plate1,Well1,Site1", "Plate1,Well1,Site2" or "Plate1,Well1","Plate1,Well2", etc).

D-DP would then (aka "this is all the unique code we would need to write for Step 7")-

-Copy all the metadata files to a new temp directory
-Copy some or all of the location files to a new temp directory (just the ones that are needed if that seems easy, otherwise all if it seems hard since those files are small)
-Copy the index file to a new temp directory, and subset it based on what is passed as the metadata subset to run
-From the subsetted index file, figure out the image files you need and then copy to a new temp directory
-If illum is True, go grab the illum file from the stated location
-If illum and/or cropping and compression is true, create new copies of the image files with whatever things we've told it to use (Could use DeepProfiler functions here or just skimage code, depending on if the DeepProfiler steps are reasonably extractable or not)
-Actually do the profiling (aka the only DeepProfiler part) "