MFT Noise calibration#6230
MFT Noise calibration#6230shahor02 merged 24 commits intoAliceO2Group:devAliceO2Group/AliceO2:devfrom MFTDecoding:feature/nightly-20210521-noise-dpl-1map-newMFTDecoding/AliceO2:feature/nightly-20210521-noise-dpl-1map-newCopy head branch name to clipboard
Conversation
nouveau fichier : Detectors/ITSMFT/MFT/calibration/include/MFTCalibration/NoiseCalibratorDigits.h nouveau fichier : Detectors/ITSMFT/MFT/calibration/include/MFTCalibration/NoiseCalibratorDigitsSpec.h modifié : Detectors/ITSMFT/MFT/calibration/src/MFTCalibrationLinkDef.h nouveau fichier : Detectors/ITSMFT/MFT/calibration/src/NoiseCalibratorDigits.cxx nouveau fichier : Detectors/ITSMFT/MFT/calibration/src/NoiseCalibratorDigitsSpec.cxx modifié : Detectors/ITSMFT/MFT/calibration/testWorkflow/README.md modifié : Detectors/ITSMFT/MFT/calibration/testWorkflow/mft-calib-workflow.cxx
shahor02
left a comment
There was a problem hiding this comment.
Hi @mcoquet642 , please see some comments below.
This version of time-slot based calibration is essentially similar to that of ITS, where the slot receives the digits or clusters fills the map. As such, it can aggregate the data both from the EPNs and from FLPs (if the decoding/clusterization is done there).
An alternative, as we discussed at the meeting, might be the FLP processing which builds the map for the chips seen on each FLP, then the time-slot-calibration receives the maps from different FLPs, merges them and sends to the CCDB populator.
| output.snapshot( | ||
| Output{clbUtils::gDataOriginCLB, clbUtils::gDataDescriptionCLBPayload, 0}, *image.get()); | ||
| output.snapshot( | ||
| Output{clbUtils::gDataOriginCLB, clbUtils::gDataDescriptionCLBInfo, 0}, info); |
There was a problem hiding this comment.
Please note that the convention for the messages sent to CCDB populator was changed since #6150 in order to avoid clashes between the inputs from different calibrators. The documentation was not fully updated, just pushed #6234.
Now it should be e.g.
output.snapshot(Output{clbUtils::gDataOriginCDBPayload, "MFT_NoiseMap", 0}, *image.get());
output.snapshot(Output{clbUtils::gDataOriginCDBWrapper, "MFT_NoiseMap", 0}, info);
There was a problem hiding this comment.
Hi Ruben, thanks for the comments. This is now corrected.
| ConcreteDataTypeMatcher{clbUtils::gDataOriginCLB, clbUtils::gDataDescriptionCLBPayload}); | ||
| outputs.emplace_back( | ||
| ConcreteDataTypeMatcher{clbUtils::gDataOriginCLB, clbUtils::gDataDescriptionCLBInfo}); |
There was a problem hiding this comment.
Respectively, in the getNoiseCalibratorSpec the outputs should be declared as
outputs.emplace_back(ConcreteDataTypeMatcher{clbUtils::gDataOriginCDBPayload, "MFT_NoiseMap"});
outputs.emplace_back(ConcreteDataTypeMatcher{clbUtils::gDataOriginCDBWrapper, "MFT_NoiseMap"});
| outputs, | ||
| AlgorithmSpec{adaptFromTask<NoiseCalibratorSpec>(useDigits)}, | ||
| Options{ | ||
| {"1pix-only", VariantType::Bool, false, {"Fast 1-pixel calibration only"}}, |
There was a problem hiding this comment.
What is the purpose of this option? It will miss the noisy channels which happen to be neighbors (e.g. noisy column).
There was a problem hiding this comment.
This was a remanent of what was in ITS code, now removed.
| bool NoiseSlotCalibrator::processTimeFrame(gsl::span<const o2::itsmft::Digit> const& digits, | ||
| gsl::span<const o2::itsmft::ROFRecord> const& rofs) | ||
| { | ||
| calibration::TFType nTF = rofs[0].getBCData().orbit / mHBFperTF; |
There was a problem hiding this comment.
The calibration may start from any orbit, so the nTF in general will not reflect the TFID. In other time-slot-based calibrations we are using auto tfcounter = o2::header::get<o2::framework::DataProcessingHeader*>(pc.inputs().get("input").header)->startTime;. This is also not really correct, but at least, once we come to agreement on the common TF counter definition, we can change all centrally. I would suggest to use the same: extract the TF counter in the CalibrationSpec and pass it to the Calibrator, e.g. like in https://github.com/AliceO2Group/AliceO2/blob/c2f765d42f324beb1e1ecda9cf5ba4acf29704cd/Detectors/TOF/calibration/testWorkflow/LHCClockCalibratorSpec.h#L51-L54
Then, you also will not need the mHBFperTF.
Actually, for you the specific slot should be irrelevant, since you have just 1 with 0 - infinity span.
There was a problem hiding this comment.
Thanks for pointing this out. Indeed I had some troubles with this. I think the same line is in the ITS code.
| mStart = ic.options().get<int64_t>("tstart"); | ||
| mEnd = ic.options().get<int64_t>("tend"); | ||
|
|
||
| mCalibrator = std::make_unique<CALIBRATOR>(onepix, probT, HBperTF); |
There was a problem hiding this comment.
You should set the length of the calibration slot, i.e. in your case infinite (e.g. std::numeric_limits<TFType>::max) since you have just 1 slot.
There was a problem hiding this comment.
Just a comment : by default this PR doesn't use the NoiseSlotCalibrator class but the NoiseCalibrator which doesn't use TimeSlots, as Jouri mentioned during the discussion that in the end ITS doesn't use the former. So I included the appropriate line but commented it out.
modifié : ../include/MFTCalibration/NoiseCalibrator.h modifié : ../include/MFTCalibration/NoiseCalibratorSpec.h modifié : ../include/MFTCalibration/NoiseSlotCalibrator.h modifié : NoiseCalibrator.cxx modifié : NoiseCalibratorSpec.cxx modifié : NoiseSlotCalibrator.cxx
| mEnd = ic.options().get<int64_t>("tend"); | ||
|
|
||
| mCalibrator = std::make_unique<CALIBRATOR>(probT); | ||
| // mCalibrator->setSlotLength(std::numeric_limits<TFType>::max); //For TimeSlot calibration |
There was a problem hiding this comment.
Since you never need multiple slots, why don't you set it in the NoiseSlotCalibrator constructor?
There was a problem hiding this comment.
Should be done now.
| auto toKeyValPairs = [](std::vector<std::string> const& tokens) { | ||
| std::vector<std::pair<std::string, std::string>> pairs; | ||
| Str strutils; | ||
| for (auto& token : tokens) { | ||
| auto keyval = Str::tokenize(token, '=', false); | ||
| if (keyval.size() != 2) { | ||
| // LOG(FATAL) << "Illegal command-line key/value string: " << token; | ||
| continue; | ||
| } | ||
|
|
||
| strutils.trim(keyval[1]); | ||
| std::pair<std::string, std::string> pair = std::make_pair(keyval[0], keyval[1]); | ||
| pairs.push_back(pair); | ||
| } | ||
|
|
||
| return pairs; | ||
| }; | ||
| std::map<std::string, std::string> meta; | ||
| auto keyvalues = toKeyValPairs(Str::tokenize(mMeta, ';', true)); | ||
|
|
||
| // fill meta map | ||
| for (auto& p : keyvalues) { | ||
| meta[p.first] = p.second; | ||
| } |
There was a problem hiding this comment.
Why don't you do directly:
std::map<std::string, std::string> meta;
auto toKeyValPairs = [&meta](std::vector<std::string> const& tokens) {
for (auto& token : tokens) {
auto keyval = Str::tokenize(token, '=', false);
if (keyval.size() != 2) {
LOG(ERROR) << "Illegal command-line key/value string: " << token;
continue;
}
Str::trim(keyval[1]);
meta[keyval[0]] = keyval[1];
}
};
toKeyValPairs(Str::tokenize(mMeta, ';', true));
There was a problem hiding this comment.
Thanks, indeed. Now fixed.
shahor02
left a comment
There was a problem hiding this comment.
approving to trigger CI check
|
Squashing/merging, CI failures are not related to the PR. @TimoWilken all 3 required tests are regularly failing: macos does not start, in o2 the compiler is killed, and fullCI fails at random stages, I suspect due to the insufficient memory. |
* First version of MFT calibration : generating noise map and writing in CCDB * Update README.md * modifié : Detectors/ITSMFT/MFT/calibration/CMakeLists.txt nouveau fichier : Detectors/ITSMFT/MFT/calibration/include/MFTCalibration/NoiseCalibratorDigits.h nouveau fichier : Detectors/ITSMFT/MFT/calibration/include/MFTCalibration/NoiseCalibratorDigitsSpec.h modifié : Detectors/ITSMFT/MFT/calibration/src/MFTCalibrationLinkDef.h nouveau fichier : Detectors/ITSMFT/MFT/calibration/src/NoiseCalibratorDigits.cxx nouveau fichier : Detectors/ITSMFT/MFT/calibration/src/NoiseCalibratorDigitsSpec.cxx modifié : Detectors/ITSMFT/MFT/calibration/testWorkflow/README.md modifié : Detectors/ITSMFT/MFT/calibration/testWorkflow/mft-calib-workflow.cxx * modifié : Detectors/ITSMFT/MFT/calibration/testWorkflow/mft-calib-workflow.cxx * Implementation of noise calibration DPL using digits * Update README.md * TimeSlot calibration * Clang format * Fix typo * Fix typo * Fix typo * Update NoiseCalibrator.h * Update NoiseCalibratorDigits.h * Updating TimeSolt implementation * Clang format * Digits/Clusters implementation * Using StringUtils * Clang Format * Fixing spaces and tabs * Changes for PR modifié : ../include/MFTCalibration/NoiseCalibrator.h modifié : ../include/MFTCalibration/NoiseCalibratorSpec.h modifié : ../include/MFTCalibration/NoiseSlotCalibrator.h modifié : NoiseCalibrator.cxx modifié : NoiseCalibratorSpec.cxx modifié : NoiseSlotCalibrator.cxx * Clang format * Changes for PR * Clang Format * Fix typo Co-authored-by: Maurice <maurice.coquet@yahoo.com>
Implementing noise scan calibration for MFT following the lines of ITS writing a single NoiseMap object in the CCDB. Additional features include an option --useDigits to make the device run in mode based on digits rather than clusters (default mode).