Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Comments

Close side panel

MFT Noise calibration#6230

Merged
shahor02 merged 24 commits intoAliceO2Group:devAliceO2Group/AliceO2:devfrom
MFTDecoding:feature/nightly-20210521-noise-dpl-1map-newMFTDecoding/AliceO2:feature/nightly-20210521-noise-dpl-1map-newCopy head branch name to clipboard
May 28, 2021
Merged

MFT Noise calibration#6230
shahor02 merged 24 commits intoAliceO2Group:devAliceO2Group/AliceO2:devfrom
MFTDecoding:feature/nightly-20210521-noise-dpl-1map-newMFTDecoding/AliceO2:feature/nightly-20210521-noise-dpl-1map-newCopy head branch name to clipboard

Conversation

@mcoquet642
Copy link
Collaborator

Implementing noise scan calibration for MFT following the lines of ITS writing a single NoiseMap object in the CCDB. Additional features include an option --useDigits to make the device run in mode based on digits rather than clusters (default mode).

MauriceCoke and others added 18 commits May 21, 2021 21:24
	nouveau fichier : Detectors/ITSMFT/MFT/calibration/include/MFTCalibration/NoiseCalibratorDigits.h
	nouveau fichier : Detectors/ITSMFT/MFT/calibration/include/MFTCalibration/NoiseCalibratorDigitsSpec.h
	modifié :         Detectors/ITSMFT/MFT/calibration/src/MFTCalibrationLinkDef.h
	nouveau fichier : Detectors/ITSMFT/MFT/calibration/src/NoiseCalibratorDigits.cxx
	nouveau fichier : Detectors/ITSMFT/MFT/calibration/src/NoiseCalibratorDigitsSpec.cxx
	modifié :         Detectors/ITSMFT/MFT/calibration/testWorkflow/README.md
	modifié :         Detectors/ITSMFT/MFT/calibration/testWorkflow/mft-calib-workflow.cxx
Copy link
Collaborator

@shahor02 shahor02 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mcoquet642 , please see some comments below.
This version of time-slot based calibration is essentially similar to that of ITS, where the slot receives the digits or clusters fills the map. As such, it can aggregate the data both from the EPNs and from FLPs (if the decoding/clusterization is done there).
An alternative, as we discussed at the meeting, might be the FLP processing which builds the map for the chips seen on each FLP, then the time-slot-calibration receives the maps from different FLPs, merges them and sends to the CCDB populator.

Comment on lines 134 to 137
output.snapshot(
Output{clbUtils::gDataOriginCLB, clbUtils::gDataDescriptionCLBPayload, 0}, *image.get());
output.snapshot(
Output{clbUtils::gDataOriginCLB, clbUtils::gDataDescriptionCLBInfo, 0}, info);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please note that the convention for the messages sent to CCDB populator was changed since #6150 in order to avoid clashes between the inputs from different calibrators. The documentation was not fully updated, just pushed #6234.
Now it should be e.g.

output.snapshot(Output{clbUtils::gDataOriginCDBPayload, "MFT_NoiseMap", 0}, *image.get());
output.snapshot(Output{clbUtils::gDataOriginCDBWrapper, "MFT_NoiseMap", 0}, info);

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Ruben, thanks for the comments. This is now corrected.

Comment on lines 161 to 163
ConcreteDataTypeMatcher{clbUtils::gDataOriginCLB, clbUtils::gDataDescriptionCLBPayload});
outputs.emplace_back(
ConcreteDataTypeMatcher{clbUtils::gDataOriginCLB, clbUtils::gDataDescriptionCLBInfo});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Respectively, in the getNoiseCalibratorSpec the outputs should be declared as

outputs.emplace_back(ConcreteDataTypeMatcher{clbUtils::gDataOriginCDBPayload, "MFT_NoiseMap"});
outputs.emplace_back(ConcreteDataTypeMatcher{clbUtils::gDataOriginCDBWrapper, "MFT_NoiseMap"});

outputs,
AlgorithmSpec{adaptFromTask<NoiseCalibratorSpec>(useDigits)},
Options{
{"1pix-only", VariantType::Bool, false, {"Fast 1-pixel calibration only"}},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of this option? It will miss the noisy channels which happen to be neighbors (e.g. noisy column).

Copy link
Collaborator Author

@mcoquet642 mcoquet642 May 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a remanent of what was in ITS code, now removed.

bool NoiseSlotCalibrator::processTimeFrame(gsl::span<const o2::itsmft::Digit> const& digits,
gsl::span<const o2::itsmft::ROFRecord> const& rofs)
{
calibration::TFType nTF = rofs[0].getBCData().orbit / mHBFperTF;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The calibration may start from any orbit, so the nTF in general will not reflect the TFID. In other time-slot-based calibrations we are using auto tfcounter = o2::header::get<o2::framework::DataProcessingHeader*>(pc.inputs().get("input").header)->startTime;. This is also not really correct, but at least, once we come to agreement on the common TF counter definition, we can change all centrally. I would suggest to use the same: extract the TF counter in the CalibrationSpec and pass it to the Calibrator, e.g. like in https://github.com/AliceO2Group/AliceO2/blob/c2f765d42f324beb1e1ecda9cf5ba4acf29704cd/Detectors/TOF/calibration/testWorkflow/LHCClockCalibratorSpec.h#L51-L54
Then, you also will not need the mHBFperTF.
Actually, for you the specific slot should be irrelevant, since you have just 1 with 0 - infinity span.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out. Indeed I had some troubles with this. I think the same line is in the ITS code.

mStart = ic.options().get<int64_t>("tstart");
mEnd = ic.options().get<int64_t>("tend");

mCalibrator = std::make_unique<CALIBRATOR>(onepix, probT, HBperTF);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should set the length of the calibration slot, i.e. in your case infinite (e.g. std::numeric_limits<TFType>::max) since you have just 1 slot.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a comment : by default this PR doesn't use the NoiseSlotCalibrator class but the NoiseCalibrator which doesn't use TimeSlots, as Jouri mentioned during the discussion that in the end ITS doesn't use the former. So I included the appropriate line but commented it out.

	modifié :         ../include/MFTCalibration/NoiseCalibrator.h
	modifié :         ../include/MFTCalibration/NoiseCalibratorSpec.h
	modifié :         ../include/MFTCalibration/NoiseSlotCalibrator.h
	modifié :         NoiseCalibrator.cxx
	modifié :         NoiseCalibratorSpec.cxx
	modifié :         NoiseSlotCalibrator.cxx
mEnd = ic.options().get<int64_t>("tend");

mCalibrator = std::make_unique<CALIBRATOR>(probT);
// mCalibrator->setSlotLength(std::numeric_limits<TFType>::max); //For TimeSlot calibration
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you never need multiple slots, why don't you set it in the NoiseSlotCalibrator constructor?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be done now.

Comment on lines 92 to 115
auto toKeyValPairs = [](std::vector<std::string> const& tokens) {
std::vector<std::pair<std::string, std::string>> pairs;
Str strutils;
for (auto& token : tokens) {
auto keyval = Str::tokenize(token, '=', false);
if (keyval.size() != 2) {
// LOG(FATAL) << "Illegal command-line key/value string: " << token;
continue;
}

strutils.trim(keyval[1]);
std::pair<std::string, std::string> pair = std::make_pair(keyval[0], keyval[1]);
pairs.push_back(pair);
}

return pairs;
};
std::map<std::string, std::string> meta;
auto keyvalues = toKeyValPairs(Str::tokenize(mMeta, ';', true));

// fill meta map
for (auto& p : keyvalues) {
meta[p.first] = p.second;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't you do directly:

std::map<std::string, std::string> meta;
auto toKeyValPairs = [&meta](std::vector<std::string> const& tokens) {
  for (auto&amp; token : tokens) {
     auto keyval = Str::tokenize(token, '=', false);
     if (keyval.size() != 2) {
        LOG(ERROR) << "Illegal command-line key/value string: " << token;
        continue;
     }
     Str::trim(keyval[1]);
     meta[keyval[0]] = keyval[1];
  }
};
toKeyValPairs(Str::tokenize(mMeta, ';', true));

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, indeed. Now fixed.

Copy link
Collaborator

@shahor02 shahor02 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approving to trigger CI check

@shahor02
Copy link
Collaborator

Squashing/merging, CI failures are not related to the PR.

@TimoWilken all 3 required tests are regularly failing: macos does not start, in o2 the compiler is killed, and fullCI fails at random stages, I suspect due to the insufficient memory.

@shahor02 shahor02 merged commit 7ea4d1f into AliceO2Group:dev May 28, 2021
cortesep pushed a commit to cortesep/AliceO2 that referenced this pull request Jun 11, 2021
* First version of MFT calibration : generating noise map and writing in CCDB

* Update README.md

* modifié :         Detectors/ITSMFT/MFT/calibration/CMakeLists.txt
	nouveau fichier : Detectors/ITSMFT/MFT/calibration/include/MFTCalibration/NoiseCalibratorDigits.h
	nouveau fichier : Detectors/ITSMFT/MFT/calibration/include/MFTCalibration/NoiseCalibratorDigitsSpec.h
	modifié :         Detectors/ITSMFT/MFT/calibration/src/MFTCalibrationLinkDef.h
	nouveau fichier : Detectors/ITSMFT/MFT/calibration/src/NoiseCalibratorDigits.cxx
	nouveau fichier : Detectors/ITSMFT/MFT/calibration/src/NoiseCalibratorDigitsSpec.cxx
	modifié :         Detectors/ITSMFT/MFT/calibration/testWorkflow/README.md
	modifié :         Detectors/ITSMFT/MFT/calibration/testWorkflow/mft-calib-workflow.cxx

* modifié :         Detectors/ITSMFT/MFT/calibration/testWorkflow/mft-calib-workflow.cxx

* Implementation of noise calibration DPL using digits

* Update README.md

* TimeSlot calibration

* Clang format

* Fix typo

* Fix typo

* Fix typo

* Update NoiseCalibrator.h

* Update NoiseCalibratorDigits.h

* Updating TimeSolt implementation

* Clang format

* Digits/Clusters implementation

* Using StringUtils

* Clang Format

* Fixing spaces and tabs

* Changes for PR

	modifié :         ../include/MFTCalibration/NoiseCalibrator.h
	modifié :         ../include/MFTCalibration/NoiseCalibratorSpec.h
	modifié :         ../include/MFTCalibration/NoiseSlotCalibrator.h
	modifié :         NoiseCalibrator.cxx
	modifié :         NoiseCalibratorSpec.cxx
	modifié :         NoiseSlotCalibrator.cxx

* Clang format

* Changes for PR

* Clang Format

* Fix typo

Co-authored-by: Maurice <maurice.coquet@yahoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.