[Bug] Possible batching bug in c++ inference api #2912
Description
Checklist
- I have searched related issues but cannot get the expected help.
- 2. I have read the FAQ documentation but cannot get the expected help.
- 3. The bug has not been fixed in the latest version.
Describe the bug
When running a segmentation model through the c++ api I get a bug. When I run batched inference, the result returned is always the first image, multiplied by the length of the batch.
So for a size 10 batch, its just the segmentation from the first batch, stretched to length 10.
If I edit the following lines in mmdeploy\common.hpp to include the offset (editing class Result_)
- T* operator->() const noexcept { return data_.get(); }
- T& operator*() const noexcept { return *data_; }
+ T* operator->() const noexcept { return data_.get() + offset_; }
+ T& operator*() const noexcept { return *(data_.get() + offset_); }It seems to fix it. Before, when we used -> to access the pointer, we were always getting the first element pointer, and when we used * to access the value we were always getting the first value. Including the offset fixes?
Therefore I think this points to a possible bug in the c++ api?
Reproduction
std::string model_path = "C:\\Users\\Ben_C\\Desktop\\mmdeploy-1.3.1\\segnext-trt\\segnext-t-fp16-batch-10";
mmdeploy::Context context;
context.Add(mmdeploy::Device("cuda"));
mmdeploy::Segmentor segmentor{ mmdeploy::Model{model_path}, context };
std::string video_path = "E:/NE_northeast.mp4";
cv::VideoCapture cap(video_path);
cv::Mat frame;
bool ret = cap.read(frame);
int count = 0;
std::vector<cv::Mat> mats;
while (ret) {
bool ret = cap.read(frame);
if (!ret) {
break;
}
mats.push_back(frame.clone());
}
int bsize = 10;
std::vector<mmdeploy::Mat> batch;
batch.reserve(bsize);
auto start = std::chrono::high_resolution_clock::now();
std::vector<std::vector<mmdeploy::Segmentor::Result>> res;
for (int i = 0; i < mats.size(); i++) {
batch.emplace_back(mats[i]);
if (batch.size() == bsize) {
check_batch(batch);
auto seg = segmentor.Apply(batch);
res.push_back(seg);
batch.clear();
}
else if (i == mats.size() - 1) {
auto seg = segmentor.Apply(batch);
res.push_back(seg);
batch.clear();
}
count += 1;
}
Environment
C++ api.