SynthDet is an open source project that demonstrates an end-to-end object detection pipeline using synthetic image data. The project includes all the code and assets for generating a synthetic dataset in Unity. Using recent research, SynthDet utilizes Unity's Perception package to generate highly randomized images of 64 common grocery products (example: cereal boxes and candy) and export them along with appropriate labels and annotations (2D bounding boxes). The synthetic dataset generated can then be used to train a deep learning based object detection model. This project is geared towards ML practitioners and enthusiasts who are actively exploring synthetic data or just looking to get started.
- SynthDet - Sample computer vision data generation project using Unity's Perception package
- 3D Assets - High quality models of 64 commonly found grocery products
- Unity's Perception package
- Unity Dataset Insights Python package
SynthDet was inspired by the following research paper from Google Cloud AI:
Hinterstoisser, S., Pauly, O., Heibel, H., Marek, M., & Bokeloh, M. (2019). An Annotation Saved is an Annotation Earned: Using Fully Synthetic Training for Object Instance Detection.
For general questions or concerns please contact the Unity Computer Vision team at computer-vision@unity3d.com.
For feedback, bugs, or other issues please file a GitHub issue and the Unity Computer Vision team will investigate the issue as soon as possible.
If you find this package useful, consider citing it using:
@misc{synthdet2020,
title={Training a performant object detection {ML} model on synthetic data using {U}nity {P}erception tools},
author={You-Cyuan Jhang and Adam Palmar and Bowen Li and Saurav Dhakad and Sanjay Kumar Vishwakarma and Jonathan Hogins and Adam Crespi and Chris Kerr and Sharmila Chockalingam and Cesar Romero and Alex Thaman and Sujoy Ganguly},
howpublished = {\url{https://blogs.unity3d.com/2020/09/17/training-a-performant-object-detection-ml-model-on-synthetic-data-using-unity-computer-vision-tools/}},
journal={Unity Techonologies Blog},
publisher={Unity Technologies},
year={2020},
month={Sep}
}
GTC 2020: Synthetic Data: An efficient mechanism to train Perception Systems
Synthetic data: Simulating myriad possibilities to train robust machine learning models
Training a performant object detection ML model on synthetic data using Unity computer vision tools