As the realms of artificial intelligence and machine learning continue to shape our world 🌏, the imperative to assess their influence on cybersecurity intensifies. While many AI/ML breakthroughs in cybersecurity revolve around defense and threat intelligence, such as intelligent 💻 SIEM systems and AI-driven malware detection, an intriguing question arises: "Can researchers harness AI/ML for offensive security, or can they outmaneuver AI/ML algorithms with innovative cybersecurity approaches?" This presents a captivating new horizon in the domain of offensive security.🚀
Within this repository lies an array of engaging CTF (Capture The Flag 🚩) challenges meticulously crafted for applications that leverage machine learning algorithms in their backend. The intent of this repository is to emphasize the need for implementation of security measures within machine learning applications, safeguarding 🛡️ them against the ever-evolving threat landscape. It serves as a guiding beacon in fortifying the convergence of technology and security🔒.
| Name | Category | Description | Difficulty | References |
|---|---|---|---|---|
| Mirage | Web - MCP Signature Cloaking | Identify backdoored server and fetch Flag:{<FLAG>} . | Medium | |
| Vault | Web - Model Inversion | Gain access to Vault and fetch Secret (Flag:). | Hard | |
| Dolos | Web - Prompt Injection to RCE | Flag is at same directory as of flask app, [FLAG].txt. | Easy | |
| Dolos II | Web - Prompt Injection to SQL Injection | Make the LLM to reveal Secret (Flag:) of user David. | Easy | |
| Heist | Web - Data Poisoning Attack | Compromise CityPolice's AI cameras and secure a smooth escape for Heist crew's red getaway car! | Medium | |
| Persuade | Web - Model Serialization Attack | Flag is at /app/InternalFolder/Flag.txt, not on the website. Find it. | Medium | |
| Fourtune | Web - Model Extraction Attack | Bypass AI Corp's identity verification to view the flag | Hard |
💭 If you want to contribute to the above list of CTF challenges (please do), submit a pull request or ping me at
Stay tuned for more challenges being added to the repo. 👀