A framework for implementing Continuous Alignment Testing (CAT) for LLM-based applications.

CAT provides the infrastructure needed to:

• Run and track CATs against LLM outputs
• Store and analyze test results over time
• Monitor changes in LLM behavior as prompts/models/data evolve
• Integrate validation into CI/CD pipelines

• team_recommender - Team Recommender
• pythonCatBot - An OpenAI powered Python console chat app

poetry run pytest

poetry run mypy -p src

The Sphinx based documentation is available at https://thisisartium.github.io/continuous-alignment-testing

• Archive - To see potentially helpful previous iterations of the code