Evaluate your LLM's response with Prometheus and GPT4 💯
-
Updated
Apr 25, 2025 - Python
#
llm-as-evaluator
Here are 13 public repositories matching this topic...
Deliver safe & effective language models
nlp artificial-intelligence benchmarks benchmark-framework model-assessment ai-safety mlops responsible-ai ml-safety trustworthy-ai ethics-in-ai ml-testing large-language-models llm ai-testing llm-test llm-evaluation-toolkit llm-as-evaluator llm-testing
-
Updated
Sep 27, 2025 - Python
[ICLR 2025] xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation
benchmark regex reliability evaluation dataset gpt phi large-language-models llm cc-by-nc-nd-4 open-compass chatglm qwen lm-evaluation llm-as-a-judge llm-as-evaluator xfinder reliable-evaluation key-answer-extraction judge-model
-
Updated
Feb 26, 2025 - Python
This is the repo for the survey of Bias and Fairness in IR with LLMs.
information-retrieval recommender-systems bias ir fairness large-language-models llm chatgpt llm4rec llm4rs llm-as-a-judge llm-as-evaluator llm4ir
-
Updated
Sep 4, 2025
Code and data for "Timo: Towards Better Temporal Reasoning for Language Models" (COLM 2024)
temporal-reasoning sota-model llms rlhf rlaif llm-as-a-judge llm-as-evaluator self-critic-framework colm2024
-
Updated
Oct 23, 2024 - Python
Code and data for Koo et al's ACL 2024 paper "Benchmarking Cognitive Biases in Large Language Models as Evaluators"
nlp evaluation bias bias-detection llm llms llm-evaluation llms-benchmarking llm-as-judge llm-as-a-judge llm-as-evaluator
-
Updated
Feb 16, 2024 - Jupyter Notebook
LLM-as-judge evals as Semantic Kernel Plugins
-
Updated
Aug 22, 2025 - C#
Use groq for evaluations
-
Updated
Jul 11, 2024 - Python
PrometheusLLM is a unique transformer architecture inspired by dignity and recursion. This project aims to explore new frontiers in AI research and welcomes contributions from the community. 🐙🌟
deep-learning mcp evaluation pipelines tracing language-model self-organization cognitive-architecture hermeneutics philosophy-of-mind gpt4 llm llmops ollama litellm llm-as-evaluator autopoietic-systems prompt-logging
-
Updated
Oct 17, 2025 - Python
A tiny, lightweight library for LLM-as-a-Judge evaluations on vLLM-hosted models.
-
Updated
Sep 29, 2025 - Python
Antibodies for LLMs hallucinations (grouping LLM as a judge, NLI, reward models)
-
Updated
Jun 13, 2024 - Python
Automated evaluation of llm generated responses on aws
-
Updated
Apr 11, 2025 - Python
project from UDACITY second module 'agentic workflow' as part of Agentic AI Nanodegree
-
Updated
Aug 16, 2025 - Python
Improve this page
Add a description, image, and links to the llm-as-evaluator topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llm-as-evaluator topic, visit your repo's landing page and select "manage topics."