In static evaluations, we have built-in evaluators for exact match and LLM judge.

Does it make sense to add semantic similarity (instead of using an LLM judge)?

The evaluator would specify an embedding model and a cutoff, then ensure similarity is within that cutoff.

How do we handle things like the hubness problem, different manifold topology for different embedding models, etc.?