Skip to main content

Answer Semantic Similarity

Evaluator: AnswerSemanticSimilarityEvaluator

Computes the semantic similarity between the answer and the ground truth using embedding-based comparison.

Required Sample fields​

  • answer - Required for semantic comparison
  • groundTruth - Required for semantic comparison

How it works​

  1. Generates embeddings for both the answer and ground truth
  2. Calculates the cosine similarity between these embeddings
  3. Returns the similarity score directly

Score interpretation​

  • Range: 0 to 1
  • Based on cosine similarity between embeddings
  • Higher scores indicate closer semantic meaning
  • Perfect score (1.0) suggests nearly identical semantic content
  • Scores above 0.8 typically indicate very similar meaning
AnswerSemanticSimilarityEvaluator evaluator = new AnswerSemanticSimilarityEvaluator(embeddingModel);
Evaluation result = evaluator.evaluate(sample);
double similarityScore = result.getValue();