Ammon News - Google contractors working to enhance its Gemini AI have reportedly been comparing its responses to outputs from Anthropic’s rival AI model, Claude, according to internal communications reviewed by *TechCrunch*. However, when asked for clarification, Google declined to confirm whether it had secured Anthropic’s permission for using Claude in these evaluations.

As tech giants race to improve AI systems, models are often assessed against competitors by running benchmarks internally. However, Google's contractors are manually evaluating Gemini’s responses against Claude’s, scoring them based on criteria such as accuracy, truthfulness, and verbosity. According to *TechCrunch*, these contractors are allocated up to 30 minutes per prompt to determine which model delivers the superior response.

Recent internal communications revealed that contractors had started noticing references to Claude within the Google platform used to evaluate Gemini. In one instance, an output explicitly identified itself as "I am Claude, created by Anthropic." Discussions among contractors highlighted Claude's emphasis on safety compared to Gemini. For example, Claude often declined to respond to prompts it deemed unsafe, such as role-playing as another AI assistant. In contrast, Gemini’s responses were flagged in some instances for severe safety violations, including inappropriate content involving nudity and bondage.

Anthropic’s terms of service prohibit clients from using Claude to develop competing products or train rival AI models without prior consent. Google, which is a significant investor in Anthropic, has not disclosed whether it obtained the necessary approval for this use.

When approached by *TechCrunch*, Shira McNamara, a spokesperson for Google DeepMind, stated that while DeepMind compares model outputs during evaluations, it does not use Anthropic’s models to train Gemini. “In line with standard industry practices, we occasionally compare outputs as part of our evaluation process,” McNamara said. “However, any claim that we have used Anthropic models to train Gemini is false.”

Anthropic declined to comment before the article's publication.