Newly-released Google's Gemini 3.8 Flash TTS tops Hume's Real-World VoiceEQ leaderboard

Voice models are advancing quickly, bringing new capabilities to a growing range of use cases. Evaluating them requires assessing a broader range of qualities—and whether models can sustain their performance over longer conversations and passages.

Hume’s Real World VoiceEQ benchmark takes this broader view of performance, measuring the human qualities of speech, including tone, expression, naturalness, speaker identity, and reliability, across real-world conditions and use cases. We recently extended our evaluations to cover voice controllability and voice replication.

Our latest benchmark update includes independent testing of preview versions of Google’s Gemini 3.8 Flash TTS and Gemini 3.8 Flash-Lite TTS. Within each evaluation, we assessed them using the same tasks, rating questions, and scoring criteria applied to the other participating models. Below, we examine where these models lead and where they improve on the previous generation. We also introduce a new overall score, a measure of frontier progress that quantifies performance over two axes, Expressivity and Reliability, assessing how well models combine expressive characteristics with accurate, consistent generations.

How we evaluated the models

Three human raters, recruited and screened through the Hume Human Feedback API, evaluated each generated TTS clip using questions tailored to the dimension being measured. Raters did not know which model produced the audio, and samples were presented in randomized order across scenarios. See our technical paper for the evaluation design and scoring methodology.

Our benchmark test set is held out by design. Its prompts and reference materials are not shared with model development teams, including researchers we work with (Hume has a non-exclusive licensing agreement with Google). This separation helps protect the integrity of the results and reduces opportunities for benchmark-specific optimization, keeping the focus on improvements that generalize beyond the test.

How Google’s latest TTS models measure up

Google’s latest TTS models are among the leaders in Real World VoiceEQ, with broad strengths in voice controllability and improved long-form stability compared with the previous generation. Speaker similarity and the precision of some voice controls remain areas for improvement.

Hume Voice EQ Benchmark

Report Card: Gemini 3.8 Flash TTS and Gemini 3.8 Flash-Lite TTS

Gemini 3.8 Flash TTS Gemini 3.8 Flash-Lite TTS top of field average lowest

Across the evaluations, four findings stand out:

• Overall TTS performance: marked gains over the previous generation. The largest gains over Gemini 3.1 Flash TTS are in extended long-form stability: scores rise from 1.22 to 2.89 for Gemini 3.8 Flash TTS and 3.03 for Gemini 3.8 Flash-Lite TTS on a five-point scale. Both models also improve in regular and our newest dimension of multilingual long-form stability. VoiceEQ results.
• Voice controllability: leading results across multiple dimensions. Gemini Flash TTS ranks first overall in Voice Design, with the highest point estimates for accent and gender and strong results for texture and prosody. Its clearest accent advantages include UK regional accents and World Englishes. Both new models also rank among the leaders at following style tags while keeping speech natural. Controllability results.
• Voice replication: speaker similarity remains less competitive. Gemini Flash-Lite TTS ranks seventh of 13 models on the same-speaker measure, scoring 3.68 out of 5 against a field average of 3.69. Gemini Flash TTS ranks eleventh at 3.53. Both score more strongly on naturalness, ranking second and fourth, respectively, illustrating the distinction between sounding human and sounding like the intended person. Replication results.
• Precision of voice customization: specific controls still need improvement. The broad controllability strength of Gemini Flash TTS does not extend equally to every attribute. Its young-adult voice accuracy is 34.0%, compared with 68.1% for ElevenLabs v3; its volume-control pass rate is 56.9%, compared with 88.9% for Inworld Voice Design. These are specific weaknesses that the overall Voice Design score can conceal. Controllability results.

Measuring expressivity and reliability together

Progress in TTS means expanding expressive range while maintaining accuracy and consistency. Strength in one does not guarantee strength in the other.

Our Expressivity–Reliability Frontier Score measures this combination using existing Real-World Voice EQ questions about delivery, accuracy, and consistency. These axes differ from the leaderboard’s individual Expressiveness and Reliability factors. Individual dimensions remain essential for understanding specific strengths, limitations, and suitability for different applications.

The Pareto frontier identifies models that no other tested model matches or exceeds on both axes while outperforming on at least one. The score uses TOPSIS to measure closeness to the ideal corner (1, 1), capturing joint performance. A model can therefore lead one axis without leading the ranking.

Gemini Flash TTS and Gemini Flash-Lite TTS rank first and second on the Frontier Expressivity–Reliability Score, at 0.920 and 0.914, respectively. Gemini Flash TTS also has the highest expressivity point estimate in this plot, while several models score higher on reliability. Their positions illustrate why the and the two axes should be examined together.

Beyond the benchmark: turning results into model improvements

For teams developing voice models, public benchmarks provide a shared point of comparison. Private evaluations help researchers investigate the failures behind those scores and test whether an intervention addresses them.

For example, a model might preserve a speaker’s identity in short clips but drift over longer passages. A targeted evaluation could test when that drift begins and how it varies with passage length, speaking style, or content.

Researchers can then test a model change on held-out examples and check whether gains generalize, while monitoring adjacent capabilities for regressions. The useful outcome is evidence about what changed, where it improved, and what remains unresolved.

The Hume VoiceEQ platform supports this workflow, from evaluation design and conversation simulation through acoustic measurements, model-based judgments, and human ratings.

Get in touch to discuss an evaluation tailored to the capabilities and failure modes your team is working to improve.