Crossing the Uncanny Valley of Voice Interaction

Sesame AI is dedicated to creating dialogue experiences with "voice presence," making AI voice interactions more natural, emotional, and human-like.

Learn More
Voice Presence

Achieving Voice Presence

At Sesame, our goal is to achieve "voice presence" - the magical quality that makes spoken interactions feel real, understood, and valued. We're creating conversational partners that don't just process requests; they engage in genuine dialogue, building confidence and trust over time.

Through this, we aim to realize the untapped potential of voice as the ultimate interface for guidance and understanding.

Emotional Intelligence

Reading and responding to emotional context

Dialogue Dynamics

Natural timing, pauses, interruptions, and emphasis

Context Awareness

Adjusting tone and style to match context

Consistent Personality

Maintaining coherent, reliable, and appropriate presence

Voice Generation Technology

Conversational Speech Model (CSM)

To create truly interactive AI companions, voice generation must go beyond producing high-quality audio—it must understand and adapt to context in real-time. Traditional Text-to-Speech (TTS) models generate spoken output directly from text but lack the contextual awareness needed for natural dialogue.

Even recent models that produce highly human-like speech still struggle with the one-to-many problem: there are countless valid ways to say a sentence, but only some fit a specific context. Without additional context—including tone, rhythm, and conversation history—models lack information to choose the best option.

To address this, we introduce the Conversational Speech Model (CSM), which frames the problem as an end-to-end multimodal learning task using transformers. It leverages conversation history to produce more natural and coherent speech.

Conversational Speech Model

Voice Interaction Demo

Visit Demo Platform

Research Results

Objective Metrics

Traditional benchmarks like Word Error Rate (WER) and Speaker Similarity (SIM) have saturated—modern models, including CSM, now achieve near-human performance on these metrics.

Objective Metrics Results

To better evaluate pronunciation and context understanding, we introduce a new set of speech transcription-based benchmarks.

  • Text understanding through homograph disambiguation: evaluating whether models correctly pronounce words with identical spelling but different meanings.
  • Audio understanding through pronunciation consistency: evaluating whether models maintain consistent pronunciation of specific words across multiple speech turns.

Subjective Metrics

We conducted two Comparative Mean Opinion Score (CMOS) studies using the Expresso dataset to evaluate the naturalness and prosodic appropriateness of CSM-Medium generated speech.

Subjective Evaluation Results

The graph shows win rates between real human recordings and CSM-generated speech samples in both studies. Without conversational context, human evaluators showed no clear preference between generated and real speech, indicating saturation in naturalness. However, when context was included, evaluators consistently preferred original recordings. These findings suggest that there remains a significant gap in prosody between generated and human speech in conversational contexts.

FAQ

What is Sesame AI?

Sesame AI is an innovative platform focused on breakthrough voice interaction technology. Through unique voice presence technology, Sesame AI makes AI voice interactions more natural and human-like.

What is the core technology of Sesame AI?

The core of Sesame AI is the Conversational Speech Model (CSM), an innovative end-to-end multimodal learning system. Through this technology, Sesame AI can generate more natural and emotional voice interactions.

Why choose Sesame AI?

Sesame AI has unique advantages in voice interaction: advanced emotional intelligence, natural dialogue dynamics, precise context awareness, and stable personality expression. This makes Sesame AI a leader in voice interaction technology.

How does Sesame AI improve user experience?

Through deep learning and emotional computing, Sesame AI provides users with more natural and humanized dialogue experiences. Each interaction is carefully optimized to ensure users get the best voice interaction experience.

What are the application scenarios for Sesame AI?

Sesame AI can be widely applied in customer service, education and training, medical consultation, and other fields. Whether for daily conversations or professional communication, Sesame AI can provide excellent voice interaction experiences.