Cloud Speech-to-Text overview
Start with speech to text
Read the overview of recognition requests and compare batch and streaming workflows.
Start with: Cloud Speech-to-Text overview
RedLectern topic
Speech and audio AI
Transcription · speech generation · speaker separation · audio understanding · voice imitation.
For: Users · builders · creators.
After this page you can: Choose and evaluate an audio workflow with appropriate consent.
Page reviewed
Part of: Artificial Intelligence › Multimodal AI
About Speech and audio AI
Speech and audio AI covers systems that turn speech into text, generate speech from text, assign turns to speakers, and interpret audio beyond its words. The right workflow depends on the output you need: a transcript, a spoken rendering, speaker-attributed turns, or an answer about sounds and speech. Some services offer separate recognition, streaming, and synthesis features, so check that the chosen workflow supports your input and intended output.
Evaluate a workflow on representative recordings and define what counts as an error before comparing results. For transcription, compare words against a reference transcript and inspect the kinds of errors; a single score does not tell you whether names, numbers, or speaker turns were handled as needed. For generated or imitated voices, check whose voice is being used, what permission covers the intended use, and what the service requires. Treat audio as potentially sensitive material: confirm who can access recordings and generated outputs before sharing them.
Start here
Learn the foundations
1.Choose the speech task and measure its outputStart by naming the output you need.
Start by naming the output you need. Automatic speech recognition turns spoken audio into text; text to speech generates spoken audio from text. Speaker diarization divides a recording into spans assigned to distinct speakers. These tasks can be combined, but a transcript can be readable while its speaker assignments are wrong.
For transcription comparisons, use reference transcripts that match your evaluation audio and state how text is normalized. Word error rate counts substitutions, deletions, and insertions relative to reference words. Review those errors in context, especially for names and numbers. A metric summarizes one comparison; it does not establish that a workflow fits every language, recording condition, or downstream task.
- Text to speech REST API
Documents text-to-speech requests and voice and language selection.
- Word Error Rate (WER)
Defines WER and describes its insertion, deletion, and substitution components.
Automatic speech recognition
Converting spoken audio into text.
Text to speech
Generating spoken audio from text.
Speaker diarization
Detecting when speakers change and labeling the individual voices in audio.
Voice cloning
Generating speech that resembles a reference person's voice.
Word error rate
Transcription errors measured relative to the number of reference words.
Try it
A transcript has the right words but assigns several turns to the wrong speaker. Which task needs improvement, and what separate check would you use for the words?
Improve speaker diarization. Check the words separately against a reference transcript, using word error rate to count insertions, deletions, and substitutions.
2.Decide when audio needs interpretationSome audio workflows return more than a transcript: they can describe or summarize an audio clip, answer questions about its content, translate speech, or iden…
Some audio workflows return more than a transcript: they can describe or summarize an audio clip, answer questions about its content, translate speech, or identify segments with timestamps. This is useful when your task asks what happened in a recording rather than only what words were spoken. Dedicated speech recognition is a separate option when the goal is transcription, including real-time transcription.
Write a small evaluation set that represents the sounds, speakers, and languages you expect to encounter. For each recording, decide what a useful answer must include, then listen back to check the response against the audio. Keep the desired output narrow: a question about a siren, a spoken phrase, or a timestamped moment calls for different checks. Do not treat a fluent description as evidence that every detail is correct.
- Cloud Speech-to-Text overview
Covers dedicated speech recognition request types, including streaming.
Try it
You need to know whether a short recording contains a doorbell, not produce a verbatim transcript. What output should you request, and how would you check it?
Request an answer about the audio content, such as whether a doorbell is audible. Listen to the recording and check that the response identifies the sound accurately.
3.Set permission boundaries for voice imitationVoice cloning uses characteristics of a speaker's voice to synthesize new speech.
Voice cloning uses characteristics of a speaker's voice to synthesize new speech. Before using voice samples, establish whose voice they contain, who has permission to provide them, what the permission covers, and where the generated speech may be used. Consent workflows differ by service: for example, one custom-voice workflow requires an explicit recorded consent statement, while another voice-cloning guide asks users to confirm they have the right and consent to clone the voice.
Keep permission records with the project and check that the permission covers the planned audience, content, and distribution. If the speaker cannot clearly authorize the use, choose a non-imitative voice or ask for permission before generating. Review generated audio before publication, since synthesized speech can say words the original speaker never said.
- Instant Voice Cloning
Its guide asks users to confirm they have the right and consent to clone the voice.
- Voice cloning: how it works
Explains that synthesized clones can say things the original speaker never said.
Try it
A creator wants to imitate a colleague's voice in a public explainer. Name two checks to make before generating audio.
Confirm that the colleague has authorized using their voice for this content and public distribution. Check the service's own consent requirements before uploading or generating.
Latest
No recent updates are available.
Books
- Speech and Language Processing
Speech and Language Processing
Daniel Jurafsky and James H. Martin · 2009
A broad reference for placing speech tasks in a language-processing workflow. The 2009 edition covers speech synthesis and recognition, so treat it as background rather than guidance on current audio models.
- Fundamentals of Speech Recognition
Fundamentals of Speech Recognition
Lawrence R. Rabiner and Biing-Hwang Juang · 1993
A focused historical reference for ASR foundations. Its 1993 edition predates current neural and generative speech systems, so use it for conceptual background rather than current model guidance.