Skip to main content Accessibility Feedback

The SpeechRecognition API

There’s an older deaf gentleman in my town who’s very friendly, and likes to communicate with rudimentary hand gestures, but I have a hard time understand what he’s trying to say.

I’d love to learn American Sign Language (ASL) at some point (shoutout King Ron), but until then, I wanted an app that could convert speech into text.

Every transcription app I found required a pretty expensive subscription… so I decided to build my own using web technologies!

Demo

Before we dive into how this all works, here’s the finished app.

Press the microphone button to listen to text and display it in the UI in real-time. You can also type/edit text manually with a keyboard.

The app works in Chrome, Edge, and Safari, but not in Vivaldi or Firefox (more on that later).

new SpeechRecognition()

The SpeechRecognition web API can take spoken audio and convert it into text.

Some browsers use a prefixed implementation, so to start, need to define SpeechRecognition and SpeechRecognitionEvent variables that account for both the unprefixed and prefixed objects.

const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;
const SpeechRecognitionEvent = window.SpeechRecognitionEvent || window.webkitSpeechRecognitionEvent;

The new SpeechRecognition() creates a new SpeechRecognition instance. We can then define a handful of properties on it.

  • continuous - If true, it converts speech as the user talks. If false, it converts in one operation when the user is done (defaults to false).
  • lang - The language to use.
  • interimResults - If true, return interim results before the session is done (defaults to false).
  • maxAlternatives - The number of possible matches to return per word or phrase.
const recognition = new SpeechRecognition();
recognition.continuous = true;
recognition.lang = 'en-US';
recognition.interimResults = true;
recognition.maxAlternatives = 1;

Listening

When the user presses the button (or in some other way toggles speech-to-text), run the .start() method to start listening and transcribing.

recognition.start();

When they press the button again (or in some other way stop listening), run the .stop() method.

recognition.stop();

Getting Results

While the SpeechRecognition instance is running, it will emit result events with an iterator of transcription .results.

Each item in .results contains an iterator of transcriptions (up to the number of maxAlternatives provided). Each one has a .transcript property with the text.

You can convert the array of .results into a string using the .map() and .join() methods. Here, I’m displaying them as text inside an element in the UI.

const result = document.querySelector('#result');
recognition.addEventListener('result', (event) => {
	result.textContent = 
		[...event.results]
			.map((result) => result[0].transcript)
			.join(' ');
});

Not really a web API

So… why doesn’t this work in Vivaldi, a Chromium browser, if it works in Chrome and Edge? And why does it work in Safari but not Firefox?

The SpeechRecognition API isn’t really a web API. Transcription isn’t built into browsers. The audio is send to a third-party service.

Chromium browsers use Google’s API. Safari uses Apple’s.

Because Google owns Chrome, they get it for free. Microsoft pays for it in Edge. Vivaldi and Firefox do not, so the “web API” doesn’t work.

There is an option for local-only mode, but it doesn’t seem to actually work anywhere.