Speech-To-Text

Speech-To-Text (STT) is the process of converting audio of spoken words into strings of text. Mycroft supports a range of Speech-To-Text engines.

Many users want to use a specific STT engine rather than the default. Like most of Mycroft's technology stack, this too can be customized.

Default Engine

For a voice assistant like Mycroft, speech recognition must be performed very quickly and with a high degree of accuracy. For this reason, Mycroft by default uses Google's STT engine.

In order to provide an additional layer of privacy for our users, we proxy all STT requests through Mycroft's servers. This prevents Google's service from profiling Mycroft users or connecting voice recordings to their identities. Only the voice recording is sent to Google, no other identifying information is included in the request. Therefore Google's STT service does not know if an individual person is making thousands of requests, or if thousands of people are making a small number of requests each.

By supporting Mozilla's DeepSpeech project we are aiming to provide a competitive open source alternative. The accuracy of DeepSpeech is not yet sufficient to provide a quality experience for Mycroft users. However we will be switching to DeepSpeech by default as soon as we have achieved an acceptable level of accuracy.

The following are some of the available STT options. Each provides details on how to get setup, and how to configure Mycroft.

Mozilla DeepSpeech

Mycroft has been supporting Mozilla's efforts to build DeepSpeech, an open Speech-to-Text technology. It is a fully open source STT engine, based on Baidu’s Deep Speech architecture and implemented with Google’s TensorFlow framework. Being open source means that if you have the hardware, it can be run within your own network providing additional privacy and control for you and your family.

Server Setup

You can test DeepSpeech using their pre-trained model by following the instructions on the DeepSpeech Github repository.

To setup a DeepSpeech server that Mycroft can use, try the deepspeech-server project on PyPI. Once you have this up and running, we can configure Mycroft to use this server.

Mycroft Configuration

Using the Configuration Manager we can edit the mycroft.conf file by running:

mycroft-config edit user

To our existing configuration values we will add the following:

  "stt": {
    "deepspeech_server": {
      "uri": "http://localhost:8080/stt"
    },
    "module": "deepspeech_server"
  }

Community Support

If you are interested in the continued development of the DeepSpeech STT engine, please join our the DeepSpeech channel on Mycroft Chat.

Kaldi

Kaldi is described as a toolkit for speech recognition written in C++ and licensed under the Apache License v2.0. It is intended for use by speech recognition researchers.

Kaldi can be run on a Linux cluster or an individual machine, making it another option for those wanting local network speech-to-text.

Server Setup

First be sure to read the system requirements in the Kaldi documentation.

The latest installation instructions can be found on the Kaldi Github repository.

Mycroft Configuration

Using the Configuration Manager we can edit the mycroft.conf file by running:

mycroft-config edit user

To our existing configuration values we will add the following:

  "stt": {
    "kaldi": {
      "uri": "http://localhost:8080/client/dynamic/recognize"
    },
    "module": "kaldi"
  }

GoVivace

GoVivace is a for-profit company that has a proprietary STT engine.

Account Setup

The software is available in both 32 and 64-bit versions for Linux, Windows, and Mac platforms. A minimum of 4GB of RAM and a 2.0GHz processor is recommended. A UniMRCP server plugin is also available.

See their website for more details.

Mycroft Configuration

Using the Configuration Manager we can edit the mycroft.conf file by running:

mycroft-config edit user

To our existing configuration values we will add the following:

"stt": {
  "govivace": {
    "uri": "https://services.govivace.com:49149/telephony",
    "credential": {
      "token": "xxxxx"
    }
  },
  "module": "govivace"
}

The credential token will be provided to you by GoVivace.

Google Cloud

The standard Google Cloud Speech-To-Text API.

Account Setup

A Google Cloud account with active billing is required. Please carefully consider the financial cost of using this service.

To obtain the required credential JSON data, you must create a Google API Console project. To do this:

• Select or create a GCP project in the Cloud Resource Manager

• Make sure that billing is enabled for your project - see documentation

• Enable the Cloud Text-to-Speech API from your GCP Console

• Set up authentication:

  • Go to the Create service account key page in the GCP Console

  • From the Service account drop-down list, select New service account.

  • Enter a name into the Service account name field.

  • Don't select a value from the Role drop-down list. No role is required to access this service.

  • Click Create. A note appears, warning that this service account has no role.

  • Click Create without role. A JSON file that contains your key downloads to your computer.

Remember to activate the API in the GCP Console

Mycroft Configuration

Using the Configuration Manager we can edit the mycroft.conf file by running:

mycroft-config edit user

To our existing configuration values we will add the following:

"stt": {
  "google_cloud": {
    "lang": "hi-in",
    "credential": {
      "json": {
        "type": "service_account",
        "project_id": "xxxxxxxxxx",
        "private_key_id": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
        "private_key": "-----BEGIN PRIVATE KEY-----nxxxxxxxxxxxxxxxxn-----END PRIVATE KEY-----n",
        "client_email": "[e-mail]",
        "client_id": "xxxxxxxxxxxxxxxxxxxxx",
        "auth_uri": "https://accounts.google.com/o/oauth2/auth",
        "token_uri": "https://oauth2.googleapis.com/token",
        "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
        "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/xxxxxx.iam.gserviceaccount.com"
      }
    }
  },
  "module": "google_cloud"
},

Google Cloud Streaming STT

A streaming STT interface for the Google Cloud Speech-To-Text API.

Account Setup

A Google Cloud account with active billing is required. Please carefully consider the financial cost of using this service.

Mycroft Configuration

Install google-cloud-speech in the Mycroft Virtual environment using:

mycroft-pip install google-cloud-speech

Then, using the Configuration Manager we can edit the mycroft.conf file by running:

mycroft-config edit user

To our existing configuration values we will add the following:

"stt": {
  "google_cloud_streaming": {
    "credential": {
      "json": # Paste Google API key JSON here
    }
  },
  "module": "google_cloud_streaming"
}

Houndify

STT provided by Houndify.

Account Setup

Create a Houndify account, then:

• Create a New Client from your dashboard

  • Give your client a name and select a platform.

• Enable the "Speech To Text Only" domain for your Client.

• Get the Client ID and Client Key from your Client Information panel.

Mycroft Configuration

Using the Configuration Manager we can edit the mycroft.conf file by running:

mycroft-config edit user

To our existing configuration values we will add the following:

"stt": {
  "houndify": {
    "credential": {
      "client_id": "xxxxx",
      "client_key": "xxxxx"
    }
  },
  "module": "houndify"
}

IBM Cloud

IBM Cloud - Watson Speech to Text is a cloud-based deep-learning speech-to-text service offered on top of the IBM Watson platform.

Account Setup

Create an account at IBM.com/cloud, then:

• Create a New Resource from your dashboard

  • Select "Speech to Text" as the product

• Retrieve the API Key and URL from the Services section of your dashboard

Mycroft Configuration

Using the Configuration Manager we can edit the mycroft.conf file by running:

mycroft-config edit user

To our existing configuration values we will add the following:

"stt": {
  "ibm": {
    "credential": {
      "token": "YOUR_API_KEY"
    },
    "url": "YOUR_URL"
  },
  "module": "ibm"
}

Microsoft Azure

STT provided by the Microsoft Azure Speech Services. Formerly known as Bing STT.

Account Setup

Create a Microsoft Azure account and get a server access token.

Mycroft Configuration

Using the Configuration Manager we can edit the mycroft.conf file by running:

mycroft-config edit user

To our existing configuration values we will add the following:

"stt": {
  "bing": {
    "credential": {
      "token": "xxxxx"
    }
  },
  "module": "bing"
}

Wit.ai

A natural language platform owned by Facebook.

Account Setup

Create an account at Wit.ai then create a new app to get your server access token. See the Wit.ai documentation for further details.

Mycroft Configuration

Using the Configuration Manager we can edit the mycroft.conf file by running:

mycroft-config edit user

To our existing configuration values we will add the following:

"stt": {
  "wit": {
    "credential": {
      "token": "xxxxx"
    }
  },
  "module": "wit"
}

Yandex SpeechKit STT

Yandex is one of the largest cloud platforms in Russia.

Account Setup

Create a Yandex Cloud account, then:

• Create a billing account - you can activate a free period in the console.

• Create first "folder" in cloud.

• Create a service account for your Mycroft instance with role editor.

• Create an API key for your service account.

See the Yandex Identity and Access Management documentation for further details.

Mycroft Configuration

Using the Configuration Manager we can edit the mycroft.conf file by running:

mycroft-config edit user

To our existing configuration values we will add the following:

"stt": {
  "yandex": {
    "lang": "en-US",
    "credential": {
      "api_key": "YOUR_API_KEY"
    }
  },
  "module": "yandex"
}