Android and Accessibility

Today I want to show you how to enhance your android applications by providing better accessibility. Android has very good built-in features to support better accessibility. In this post I want to concentrate on the Text to Speech Feature and the Speech Recognition Feature. As there are already articles and API demos on the google android developer pages, I will only talk about the basics and provide you with an aggregated example of an Text to Speech and Speech Recognition application.

Text to Speech

With version 1.6 (Donut), android introduced the Text to Speech capability to the platform. The TTS Engine supports several languages: English, French, German, Italian and Spanish. Even accents are supported for british or american english.

To check if TTS is installed on your device you can use the intent mechanism of android by calling the startActivityForResult() method with predefined intent.

In your onActitivityResult() method you can instantiate a new TextToSpeech object afterwards. For the case that TTS is not installed on your device, google already provides an installation intent which you can trigger.

Use the article example provided by google for your integration and for further detailed information about the Text to Speech Feature. It can be found here.

Note that you have to implement the OnInitListener interface to determine when the TTS Engine is initialized. Afterwards you can configure it to your needs by providing a certain Locale for example.

Before configuring the engine for a specific Locale make sure to check if it is supported by your system and if the corresponding resources are installed.

If everything is set up, it is time to let your device talk to YOU for a change.

This can be done by calling the speak() method on your TTS object. Note that it has three parameters.

The first parameter is the text you want to have spoken.

The second parameter is the queue mode.

The TTS Engine holds a queue for the texts which have to be spoken and works through it one after another. If you want to just simply add a new text to the queue you would use the TextToSpeech.QUEUE_ADD mode. If you want to clear the current queue to start a new one you use the TextToSpeech.QUEUE_FLUSH mode. 

With the third parameter you can define optional parameters.

As the Text to Speech feature relies on a shared service it is crucial that you call the shutdown() method when you are finished.

Speech Recognition

The Speech Recognition capability of android is based on the Google Voice Search application. It is preinstalled on many android devices. Again you should check for its availability on the device before trying to use it. With the help of the PackageManager you can search for Activities which can handle a ACTION_RECOGNIZE_SPEECH Intent.

PackageManager pm = getPackageManager(); 

List activities = pm.queryIntentActivities(

new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH), PackageManager.MATCH_DEFAULT_ONLY);

if (activities.size() != 0) {

    //available

}

If the Voice Search application is available you can call it with the startActivityForResult() method.

You will see a progress indicator followed by a dialog which records your speech.

The data is sent to Google's servers for processing.

When the data has been processed, your onActivityResult() method is called with an Intent object which holds the result data. The result is represented as an ArrayList of type String which can hold multiple matches.

As an additional feature Google introduced the Voice Search as an input method for any text based input field in the android platform from version 2.1(Eclair) upwards. To use this feature you have to enable it on your soft-keyboard. Just press the cogwheel/settings icon on your keyboard and activate the voice input. Afterwards you will see a small microphone icon on the soft-keyboard. Just tap it and you can use the voice recognition as an input method.

Here is the example code:

package devorama.marioboehmer.accessibilityspeechexample;

import java.util.ArrayList;

import java.util.List;

import java.util.Locale;

import android.app.Activity;

import android.content.Context;

import android.content.Intent;

import android.content.pm.PackageManager;

import android.content.pm.ResolveInfo;

import android.os.Bundle;

import android.speech.RecognizerIntent;

import android.speech.tts.TextToSpeech;

import android.speech.tts.TextToSpeech.OnInitListener;

import android.view.View;

import android.view.View.OnClickListener;

import android.widget.Button;

import android.widget.EditText;

import android.widget.Toast;

/**

 * Simple demo for better accessibility. Serves as an example for text to speech capabilities of android which is supported

 * from version 1.6 upwards and for speech recognition.

 * 

 * @author mboehmer

 */

public class SpeechActivity extends Activity implements OnInitListener {

private Button speakButton;

private Button listenButton;

private EditText editText;

private TextToSpeech mTts;

private static final int TTS_ACTIVITY_RESULT_CODE = 1;

private static final int SPEECH_RECOGNITION_ACTIVITY_RESULT_CODE = 2;

private boolean tTSInitialzed;

private boolean speechRecognitionInstalled;

private Context context = this;

@Override

public void onCreate(Bundle savedInstanceState) {

super.onCreate(savedInstanceState);

setContentView(R.layout.main);

speakButton = (Button) findViewById(R.id.speak_button);

listenButton = (Button) findViewById(R.id.listen_button);

editText = (EditText) findViewById(R.id.edittext);

// check for TTS

Intent checkTTSIntent = new Intent();

checkTTSIntent.setAction(TextToSpeech.Engine.ACTION_CHECK_TTS_DATA);

startActivityForResult(checkTTSIntent, TTS_ACTIVITY_RESULT_CODE);

// check for speech recognition

PackageManager pm = getPackageManager();

List<ResolveInfo> activities = pm.queryIntentActivities(

  new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH), PackageManager.MATCH_DEFAULT_ONLY);

if (activities.size() != 0) {

speechRecognitionInstalled = true;

}

speakButton.setOnClickListener(new OnClickListener() {

public void onClick(View v) {

if (tTSInitialzed) {

// check if your locale is supported by tts

if (mTts.isLanguageAvailable(Locale.getDefault()) == TextToSpeech.LANG_COUNTRY_AVAILABLE) {

mTts.setLanguage(Locale.getDefault());

mTts.speak(editText.getText().toString(),

TextToSpeech.QUEUE_FLUSH, null);

// use the tts queue for following texts

// mTts.speak(editText.getText().toString(),

// TextToSpeech.QUEUE_ADD, null);

// or flush the queue to cancel current spoken text and

// start new text

// mTts.speak(editText.getText().toString(),

// TextToSpeech.QUEUE_FLUSH, null);

} else {

Toast localeNotSupportedToast = Toast.makeText(context,

R.string.locale_not_supported_text,

Toast.LENGTH_SHORT);

localeNotSupportedToast.show();

}

} else {

Toast ttsNotInitializedToast = Toast.makeText(context,

R.string.tts_not_initialized_text,

Toast.LENGTH_SHORT);

ttsNotInitializedToast.show();

}

}

});

listenButton.setOnClickListener(new OnClickListener() {

public void onClick(View v) {

if(speechRecognitionInstalled) {

Intent recognizeSpeechIntent = new Intent();

recognizeSpeechIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,

                RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);

recognizeSpeechIntent.putExtra(RecognizerIntent.EXTRA_PROMPT, getText(R.string.speech_recognition_prompt_text));

recognizeSpeechIntent.setAction(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);

startActivityForResult(recognizeSpeechIntent, SPEECH_RECOGNITION_ACTIVITY_RESULT_CODE);

}

}

});

}

@Override

protected void onActivityResult(int requestCode, int resultCode, Intent data) {

if (requestCode == TTS_ACTIVITY_RESULT_CODE) {

if (resultCode == TextToSpeech.Engine.CHECK_VOICE_DATA_PASS) {

// success, create the TTS instance

mTts = new TextToSpeech(this, this);

} else {

// missing data, install it

Intent installIntent = new Intent();

installIntent

.setAction(TextToSpeech.Engine.ACTION_INSTALL_TTS_DATA);

startActivity(installIntent);

}

} else if (requestCode == SPEECH_RECOGNITION_ACTIVITY_RESULT_CODE) {

ArrayList<String> matches = data.getStringArrayListExtra(

                    RecognizerIntent.EXTRA_RESULTS);

if(matches.size() != 0) {

// we only want to show the best matching result

editText.setText(matches.get(0));

} else {

Toast speechNotRecognized = Toast.makeText(context,

R.string.speech_not_recognized_text,

Toast.LENGTH_SHORT);

speechNotRecognized.show();

}

}

super.onActivityResult(requestCode, resultCode, data);

}

@Override

public void onInit(int status) {

if (status == TextToSpeech.SUCCESS) {

tTSInitialzed = true;

}

}

@Override

protected void onDestroy() {

if (mTts != null) {

mTts.shutdown();

}

super.onDestroy();

}

}

The complete project for this example can be found on github.

Further information about the feature can be found in this android article.