Objc - Speech Sample

objc - Speech Sample

Objc - SpeechSample.zip (1.9 KB)

Codea Version

3.6 (In Beta)

Description

A simple example of using the SFSpeechRecognizer to control a character using voice commands. This sample uses the en_US locale but can easily be udpated to support a different language.

The SFSpeechRecognizer cannot easily recognize single words and thus the voice commands included are all two words long.

Here’s the list of voice commands included with the sample:

  • Go up
  • Go down
  • Go left
  • Go right
  • Say hello
  • Say goodbye

API Covered

objc.cls
objc.enum

1 Like

This is so cool! Feels like a great prototype for the basis of a virtual pet type game where you can talk to your pet

Another fun use of this would be to make a virtual magic 8-ball, dictate questions and get flippant advice

1 Like

A very creative example! very interesting!

I have a little problem, How to make it display text at the same time, can also produce sound to say hello?

I tried to modify the following code, but it didn’t work. It looks as if I’m not allowed to make a sound with the speech.say() while doing speech recognition

        ["Hello"] = function() say("Hello!"); speech.say("Hello!") end,
1 Like

@jfperusse - very neat, is the intention to include this facility within Codea or do we have to use these external C libraries?

@John - just a quick note on an observation that I have seen in V4 which seems to pop up intermittently. That is the stop execution button on the controls in the BRHS of the screen. I ran this demo, tapped the screen a few times and then tried to close the project - it was very slow to respond after several attempts.

Closing the project, restarting it and immediate closing worked on one tap quickly. But, a repeat and after several on screen taps before tapping the close control again took ages and seemed not to respond initially.

I don’t think this is limited to this project.

Feels like the parsing of the controls is not in the main touch loop.

Edit: weird, came out of Codea after posting this. Re-started Codea and loaded this project and ran it, but this time I noticed the speech bubble posted up (wasn’t there before) and the touch response was excellent. Looks like it may be one of those something left in memory interfering with a newly loaded project. Or, since I downloaded this project and ran it directly, some interference from the downloaded project.

On that topic - it seems a bit odd that you need to set up a new project to access the forum website from the Codea Project Editor menu - would it be better to access the forum website via the project files window and download and store before running projects?

Thanks! It’s quite possible the audio engine used for the speech recognition is preventing the ability to play speech audio. I will have to investigate if that’s an issue with how it’s configured or if both simply cannot work together at the same time. If it’s the later, then one approach which should work would be to enable/disable the speech recognition based on inputs. For example, you could have a “Talk” button which you have to hold to use speech recognition, release it when you’re done talking to process the command, and then play the audio using the speech api.

At the moment, the main intention of this sample is to show how the objc feature can be used to access speech recognition, but it should be easy to hide most of the low-level objc functionnality behind an higher level Lua library using the code I shared.

I’ve learned that, like some social software, there should be a state-switching mode that turns on speech recognition and disables voice playback when you speak. When voice input is stopped, the voice recognition function is turned off and the voice is allowed to play. Thank you for the explanation.