Solved

resetting recognition engine to fewer phrases...

Posted on 2011-03-13
16
1,030 Views
Last Modified: 2012-06-27
The code attached attempts to adjust the speech recognizer so it will only recognize one phrase.
Once that phrase is recognized, that grammer will be unloaded and another grammer, containing many more commands, will be loaded into the recognizer to await further user input.

(The idea is the speech engine only responds to one phrase at first, then once it hears that phrase, it will respond to additional phrases.)

My problem is, despite calling recognizer.UnloadAllGrammers, the recognizer still responds to phrases that were in its previously-loaded grammer.  That is not the behavior I need.

Debugging, I was able to confirm that the new grammer only contains the one phrase to which I want the engine to respond after running the Reset code attached.

Can anyone tell me what it is I may be doing wrong here?



Public Function Reset(ByRef lex As LastException) As Boolean

        Dim retval As Boolean = True

        If Not _recnizer Is Nothing Then
            _recnizer.PauseRecognizerOnRecognition = True
            _recnizer.UnloadAllGrammars()
        End If

        _commands = Nothing


        _commands = New Speech.Recognition.Choices

        _commands.Add(My.Settings.Attention_Phrase)
        
       

        _gbldr = New System.Speech.Recognition.GrammarBuilder(_commands)


        If Not _grm Is Nothing Then
            _grm = Nothing
        End If

        _grm = New System.Speech.Recognition.Grammar(_gbldr)


        If Not _recnizer Is Nothing Then
            _recnizer.LoadGrammar(_grm)
        End If


        LastCommand = ""


        Return retval
    End Function

Open in new window

0
Comment
Question by:codefinger
  • 8
  • 8
16 Comments
 
LVL 96

Assisted Solution

by:Bob Learned
Bob Learned earned 500 total points
Comment Utility
This is where it gets kinda tricky...the recognition engine is asynchronous, multi-threaded, and timing is everything.  .NET is non-deterministic, and setting _commands = Nothing doesn't happen right away.  Where are you calling the Recognize method?  I am not exactly sure how to ask the recognizer engine what phrases it works with...
0
 

Author Comment

by:codefinger
Comment Utility
Ah, so this time it isn't because of some code I wasn't aware of....like maybe there was a refresh() call on the recognizer or something like that needed after UnloadAllGrammers.

I am not exactly sure how to ask the recognizer engine what phrases it works with...

I have code in the Speech_Recognized event to acknowledge when it hears something it recognizes.
So when my grammer doesn't include a phrase and it reacts anyway, I have a problem. (Debugging confirmed it did not just mishear something else...it actually heard and recognized the earlier phrase that was not supposed to be available.)

I had thought earlier that instead of unloading and loading grammers, I should just load all that I would need and enable and disable them as called for.  The application should react a little faster and MAYBE that approach will resolve the timing issue as well.  I will be working on that this evening.  

Thanks for the input.  I'll let you know how it turns out.
0
 
LVL 96

Expert Comment

by:Bob Learned
Comment Utility
What happens if you start with the 2nd set of phrases, without loading the 1st set of phrases, does it have the same recognition problem?
0
 

Author Comment

by:codefinger
Comment Utility
I think I found part of the solution...there is a property of the recognizer called PauseOnRecognition that can be set to true.   Its too soon to tell for sure, but so far, that seems to have helped.

My new approach is not working out quite like I planned.  I don't think I quite understand how to work with multiple grammars loaded at the same time, but separating the two groups of phrases into two different grammars and unloading and loading them as needed also seems to be helping.

So I am getting there...I won't have a chance to work on this again until Thursday.  Again, thanks for the input.  I will let you know how it goes.
0
 
LVL 96

Expert Comment

by:Bob Learned
Comment Utility
I thought you already had the pause in the code:

        
        If Not _recnizer Is Nothing Then
            _recnizer.PauseRecognizerOnRecognition = True
            _recnizer.UnloadAllGrammars()
        End If

Open in new window


Are you saying that you moved that call to a different point in the process?
0
 

Author Comment

by:codefinger
Comment Utility
DOH!  I thought I just added that pause for the first time last night.   Did not even realize it was already in the orignal code, so I guess that pause is not what is helping after all....sorry, guess I am getting a bit senile....please try to be patient as I dodder along here....
0
 
LVL 96

Expert Comment

by:Bob Learned
Comment Utility
I have never worked with multiple grammars, either, so I am very interested in this process, and the problems that you are experiencing.
0
 

Accepted Solution

by:
codefinger earned 0 total points
Comment Utility
"Timing is everything" has put me on the right track to a solution.  I was adjusting the same grammar to different choices, which did not happen right away, so I got some unexpected results.  The key is to load different grammars as they are needed.  (Its probably possible to work with multiple grammars loaded at the same time, but I had no real need to do it that way.)   This way, too, if you have a grammar that does not change, you only have to load its choices one time.

I will post some of the code involved when I am finished testing it.
0
Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

 
LVL 96

Expert Comment

by:Bob Learned
Comment Utility
I would love to hear why you need multiple grammars, and to switch between them.
0
 

Author Comment

by:codefinger
Comment Utility
Imagine that you cannot use your hands.  You cannot push a mute button on your microphone or reach up to take off your headset.   When you are controlling your computer, the microphone must be listening all the time.  But sometimes you need to talk to someone else in the room.  How does the computer "know" when/if you are talking to it and when you are talking to someone else?

My approach is to use two grammars.   The first grammar can only recognize an attention phrase (user defined), like "Computer" or "Attention" and a couple of cancel phrases "Never mind" or "Not you".

When an attention phrase is recognized, the second, more extensive grammar is loaded, and interacts with the user until it is no longer needed.  Then it is unloaded and the first grammar is loaded again to wait quietly until the second grammar is needed again.  The first grammar acts as the gatekeeper.



0
 
LVL 96

Expert Comment

by:Bob Learned
Comment Utility
I can't quite remember, but are you using SpeechRecognitionEngine or SpeechRecognizer?
0
 

Author Comment

by:codefinger
Comment Utility
Public WithEvents _recnizer As New Speech.Recognition.SpeechRecognizer

0
 
LVL 96

Assisted Solution

by:Bob Learned
Bob Learned earned 500 total points
Comment Utility
With the SpeechRecognitionEngine, you can start and stop the recognition process:

SpeechRecognitionEngine.RecognizeAsync Method ()
http://msdn.microsoft.com/en-us/library/ms554581(v=vs.85).aspx

Private Sub _startRecogButton_Click(sender As Object, eventArgs As EventArgs)
	If _startRecogButton.Text = "Start Speech Recognition" Then
		'Toggle Speech Recogniztion on
		_startRecogButton.Text = "Stop Speech Recognition"
		If _useMultiple Then
			_recognitionEngine.RecognizeAsync(RecognizeMode.Multiple)
		Else
			_recognitionEngine.RecognizeAsync(RecognizeMode.[Single])
		End If
	Else
		'Toggle Speech Recognition off
		_startRecogButton.Text = "Start Speech Recognition"
		If _friendlyStop Then
				'Stop after current phrase is finished.
			_recognitionEngine.RecognizeAsyncStop()
		Else
				'Stop before current phrase is finished.
			_recognitionEngine.RecognizeAsyncCancel()

		End If
	End If
End Sub

Open in new window


I haven't found a way to do that with the SpeechRecognizer (not to say that it isn't possible).
0
 

Author Comment

by:codefinger
Comment Utility
That would probably work, but would not the user have to click a mouse?  (No hands, remember?)

Anyway, my approach of loading and unloading the needed grammars seems to work.  I have attached some useful code:  The function attached displays the available phrases from the currently loaded Grammar.   (Its always Grammar(0), since I only use one grammar at a time.)

(FYI, sc is my SpeechClass.)


Public Function DisplayCurrentCommandList(ByRef lex As LastException) As Boolean
        Dim retval As Boolean = True
        Dim strsplit() As String
        Dim cntr As Integer = 0
        Dim str_commands As String = ""
        Dim currentgrammar As System.Speech.Recognition.Grammar



        Try
            currentgrammar = sc._recnizer.Grammars(0)
            Select Case currentgrammar.Name
                Case "Attention Phrases"
                    strsplit = sc.AttentionCommands.ToGrammarBuilder().DebugShowPhrases().Split(",")
                Case "User Phrases"
                    strsplit = sc.UserCommands.ToGrammarBuilder().DebugShowPhrases().Split(",")
               
            End Select

            'eliminate the opening and closing brackets...
            strsplit(0) = Right(strsplit(0), strsplit(0).Length - 1)
            strsplit(strsplit.Length - 1) = Left(strsplit(strsplit.Length - 1), strsplit(strsplit.Length - 1).Length - 1)


            For cntr = 0 To strsplit.Length - 1
                str_commands = str_commands & strsplit(cntr) & vbCrLf
            Next

            frm.TextBoxCommands.Text = str_commands

        Catch ex As Exception

            lex.ErrorMessage = ex.Message
            If Not ex.InnerException Is Nothing Then
                lex.ErrorMessage = lex.ErrorMessage & vbCrLf & ex.InnerException.Message
            End If
            retval = False
        End Try

        Return retval
    End Function

Open in new window

0
 
LVL 96

Assisted Solution

by:Bob Learned
Bob Learned earned 500 total points
Comment Utility
Heck, my job, in this case, is to be an active observer.  That was only an example that shows the starting and stopping, and was not meant to show the trigger mechanism.  The SpeechRecognitionEngine has a few events that you can attach to, such as the SpeechRecognized event.

The event passes SpeechRecognizedEventArgs, which has the Result property (RecognitionResult).  RecognitionResult has the Words property, which is a ReadOnlyCollection(Of RecognizedWordUnit).  RecognizedWordUnit has different properties, including the Text property.

Example:

      Private Sub engine_SpeechRecognized(sender As Object, e As SpeechRecognizedEventArgs) Handles SpeechRecognitionEngine.SpeechRecognized
	Dim wordList As ReadOnlyCollection(Of RecognizedWordUnit) = e.Result.Words

	For Each word As RecognizedWordUnit In wordList
    	     Dim wordText As String = word.Text
	Next word
      End Sub

Open in new window

0
 

Author Closing Comment

by:codefinger
Comment Utility
Thanks for all the useful input.   It got me past my problem and on to my next challenge.  I will keep this discussion in my history and will probably be referring to it in the future.
0

Featured Post

How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

Join & Write a Comment

Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
When you start your Windows 10 PC and got an "Operating system not found" error or just saw  "Auto repair for startup". After a while, you have entered a loop for Auto repair which does not fix anything and you will be in a  panic as all your work w…
When you create an app prototype with Adobe XD, you can insert system screens -- sharing or Control Center, for example -- with just a few clicks. This video shows you how. You can take the full course on Experts Exchange at http://bit.ly/XDcourse.
This video explains how to create simple products associated to Magento configurable product and offers fast way of their generation with Store Manager for Magento tool.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now