EP3057092B1

EP3057092B1 - Methods for training a speech recognition system

Info

Publication number: EP3057092B1
Application number: EP16154356.6A
Authority: EP
Inventors: John Pecorari
Original assignee: Hand Held Products Inc
Current assignee: Hand Held Products Inc
Priority date: 2015-02-11
Filing date: 2016-02-04
Publication date: 2017-07-26
Anticipated expiration: 2036-02-04
Also published as: US20160232891A1; EP3057092A1; US10121466B2

Description

FIELD OF THE INVENTION

The present invention relates to voice-directed workflow and, more specifically, to a speech recognition system with voice templates that are helped made distinct by a dynamic training analysis.

BACKGROUND

Voice-directed workflow systems allow workers to communicate verbally with a computer system. These systems may be used in warehouses or distribution centers to improve safety and efficiency for tasks such as picking, receiving, replenishing, and/or shipping.
Voice-directed workflow systems typically require a worker to wear a headset equipped with a microphone and earphone. Voice commands are transmitted to the worker via the earphone and spoken responses from the worker are received by the microphone. In this way, a worker may be directed to perform a task and respond with their progress by speaking established responses into the microphone at certain points in an established workflow dialog.
Speech recognition is part of a voice-directed workflow system. Speech recognition is the translation of spoken words into text/data via a computing device. A computing device configured for speech recognition is known as a speech recognizer.
Speech recognition is a challenging problem for a variety of reasons. First, the speech recognizer must detect speech versus background noise. For example, the speech recognizer must recognize that a sound represents speech rather than a breath. Next, the speech recognizer must compare the speech input to words and/or phrases in a vocabulary typically specific to the application (i.e., application vocabulary). Here, the speech recognizer may use the workflow dialog to help determine what was said.
Often, for a particular workflow dialog, the expected responses are limited to a range of possible responses, or even a single expected response. For example, if a worker is given a picking task with the prompt, "pick two," and the worker is expected to confirm the picking task with the response "two," then the speech that occurs after the prompt may be expected to match a voice template for "two." In general, a workflow has an associated application vocabulary consisting of voice templates for the vocabulary words, sounds, or phrases necessary to carry out the tasks associated with workflow.
Voice templates (i.e., speech templates or templates) are voice patterns for particular words or phrases stored in memory. The voice templates may be specific to a user in speaker-dependent recognition systems. Alternatively, the voice templates may be for all users (i.e., generic) in speaker-independent recognition systems. In either case, the speech recognizer determines how closely the received speech matches a stored voice template to determine what was most likely spoken.
Since everyone's speech may be different, custom voice templates may be created. To create a custom voice template for a word, a user may be prompted (e.g., through a display) to provide speech samples (e.g., by repeatedly saying a word). It is common to require workers new to a voice-directed workflow system to train the system for their voice by creating voice templates for a variety of words and/or sounds.
A problem arises when the voice templates created by a worker are not distinct enough for a speech recognizer to distinguish it from other words in the application vocabulary. For example, some workers may pronounce the word, "five," and the word, "nine," similarly. This may result in voice templates created for the word, "five," that are very similar to the voice template, "nine."
Voice template similarity may erode the speech recognizer's performance. For example, a worker may be asked to repeat what they have said which may reduce productivity and cause frustration. Errors may also occur as numbers may be transposed (e.g., a 5 recorded when a 9 was intended, or vice-versa).
Therefore, a need exists for analysis during the creation of a voice template (i.e., during training) to insure that a created voice template is not similar to (or does not match with) any other stored voice templates. If a similarity is found, then a user may be prompted to create a new, more distinct, voice template for the word. This dynamic training analysis may improve user experience and accuracy for voice-directed workflow systems.
European Patent Application EP1 079 370 A2 discloses a method for training a speech recognitions system, where the system alerts the user if a new word is confusingly similar to an existing word using reference models.

SUMMARY

Accordingly, in one aspect, the present invention embraces a method for creating a voice template for a speech recognition system as set forth in claim 1. Preferred embodiments are set forth in the dependent claims.
The foregoing illustrative summary, as well as other exemplary objectives and/or advantages of the invention, and the manner in which the same are accomplished, are further explained within the following detailed description and its accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 (Fig. 1) illustrates a perspective view of a worker using a speech recognizer in a typical work environment according to an embodiment of the present invention.
Figure 2 (Fig. 2) is a flowchart illustrating a method for creating a voice template for a speech recognition system according to an embodiment of the present invention.
Figure 3 (Fig. 3) is a flowchart illustrating a method for re-training a speech recognition system.

DETAILED DESCRIPTION

Voice-directed workflow systems (e.g., used in warehouses or distribution centers) may benefit from speech recognition. Speech recognition systems help workers perform tasks (e.g., picking or restocking) without the need for paper or displays. As a result, the worker's hands and eyes are free to perform a task.
In these systems, each worker uses a speech recognition system communicatively connected to a host computer running software that supervises the workflow. A task prompt for a worker may be created by the host computer and then sent wirelessly to a speech recognition system worn by a worker. The speech recognition system may then convert the text/data task prompt into speech (e.g., using a speech synthesizer) and relay the spoken task prompt to the worker via a speaker (e.g., an earphone). The worker's spoken responses may be collected via a microphone, recognized as speech, converted into data/text, and then transmitted back to the host computer wirelessly.
Fig. 1 illustrates a perspective view of a worker using an exemplary speech recognition system. The speech recognition system has an audio input/output (I/O) device for receiving/transmitting audio. The audio I/O device shown in Fig. 1 is a headset worn by a worker 1. The headset is configured with an earphone 5 for transmitting sounds and speech to the worker 1 and a microphone 4 for receiving voice input from the worker 1.
The audio I/O device is communicatively coupled to a computing device 7. In some possible embodiments, the audio I/O device is integrated with the computing device 7 into a headset. In others, like the embodiment shown in Fig. 1, the computing device 7 is worn on a worker's body (e.g., via a belt 3). In some embodiments, the computing device may be wirelessly connected (e.g., BLUETOOTH™, near-field communication, etc.) to the headset, while in others the computing device may be connected via a cable 6.
The computing device 7 may be a single-purpose device, multipurpose device (e.g., barcode scanner), or may be a general purposed device like a smartphone. The computing device 7 may include variety of means for input/output (e.g., a display, buttons, touchscreen, etc.) and may have connectors 2 that enable peripheral input/output devices to be attached either temporarily or permanently.
The computing device 7 typically has some means of storage or memory (e.g., RAM, ROM, CD, DVD, hard-drive, solid state drive, etc.). Software programs and data may be stored in the memory and accessed by a processor (e.g., one or more controllers, digital signal processor (DSP), application specific integrated circuit (ASIC), programmable gate array (PGA), and/or programmable logic controller (PLC)).
The software programs stored in the memory and accessed by the processor may enable the speech recognition system to convert a digitally sampled voice waveform signal into text/data that represent the speech's intended meaning.
To accomplish speech recognition, the speech recognizer must first detect that something was spoken rather than some other sound (e.g., breath, wind, background noise, etc.). Next, the waveforms for the spoken words/phrase may be compared to a selected set of voice templates. The selected set of voice templates may be voice templates for expected words/phrase determined by the workflow dialog. For example, the response to a yes/no question is expected to be "yes" or "no." The speech recognizer determines which word/phrase from the selected set best matches what was spoken. For example, a similarity score may be computed between the spoken word and a voice template. If this similarity score is above a threshold then the spoken word may be considered an acceptable match to the voice template.
A voice template is representative voice waveform for a particular word. An application vocabulary is a collection of voice templates representing the words in the vocabulary. These voice templates may be unique to each user (i.e., custom) or may be generic for all users. Creating a custom voice template requires training.
Training allows each worker to create custom voice templates for the words in the application vocabulary. For example, a new worker may be required to train a speech recognition system before use (i.e., enrollment training). During a training session, a word or phrase may be presented to a worker via a display (e.g., on a display temporarily attached to the computing device 7). The worker may read the word aloud several times into the microphone 4. A program running on the computing device 7 may receive the speech signals and compute a statistical average of the word to form a voice template. The voice template may then be stored in the memory as part of the application vocabulary.
Custom voice templates are used in speaker-dependent speech recognition systems, while generic voice templates are used in speaker-independent speech recognition systems. Some speech recognition systems, however, may have both generic and custom voice templates to improve accuracy for a particular user on words that may sound alike.
Re-training (i.e., update training) a voice recognition system is sometimes necessary. In some cases, a speech recognition system will have poor performance on a particular word. For example, a user may notice that the system often requires the user to repeat the word, or a user may notice that the system falsely recognizes one word as another. Here, the worker may initiate re-training in order to create a new voice template for the word. In some examples, the detection of poor performance and/or the re-training may be done automatically by the speech recognition system.
One cause of poor recognition performance is voice template similarity. Similar voice templates make template matching difficult. Similar voice templates are common for words that sound similar (e.g., "five" and "nine"). It is especially troublesome for words of the same class (e.g., numbers), words that may be spoken together, and/or words that are equally expected at a dialog response points. Sometimes the similarity can be corrected by better enunciation or different pronunciation of the word/phrase.
The present invention embraces methods that prevent voice template similarity from resulting during training or re-training. These methods proactively prevent workers from completing training of an application vocabulary with voice templates for words that could otherwise confuse the speech recognition system.
Figure 2 illustrates a method for creating a voice template for a speech recognition system according to an embodiment of the present invention.
The method begins with the step of acquiring a speech sample 8. This speech sample is typically a spoken word but could also be a set of spoken words (i.e., phrase). The speech sample may be a word/phrase spoken once or may be a word/phrase spoken repeatedly. The word/phrase is part of an application vocabulary that includes voice templates for different words/phrases. The voice templates for word/phrases in the application vocabulary may be generic voice templates for all users or may be custom voice templates for a single user.
A voice template is created 10 for the spoken word from the speech sample. The voice template may be a file of data points representing the digital samples of the voice waveform created when the word is spoken into the microphone 4 and digitized by the computing device 7.
The voice template for the word is compared to voice templates from the application vocabulary 15. This comparison may yield a similarity score that may be used as the basis for determining if the voice template for the word is too similar to other words already in the application vocabulary. Various methods such as dynamic time warping (DTW) may be used to evaluate this similarity. For example, a similarity score may be created and compared to a threshold to determine if two words match.
The created voice template may be compared to the all words in the application vocabulary or a subset of words in the application vocabulary. For example, a subset of words may be words that sound alike or words from the same type (e.g., rhyming words) or class (e.g., numbers).
In speaker-dependent speech recognition systems, template similarity may be found if the created voice template matches the wrong word or the correct word and at least one other word's custom template. For speaker-independent speech recognition systems, template similarity may occur when the created voice template matches multiple generic voice templates or the wrong generic voice template. When similarity is found 20, then the user may be prompted to create a new template for the word in a way that is more likely to create a voice template for the word that is less similar to the other words in the application vocabulary. This prompt may be embodied as a voice message on a speaker and/or a text/graphical message on a display.
The method also includes the step of providing instructions (i.e., prompts) to a worker to help the worker create a less similar voice template for the word 30. These instructions may include a list of possible alternate words that could be used in place of the word. For example, the alternate word "fiver" might be suggested for use in place of the word "five." In another embodiment, the instructions provided could include prompts to help a worker enunciate the word more clearly or to emphasize the word differently (e.g., emphasize the "f" in "five"). In still another example, a user may create their own word or sound to represent the word. This option may be especially useful for workers that have a native language that is different from the application dialog language. For example, a worker may choose to say "cinco" for the word "five."
The method continues when a user applies the instructions and creates a new template for the alternate word. Here, the method may repeat creating alternate voice templates until a suitable (i.e., no template similarity) is found. When a suitable alternate (i.e., substitute) word has been found, training for that word ends and the substitute word's voice template is stored in the application vocabulary 25. Form that point on, the substitute word represents the dialog word in the application vocabulary. For example the method may result in the voice template for "fiver" stored in the application library for the word "five". At this point, other words may be trained or the training of the speech recognition system may conclude.
Sometimes re-training a speech recognition system on a word is required. A flowchart for a method for re-training a speech recognition system according to an example is shown in Fig. 3.
A speech recognition system may periodically evaluate its performance 35. If the speech recognizer is performing poorly (e.g., on a particular word) then the re-training may be initiated automatically. In some possible embodiments, the re-training may be initiated manually by a user. This initiation of re-training may be based on a user's evaluation or perception of the system's performance or may be for other reasons.
Re-training a speech recognition system begins with acquiring a speech sample (e.g., phrases that use the word) of a word 40. The speech sample is compared to voice templates for words (e.g., words that sound similar to the word) from an application vocabulary 45. If there the speech sample matches the wrong word or matches multiple words in the application vocabulary then the user is prompted (e.g., via graphic/text on a graphical user interface display) to retrain the system using an alternate word 55, 60. In one possible example, the alternate word includes words chosen from a list of suggested words. In another possible embodiment the alternate word includes a set of words (i.e., phase) to represent the word. For example, the word "five" could be replaced with the word "number five."
In some examples, choosing alternate words 55, re-training 60, and comparing the alternate word to the application vocabulary 45 may continue until a suitably different voice template is created for the word. When a suitable alternate word is found, the voice template for this alternate word is inserted into the application vocabulary for the word and the re-training ends.
[deleted]
In the specification and/or figures, typical embodiments of the invention have been disclosed. The present invention is not limited to such exemplary embodiments. The use of the term "and/or" includes any and all combinations of one or more of the associated listed items. The figures are schematic representations and so are not necessarily drawn to scale. Unless otherwise noted, specific terms have been used in a generic and descriptive sense and not for purposes of limitation.

Claims

A method for creating a voice template for a speech recognition system, the method comprising:
acquiring, using the speech recognition system, multiple samples of a spoken word from a user, the spoken word representing a vocabulary word from an application vocabulary stored in a memory;

creating a voice template for the spoken word from the multiple samples of the spoken word;

comparing the voice template for the spoken word to other voice templates for other words from the application vocabulary; and

if the voice template for the spoken word is similar to at least one of the other voice templates for the other words, then providing information to the user, wherein the information comprises:
(i) a prompt to create a new voice template for the spoken word, and

(ii) instructions for adjusting the spoken word to make the new voice template for the spoken word less similar to the other voice templates for other words, wherein the instructions for adjusting the spoken word comprise:
prompts to help the user to enunciate the spoken word differently, or

prompting the user to utter an alternate word to represent the spoken word, comprising presenting the user with a set of possible alternate words.
The method according to claim 1, wherein the information provided to the user is displayed on a screen.
The method according to claim 1, wherein comparing the voice template for the spoken word to the other voice templates from other words from the application vocabulary comprises computing a similarity score and comparing the similarity score to a threshold.
The method according to claim 1, wherein the other voice templates for other words comprise custom voice templates created for a specific user.
The method according to claim 1, wherein the other voice templates for other words comprise generic voice templates created for any user.