DE102011121110A1 - Method for operating voice dialog system in vehicle, involves determining system status of voice dialog system, assigning color code to determined system status, and visualizing system status visualized in color according to color code - Google Patents

Abstract

The method involves determining a system status of voice dialog system (1), where the system status is associated to a voice dialog. A color code is assigned to the determined system status. The system status is visualized in color according to the color code. The system status is directly visualized in a region in a windscreen and/or in an interior region of a vehicle. The system status is determined based on a phase of the voice dialog and/or recognition degree of a detected speech input in comparison with a stored language command. An independent claim is also included for a voice dialog system in a vehicle.

Description

The present invention relates to a method for operating a speech dialogue system in a vehicle, in which a system state of the speech dialog system belonging to a speech dialog is determined. The present invention further relates to an associated speech dialogue system and a vehicle having such a speech dialogue system.

When operating systems in a vehicle must be taken into account that the operator action must often be carried out by the driver himself, without distracting him from the road. The availability of a speech dialogue system therefore offers the possibility, in particular of the driver of a vehicle, to operate various functional devices in the vehicle intuitively via voice inputs without manual interaction. The driver is guided by means of a voice dialogue to the operating functions to be executed. The voice control can alternatively or in addition to other input options, eg. As a manual input, are offered so that the driver can select the most comfortable input option for him depending on the situation. This then contributes to the improvement of road safety.

In speech dialogue systems, the user's speech inputs are typically compared to database entries stored in a database. The relevance of each database entry to the captured speech input is then calculated by an algorithm and a recognition level, e.g. B. in the form of a confidence measure that quantifies the relevance of a potentially suitable for voice input database entry. If the search result is unique, the corresponding database entry is output or an associated function is executed on its basis. If the search result is not unique, then either a hit list or a system-side query is typically output. In this case, user input and system output form a dialog-like scheme, in particular if a system output responding to a user input requests another user input.

In order to make a language dialogue as effective as possible, in particular in order to keep the number of system-related queries within reasonable limits, visual support of the speech dialogue has proven to be advantageous. In the DE 100 08 226 C2 For example, a voice control for a device, in particular in a vehicle, is described, in which various pictorial indications are displayed on a display, depending on the state of the device and the voice outputs of the device.

For effective dialogue, it is also important that the user remains informed about the state of his "conversation partner", ie the speech dialogue system. The DE 103 38 512 A1 describes a support method for speech dialogues in a vehicle, in which the state of the speech dialogue system is communicated to the driver during a speech dialogue. In this case, the user is signaled via an acoustically differentiable signal such phases in which the speech dialogue system can capture inputs, processes inputs or outputs its own expenditure.

It is the object of the present invention to provide a method for operating a speech dialogue system and an associated speech dialogue system of the type mentioned, which is intuitive to use and by means of which the fastest possible and goal-oriented operation is possible.

This object is achieved by a method having the features of claim 1 and by a speech dialogue system having the features of claim 8. Advantageous training and further developments emerge from the dependent claims.

The method according to the invention is characterized in that a color code is assigned to the determined system state and the system state is color-visualized according to the color code. The system state visualized by means of a color code can be detected simply and reliably without any longer focusing.

A color code is defined within the meaning of the invention by one or more colors. The colors are generated in particular by correspondingly colored light sources, for. B. by illuminated displays or LEDs. In this case, temporal patterns of the color code can be included, for. B. alternately flashing in two different colors light source.

The visualization of the system state is advantageously carried out in an area in the windshield and / or in an immediately adjacent inner area of the vehicle. This area has the smallest distance of the driver's eyes to the windshield in the direction of the relevant road traffic and is in the field of vision or at least in the peripheral field of view of the driver. In this case, the driver needs to perform no head movement during a normal driving situation. In particular, the color visualization in this area can quickly perceive.

According to one embodiment of the method according to the invention, the system state is determined as a function of the phase of the speech dialogue and / or as a function of the degree of recognition of a recorded speech input in comparison with a stored speech command. In this way, the user can be informed on the one hand, the phase of the general speech dialogue, on the other hand, but also references to the chances of success of the speech dialogue specifically carried out. The general system states include z. B. information about whether the speech dialogue system is even turned on or active, and if so, whether just the system side a voice input expected or processed or a voice output is prepared or output. The degree of recognition of a detected speech input includes, for example, the quantitative evaluation by means of a confidence measure with which probability a particular voice command was recognized.

In one embodiment of the method according to the invention, it is provided that two distinguishable color sets are defined. The first color set is used to visualize different phases of the speech dialogue and the second color set is used for the degree of recognition of a detected speech input as compared to a stored speech command. As a result, the user can be informed of the risk of confusion by not only the risk of confusion, but also of the rough course of the dialogue as well as of the particular progress made with regard to a voice input that has just been made.

In such a case, relatively unobtrusive colors are suitable for the first color set because the general system condition in a specific situation need not even be relevant to the driver of the vehicle. As a minimum requirement, it only has to be ensured that the driver can quickly grasp the system state if he wants to actively know this state. For example, color codes based on white and blue light are generated.

According to an advantageous embodiment of the method according to the invention, it is provided that a voice input is detected and compared with stored voice commands and two different limit values are determined for the recognition level, which quantify whether the voice input can be assigned to a specific stored voice command. The visualization then takes place in a first color, in particular green, if the recognition degree is greater than both limit values, the visualization takes place in a second color, in particular red, if the recognition level is less than both limit values, and the visualization takes place in a third color, especially yellow, if the degree of recognition is greater than or equal to the smaller limit but less than or equal to the larger limit. The appearance in three different colors, especially the traffic light colors, have the property that they attract the attention of the driver. In the case of an ambiguous voice input or erroneous input of the driver this is thus increasingly pointed out whether his voice input has been processed correctly. The use of the traffic light colors as the color code for the degree of recognition can preferably be combined with an aforementioned color code for the general system states.

According to the invention, recorded speech inputs are alternatively or additionally displayed as text. In the voice input while a key word relevant for a stored voice command is detected and the recognized keyword is highlighted. The visualization takes place in particular in the aforementioned area of the windshield and / or the adjacent area. As tests have shown, smaller quantities of text in the peripheral field of vision can still be detected relatively quickly. As a result, the user is provided with a tool with which he can view the system recognized as text and check for accuracy. By highlighting the recognized keywords in his speech input, the user also has the opportunity to learn the relevant keywords during the operation of the speech dialogue system.

Highlighted keywords are visually differentiated in one embodiment of the invention depending on the degree of recognition. The representation size, representation sharpness and / or the degree of transparency are changed, for example, as a function of a quantifiable measure of the degree of recognition. This makes it easier for users to perceive keywords more easily the safer they are recognized.

The speech dialogue system according to the invention in a vehicle comprises means for detecting speech inputs for a speech dialogue. The means for detecting speech inputs comprise in particular a microphone known per se with associated signal processing functions, eg. B. voice activity detection and noise reduction. Furthermore, means for outputting speech output, e.g. As a speaker can be provided. The speech dialogue system according to the invention further comprises a control unit, which are connected to the means for detecting speech inputs, for determining the system state of the speech dialogue system associated with the speech dialogue and visualization means connected to the control unit for visualizing the system state. The speech dialogue system according to the invention is characterized in that a color code can be assigned to the determined system state by means of the control unit and the visualization means are activatable such that the system state can be visualized in color according to the color code. The speech dialogue system according to the invention is suitable for carrying out the method according to the invention. It thus also has the advantages of the method according to the invention.

In one embodiment of the speech dialogue system, the visualization means are arranged in an area in the windshield and / or in an inner region of the vehicle immediately adjacent thereto, in particular in a region of the windshield of the windshield and / or in the A-pillar of the vehicle. Here they can be easily integrated into the cockpit design without major design dependencies and do not adjust the driver's view to the front.

The area in which the system state is visualized in color preferably has such an extent that it always lies at least in its peripheral field of vision even when the driver's variable angle of view. The visualization means therefore preferably comprise a light bar, the z. B. is illuminated on the basis of LEDs. This requires on the one hand little space, but on the other hand can be arranged along the window sash of the windshield so that it is visible for each angle forward.

Alternatively or additionally, a head-up display in the windshield can be used for visualization. In addition, the visualization of the system states can be displayed on other displays in the cockpit. B. in a multifunction display or combi display in the dashboard of the cockpit.

According to the invention, a vehicle is equipped with such a speech dialogue system.

The invention will now be explained in more detail by means of embodiments with reference to the figures.

1 shows schematically the structure of a speech dialogue system according to an embodiment of the invention,

2 shows the cockpit of a vehicle in which a speech dialogue system according to an embodiment of the invention is arranged,

3 shows a flowchart of a voice dialogue according to an embodiment of the invention and

4 shows a text output on a pane root display, as produced according to an embodiment of the invention.

In the 1 schematically is the structure of a speech dialogue system 1 illustrated according to an embodiment of the invention. The speech dialogue system 1 includes a microphone 4 to receive voice input. The microphone 4 is connected to a software module for voice activity detection and with an adaptive filter for noise reduction and can situation specific, z. B. be activated by means of a so-called push-to-talk button (not shown). For system-side voice output, a speaker can be optional 5 be provided.

The microphone 4 and the speaker 5 are with a control unit 6 connected. By means of the control unit 6 If necessary, the speech dialogue system dependent state of the speech dialog system can be determined and detected speech inputs are compared with stored voice commands, as with reference to the 3 and 4 will be explained in more detail. By means of the control unit 6 In particular, a color code can be assigned to each system state.

The control unit 6 is over the data bus 7 of the vehicle 15 with a functional device 8th of the vehicle 15 connected via a voice dialogue of the speech dialogue system 1 is operable. The to the functional system 8th associated voice commands are either in a data store of the functional system 8th to which the control unit 6 Has access, or in the control unit 6 saved yourself. The voice commands can also be composed of various commands and attributes, eg. B. "Calculate route to Brunswick". In the functional device 8th is it z. B. to a navigation system, an infotainment device or the air conditioning system of the vehicle 15 ,

The control unit 6 is furthermore with an LED strip 2 (LED = Light Emitting Diode) connected by the control unit 6 is controllable such that the system states are color visualized according to the color code. The LED strip 2 includes a variety of LEDs that are suitable for additive color mixing of the required colors. The control unit 6 is alternatively or additionally with a lens root display 3 connected on which the from the control unit 6 recognized voice inputs can be displayed as text.

The LED strip 2 and the lens root display 3 are at the bottom of the windshield 9 , the so-called disk root 10 , arranged ( 2 ). The LED strip 2 can Alternatively, in the driver side and possibly also the passenger side A-pillar 11 be arranged. According to an embodiment variant, several LED strips 2 provided, or the LED bar 2 extends between the A-pillars 11 over the entire lens root 10 , The microphone 4 is integrated in a suitable place in the cockpit, z. B. in the headliner near the head positions of the driver or passenger (not shown).

The inventive method will now be described with reference to embodiments with reference to the 3 and 4 explained in more detail. For carrying out the method, in particular that with reference to the 1 and 2 described speech dialogue system 1 be used. In the following it is assumed that the user of the speech dialogue system 1 the driver of the vehicle 1 is. Likewise, the user could be another vehicle occupant, especially the passenger.

In the 3 FIG. 3 is a flowchart of a voice dialog according to an embodiment of the invention. The speech dialog begins with a system activation 20 , There will be more system states 21 - 26 of the speech dialogue system 1 which are determined depending on the phases of the speech dialogue and on the system-side degree of recognition of the detected speech input. In this case, according to the invention in the control unit 6 the system state concerned 20 - 26 determined. The different system states 20 - 26 in each case a color code is assigned and the system status 20 - 26 will be via the LED bar according to the assigned color code 2 and if necessary via the lens root display 3 in the disk root 10 visualized in color.

After activation in phase 21 the LED strip is illuminated with white light so that the driver, without looking away, realizes that he is operating the functional system 8th can now operate by voice. In the phase 21 but no voice signals are identified yet.

When the driver starts talking, they are in phase 22 by means of the speech recognizer in the control unit 6 Voice signals are identified and recorded. The driver will see this in the LED bar 2 This signals that one or more adjacent LEDs of the LED strip 2 light up blue. In the process, different LEDs are activated over time, giving the visual impression that a blue dot or bar is moving slowly from left to right across the LED strip 2 running. The running speed of the bar is chosen so that the driver is signaled the remaining talk time for voice input.

A recorded voice input is in phase 23 processed and compared with stored voice commands. During speech processing, the blue dot or bar will appear on the LED bar 2 stopped. In order to signal the driver the system-related break in the speech dialogue, a suitable light pattern can be selected, for. B. a rhythmically extending and shortening blue bar.

The previously recorded voice input will be in phase 24 a text output on the lens root display 3 generated.

The input of the speech input can be done for a whole sentence or word by word, so that the phases 22 - 24 if necessary, repeated until a sentence or the voice input is finished.

After completion of the speech input and the associated language processing takes place in the phase 25 a system-side speech output, in which a function execution is announced or a consultation takes place, in which the speech dialogue system 1 into the phase 21 returns. During speech output, the LED bar flashes 2 blue and white to visually alert the driver to the voice output.

In the phase 26 the function execution terminates the speech dialogue in question. During the beginning function execution will be on the LED bar 2 creates a guy light pattern, e.g. B. a fast left to right running white light bar. The executed function is optionally available as text output on the lens root display 3 displayed.

The mentioned color visualization of the system states is optionally also available via the window root display 3 generated.

According to one embodiment, during the phase 23 The recognition level determines whether or to what extent a stored voice command can be assigned to the voice input. The degree of recognition is calculated using a confidence measure, which quantifies the agreement of a voice input with a stored voice command via specified criteria.

For the degree of recognition, two different confidence values are determined and thus differentiated into three categories of the degree of recognition. The confidence values are calculated, for example, in a manner known per se as a percentage, as is known to search engines when searching for a specific search query. The visualization for the first category takes place in green when the Detection level is greater than 95% and no further hit candidate was found with such a high degree of recognition. The visualization for the second category is yellow if the recognition level is greater than or equal to 50% but less than or equal to 95%. The visualization for the third category is in red if the recognition level is less than 50%.

In the first category, the voice input can be clearly assigned to a voice command. In the second category, the voice input can not be unambiguously assigned to a voice command, but a particular voice command or attribute is most likely to appear on the system side. For this second category, the function execution proposal can be issued on the system side based on this identified voice command, which is then only to be confirmed by the driver. In the third category, the voice input is so unclear that either a large number or no voice command can be assigned to this voice input. For this third category, the speech dialogue system requests a renewed speech input. The in text output in phase 24 displayed text is highlighted according to the assigned colors. In addition, it is provided in this embodiment that in phase 25 the voice output the LED bar 2 Depending on the detected detection level, green, yellow or red lights up.

Related to the 4 a further embodiment of the method according to the invention is shown, in which a text output belonging to the speech input on the screen root display 3 is issued. It may be in particular a text output in the phase 24 with reference to the 3 act described dialogue process.

The voice input is recorded word by word and on the screen root display 3 as text 24A - 24F displayed. Each word in the speech input is checked to see if it matches or is at least related to a keyword for a stored voice command. If this is the case, the recognized keyword is highlighted. In the case shown are the words captured 24A . 24D has been clearly recognized as a keyword for a navigation function, namely the computation of a navigation route. The words 24A . 24D will be displayed in bold letters. The word 24F was recognized as a possible attribute for the navigation function. Due to the background noise, it could not be clearly identified, but only with a confidence level below 95%. As a result, the potential keyword is displayed large but with a lower contrast or transparency level to give the driver a quick overview of the word 24F may not have been recorded correctly.

LIST OF REFERENCE NUMBERS

1

Voice control system

2

LED strip

3

Disk root display

4

microphone

5

speaker

6

control unit

7

bus

8th

functional unit

9

Windshield

10

Windshield windshield

11

A column

15

vehicle

20

system activation

21

receptiveness

22

voice recording

23

language processing

24

text output

24A-24F

Texts

25

speech

26

function execution

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 10008226 C2 [0004]
• DE 10338512 A1 [0005]

Claims (10)

Method for operating a speech dialogue system ( 1 ) in a vehicle ( 15 ), in which a system state belonging to a speech dialogue ( 20 - 26 ) of the speech dialogue system ( 1 ), characterized in that - the determined system state ( 20 - 26 ) a color code is assigned and - the system state ( 20 - 26 ) is visualized in color according to the color code. Method according to claim 1, characterized in that the system state ( 20 - 26 ) in an area in the windshield ( 9 ) and / or in an immediately adjacent interior area ( 10 ; 11 ) of the vehicle ( 15 ) is visualized. Method according to claim 1 or 2, characterized in that the determination of the system state ( 20 - 26 ) depending on the phase ( 20 - 26 ) of the speech dialogue and / or depending on the degree of recognition of a recorded speech input ( 23 - 25 ) in comparison with a stored voice command. A method according to claim 3, characterized in that - two distinguishable color sets are defined, - the first color set for visualizing different phases ( 20 - 26 ) of the speech dialogue is used and - the second color set for the degree of recognition of a recorded speech input ( 23 - 25 ) is used in comparison with a stored voice command. A method according to claim 3 or 4, characterized in that - a voice input is detected and compared with stored voice commands ( 22 . 23 ), - two different limit values are determined for the recognition level, which quantify whether the speech input can be assigned to a specific stored speech command, and - the visualization in a first color, in particular green, if the recognition degree is greater than both limit values, the visualization in a second color, in particular red, if the degree of recognition is less than both limits, and the visualization is a third color, in particular yellow, if the degree of recognition is greater than or equal to the smaller limit but less than or equal to the larger limit. Method according to one of the preceding claims, characterized in that - a recorded speech input as text ( 24A - 24F ) is displayed, - in the voice input a keyword relevant to a stored voice command is recognized and - the recognized keyword is highlighted ( 24A . 24D . 24F ). A method according to claim 6, characterized in that a highlighted displayed keyword is visually differentiated depending on the degree of recognition ( 24A . 24D ; 24F ). Speech dialogue system ( 1 ) for a vehicle ( 15 ), comprising - means ( 4 ) for detecting voice input for a voice dialogue, - a control unit ( 6 ), which with the means ( 4 ) for detecting speech input, for determining the system state associated with the speech dialogue ( 20 - 26 ) of the speech dialogue system ( 1 ) and - with the control unit ( 6 ) associated visualization means ( 2 ; 3 ) for visualizing the system state ( 20 - 26 ), characterized in that - by means of the control unit ( 6 ) the determined system state ( 20 - 26 ) a color code can be assigned and - the visualization means ( 2 ; 3 ) are controllable such that the system state ( 20 - 26 ) is color-visualizable according to the color code. Speech dialogue system ( 1 ) according to claim 8, characterized in that the visualization means ( 2 ; 3 ) in an area in the windshield ( 9 ) and / or in an immediately adjacent interior area ( 10 ; 11 ) of the vehicle ( 15 ), preferably in a region of the lens root ( 10 ) of the windscreen ( 9 ) and / or in the A-pillar ( 11 ) of the vehicle ( 15 ) are arranged. Speech dialogue system ( 1 ) according to claim 8 or 9, characterized in that the visualization means ( 2 ) comprise a light bar.

Images