WO2008107832A1

WO2008107832A1 - Stress estimation

Info

Publication number: WO2008107832A1
Application number: PCT/IB2008/050757
Authority: WO
Inventors: Ralph Kurt; Frank Wartena; Ronaldus M. Aarts; Petronella H. Pelgrim; Joanne H. D. M. Westerink; Wilhelmus J. J. Stut
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2007-03-07
Filing date: 2008-03-03
Publication date: 2008-09-12

Abstract

A system is described for estimating a level of stress or tiredness that a person is experiencing. The system comprises measurement means (10) for performing a plurality of eye movement measurements of the person during a time interval and accumulating means (20) for accumulating the measurements and determining a spectral distribution thereof. It further comprises analyzing means (30) for estimating the level of stress based on the spectral distribution.

Description

Stress estimation

BACKGROUND OF THE INVENTION

Technical field

The present invention relates to a system, method and computer program for estimating a condition of a person, in particular a level of stress or tiredness that a person is experiencing.

DESCRIPTION OF RELATED ART

Many medical experts agree that stress management is an important factor in disease prevention and recovery. For this reason, many developments are going on in the field of detection and prevention of stress. Especially devices for stress detection/monitoring, for stress prevention, devices to treat stressed people, but also integrated solutions such as embedded in a PC have become available, recently. These devices use measured vital body signs of a person to calculate a measure of the person's stress.

In US 7,071,831 B2 an alertness monitoring system is disclosed, that uses reflectance measurement of infrared light pulses to monitor eye movement of a vehicle or machine operator. The data is used to measure eye and eyelid movement for determining the presence of slow or drifting eye movements, and/or eyelid closure and optionally one or more of, absence of saccadic movement, loss of eye co-ordination, wavering eyelids, partial eye closure, and drooping eyelids as indicators of loss of attention and drowsiness. The eye movements of a subject are analyzed using reflected infra red light to obtain measures of the blink amplitude velocity ratio and the saccade amplitude velocity ratio. The eye movements are recorded over a period of some seconds. The average values of the eye movement measurements over this period are determined and are continuously compared with eye movement data indicative of degrees of alertness which correlate to blood alcohol content values so that the monitor provides an indication of the operator's legal fitness to operate the vehicle or machine from the point of view of drowsiness no matter how it is caused. The system is focused on the alertness of the operator, which is a property that can change very rapidly and requires therefore intrinsically a short measurement (typically within seconds). It is an object of the invention to provide a system and method for determining a person's condition over an extended period of time.

SUMMARY OF THE INVENTION This and other objects of the invention are achieved by a system according to claim 1, a method according to claim 13 and a computer program according to claim 14. Favorable embodiments are defined by the dependent claims 2-12 and 15-16.

According to an aspect of the invention a system is provided for estimating a condition of a person, in particular a level of stress or tiredness that a person is experiencing. The system comprises measurement means for performing a plurality of eye movement measurements of the person during a time interval and accumulating means for accumulating the measurements and determining a spectral distribution thereof. It further comprises analyzing means for estimating the condition based on the spectral distribution. In this way, a good estimation can be made of a person's condition, in particular the "accumulated" stress of a person over an extended period of time.

The invention is based on the insight that extreme situations such as a staring view, very concentrated or fixed viewpoint, very monotonous eye movements (with a fixed speed or frequency) as experienced when driving along a straight road or in a mass production line, or very fast eye movements as required e.g. in a race car, when playing a computer game or working in a stock exchange office are perceived as stressful and they are experienced as extremely tiring when performed for an elongated/extended period of time. In all these extreme situations, the spectral distribution of fast, slow and medium speed eye movements is not balanced. On the other hand situations in which fast, slow and medium speed eye movements are in a good balance over time are perceived as relaxed and less tiring.

According to an embodiment the accumulating means are adapted for classifying the measurements in a class or interval in order to obtain the spectral distribution. This is an easy way of determining the spectral distribution without the need of any complicated algorithms or calculations. According to an embodiment, at least 100 but preferably at least 500 or even

5000 eye movement measurements are used to estimate the level of stress of a user. Thereto the eye movements are measured during a prolonged period of at least several minutes or even hours. According to an embodiment the analyzing means are adapted for categorizing the level of stress as high or normal. In this way a clear and easily understandable feedback about a person's stress level can be given. In case of an even spectral distribution of the measurements the level of stress is categorized as normal. In case of an uneven or unbalanced distribution of the measurements or in case of peaks in the distribution of the measurements the level of stress is categorized as high.

According to a further embodiment the system comprises output means for generating a signal depending on the estimated level of stress. In case of a high stress level these output means may send a warning, such as an acoustic signal to the user or a signal forcing the user make a break for relaxation exercises.

According to a still further embodiment the measurement means are adapted to measure the eye movement, by measuring at least one of the following parameters: speed of eye movements, frequency of eye movements, amplitude of eye movements, direction of eye movement, focus distance (measured by eye curvature) and pupil size variation (reacts on different brightness/ color levels). These parameters are very suitable for estimating the stress of the user.

The measuring means for measuring the eye movements may comprise an optical sensing system, such as a camera mounted on a PC in an office environment or in a car. They also may comprise suitable sensing means, such as optical, mechanical or electrical sensors integrated in specially designed goggles or glasses.

According to a still further embodiment the system comprises a movement sensor for sensing the movement of the head of the person. It is advantageous to take the movement of the head into account, allowing the eye movement detection to be more robust.

According to a still further embodiment the measurement means comprise multiple eye movement trackers and are adapted to combine the measurements of these trackers. As a result of combining the measurements the system becomes more robust.

According to a further aspect of the invention a method is provided for estimating a condition of a person, in particular a level of stress or tiredness that a person is experiencing comprising the following steps: - performing a plurality of eye movement measurements of the person during a time interval, accumulating the measurements and determining a spectral distribution thereof, and estimating the condition based on the spectral distribution. The method may be implemented by means of a suitable computer program.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be better understood and its numerous objects and advantages will become more apparent to those skilled in the art by reference to the following drawings, in conjunction with the accompanying specification, in which:

Figure 1 shows a block diagram of the system according to the present invention,

Figure 2 shows a block diagram of the measurement means, and

Figure 3 shows a diagram of the distribution of the values of the accumulated measurements in typical situations.

Throughout the figures like reference numerals refer to like elements.

DETAILED DESCRIPTION OF THE PRESENT INVENTION

Figure 1 shows a non-obtrusive eye movement detection system that allows to deduce/calculate/estimate the level of stress that an analyzed person is experiencing based on the analysis of accumulated measurement signals. It comprises measurement means 10 for measuring one or more parameters indicative of eye movements of the person during a certain time interval, accumulating means 20 for accumulating the measurements an determining a spectral distribution thereof, analyzing means 30 for estimating the stress level of the person based on the spectral distribution and output means 40 for generating a signal depending on the estimated level of stress. Optionally, the system comprises further measurement means 50 for measuring the movements of the head of the person.

The measurement means 10 are adapted for detecting the eye movement pattern of a person. Thereto, the measurement means may comprise suitable optical sensing means such as a camera. Using a camera for eye movement detection is well known in the art and is feasible, low cost and reliable. At the present there basically exist two types of eye movement detection systems on the market, using a camera: outside-in systems and inside- out systems. In outside-in systems one or more cameras record the eye of the participant and determine eye-movement parameters through imaging algorithms. The cameras are positioned in front of the participant. One of the advantages of these systems is given by the fact that the camera can be integrated into a monitor, and therefore is basically invisible to an analyzed person. In inside-out systems the person has to wear a special device on the head. The image of the eye is led into a mini-camera by using mirrors. This mini-camera records the eye and the actual line of vision is found out through imaging algorithms.

Alternatively, or additionally, the measurement means 10 may comprise other suitable sensing means such as mechanical or electrical sensors. The use of such sensors for eye movement detection is well known in the art. The sensors may be mounted near the eye. They may be for example included in smart electronic contact lenses or specially designed goggles or glasses. Parameter such as the eye movement speed, frequency of movements, amplitude (x and y direction) of the movements, direction of eye movement, focus distance (measured by eye curvature), pupil size variation (reacts on different brightness/ color levels) and others can be detected and used for estimating the level of stress.

Figure 2 shows a block diagram of measurement means 10 comprising IR sensors. Using IR sensors for eye movement tracking is well known in the art. The measurement means 10 comprise an IR pulse generator 12, generating IR- light in the direction of the eye and IR sensors or detectors 13,14 detecting the IR light reflected by the eye. Detectors 13,14 detect reflected light in different directions (e.g. horizontally and vertically) across the eye. This reflected light corresponds to the movement of the eye ball. From the movement of the eye ball, the speed of the eye movement may be derived, according to algorithms well known in the art. The measurement signals are transferred to the accumulating means 20 by means of wireless transmitter 16. Preferably, the reflected light of both eyes is measured (not shown). For a more detailed description of eye movement measurement by using the reflection of IR-signals from the eye reference is made to the earlier cited patent US 7,071,831 B2.

The accumulating means 20 and the analyzing means 30 may be implemented by a suitably programmed processor and a memory. The accumulating means 20 accumulate the measurements over a prolonged time period of several minutes or even hours.

The measurements may exist of a plurality of measurements of a certain parameter representative of the eye movement, each of them being averaged over a certain time interval. By way of example, the total time period of the measurements may be 1 hour. This time period is divided in a plurality of sub-periods of for example 2 seconds each, resulting in a total of 1800 sub-periods. The parameter is measured during each sub-period and the mean value thereof is determined. The 1800 measurements thus obtained are each classified in a suitable interval or class. In this way, a spectral distribution (histogram) of the measurements is created. Of course different values for the total time interval and the sub-periods than the ones given herein above may be selected, depending on the circumstances. The number of sub periods preferably is at least 100 but it may also be much higher for instance at least 5000. The analyzing means 30 perform an analysis of the spectral distribution in order to estimate the stress of a person.

Figure 3 shows the spectral distribution of the values of the accumulated measurements in typical situations. The diagram shows the accumulated intensity I, or in other words the occurrence rate of the values of the frequency f of eye movements over a certain measurement period. A first typical spectrum is indicated as "a" and is a typical spectrum of a person that is steering or is very concentrated. The spectrum includes a high accumulated intensity of low frequencies of eye movements. This means that low frequencies of eye movements occurred much more often during the time period of the measurements than high frequencies of eye movements The spectrum "b" is a typical spectrum of a person during a relatively monotonous (eye) activity such as reading or working in an assembly line production. In this case the measurement values are more evenly distributed but very high and very low frequency values are absent. The spectrum "c" is a typical spectrum of a person doing an activity for which very fast eye movements are required as e.g. during car racing or computer games. The spectrum includes a high accumulated intensity of high frequencies of eye movements. The spectrum "d" is a typical spectrum of a person performing normal

(office) activity. In this case the measurement values are evenly distributed, including very high and very low frequency values. The latter we consider a relaxed pattern. It is characterized by a relatively broad spectrum and a big FWHM (full width at half-maximum) as indicated by the arrow. In contrast the spectra "a", "b" and "c" have a much shorter FWHM (see corresponding arrows) and sharper peaks, indicating a stress situation if performed for a longer time.

The system here described is based on the insight that all extreme situations including staring view, very concentrated or fixed viewpoint, very monotonous eye movements (with a fixed speed or frequency) as experienced when driving along a straight road or in a mass production lines, and very fast eye movements as required e.g. in a race car, when playing a computer game or working in a stock exchange office are perceived as stressful and they are experienced as extremely tiring when performed for an elongated/extended period of time. On the other hand situations in which fast, slow and medium speed eye movements are in a good balance in time are perceived as relaxed and less tiring.

In order to estimate the stress level of a person the analyzing means 30 determine certain characteristics of this distribution, which are representative of the stress level. Such characteristics are for example the full width at half-maximum of the distribution, the number, the height, the width, and slope of its peaks, the absence of certain values, its median, the kurtosis, etc. A relatively broad and balanced spectrum is considered to be normal. In this case the accumulated stress level is categorized as normal. However, if the system detects high, sharp peak(s) or an unbalance in the spectrum, meaning that fast, slow and medium speed eye movements are not in a good balance in time, the accumulated stress level is categorized as high. In this case, the output means 40 can send a warning to the user, such as an acoustic signal or in some cases it could force you to make a break for relaxation exercises similar as know from RSI.

In case of digital content (on a PC monitor and the like) the system according to the invention could be linked to the monitor and display the presented information in a way forcing counterbalancing of the eye movement (e.g. by altering speed of scrolling, movement on screen, dataflow, light level, font size etc.)

Figure 3 shows how the distribution of the frequency of eye movements may be used to estimate the level of accumulated stress of a person. However, also the distribution of other parameters related to eye movements may be used, such as the eye movement speed, the amplitude of the eye movements in horizontal and vertical direction, the direction of eye movement, the focus distance (measured by eye curvature) or the pupil size variation (reacts on different brightness/ color levels).

The system according to the invention could according to a preferred embodiment also perform smart background or baseline subtraction (from time to time). The spectrum could be fit by a lower order polynomial function and subtracted, i.e. this becomes the new zero line. After that subtraction all positive values (counts) in the classes (said predefined intervals) remain, whereas negative counts are set to zero. From here clustered accumulation continues. One could in this way easier detect peaks and also detect, when a certain monotonous behavior occurs that falls in a class that was underrepresented in the previous time period.

The spectral distribution shown in figure 3 may be obtained by measuring the (mean) frequency of the eye movements for each sub-period and classifying the values thus obtained as explained herein above. However, the spectral distribution of the frequency of movements may also be obtained by measuring the eye movement over the whole time interval in the time domain and converting this measurement to the frequency domain by a Fast Fourier Transform. In this way, the spectral distribution shown in figure 3 is obtained without classification of measurement values of sub periods. The system may use multiple eye movement trackers, for example a stationary camera and a sensor on the head and combine the signals generated by these trackers to achieve a more robust eye movement detection. The combination of both signals can be performed with a so-called Kalman filter. This is very suitable for parameter estimation and parameter tracking and is able to remove artifacts. If there is no known relation between both trackers, a look up table can be implemented that has a specific entry for each possible value of the outputs of the reference devices (trackers).

For parameter estimation, a Kalman filter, an extended Kalman filter, or a particle filter can be used. These filters are adaptive and are very suitable for parameter estimation and parameter tracking, they are well-known and widely used in various domains, like robotics, physics, chemistry and engineering, see IEEE Signal Processing Magazine 20(5), Sept. 2003, pp.19-38 for an overview.

Because these filters are adaptive, artifacts can be removed even in systems with a non-stabile relation between the tracking parameters. The described system could also be used as learning tool, to analyze your day or the last hour helping you to improve e.g. your time management. Thereto the system could be linked to a user's agenda of this day.

Another interesting extension of the system is the use of a movement sensor 50, for example an accelerometer on the head of the user. This movement sensor 50 may be integrated into the user's glasses or goggles. This allows the detection of head movements. Taking these head movements into account makes the eye movement detection more robust. This input can also be used with the Kalman filter, which is mentioned above.

As will be recognized by those skilled in the art, the innovative concepts described in the present application can be modified and varied over a wide range of applications.

Accordingly, the scope of patented subject matter should not be limited to any of the specific exemplary teachings discussed, but is instead defined by the following claims.

Any reference signs in the claims shall not be construed as limiting the scope thereof.

Claims

CLAIMS:

1. System for estimating a condition of a person, in particular a level of stress or tiredness that a person is experiencing comprising: measurement means (10) for performing a plurality of eye movement measurements of the person during a time interval, - accumulating means (20) for accumulating the measurements and determining a spectral distribution thereof, and analyzing means (30) for estimating the condition based on the spectral distribution.

2. System according to claim 1, wherein the accumulating means (20) are adapted for classifying the measurements in a class or interval in order to obtain the spectral distribution.

3. System according to claim 1, wherein the number of eye movement measurements is at least 100.

4. System according to claim 1, wherein the condition is the level of stress and the analyzing means (30) are adapted for categorizing the level of stress as high or normal.

5. System according to claim 4 wherein in case of an even spectral distribution the analyzing means (30) are adapted to categorize the level of stress as normal.

6. System according to claim 4 wherein in case of unbalanced spectral distribution or in case of peaks in the spectral distribution the analyzing means (30) are adapted to categorize the level of stress as high.

7. System according to claim 1, wherein the condition is the level of stress and the system further comprising output means (40) for generating a signal, preferably an acoustic signal, depending on the estimated level of stress.

8. System according to claim 1 wherein the measurement means are adapted to measure the eye movement, by measuring at least one of the following parameters: speed of eye movements, - frequency of eye movements, and amplitude of eye movements, direction of eye movement focus distance (measured by eye curvature) pupil size variation (reacts on different brightness/ color levels).

9. System according to claim 1 wherein the measurement means (10) comprise an optical sensing system, such as a camera.

10. System according to claim 1 wherein the measurement means (10) comprise one or more sensors (13,14) in the person's glasses or goggles.

11. System according to claim 1 comprising a movement sensor (50) for sensing movements of the head of the person.

12. System according to claim 1 wherein the measurement means (10) comprise multiple eye movement trackers and are adapted to combine the measurements of these trackers.

13. Method for estimating a condition of a person, in particular a level of stress or tiredness that a person is experiencing comprising the following steps: performing a plurality of eye movement measurements of the person during a time interval, accumulating the measurements and determining a spectral distribution thereof, and - estimating the condition based on the spectral distribution.

14. A computer program comprising computer program code means adapted to perform the following steps, when said program is run on a computer: accumulating a plurality of eye movements measurements of a person made during a time interval and determining a spectral distribution thereof, and estimating a condition of the person, such as the level of stress, based on the spectral distribution.

15. A computer program as claimed in claim 14 embodied on a computer readable medium.

16. A carrier medium carrying the computer program of claim 14.