WO1999023626A1

WO1999023626A1 - Method for analysing a video image for a fixed camera

Info

Publication number: WO1999023626A1
Application number: PCT/FR1998/002303
Authority: WO
Inventors: Denis Le Goff; Laurent Hervot
Original assignee: Telediffusion De France
Priority date: 1997-11-03
Filing date: 1998-10-28
Publication date: 1999-05-14
Also published as: FR2770724A1

Abstract

The invention concerns a method for processing a current video image (1) which consists in producing first signals (5) associated with the image zones and representing the fixed and varying characteristics of the image at a given time and in producing a second signal representing the time interval during which the first signal associated with the zone indicates a fixed characteristic or a varying characteristic of the current image (1). The current image (1) is processed on the basis of the second signal value. The invention is particularly adapted to monitoring systems using fixed cameras.

Description

Video image analysis method for fixed camera.

The invention relates to a video image analysis method for a fixed camera. The invention applies to the field of remote monitoring.

Video surveillance is used in various fields of activity, particularly with regard to the surveillance of goods and people: surveillance of industrial sites, surveillance of public places, surveillance of road traffic.

The invention relates more particularly to surveillance of places in order to detect intrusions using a fixed camera. In the state of the art, it is known to use fixed or mobile cameras connected to a console of control screens which an operator permanently watches. In practice, operators do not have their attention fixed on the screens all the time. In addition, such a system requires having as many screens as the number of cameras.

One solution is to display on a reduced number of screens, sequentially, what each camera films. This solution makes it possible to reduce the number of screens and to make screen monitoring less monotonous in order to increase the vigilance of the person who monitors the screens.

An improvement consists in doubling each camera with a presence or event detector in order to automatically display the images of a camera whose detector detects movement on the image. The operator will then view the situation using a control screen. In order to remove the detectors associated with each camera, it is possible to use image processing to detect movements on the image with respect to a reference image. Conventionally, this is done for fixed cameras that continuously film the same place at a single angle. A reference image corresponding to the place to be monitored is digitized and each filmed image is also digitized and then compared with the reference image. The comparison of the filmed image with the reference image proceeds as follows:

- point by point differentiation of the filmed image and the reference image to obtain a difference image; - triggering of an alarm if the image contains moving elements, that is to say differences, and display of the image on a control screen so that the operator visually controls the place being monitored.

Such a system has some shortcomings. A first defect comes from the fact that the monitored place must be homogeneously lit which is very suitable for places lit with artificial light (for example a room without window) or indirectly (for example a corridor lit by reflection of light on a wall). On the other hand, if the place to be monitored is directly lit by sunlight or if it is an exterior, many false alarms can be triggered thus reducing the vigilance of the operator who will no longer control screens whose alarms are continuously triggered. A second problem also comes from false alarms which can be triggered by a change in the decor of an office (poster, plant, etc.). A third drawback comes from false alarms due to a reference threshold that is too low compared to the quality of the image (noise caused by a thunderstorm).

The invention proposes a solution which will considerably reduce this type of false alarms by using, on the one hand, a variation threshold and, on the other hand, one or more time thresholds to trigger an event if necessary. The subject of the invention is a method for processing a current video image in which first signals are produced associated with areas of the image and representative of the characters of fixity and variation of said areas at a given instant with respect to a reference image, for at least one zone, at least one second signal is produced representative of the duration during which the first signal associated with the zone indicates a fixity or a variation of the current image with respect to the reference image, and the current image is processed as a function of the value of the second signal.

The invention proposes to process the image by blocks with which indicators are associated in order to carry out a pattern recognition to produce different types of alarms or decision, the discrete transform being carried out on blocks of the image area, said blocks being reduced in size and identical to each other. An image state indicator table is used which includes at least one of the following indicators in the active or inactive state:

- a variability indicator representative of a block becoming active as soon as a first duration of variation of the image has been detected for this block, a variability saturation indicator representative of a block becoming active as soon as a second duration variation of the image was detected for this block,

- a chaining indicator representative of a block becoming active if at least one variability indicator is active for this block and for an adjacent block.

The invention will be better understood and other particularities and advantages will appear on reading the description which follows, the description referring to the ^' appended drawings among which: FIG. 1 represents a functional diagram of the invention, and FIG. 2 represents a table of indicators according to the invention.

A common video image 1 and a reference image 2 can be distinguished in FIG. 1. The current video image 1 and the reference image 2 are of identical sizes, for example 256 x 256 dots (or pixels). In the preferred example, said video and reference images 1 and 2 are divided into 1024 blocks of 8 x 8 points. Each point corresponds to a digital datum which corresponds for example to a gray level coded on an integer for example of 12 bits.

The reference image 2 is subtracted point by point from the current video image 1 in order to obtain a resulting image. The resulting image then undergoes a two-dimensional discrete transform performed block by block in order to obtain a transformed image 3 representative of the differences between the current video image 1 and the reference image 2. The transformed image

3 is performed block by block for two reasons which are on the one hand the limitation of the computing power and on the other hand a block processing of the image which will be discussed later. It is however possible to carry out a global transform of the entire image which will be assimilated to a single block. The transform carried out in the preferred example is a discrete cosine transform (hereinafter TCD). Other transforms can be used, among others the discrete spatial Fourier transform, however the TCD has the advantages of obtaining a number of coefficients identical to the number of points and allows direct extraction of the average brightness of the block being processed.

Each image block is represented by a matrix

4 of Cij coefficients obtained by applying the following known formula:

with f (k) - if k = 0 and f (k) = 1 if k ≠ 0

- in which N corresponds to the maximum number of rows and columns of points in a block, in which i and j are the indices of the coefficients of the block and are between zero and N - 1, and χ (nη_, n2) corresponds at the value of the coordinate point nη_, Ώ.2 in the block before transformation.

For each block, a signal representative 5 of the variation or fixity is extracted which corresponds to the spatial activity A _s of said block. The spatial activity A _s is defined as the sum of the squares of all the coefficients Cij of the matrix 4 resulting from the TCD, except the coefficient C00 which is representative of the average brightness of the block of the corresponding resulting image.

For each block, a comparator 6 will receive the representative signal 5 to compare it with a reference value 7 in order to produce a mobility bit in a first state if the representative signal 5 is greater in absolute value than the reference value 7 or in a second state if the representative signal 5 is less in absolute value than the reference value 7. The reference value 7 must be very low and corresponds to the detection sensitivity. One can for example fix this reference value at 0.1% of the maximum variation of the representative signal 5. The first state means that the block considered of the current video image 1 is different from the corresponding block of the reference image 2 and therefore that this block contains mobile elements. The second state means that the block in question of the current video image 1 is identical to the corresponding block of the reference image 2 and therefore that this block does not contain moving elements. For each block, first and second counters

9 and 10 carry out the integration over time of the information provided by the mobility bit respectively for the immobility and the movement of the image. Each of these counters 9 and 10 has a validation input EN, a reset / RST input, a clock input CLK, and a count output. The reset / RST input is of the synchronous type, it is activated when the input signal corresponds to the first state and it resets the counter 9 or

10 to zero when activated during an active edge of the clock signal. The validation input EN is activated when the input signal corresponds to the second state and it authorizes the increment of the counter 9 or 10 at each clock edge when this input is activated. The counting output provides data, for example on eight bits, which corresponds to the state of counter 9 or 10. When one of the counters 9 or 10 reaches its maximum value, it remains blocked. The first counter 9 receives on its validation input EN and on its reset / RST input the mobility bit. An inverter 11 also receives the mobility bit to supply the complement of this mobility bit on the validation EN and reset / RST inputs of the second counter 10. When the mobility bit is in the second state (when 1 ' image is fixed), the first counter 9 is set to zero and the second counter 10 is incremented by 1; the output of the first counter 9 displays a zero and the output of the second counter 10 provides a number representative of the duration during which the mobility bit has successively indicated the second state. When the mobility bit is in the first state (when the image is moving), the first counter 9 is incremented by 1 and the second counter 10 is set to zero; the output of the first counter 9 provides a number representative of the duration during which the mobility bit has successively indicated the first state and the output of the second counter 10 displays a zero.

A first comparator 12 having two inputs and an output receives on one of its inputs the number provided by the first counter 9 and on the other of its inputs a first threshold 13. The output of the first comparator 12 provides an indicator of immobility which is in a first state when the number at the output of the first counter 9 is less than the first threshold 13. The output of the first comparator 12 provides the indicator of immobility in a second state when the number at the output of the first counter 9 is greater than or equal to the first threshold 13. When the immobility indicator is in the second state, this means that the image block has been stationary for a certain duration greater than or equal to the value of the first threshold multiplied by the signal period of the first counter 9.

Second and third comparators 14 and 15 each having two inputs and one output receive on one of their inputs the number provided by the second counter 10 and on the other of their inputs respectively of the second and third thresholds 16 and 17. The output of second comparator 14 provides a variability indicator which is in a first state when the number at output of second counter 10 is less than the second threshold 16. The output of second comparator 14 provides a variability indicator which is in a second state when the number at the output of the second counter 10 is greater than or equal to the second threshold 16. When the variability indicator is in the second state, this means that the image block has been movable for a duration greater than or equal to the value of the second threshold 16 multiplied by the period of the clock signal of the second counter 10. The output of the third comparator 15 provides an indicator saturation of variability which is in a first state when the number at the output of the second counter 10 is less than the third threshold 17. The output of the third comparator 15 provides an indicator of variability saturation which is in a second state when the number at the output of the second counter 10 is greater than or equal to the third threshold 17. When the variability saturation indicator is in the second state, this means that the image block has been movable for a duration greater than or equal to the value of the third threshold 17 multiplied by the second counter 10 clock signal period.

To use the aforementioned indicators, an indicator table 18 is used, organized in a matrix as shown in FIG. 2. The indicators are arranged in groups 19 relating to each block in locations corresponding to the position of the block in the image. In FIG. 2, it can be seen that each group is composed of first to fourth bits 20 to 23. The first to third bits 20 to 22 correspond respectively to the indicators of immobility, saturation of variability, and variability defined previously, the "0" corresponding to the first state, and the "1" corresponding to the second state. The fourth bit 23 corresponds to a chaining indicator which is set to "1" when the variability bit of its group is at "1" and at least one variability bit of a neighboring group is also at "1". This is obtained by a program which, each time it modifies the variability indicators, checks for each variability indicator to "1" that, in at least one of the eight neighboring blocks at least one of the variability bits is also at " 1 ".

From the indicator table 18, it is possible to make different types of decisions. By way of nonlimiting example, it is considered that the duration corresponding to the first threshold 13 is five minutes, the duration corresponding to the second threshold 16 is half a second, and the duration corresponding to the third threshold is ten minutes. It is possible to have a system which take the following decisions: - an alarm signal occurs if a number of blocks associated with active chaining indicators is greater than a predetermined number or if blocks associated with indicators of Active chaining has a shape that. covers a predefined shape; - the block of this reference image is replaced in the reference image by the corresponding block of the current image if the variability saturation indicator associated with said block is active and, for security reasons, if an operator causes it; - the block of this reference image is replaced in the reference image by the corresponding block of the current image if the immobility indicator is active.

Of course many decisions are possible and can take into account the block positions associated with the indicators if the different parts of the image are to be treated differently. Those skilled in the art will also understand that it is possible to use only part of the indicators depending on the use that he wishes to make of the system. Such a system can be sensitive to noise from a video image. For this, the signal representative of the variation and of the fixity 5 can be calculated by taking into account only the low harmonics of the transform, in order to suppress the variation of the image due to noise. One removes for example the two lines and the two columns of the matrix 4 of coefficients which correspond to the highest spatial harmonics.

Those skilled in the art will better understand the various advantages of the invention with the aid of the examples which follow. Updating the reference image for one day.

During a day, the movement of the sun creates moving shadows in front of the camera lens and variations in light intensity. As explained above, variations in average light intensity on a block are not detected as being movements. Regularly, refreshing a block of the reference image, each time it is considered to be stationary for a predefined period, makes it possible to have a reference image closer to the real image.

The shadows move at a variable speed during the day. In addition, the speed also depends on the orientation of the surface on which the shadow is cast relative to the position of the sun. Shadow movements are either slow, linear, or both. Slow movements are filtered by thresholding and the corresponding blocks are regularly updated. Shadow movements at the shadow border having a linear structure, it follows that few blocks are chained before the variability saturation indicators are active. Modification of the decor. The surveillance of places frequented by the public can change scenery from one day to the next.

If the modification is minor, moving a pen on a desk or growing a plant, the blocks of the reference image concerned will be updated without the change being detected.

On the other hand, if the modification is significant (addition of a poster or a fairly large object), several blocks will have their variability saturation indicators which will be active simultaneously, it will be possible to substitute them in the reference image with possibly validation by an operator. Camera depointing.

The defocusing of a camera is detectable when a very high number of blocks distributed homogeneously on the image becomes mobile. The system can therefore notify the operator that the camera has moved. Having this timing information allows the operator to take action if it has not been told that the camera has been moved in a maintenance operation. Intrusion detection.

Intrusion detection can be done in different ways.

The simplest way is to determine the minimum number of blocks that are made mobile when a person is present in the field of the camera. During each image sampling, it suffices to count the number of chaining indicators and to trigger an alarm each time the number of active chaining indicators is greater than the minimum number.

Person detection can also be done in a more or less sensitive way depending on the position in the image. So if the camera points a long enough corridor

(for example between two buildings), it is possible to take into account the perspective by defining several minimum numbers on different areas of the image. The system takes the minimum number which corresponds to the area of the image in which the chained blocks are located to determine the presence of people. A finer way is to define minimal shapes. Each minimum shape must correspond to a minimum number of active blocks when a person is present in the camera field. Several blocks must be defined in order to correspond to several attitudes of a person. Then each time that chaining indicators are active, we compare the form constituted by the blocks whose chaining indicators are active with all the minimal forms. If a minimal form can be contained in the form constituted by the chained blocks then an alarm is triggered.

In the various examples cited above, it is always a question of having an operator. However for certain applications, the operator is not necessary. As an example, we can cite the case of a camera which points towards the entry airlock of a bank. The system will have the function of recording on magnetic tape automatically all the people who come in or go out during the day and stop the recording when the airlock is empty in order to reduce the number of video cassettes necessary to perform this task. . The system can update automatically for all the cases indicated above and trigger the recording as soon as a minimum number of chained blocks is detected. A variant consists in using only one area of the image. Indeed, it is possible that part of the image is constantly mobile and uninteresting. It suffices to define a useful area of the image on which the image processing will be carried out. If on the other hand one wishes to carry out a global processing of the image, it is necessary to modify the image resulting from the difference of the current video image and the reference image by making null all the points lying outside the useful area before performing the discrete image transformation. If an image divided into blocks is used, it is possible to carry out the transformation and the processing only on the blocks of the useful area of the image, or else to carry out the processing on all the blocks of the difference image in which all points outside the useful area are canceled. It is also possible to define several areas of interest on the image and to process them independently of each other.

Regarding the implementation of the system, the skilled person has many possibilities of implementation. The system can be entirely implemented in software or entirely implemented using dedicated electronic components. Preferably, one or more signal processors (more commonly known as DSP from the English Digital Signal Processor) will be used to perform the image subtraction, the cosine transforms of all the blocks and to calculate the spatial activity of each block. The comparators, counters, indicator table and decision-making will be programmed on the DSP or on a microcomputer. It is important to note that the microcomputer must process information in real time. In other words, at each sampling of current video image, the DSP (s) must carry out their calculations then transfer the spatial activities of each block to the microcomputer which will in turn have to carry out the comparisons, the counts and the decision-making before the sampling of the next video image. Note that a frame rate of four frames per second is more than enough for this kind of application and that it is even possible in some cases to decrease the frame rate to one frame per second.

Claims

1 - Method for processing a current video image (1) in which

first signals (5) associated with zones of the image and representative of characters of fixity and variation of said zones are produced at a given instant with respect to a reference image (2),

- for at least one zone, at least one second signal is produced representative of the duration during which the first signal associated with the zone indicates a fixity or a variation of the current image (1) relative to the reference image ( 2),

- And the current image (1) is processed as a function of the value of the second signal.

2 - Method according to claim 1, characterized in that: a two-dimensional discrete transform of image signals is carried out from a resulting image area corresponding to the difference between the current image and the reference image, and we take as the first signal (5) representative of the fixity or the variation of the current image the sum of coefficients

(Cij) of this discrete transform apart from the coefficient representative of the average brightness (C00) of the resulting image area, and - the fixity or the variation of the image is indicated by comparing this sum with a reference signal (7 ).

3 - Method according to claim 2 characterized in that

- the signal representative of the fixity or of the variation (5) of this resulting image area is taken the sum of the coefficients (Cij) representative of the low harmonics of this discrete transform.

4 - Method according to one of claims 1 to 4, characterized in that the value of the second signal is compared with at least one threshold (13, 16, or 17) and the current image (1) is processed as a function of this comparison with this threshold.

5 - Method according to one of claims 1 to 4, characterized in that in the reference image (2) the area of this reference image is replaced by the corresponding area of the current image (1) if the area processed from the current image is fixed for a duration greater than a first threshold (13).

6 - Method according to one of claims 1 to 5, characterized in that the discrete transform is a transform in discrete cosine.

7 - Method according to one of claims 1 to 6, characterized in that the discrete transform is performed on blocks of the image area, said blocks being of reduced size and identical to each other.

8 - Method according to one of claims 1 to 7, characterized in that a table (18) of image state indicators is created comprising at least one of the following indicators in the active or inactive state

- a variability indicator (22) representative of a block becoming active as soon as a first duration of variation of the image has been detected for this block, - a saturation indicator (21) of variability representative of a block becoming active as soon as a second image variation duration has been detected for this block,

- a chaining indicator (23) representative of a block becoming active if at least one variability indicator is active for this block and for an adjacent block.

9 - Method according to claim 8, characterized in that - an alarm signal is produced if a number of blocks associated with active chaining indicators (23) is greater than a predetermined number.

10 - Method according to one of claims 8 or 9, characterized in that an alarm signal is produced if blocks associated with chaining indicators (23) active have a shape which covers a predefined shape.

11 - Method according to one of claims 8 to 10, characterized in that

- the block of this reference image is replaced in the reference image (2) by the corresponding block of the current image (1) if the variability saturation indicator (21) associated with said block is active.