WO2006111911A2

WO2006111911A2 - Statistical study of reliability in a noisy environment with application to noise reduction

Info

Publication number: WO2006111911A2
Application number: PCT/IB2006/051169
Authority: WO
Inventors: Nalliah Raman; Calina Ciuhu
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2005-04-22
Filing date: 2006-04-13
Publication date: 2006-10-26
Also published as: WO2006111911A3

Abstract

A method and an apparatus for determining a pixel value for a central pixel in a window of pixels is disclosed. The reliability of pixel values in the window surrounding the central pixel are determined. The value of the central pixel is calculated using values of surrounding pixels which have been determined to be reliable.

Description

Statistical study of reliability in a noisy environment with application to noise reduction

This invention pertains in general to the field of signal processing. More particularly the invention relates to noise reduction in signal processing and more particularly to a reliability factor, which is used in the evaluation or estimation of a pixel in a windowed environment.

It is known that current filters used in signal and image processing utilise the neighbouring pixels/samples as a means of estimating/determining the central pixel's or sample's appropriate value (e.g. Sobel operator, median filters, low-pass filters, second moments of the luminance, cost functions such as the sum of absolute differences, etc). This method of estimation relies on the assumption that the neighbouring pixels or samples are reliable. However, in noisy environments, such assumptions do not normally hold and may lead to erroneous results.

The use of neighbouring pixels in estimating/calculating the central pixel's value may lead to erroneous results in the presence of noise. Smoothing filters such as the simple low-pass filter may yield excessive smoothing while gradient operators such as the Sobel operator may output large extreme values. This undesirable effect can be significantly reduced by ignoring neighbouring pixels, which are deemed to be unreliable. Similarly, the robustness of temporal/spatial filters used in motion estimation can be enhanced by eliminating these unreliable pixels from the decision process.

Thus, there is a need for a new method and apparatus for performing noise reduction during signal processing. It is the aim of this present invention to propose an improvement to the current approaches, being more advantageous. This is done by including a reliability iactor associated with the extreme values in the neighbouring pixels or samples.

Accordingly, the present invention preferably seeks to mitigate, alleviate or eliminate one or more of the above-identified deficiencies in the art and disadvantages singly or in any combination and solves at least the above mentioned problems by providing an apparatus, a method, and a computer-readable medium, that creates a reliability factor which is used in the evaluation and estimation of a pixel in a windowed environment according to the appended patent claims.

The general solution according to the invention is to determine the reliability of surrounding pixels and only use the pixels, which are deemed to be reliable when calculating the value of a central pixel.

According to aspects of the invention, a method, an apparatus, and a computer-readable medium for determining a pixel value for a central pixel in a window of pixels are disclosed, whereby only pixel values in the window, which are deemed to be reliable are used in determining the pixel value of the central pixel. According to one aspect of the invention, a method is provided for determining a pixel value for a central pixel in a window of pixels. The method comprises the steps of determining reliability of pixel values in the window surrounding the central pixel, and calculating value of the central pixel using values of surrounding pixels, which have been determined to be reliable. The step of determining reliability of values of pixels in the window iurther comprises the steps of ranking pixels within window using a rank-order filter according to their properties such as their intensity values, dividing pixels into lower rank N_10W and an upper rank N_upby specifying the rank order of the pixels within a range from 0 to N_10W and from W-N_up to W where W is the highest rank; calculating contract value C_low for each pixel in the lower ranks, calculating contrast value C_up for each pixel in the upper ranks, comparing contrast values C_low and C_up for each pixel to a local contrast threshold, discarding pixels that have contrast values C_low and C_up above the local contrast threshold, and determining which remaining pixels values are reliable.

According to another aspect of the invention, an image processing apparatus for determining a pixel value for a central pixel in a window of pixels is disclosed. The image processing apparatus comprises means for determining reliability of pixel values in the window surrounding the central pixel; means for calculating value of the central pixel using values of surrounding pixels which have been determined to be reliable, means for ranking pixels within window using a rank-order filter according to their properties such as their intensity values, means for dividing pixels into lower rank N_low and an upper rank N_upby specifying the rank order of the pixels within a range from 0 to N_low and from W-N_up to W where W is the highest rank, means for calculating contrast value C_low for each pixel in the lower ranks, means for calculating contrast value C_up for each pixel in the upper ranks, means for comparing contrast values C_low and C_up for each pixel to a local contrast threshold, means for discarding pixels that have contrast values C_low and C_up above the local contrast threshold, and means for determining which remaining pixels values are reliable.

According to a further aspect of the invention, a computer-readable medium having embodied thereon a computer program for processing by a computer is provided. The computer program comprises a first code segment for determining reliability of pixel values in the window surrounding the central pixel, a second code segment for calculating value of the central pixel using values of surrounding pixels which have been determined to be reliable, a third code segment for ranking pixels within window using a rank-order filter according to their properties such as their intensity values, a fourth code segment for dividing pixels into lower rank N_low and an upper rank N_up by specifying the rank order of the pixels within a range from 0 to N_low and from W-N_up to W where W is the highest rank, a fifth code segment for calculating contrast value C_low for each pixel in the lower ranks, a sixth code segment for calculating contrast value C_up for each pixel in the upper ranks, a seventh code segment for comparing contrast values C_low and C_up for each pixel to a local contrast threshold, an eighth code segment for discarding pixels that have contrast values C_low and C_up above the local contrast threshold, and a ninth code segment for determining which remaining pixels values are reliable.

The present invention has the advantage over the prior art that it evaluates the reliability of surrounding pixels (samples), which are used in estimating the central pixel (sample) value.

These and other aspects, features and advantages of which the invention is capable of will be apparent and elucidated from the following description of embodiments of the present invention, reference being made to the accompanying drawings, in which

Fig. 1 is a simplified block diagram of an image processing system to which the embodiments of the invention are applicable;

Fig. 2 illustrates a pixel window according to one embodiment of the invention; Fig. 3 is a chart plotting the reliability probabilities versus Pf_K according to one embodiment of the invention;

Fig. 4 is a flow chart illustrating an algorithm employing the reliability concept in noise removal using a rank-order filter according to their properties such as intensity values according to one embodiment of the invention; Figs. 5a-c illustrate images affected by impulse noise;

Figs. 6a-b illustrate images with improved impulsive noise reduction characteristics according to embodiments of the invention;

Fig. 7 illustrates a computer-readable medium according to one embodiment of the invention; and

Fig. 8 illustrates a processing system according to one embodiment of the invention.

The following description focuses on an embodiment of the present invention applicable to a detailed statistical model describing the reliability of the extreme values within the neighbouring pixels or samples in the presence of impulsive noise as well as an example in reducing it. However, it will be appreciated that the invention is not limited to this application but may be applied to many other calculations, which utilize neighbouring pixels or samples including for example noise-robust edge enhancement and motion estimation.

One embodiment of the invention focuses on impulsive noise in images (concept can be easily extended to Id signals or other types of noise such as multiplicative noise) as they are deemed to be one of the most challenging to remove without introducing significant blurring. This type of noise is known to plague imaging as well as in communication systems for example; the readout of data from a CCD sensor, and data transmission through cable and in freespace.

Fig. 1 illustrates an image-processing system 10 whereto the embodiments of the invention may be applicable. As shown in Fig. 1, the system 10 includes one or more video/image sources 12, one or more input/output devices 14, a processor 16, a memory 18, and a display device 20. The input/output devices 14, processor 16, and memory 18 may communicate over a communication medium 22, which may represent, i.e., a bus, a communication network, one or more internal connections of a circuit, circuit card or other device, as well as portions and combinations of these and other communication media. The memory 18 may represent, i.e., disk-based optical or magnetic storage units, electronic memories, as well as portions or combinations of these and other memory devices. Note that various functional operations associated with the image-processing system 10 may be implemented in whole or in part in one or more software programs/ signal processing routines stored in the memory 18 and executed by the processor 16. In other embodiments, however, hardware circuitry, such as an application-specific integrated circuit may be used in place or, or in combination with software instructions to implement the invention.

In addition, the explicit use of the term "processor" or "controller" should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor hardware, read-only memory for storing software, random access memory and non-volatile storage. Other hardware, conventional and/or custom, may also be included. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.

First, mathematical models and definitions of the various thresholds used in the statistical analysis will be described. The description of the algorithm used in a noise reduction application will be provided further below.

The impulsive noise model used in this analysis will now be described. Given an uncorrupted n-bit 2-dimensional signal which can be expressed as,

/(i,y) = t₁(i,y) * 2^ΪI-¹ + t₂(i,Λ * 2"-² + - + t_ϊl_i0^",Λ * 2 + *_>I(i,Λ (i) where k_m(ij)e {0,1 }, for all l≤m≤n and all ij, the noise corrupting the signal can be generalised such that each coefficient of the signal (for example

is corrupted with probability pu_t independently of all the coefficients. Consequently, the corrupted signal can be represented as,

1(UJ) = k\ (UJ) * 2^n~X + k\ (ij) * 2^n~2 + ^{■ ■ ■} + AV₁ (ij) * 2 + k\ (UJ) (2)

u _/ . ,- -_Λ f ^km (f>j) with probability l-p_bit 1 where, k'_m (ι,j) = \ (3)

[1 - K (h J) w¹* probability p_hit J

This formulation of the impulsive noise model is regarded to be more general and realistic than the commonly used 2-level "salt & pepper" noise model. Having described the impulsive noise model, let us now consider the estimation/evaluation of a central pixel's, x^c intensity value within a window as shown in Fig. 2.

To obtain a ranked-order set, all the pixels intensity values within the window are sorted in ascending order. Mathematically, this rank-order set can be expressed as,

X₁ ≤ x₂ < ^{• • •} < X^₁ ≤ x_w (4) where xι correspond to the minimum value and x_w to the maximum value in the window.

Briefly, the system 10 ranks the pixels within the window using a rank-order filter (according to their properties such as their intensity values in step 400. The pixels are then divided into two groups for testing, pixels X(i) to X(Niow) and pixels X(_w-Nup) to X(_wy To enable the system 10 to quantify the reliability of these probable extreme values (^₁) and X(_W)) within the window according to the algorithm illustrated in Fig. 4, a threshold is used. Two additional parameters, iV_low (number of lower ranks (e.g. iV_low = 2; ^₁), xp))) and N_up (number of higher ranks (e.g. N_up = 3; X(_W), X(_w-i), *(w-2)) ), are used to enable the system 10 to evaluate whether samples of the higher and lower ranks are reliable instead of just considering the highest and lowest rank samples. The rank order of the pixels is specified within a range from 0 to N_low and from W-N_up to W where W is the highest rank. These parameters can take any values between one to the size of the window, where the upper limit is constrained such that the sum of N_up and M_ow cannot exceed it. In this embodiment, the threshold used, pf_K, is based on the concept of modulation contrast and by definition lies between 0 and 1. The system 10 computes the contrast values in equation (5) in steps 401, 402 and compares the contrast values to the predefined modulation local contrast threshold, pf_K to classify the reliability of the outliers in steps 403, 404.

„ _ ^X(w-N_up) ^{~ X}m ^^{) +} ** ₍₅₎

x_w__Nu and x_m in equation (5) are the k and m ranked values of the rank-order set with k>m.

From initial investigations, it was found that the best estimate for the parameters iV_low and N_ap for impulsive noise is pw/2. This implies that k, typically lies in the range [w - [pw/2], w] and m between [l , [/m>/2]] . In computing (5), a high value of C_low or C_up would imply that there is a very strong likelihood that the extreme values are noisy samples. In this embodiment of the invention, the samples that have a contract value above the predetermined local contrast threshold are deemed to be unreliable in steps 405, 406 while the samples that have contrast values below the predetermined local contrast threshold are deemed to be reliable in steps 407, 408. It is also noted that the fixed modulation contrast threshold may not yield desired results in textured regions (due to the large variance in intensity values) and as such, the threshold may be refined to be adaptive to its region. This adaptive modulation contrast threshold is called the rejection ratio p_adP and it is defined as,

P^ *^ ^" (6)

X(w-N_up) ^{+ X}(N_low+l) Similar to the fixed modulation contrast threshold, the system 10 may evaluate the expressions in equation (5) and compare it with p_aclp to classify whether the extreme values are reliable.

The statistical properties of the reliability of extreme values within a window will be outlined according to the algorithm illustrated in Fig. 4. This is performed to guide a person in understanding how the various parameters affect the statistical reliability of the outliers in the sampled window. The statistical equations are developed here using the fixed modulation contrast threshold for simplicity. In this context, the probabilistic equations developed here may be viewed as the worst-case scenario. In this embodiment, a window of size w will now be considered. The probability that X_k is an unreliable outlier in this window given that there are N_ap probable samples that are unreliable is given by, p(x_k is unreliablel N_up)

)

→ />( Vs unreliable/N up ) > = t > -) (7)

';„<?*» ',"p (ox - X Y™^'"""^'"^ - (OX - X - n^<-^w-"^ι°"-^N"^p)

(x - x -i^Λ⁾

(8)

where N_up is a predefined number of higher order ranked samples, L is the number of grey levels, w is the window size, and x_{(N +1)} and x_(w__{N )} are the lower and upper boundaries of the probable reliable samples in a system with L intensity levels. Now, the number of

can take any value between x\ up to x_(w__{N )} and as such, all these possibilities have to be summed up.

A similar expression holds for the probability that the lower ranked samples are unreliable extreme values. Based on the above equations, the reliability probabilities versus Pf_Kis plotted in Fig. 3. The asymmetry exhibited in the reliability plots of the lower and higher order ranked samples for small L values is due to the asymmetry in the range of ^_(ΛΓ^_+I) a^d X_(W-_NU > values (0 to L instead of -L to L) chosen. It may be observed that this asymmetric behaviour becomes less pronounced with increasing L because the number of samples within the range of *_{(# +}i₎ ^d ^X _(W-_{NU )} values increased, tending towards a continuous set. It may also be observed from the plot that as Pa_x increases, the lower and higher order ranked samples become more reliable. This plot also suggests that having a fixed modulation contrast threshold may not be ideal in regions with a high intensity variance (texture) as the reliability probability has a monotonic relationship with respect to Pa_x. From this information, the system 10 identifies which samples are reliable.

In the preceding sections, the concept of reliability was introduced and its statistical properties were described with regard to samples within a window. An application of this concept in noise reduction will now be described. A rank-order filter has been used in the investigations, as they are known to perform well with regard to impulsive noise reduction as compared to linear filters. The following experiments use a value of 0.3 for p^n, 25 for w and 2 for JV_low and N_ap. Although an intuitive argument earlier on the improved performance of the adaptive over the fixed modulation contrast threshold was provided, Fig. 6 shows experimental evidence of the improvement.

Figs. 5 a, b and c show an image of 'Lena' affected by impulsive noise and the flagging of the unreliable extreme values using pf_K and p_adP in colour respectively. A comparison of Fig. 5b and 4c shows that the adaptive modulation contrast threshold performs better in detecting unreliable extreme values in the image as indicated in the circled regions. The Fig.s also show that with pf_K, pixels near the edges are more than often erroneously flagged as unreliable. This implies that extreme values are more likely to be unreliable in a smooth area (p_adP small) than in a highly textured area (p_adP large).

Having established that the adaptive modulation contrast threshold provides a better performance, we have used it with respect to reducing the impulsive noise affecting the images shown in Figs. 5, using the algorithm outlined in Fig. 4. Fig. 6 clearly shows the improved impulsive noise reduction characteristics of the filter with reliability consideration. Although, the potential of using reliability concept has been demonstrated with respect to impulsive noise reduction, the reliability concept may also be extended to any form of computation utilizing neighboring pixels or samples. For example, the reliability concept may be used in noise-robust edge enhancements and motion estimation. Another application is in motion estimation algorithms, where typically one has to evaluate a cost function (or Sum of Absolute Differences) over a group of pixels. The computation of this cost function for the motion vectors will be more robust if calculated with only reliable pixels. The reliability concept may also be used in classifiers.

Although it has been shown that the reliability concept can removed impulsive noise effectively, other types of noise can also be tackled effectively. In additive noise, a similar approach as the one outlined in the Fig. 4 can be used with JV_low and N_up modified accordingly. For multiplicative noise, the image may be "linearize" through a simple logarithmic transform, and perform the filtering as in the additive noise case (see equations below): y =x\ (x)x ln(j;) = ln(η (X)) + ln(x) ⁽⁹⁾ where y is the sampled image, x is the original image without noise and η(x) is the noise affecting the image.

In another embodiment of the invention according to Fig. 7, a computer readable medium is illustrated schematically. A computer-readable medium 100 has embodied thereon a computer program 110 for determining a pixel value for a central pixel in a window of pixels for processing by a computer 113. The computer program comprises a code segment 114 for determining reliability of pixel values in the window surrounding the central pixel, a code segment 115 for calculating value of the central pixel using values of surrounding pixels which have been determined to be reliable, a code segment 116 for ranking pixels within window using a rank-order filter according to their properties such as their intensity values, a code segment 117 for dividing pixels into lower rank N_low and an upper rank N_upby specifying the rank order of the pixels within a range from 0 to N_low and from W-N_up to W where W is the highest rank, a code segment 118 for calculating contrast value C_10W for each pixel in the lower ranks, a code segment 119 for calculating contrast value C_up for each pixel in the upper ranks, a code segment 120 for comparing contrast values C_10W and C_up for each pixel to a local contrast threshold, a code segment 121 for discarding pixels that have contrast values C_low and C_up above the local contrast threshold, and a code segment 122 for determining which remaining pixels values are reliable.

Fig. 8 illustrates an exemplary processing system 200 according to a further embodiment of the present invention. According to the embodiment, the processing system 200 comprises means 202 for determining reliability of pixel values in the window surrounding the central pixel, means 204 for calculating value of the central pixel using values of surrounding pixels which have been determined to be reliable, means 206 for ranking pixels within window using a rank-order filter according to their properties such as their intensity values, means 208 for dividing pixels into lower rank N_low and an upper rank N_Upby specifying the rank order of the pixels within a range from 0 to N_low and from W-N_up to W where W is the highest rank, means 210 for calculating contrast value C_low for each pixel in the lower ranks, means 212 for calculating contrast value C_up for each pixel in the upper ranks, means 214 for comparing contrast values C_low and C_up for each pixel to a local contrast threshold, means 216 for discarding pixels that have contrast values C_low and C_up above the local contrast threshold, and means 218 for determining which remaining pixels values are reliable. Said means 202-218 are preferably electronic components operatively connected to each other in a suitable way. Other components of the processing system are not illustrated or discussed in detail.

The invention can be implemented in any suitable form including hardware, software, firmware or any combination of these. However, preferably, the invention is implemented as computer software running on one or more data processors and/or digital signal processors. The elements and components of an embodiment of the invention may be physically, functionally and logically implemented in any suitable way. Indeed, the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the invention may be implemented in a single unit, or may be physically and functionally distributed between different units and processors.

Although the present invention has been described above with reference to specific embodiments, it is not intended to be limited to the specific form set forth herein. Rather, the invention is limited only by the accompanying claims and, other embodiments than the specific above are equally possible within the scope of these appended claims.

In the claims, the term "comprises/comprising" does not exclude the presence of other elements or steps. Furthermore, although individually listed, a plurality of means, elements or method steps may be implemented by e.g. a single unit or processor.

Additionally, although individual features may be included in different claims, these may possibly advantageously be combined, and the inclusion in different claims does not imply that a combination of features is not feasible and/or advantageous. In addition, singular references do not exclude a plurality. The terms "a", "an", "first", "second" etc do not preclude a plurality. Reference signs in the claims are provided merely as a clarifying example and shall not be construed as limiting the scope of the claims in any way.

Claims

CLAIMS:

1. A method for determining a pixel value for a central pixel in a window of pixels, comprising: determining reliability of pixel values in the window surrounding the central pixel; and - calculating value of the central pixel using values of surrounding pixels which have been determined to be reliable.

2. The method according to claim 1, wherein said determining reliability of values of pixels in the window, comprises: - ranking pixels within window according to at least one property of the pixels; dividing pixels into lower rank N_low and an upper rank N_up; calculating contrast value C_low for each pixel in the lower ranks; calculating contrast value C_up for each pixel in the upper ranks; comparing contrast values C_low and C_up for each pixel to a local contrast threshold; discarding pixels that have contrast values C_low and C_up above the local contrast threshold; and determining which remaining pixels values are reliable.

3. The method according to claim 2, wherein number of lower rank and upper rank pixels is fixed.

4. The method according to claim 2, wherein number of lower rank and upper rank pixels varies depending on the size of the window.

5. The method according to claim 2, wherein number of lower rank and upper rank pixels is determined based on noise statistics.

6. The method according to claim 2, wherein the local contrast threshold is fixed.

7. The method according to claim 2, wherein the local contrast threshold is dependent on the local contrast of the window (p_adp) according to

, where N_up and iV_low are the number of higher and lower rank samples of the rank-order set of the pixels within the window.

8. The method according to claim 2, wherein the pixels are ranked according to each pixel's intensity value.

9. The method according to claim 2, wherein a rank order filter is used to rank the pixels.

10. The method according to claim 2, wherein the pixels are divided by specifying the rank order of the pixels within a range from 0 to N_low and from W-N_up to W where W is the highest rank.

11. An image processing apparatus (200) for determining a pixel value for a central pixel in a window of pixels, comprising: - means for determining (202) reliability of pixel values in the window surrounding the central pixel; means for calculating (204) value of the central pixel using values of surrounding pixels which have been determined to be reliable.

12. The image processing apparatus (200) according to claim 12, further comprising: means for ranking pixels within the window according to at least one property of the pixels; means for dividing pixels into lower rank N_low and an upper rank N_up; - means for calculating contrast value C_low for each pixel in the lower ranks; means for calculating contrast value C_up for each pixel in the upper ranks; means for comparing contrast values C_low and C_up for each pixel to a local contrast threshold; means for discarding pixels that have contrast values C_low and C_up above the local contrast threshold; and means for determining which remaining pixels values are reliable.

13. A computer-readable medium ( 100) having embodied thereon a computer program (110) for processing by a processing device (113), the computer program comprising code segments for determining a pixel value for a central pixel in a window of pixels, comprising: a first code segment (114) for determining reliability of pixel values in the window surrounding the central pixel; a second code segment (115) for calculating value of the central pixel using values of surrounding pixels which have been determined to be reliable.

14. The computer-readable medium according to claim 14, further comprising: - a third code segment (116) for ranking pixels within the window according to at least one property of the pixels; a fourth code segment (117) for dividing pixels into lower rank N_low and an upper rank N_up; a fifth code segment (118) for calculating contrast value C_low for each pixel in the lower ranks; a sixth code segment (119) for calculating contrast value C_up for each pixel in the upper ranks; a seventh code segment for comparing contrast values C_low and C_up for each pixel to a local contrast threshold; - an eighth code segment (120) for discarding pixels that have contrast values

C₁₀W and C_up above the local contrast threshold; and a ninth code segment (121) for determining which remaining pixels values are reliable.