Block boundary artifact reduction for block-based image compression

(54) BLOCK BOUNDARY ARTIFACT

REDUCTION FOR BLOCK-BASED IMAGE
COMPRESSION

(75) Inventor: Larry Alan Westerman, Portland, OR (US)

(73) Assignee: Sharp Laboratories of America, Inc.,

Camas, WA (US)

( * ) Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.C. 154(b) by 0 days.

(21) Appl. No.: 09/241,829

(22) Filed: Feb. 1, 1999

(51) Int. C I. G06K 9/40

(52) U.S. CI 382/275; 382/268

(58) Field of Search 382/254, 268,

382/275, 232, 233, 294; 358/426, 432,

433

(56) References Cited

U.S. PATENT DOCUMENTS

4,941,043 A 7/1990 Jass 358/133

4,975,969 A 12/1990 Tal 382/2

4,991,223 A 2/1991 Bradley 382/17

5,454,051 A 9/1995 Smith 382/233

5,475,434 A 12/1995 Kim 348/420

5,495,538 A 2/1996 Fan 382/233

5,521,841 A 5/1996 Arman et al 364/514 A

5,563,662 A 10/1996 Kishi 348/420

Block boundary artifact reduction in decompressed digital images is accomplished by filtering the intensity of pixels in the vicinity of the block boundary. The filter utilizes filter coefficients selected from tables on the basis of the distribution of a scalar quantity describing the pixels neighboring the pixel to which the intensity adjustment is to be applied. The intensities of pixels at the boundary and one pixel removed from the boundary are adjusted. If interpolation pixels in each of neighboring blocks are of relatively constant intensity, the intensities of additional pixels, more remote from the boundary, are adjusted. Filter coefficients can be selected from different arrays for pixels on the boundary or removed from the boundary or if the pixel intensity adjustment is based on the intensities of pixels in a horizontal row or vertical column of interpolation pixels.

8 Claims, 2 Drawing Sheets

1

BLOCK BOUNDARY ARTIFACT
REDUCTION FOR BLOCK-BASED IMAGE
COMPRESSION

BACKGROUND OF THE INVENTION 5

The present invention relates to digital image reproduction and more particularly to a technique for reducing block boundary artifacts in digital images which have been subjected to block based image compression and decompres- 10 sion.

In various applications, including high definition television and video conferencing systems, it is desirable to store and transmit images in a digital form. The amount of data generated by converting images from analog to digital form ^ is so great that digital transmission and storage of images would be impractical if the data could not be compressed to require less bandwidth and data storage capacity. As a result, a number of techniques have been developed to compress the digital data representing an image and a video sequence. 2o

Several image and video compression techniques process digital image data by first dividing the image into 2-dimensional blocks. A common block size for block-based compression is an area eight pixels by eight pixels in size. Images and video may be processed in 8x8 blocks by 25 algorithms specified by the ISO "JPEG" specification for still images, the CCITT "H.261" recommendation for video conferencing, and the ISO "MPEG" specification for video compression. After the image is divided into blocks, compression of the data proceeds on a block-by-block basis. 30 There are several methods for processing the intrablock data and more than one method may be used. A common compression technique includes a transformation step where a transform, such as a discrete cosine transform (DCT), is applied to the intrablock image data to generate transform 35 coefficients for each block of picture elements or pixels. The transformation is often followed by a quantization step where the transformed intrablock image data is replaced by quantized data which usually requires fewer data bits. The quantized data may be processed further and then stored or 40 transmitted as compressed image data. When the compressed image data is received, the receiver reverses the process, decompressing the data and reconstructing the image.

The image compression methods with the highest com- 45 pression efficiency are so called "lossy" methods. An image often contains considerable information, such as small detail, that contributes very little to a real observer's appreciation of the reconstructed image. When transformed this information appears as intermediate or higher frequency 50 components which can be discarded following transformation. In addition, some degradation of the image may be acceptable in certain circumstances as a trade-off to reduce the amount of data which must be transmitted or stored. Discarding or losing "excess" information during the quan- 55 tization of the transformed data is the core of the data compression effectiveness of lossy compression methods. On the other hand, much of the information discarded in block-by-block compression is data relevant to interblock relationships. Processing image data on a block-by-block 60 basis and discarding information related to interblock relationships often results in a noticeable change in pixel intensity across the boundary between blocks of the reconstructed image. The "blocky" appearance of a decompressed image has been consistently recognized as a problem when 65 using lossy compression methods for digital image processing.

2

Several strategies have been developed to reduce the block boundary artifacts in a decompressed digital image. For example, the H.261 compression—decompression technique incorporates an optional low pass filter that can be applied to all pixels of the image except those at the edge of the image. Applying the filter to all pixels of the image is time consuming and reduces the apparent image resolution. Another technique is the projection on convex sets (POCS) method which attempts to estimate the characteristics of the pre-compression image from the compressed data together with a model of the statistical characteristics of the image before compression. This method is successful in producing high quality decompressed images and eliminating block boundary artifacts. However, the computational complexity of the method makes it impractical for real time video applications.

Another strategy to reduce block boundary artifacts involves retaining some of the interblock correlation data when the image is compressed. Lossy compression commonly involves a transformation step where spatial domain image data is converted to a frequency domain representation by application of a mathematical transform, such as a discrete cosine transform (DCT). The DCT coefficients have been used to determine whether to apply a filter to the image data and, if so, to determine which filter to apply. The interblock correlation data can either be embedded in the transform coefficients or transmitted separately. However, the DCT coefficients which are not incorporated in the compressed data must be transmitted and stored separately from the image data reducing the efficiency of the data compression. Further, integration of this additional data in existing standards, such as the JPEG standard, is difficult or impossible.

Komuro et al., U.S. Pat. No. 5,675,666, disclose a technique for preprocessing image data before compression to compensate for the "block effect" artifact that would normally appear upon decompression of the image data. Error correction data is developed by a test compression and decompression of the image followed by computation of the error correction data from the differences between the input image data and the data for an output test image. The error correction data is added to the input image data so that when the image is compressed again and decompressed, subsequently, adjustments to reduce block boundary artifacts will have been made in the image data for pixels at the block boundaries. Preprocessing can be effective in reducing the boundary artifacts but requires access to the image content before compression.

Images may originate from many sources which do not have access to preprocessing capability. As a result, techniques of postprocessing the image to reduce block boundary artifacts after decompression in order to restore or enhance image quality have been proposed. One postprocessing technique of boundary artifact reduction is disclosed by Kim, U.S. Pat. No. 5,694,492. In this technique the compressed image data is received, decompressed and stored. A target pixel is identified and its data is filtered repeatedly until the absolute difference between the filtered and unfiltered data exceeds a predefined threshold. The process is repeated for each pixel in the image. This technique relies on repetitive filtering of data with a low pass filter having fixed coefficients. The iterative nature of the process is time consuming and computationally expensive. In addition, the fixed filter coefficients do not respond to local signal variations in the image resulting in less than optimal treatment of some discontinuities.

Chan et al, in a paper entitled A PRACTICAL POSTPROCESSING TECHNIQUE FOR REAL-TIME BLOCK