EP0428624A1

EP0428624A1 - Intelligent scan image processor

Info

Publication number: EP0428624A1
Application number: EP89910797A
Authority: EP
Inventors: Ran Ginosar; Oliver Hilsenrath; Yehoshua Y. Zeevi
Original assignee: iSight Inc
Current assignee: ISIGHT, INC.
Priority date: 1988-08-02
Filing date: 1989-08-02
Publication date: 1991-05-29
Also published as: EP0428624A4; WO1990001750A1

Abstract

L'invention concerne un réseau de triage destiné à traiter des pixels avoisinants dans chacune des zones d'un ensemble de zones d'intérêt. Le réseau comprend un appareil produisant des valeurs maximum, minimum et intermédiaires du voisinage (24), ainsi qu'un appareil permettant la transformation morphologique des valeurs (26) de pixels du voisinage.The invention relates to a sorting network intended to process neighboring pixels in each of the zones of a set of zones of interest. The network includes an apparatus producing maximum, minimum and intermediate values of the neighborhood (24), as well as an apparatus allowing the morphological transformation of the values (26) of pixels of the neighborhood.

Description

INTELLIGENT SCAN IMAGE PROCESSOR

FIELD OF THE INVENTION

The field of the invention relates to the field of image processing generally end to the field cf image processing Very Large scale Integration (VLSI) architecture in particular, BACKGROUND OF THE INVENTION

Conventional image processing systems acquire images from 'th image acquisition unit, such as a camera, which producer each image in a rastar-scan format, i.e. the pixels of au image are transmitted to the image processing units beginning from the top left corner and proceeding across each line until reaching the bottom right corner. The video signal thus produced is typically forwarded to a pipeline of preprocessors . typically consisting of linear neighborhood processors such as convolvers, non-linear neighborhood processors and other processors , such as histogram transformers or Fast Fourier Transformerr . In addition the preprocessors typically generate their output in & raster-mean format so that one preprocessor can directly feed into the next one.

A novel non-raster scanning method, modelled on the human visual system, has been developed which selectively acquires and processor the data pertaining to specific areas of interest in the image, as defined according ts predetermined enteric. The non-raster scanning method it disclosed in applicants contnding unpublished israel Patent Application 7948 common owned by the common owners of the present scanning method. An ares of interest is the image in identified through processing on a low resolution version of the image, typically utilising one-sixty fourth of the pixels in the image, and the random access image sensor is instructed to acquirs the remaining pixels in the area of interest at higher resolution levels . The random access image sensor is disclosed in applicant' s copending unpublished Isrmal Patent Application 83213 , which is incorporated herein by reference.

U.S. Patent 4,601 ,055 discloses a low-level image processor with many internal stages. An input stream of pixels typically comprises an entirety of an image although regions of interest can be defined externally using 1 bitsmap.

U.S. Patent 4,541,116 discloses a neighborhood transform stage for implementing filter operations. The value of each pixel but the central one in 6 square of nine pixels is compared with the central pixel. This operation uses tight comparators and the result is used to form a vector to address the appropriate element in a programmable lookup table . The value of the central pixel is maintained or transformed to that at one of the other pixels in the square depending on the value of the appropriate element in the programmable lookup table. The neighborhood trans form stage is typically used for filtering.

U.S. Patent 4,718,101 discloses segmentation apparatus suitable for recognising a pattern in an image , for labelling each pattern , for detecting the area at each label , for calculating top or vertex coordinates and for determining nesting relationships between patterns.

U.S. Patent 4 ,417 , 685 discloses image analyzing apparatus which employs techniques of integral geometry to classify patterns in 2- and 3-dimansional raster-scanned images where 3-dimensional images refer to multi -valued images and 2-dimensional images refer to binary-valued images . For 3-dimensional images, the apparatus selects the maximum value associated with each 5 x 3 neighborhood in the image. The values of the 3 x 3 neighborhood are typically modified by neighborhood contribution circuitry.

K. Batcher discloses , in an article entitled , "Sorting Networks and Their Applications , " , published in the Proceedings of t he 1968 Spring Joint Computer Conference , AFIPS, Washington, S.C. , a sorting network for an amount of elements which is a power of 2.

P. A, Ruetr and H.W. Brodarsen, in an article entitled, "Architecture and Design Techniques for Real Time Image Processing IC ' t , " published in the IEEE Journal of Solid Stats Circuits, vol . 8022 , number 2 , of April 1987, describe a sorting network which is capable of s electing one of the maximum, the minimum or a value close to the median of 9 elements . SUMMARY OF THE INVENTION

The present invention seeks to provide an image processing architecture for processing images acquired with via non-raster scanning means .

There is thus provided , in accordance with the present invention, an apparatus for processing images acquired as a plurality of areas of interest end comprising apparatus for defining a neighborhood for each pixel in each of the plurality of areas of interest and image processing apparatus which uses a neighborhood generated by the apparatus for defining a neighborhood.

Additionally, there is provided in accordance with the present invention , an apparatus for processing images which comprises apparatus for defining a neighborhood of each incoming pixel and apparatus for image processing including apparatus for producing the maximum, minimum and median values of the neighborhood.

Additionally, in accordance with an embodiment of the present invention, the apparatus for defining a neighborhood includes apparatus for padding the : neighborhood of incoming pixels defined as belonging to the border of one of the areas of interest.

Further, in accordance with an embodiment of the present inventions the apparatus for defining a neighborhood produces a portion of the neighborhood of a pixel and the apparatus for image processing reconstructs the complete neighborhood from the portion of the neighborhood and from its own storage apparatus.

The portion of the neighborhood is the North, Center and South pixels of a neighborhood of on incoming pixel.

Still further, in accordance with an embodiment of the present invention , the apparatus for producing comprises seven three element mortars .

Moreover, in accordance with an embodiment of the present invention, the image processing apparatus includes a morphological operator. The morphological operator may determine the maximum, medium and minimum values of the pixel values of the neighborhood. It additionally may perform in extended neighborhood transfers which can be erosion or dilation n other suitable transform.

Additionally , in accordance with an embodiment of the present invention, the areas of interest may be square or non-square shapes.

There is provided in accordance with an embodiment of the present invention, a sorting network for producing the maximum median and minimum values of nine input elements . The network comprises seven three element sorters. BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be understood and appreciated more fully from the following detailed description taken in conjunction with the drawings in which:

Fig. 1 is a block diagram illustration of a system which incoporates the area of interest image processor of the present invention.

Fig. 2 is a block diagram illustration of the basic unite of the area of interest image processor of the present invention,

Figs . 3A and 3B are pictorial illustrations of two scan methodologies.

Fig. 4A is a pictorial illustration of an image with two areas of interest;

Fig. 4B is an illustration of the interfacing signals resulting from the image of Fig. 4A.

Fig. 5 is a block diagram illustration of a neighborhood generator useful in the area of interest image

Fig. 6 is an illustration of a finite state machine logic useful in a data output server of the neighborhood generator of Fig. 5.

Fig. 7 is a pictorial illustration of the operation of & processing unit of the area of interest image processor of Fig. 2.

Fig. 8 is a block diagram illustration of a processing unit useful for the area of interest image processor of Fig. 2.

Figs . 9A - 90 are block diagram illustrations of elements of a sorter useful in the processing unit of Fig. 8.

Fig. 10 is a block diagram illustration of an extended neighborhood transfort block useful in the processing unit of

Fig. 8. and

Fig. 11 is a simpli fied block diagr am of a neighborhood generator useful in the area of interest processor of Fig. 2.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT

Reference is now made to Fig. 1 which illustrates an image processing system utilizing an area of interest image acquisition method. The system comprises an image content responsive image acquisition unit 10, such as a photodiode array camera with a random access image ; sensor 11 as described in the previously mentioned Isrm al Patent Application 83213, to acquire analog pixel intensities of a current area cf interest in the image, where the current area of interest is typically defined by a system user. An A/D converter 12 converts the analog pixel intensities to digital values and a plurality of Area of Interest Image processors (AllP) 14 , such as convolvers or morphological and other non-linear operators , process the data acquired by the image acquisition unit 10. The system additionally comprises an image content responsive scan controller 16 to analyze the data, select a new area of interest and output the relevant coordinates of the new area of interest to the image acquisition unit 10 which than acquires the analog pixel intensity values of the new area of interest, The coordinate information, denoted "control" in Fig. 1, ia additionally passed from the image acquisition unit 10 to the first AIIP 14. Algorithmic blocks 18 and 20 respectively, receive the processed data free the plurality of AIIPs 14 and Additionally process the date, typically creating a new definition of the area of interest.

Fig. 2 illustrates the major elements and the input and output signals of an AIIP 14. Each AIIP 14 preferably comprises a neighborhood generator 22 and a processing unit 25 typically comprising a neighborhood operator 24 and an extended neighborhood transform block 26. The neighborhood generator 22 receives as input a signal P_in which contains the digital pixel intensities of the entirety of pixels within an area of interest, a signal X_in which contains the x-coordinate of each pixel , and control information for each pixel as described hereinbelow. For each input pixel , the neighborhood generator 22 generates a neighborhood , typically of nine pixels , where the current input pixel is the center pixel of the neighborhood , typically for use by the neighborhood operator 24 of processing unit 23. The neighborhood generator 24 additionally produces control information and the x -coordinate of the pixel, denoted X_out , for use by other processors which may succeed AlIP 14 in processing the image data. As will be discussed in more detail hereinbelow, the neighborhood generator 22 also detects the border pixels of each eres, of interest and provides the appropriate padding to fill the invalid components of the neighborhood , in accordance wit h process parameters initially downloaded into it.

According to an embodiment of the present invention, the neighborhood operator 24 is typically % 3 x 3 mor phological operator, although other 3 x 3 operators , such as convolvers or non-linear operators . are also possible.

The neighborhood operator 24 produces the result a of the morphological operation and typically a modified copy of the intensity of the center pixel , P_in for use by the extended nei ghborhood transport block 26,

In accordance with an alternative embodiment of the present invention, the neighborhood operator. 24 performs two-dimensional linear functions, such as convolutions. These operations typically do not preserve the characteristics of the area of interest; however , they produce information useful in the processing of an image .

The extended neighborhood transfers is typically obtained in the extended neighborhood transfers block 26 by performing a general function , f , on some combination of the inputs from the neighborhood operator 22, P_in and 0, and external dats , such as a constant value or the value of a related pixel from a second image . The general function f can be addition, subtraction, identification of the maximum or the minimum, or a threshold operation. The output of block 26 , which is the output of the processing unit 23, is the signal P_out which contains the transtformed intensity values of the pixels of the area of internet.

In accordance with an alternative embodiment of the present invention, extended neighborhood transfers block 26 it not included in processing unit 22. Thus , the processing unit 23 produces the outputs of neighborhood operator 22, P_in and G.

Eash AIIP 14 is typically built on a single VLSI chip typically designed as two functional, units , the neighborhood generator 23 and the processing unit 23. For standard image processing hardware , the abovementioned configuration would require 72 interconnecting wires to transmit the nine 6-bit intensity values of the neighborhood free the neighborhood generator 22 and the processing unit 28 . In accordance with the preferred embodiment of the present invention , a novel seness of providing only a third of the neighborhood intensities to the processing unit 23 at one time reduces the number of interconnecting wires while requiring very little additional hardware,

As mentioned hereinabove , the output of each AIIP 14 is sent to the image content respons ive scan controller 16. A preferred embodiment of the image content responsive scan controller 16 typically initially analyzes a low resolution version of the image and identifies areas of interest in it. The controller 16 instructs the image acquisition unit 10 to acquire the intensities of the pixels of the areas of interest at the next highest resolution , where a higher resolution indicates that the chosen pixels are closer to each other. The data from the areas of interest are processed by the AIIPs 14 and the controller 16 analyses the results to identify areas within the areas of interest which require still further resolution to fully define them. The process repeats itself until fill sections of the image are acquired it the highest resolution necessary ,

It will be appreciated that other algorithms which incorporate random access acquisiti on of pixels can be implemented in controller 16, In all cases , the output of the controller 16 is a stream of pixel coordinates in the areas of interest typically organized as followa and according to Fig . 3A. Since an area of interact is rarely square or rectangular, it may first be made rectangular by adding additional pixels . This it done to simplify the organiza tion of the data . Subsequently , the areas of interest in the image are scanned where the scanning methodology utilises the fact that the image sensor 11 described in previously mentions Isrmal Patent Application 83213 internally obtains the intensity values pertaining to the entirety of pixels in a certain row and only afterwards executes the x coordinate selection. Thus, the scanning is typically performed as shown in Fig. 3A.

An alternative embodiment of the scanning methodology scans the areas of interest in order, each area is scanned from left to right and from top down, as shown in Fig, 3B.

Since the neighborhood operator 24 operates on small neighborhoods and since the image sensor 11 scans the image in the x direction (i.g . from left to right) , only the sequence of the scan lines is important, the y coordinate, or the row position, is not necessary. Each arrow in Fig. 3A corresponds to a sequential access to pixels of a given area of interest that share the same y coordinate. Since the pixels thus marked can be accessed sequentially with no interruptions, they constitute an xburat. The dashed lines correspond to breaks in the date flow for acquiring the areas of interest and it will be appreciated that the breaks should be skipped as fast as possible in order to maintain system performance. Breaks occuring in the same row are denoted xbreaks and breaks occurring between rows are denoted ybreaks. An xbreak may typically cause a date time equivalent to the access time of at most two pixela l a ybreak may typically cause a date time equivalent to the access time of at most ten pixels.

According to a preferred embodimen t of the present invention. set of interfacing signals a described hereinbelow, are received by the neighborhood generator 22 - from the image acquisition unit 1C, Any other similar not of control signals which indicate to the neighborhood generator 22, for each incoming pixel , its coordinates in the image and its location in the area of interest, can be used .

The preferred set of interfacing signals to the neighborhood generator 22 typically is.

P_in The intensity of the image at the current absolute coordinate inside an xburst. Absolute coordinates refer to the location of the pixel in the image .

X The absolute abscises coordinate x of the pixel in the image. X is valid only inside an xburst and a series of Xs is typically monotonically increasing, thus simplifying the hardware of the neighborhood generator 22.

LV A line valid signal which allows , when active , a set of 'xbursts to occur. The riming edge of the signal marks the end of a scanning row and the filling edge marks the Beginning of the next.

xs , xe Signals to indicate the beginning and end of in xburst, xs becomes active during one pixel time (i.e. the time requited to acquire one pixel) to indicate the start of an xburst. xe becomes active during one pixel time to indicate the end of en xburst. The pixel arriving at the same time that xs it mesurted it called the leftmost edge and the one that arrives with xs it called the rightmost edge . Typically , xs and xe are required act to become active at the same time .

F Indicates a valid image framo . The rising edge of the signal indicates the and of a first image and the falling edge marks the beginning of a second image.

ys ,ye Signals to indicate the topmost and bottomost pixels of an area of interest. If ye is active , all pixels received are topmost pixels. If ye is active, the pixels received are bottomost pixels . If neither is active, then an internal pixel is being received. It is typically required that ye and ye not become active at the same time .

The operation of the interfacing signals described hereinabove can be understood with regard to Figs. 4A and 4B which show an image with two areas of interest and the timing signals resulting from the image, respectively.

The bullets in Fig_. 4A indicate the pixels being scanned. Across the top of Fig. 4A is an indication of the x coordinate; it should be noted that the leftmost area ef interest is located between x coordinates 21 and 33 and the rightmost area of interest it. located between x coordinates 101 and 105 , The y coordinates are not marked as they are not necessary. lt should also be noted that xbursts are numbered in the order that they are scanned.

The neighborhood generator 22 typically receives the abovementioned input signals from controller 16, via image acquisition, unit 10, and utilises them to generate the neighborhood of the pixel (x,y) whose intensity is P_in. Since the neighborhood generator 22 produces as output only three pixels of the neighborhood of a pixel (x, y) as any instant (i.e . a North, a

Center and a South pixel ) , it will be appreciated that each pixel (x ,y) it a potential element of threr neighborhoods as the topmost, the central and the bottom ost pixel, Thus , the neighborhood generator 22 stores each pixel (x,y) in three separate locations in order to properly generate the three neighborhoods to which each pixel (x,y) belongs , It should also be noted that the neighborhood operator 24 must be organized such that the pixels thus transmitted can be used as the rightmost column , the central column and the lef tmost column of thres consecutive neighborhoods . Additionally, the neighborhood generator 22 must add a dummy pixel to the right of a finishing xburst and to the left of a beginning xburst, a process known as padding, to ensure that the entirety of pixels in the area of interest can be utilised in the processing. This addition of two extra pixels between xbursts costs the sys tem two pixels of dead time.

An alternative embodiment of the padding, which reduces the dead time between pixels , comprises an additional output signal, denoted edge , which becomes active whenever an xburst starts and one clock after it ends. Thus, the dead time is reduced to one pixel time.

Reference is now made to Fig. 5 which illustrates , in block diagram form, the architecture of the neighborhood generator 22 comprising a date input system 30 for organizing the flok of input information, a FIFO 32 for storing the incoming pixels in en organized manner , and a data output server 34 for reading the date free the FIFO 32 and for generating the properly padded neighborhood and the output control signals . A controller 36 control, the operation it the data input system 30 and the data output server 34.

The neighborhood g enerator 22 receives , for each pixel

(x,y) of an area of interest, the interface signals . xs _in, xe _in in ys _in , ye _in, LV_in , and FV_in , as wall as the x coordinate x_in and the intensity P_in of the incoming pixel . It produces , for each pixel (xfy) , the interface signals xs _out, xe _out , ys _out , ye _out ,LV _out , and PV _out , and the north, center, south and edge signals .

The data input system 30 directs the input data to the appropriate locations within the neighborhood generator 22 for storage or for computations .

The FIFO 32 is organized in 5 column arrays 8, C, N, X, and 7 corresponding, respectively, to south pixels , center pixels, north pixels, x coordinate values , and 3-bit interface tags described in detail hereinbelow . The arrays are typically filled by the data input system 30 to ensure that the output information for a given pixel (x,y) is located at the same location in each array, A head pointer h paints to the currant available location and is increased by one every time a pixel intensity is entered into the FIFO 32. Thus , if the intensity of an incoming pixel is stored as the central pixel in location C[h] , than its south neighbor ie stored in 8[h] , its north neighbor is stored in N[h] , its a coordinate is stored in X[h], and its 3-bit tag is stored in Tag[h] .

The 3-bit. tag is typically included to pass the informatio n of the location of the pixel in the area of interest on to ether succeeding processors. It is generated according to the following table where xborder indicates eithhr xs or xe and yborder indicated either ys or ye . Tag Represents

0 internal pixel

1 not xborder but ys

2 not xborder but ye

3 xborder but hot yborder

4 xborder and ys

5 xborder and ye

6 line valid falling edge indicator

7 freme valid falling edge indicator

The input datt flow through the neighborhood generator

22 is commanded by the controller 36 and is as follows for an internal pixel (i.e. one which is not on a border of the area of interest) . The current intensity p_in of pixel (x,y) is stored in FIFO 32 at C[h] as indicated by pointer A. It is additionally stored in the proper locations in FIFO 32 to become the south neighbor of the pixel in the line above it end the north neighbor for the pixel below it. Accordingly , the value of h for the X_in pixel of the previous line it stored in the X_in location of a line pointer column array, L_Pointer , of a two column array RAM

38 and is used to indicate the appropriate location in 8 for storing the current intensity value P_in. The current value of h is then scored in L _Pointer[X_in] for use in locating in column 3 the pixel at x coordinate X_in in the subsequent line .

P_in is also stored in the X_in location of a previous pixel column erray , L_Pixel, of the RAM 38. However , before it is stored in that location , the value previously stored there 10 copied into N[h] . Ones the operations described hereinabove are performed . h is increased by one and the process repeats itself.

A topmost border pixel has no north neighbor and is not 6 south neighbor for any ether pixel and thus , the pixel intensity P_in is not stored in column 8 and no pixel intensity value is copied from column L_Pixel to column N. However, P_in is stored in columns O and L_Pixel , its x coordinate is stored in column X and its tag is stored in column TAG.

A bottomost border pixel has a north neighbor for which it is a south neighbor. To simplify the storing logic, the entirety of operations described for an internal pixel occur for the bottomost border pixel; P_in is placed at C[h] , the intensity of its north neighbor is taken from the appropriate location in

L_Pixel and P_in is stored at the appropriate location to be a south neighbor. P_in is also placed in the appropriate location in L_Pixel to be a north neighbor for a pixel of the following line , although no pixels will enter the FIFO 32 which will utilize the value stored in L_Pixel .

Additionally, at the end of an xburst, or at the end of a frame, the appropriate tag value is stored and the location b is increased by 1.

A program embodiment of the logic described hereinabove , typically incorporated into controller 36, can be found in Annex A . It should be noted that the program also records. the number of currently scored lines , in variable

Stored Lines , and the previous a coordinate value , in variable Last_X.

The neighborhood generator 22 also comprises a process

current line, the processing is finished.

The two process parameters padmods_k and padmode_y

Λ W to _

west and north-south) . The two options are duplicate padding and constant, or register , padding. For constant padding, a constant stored in a special register is used as the padding value.

Typical values of the constant are the values of white or black.

Duplicate padding consists of using the value of the border pixel as the value of the padded pixel .

A programmable delay block 44 is typically incorporated to implement a programmable delay on the output interface signals, xs_ou, xs _out , ys_out, ys_out, LV_out, TV_out and X_out Thus , several processors with different pipeline stages are accomodated.

According to a preferred embodiment of the invention, the operation of the data output server 34 , is best described by a Mealy finits stats machine as defined in the book, Switching and Finitr Automata Theory by Z. Kohavi, Mouraw-Hill, New York, 1970. Fig. 6 shows the embodiment of the finite state machine incorporated into the controller 36, Fig. 6 will be better understood with the following definitions of the input signals to the finite stats machine.

Empty If the empty flag a is set, indicating that no data resides in the FIFO 32, no action is taken, Otherwise, the contents of the FIFO 32 at the address indicated by pointer t are read into internal registers of the data output server 34 for further processing.

Xbordes Active when Tag[t] (i. g. the value of column Tag at the address indicated by pointer t) indicates an xborder. The odd xborders are typically considered as xs and the even xborders are typically considered as xs.

Newlins Active when Tag[t] spacifies a change in either the LV or PV signals.

Op Allows the output of a given xburst. The signal is defined according to the following condition.

where stored_Liner is a variable indicating the number of lines presently stored in FIFO 32 and Last_X is a variable indicating the previous x location. Stored_Lines and Last _X are calculated by controller 36 and are used in the program of Annax A.

cap Causes the insertion of a gap, as defined in the process parameters register 40, with the appropriate padding before an xburst starts and after it finishes .

The finite state machine produces the following output signals.

Putdass Osuses the contents of Tag[t] , N[t] . C[t] , g[t] and X[ t] to be written at the data outputs ys_out , ys_out, x_out, North, center and South. If ys is true, the North output will ce padded. If ye is true , the south output will be padded.

It should be noted that the padding will be performed according to the value of tne process parameter register padmode_y .

Xpad If this signal is active and the padding: mode it register , all North , center and South wires will contain the

g_x,y , and the extended neighborhood transform block 26 performs a subtration between g_x,y and a_x,y producing the output intensi ty a_x,y.

The fourth type of operation is topically performed between a senc eant K and at least one of the abovementioned input, to the extended neighborhood transform block 26 , specifically, at least one of the pixel intensities a_x,y , b_x,y or the output of the neighborhood operator 24 a_x,y . A typical operation is a threshold operation.

Reference is now made to Fig. 8 which illustrates , in detailed block diagram form, the elements of the processing unit 25. Neighborhood operator 24 typically comprises an input register so for receiving, at each clock, one three element column of a neighborhood and for producing, at each clock, a nine element neighborhood , a mask register 32 for storing a nine element mask, a mask 34 for performing a masking operation to produce a difntion or erosion operation on the nine element neighborhood, or, alternatively , to perform a linear convolution, and a sorter 36 to find the maximum, minimum and median values of the masked nine elements of the neighborhood and to produce one of the maximum, minimum or median values as the result g_{x ,y} .

The extended neighborhood transform block 25 has four input ports , receiving the input pixel intensity a_x,y , the output r_{x, y} of the neighborhood operator 24 , the pixel intensity value b_{x, y} , and the constant value K . The result of block 26, and or the preceding unit 25 , is the value r_{x, y} . The type of operation performed by block 26 and the inputs to be used for i t are indicated to the block 26 by a system operator.

The input register 30 is comprised of any suitable hardware which accepts a three element column as input, stores it and produces it three times on output, as the rightmost, center and leftmost column of three neighborhoods. Additionally, the hardware must respond to edge information indicating the edge of an area of interest such that edge columns or columns next to edge columns are used for output only once or twice , respectively.

The masker 34 is composed of suitable hardware to perform a mesking operation. The mask , stored in mask register 32 , typically comprises nine 8-bit elements whose values are either 1 or 0 where only the lowest significant bit is utilized for the masking operation. The remaining bits are included so that, in the alternate embodiment where masker 34 performs a linear convolution, the mask register 32 need not change. Masker 34 operates as follows, for each mask element whose lowest significant bit is 1 , masker 34 passes the pixel intensity value on which the mask element operates to the sorter 36 unchanged. For a mask element whose lowest significant bit is 0, makear 34 removes the pixel intensity value from the sorting operation of sorter 36 by passing & large value, typically 255 (all 8 bits equal to 1), when the sorter 36 is configured to find the minimum value or by passing a small value typically 0 (all 8 bits equal so 0) , when the sorter 36 is configured to find the maximum value , When the sorter 36 it configured to find the median value, the masker 34 typically produces & large value, although a small value can also be produced. Reference is now made to Fig. 9A which illustrates , in block diagram form, the sorter 36 which identifies the maximum, minimum and median of the nine masked pixel values received from masker 34. Sorter 36 is typically comprised of 7 minimum-maximum - median (M³ ) sorters 60 each of which finds the minimum, maximum and median of three input signals .

Fig. 9B illustrates the operation of an individual M³ sorter 60. Three masked pixel intensity values , p , q and r, are sorted by three max-min sorters 65 , into the maximum, median and mini mum values as follows , it will be appreciated that the signals are positive.

Max-min sorter 62a receives masked Pixel values p and q and produces the larger value as value' s and the smaller value as value t. Values t and r are compared by max-min sorter 63b, the larger one is produced as value u and the smaller, which is the smallest value of the three masked pixel values , is produced, on output, as the minimum value. The two larger values s and u are compared by max-min sorter 62c , the larger value is produced as the maximum value of ths three input signals p, q and r and the smaller value is produced as ths median value.

Fig. 90 illustrates the elements of ths max-min sorters 6B. Each max-min sorter 62 is comprised of a subtrac tor 64 and a multiplexer 66. The subtractor 64 produces the signal k , the substration of w from v. If a is positive , the multiplexer 66 produces v as the maximum value and w as the minimum value . If x is negative , the multiplexer 66 produces v. as the minimum value and w as the maximum value, t_{x ,y} o f the neighborhood operator 24 , and a constant K, and performs between two of the inputs a two-element operation, such as addition, subtraction, calculation of the maximum or minimum, or a threshold operation.

Block 26 typically comprises four selectors 70 and an adder/subtractor 72. Selectors 70a and 70b select the two input values in accordance with the values of two input select signals.

As shown , selector 70a selects one of g_{x, y}, a_{x, y} or b_{x, y} and selector 70b selects between K end b_{x, y} . The output of selectors 70a and 70b is sent to adder/subtractor 72 and to selector 70c.

To produce the SUB or difference of any two values of the four input signals , selectors 70a and 70b pass the selected values to adder/subtractor 72 and its output is selected by selector 70d.

To produce the maximum, minimum or median of the neighborhood of a_{x, y}, t_{x, y} it directly passed to the output vie selectors 70a , 70c and 70d . For this output, selector 70b is not operative.

To apply a binary threshold on either a_{x, y} , b_{x, y} or g_{x, y} adder /sub tree tor 72 receives the selected signal and the threshold value k via selectors 70a and 70b respectively , The borrow signal of adder/subtractor 72 is utilised to control selector 70a to pass either a 1 , a 0, or any other desirable valυe, to the output via selector 70d.

It Kill be appreciated by persons skilled in the art that the present invention is not limited by what has been particularly shown and described hereinabove . The scope of the present invention is defined only by the claims which follow.

Claims

C L A I M S

1. Apparatus for processing images acquired as a plurality of areas of interest comprising .

means for defining a neighborhood for each pixel of each of said plurality of areas of interest; and

image processing means using a neighborhood generated by s aid means for defining a neighborhood.

2. Apparatus for processing images comprising:

means for defining a neighborhood of each incoming pixel; and

means for image processing including;

means for producing the maximum, minimum and median values of said neighborhood.

3. Apparatus for processing images according so claims 1 and 2 and wherein said means for defining a neighborhood include means for padding the neighborhood of incoming pixels defined as belonging to the border of one of said areas of interest by including in the neighborhood pixels which belong outside of said areas of interest and by giving said outside pixels values determined by said means for defining a neighborhood. 4. Apparatus for processing images according to any of the previous claims and wherein said means for defining a neighborhood produce and transmit a portion of s aid neighborhood of a pixel and wherein said means for image processing reconstruct a complete neighborhood from said portion of said neighborhood and from portions of said neighborhood previously transmitted from said means for defining a neighborhood,

5. Apparatus for processing imaged according to claim 4 and wherein said means for defining a neighborhood produce the North , Center and South pixels of a neighborhood of a pixel .

6. Apparatus for processing images according to claim 2 and wherein said means for producing comprises seven three element sorters .

7. Apparatus for processing images according to claim 1 and wherein said image processing means includes a morphological operator.

8, Apparatus for processing images according to claim 7 and wherein said morphological operator determined the maximum, median and minimum values of the pixel values of said neighborhood.

9. Apparatus for processing images according to claim 7 and wherein said morphological operator performs an extended neighborhood transform.

10. Apparatus for processing images according to claims 7 and & and wherein said morphological operator perform s erosion.

11. Apparatus for processing images according to claims 7 and 9 and wherein said morphological operator performs dilation.

11. Apparatus for processing images according to claim 1 and wherein said areas of interest are of non-square shapes .

13. Apparatus for processing images acoording to claim 1 and wherein said areas of interest are of square shapes.

14. A sorting network for producing the maximum, median and minimum values of nine input elements.

15. A network according to claim 14 and comp rising seven three element sorters .