CA2000012C - Computer-implemented method for automatic extraction of data from printed forms - Google Patents
Computer-implemented method for automatic extraction of data from printed formsInfo
- Publication number
- CA2000012C CA2000012C CA002000012A CA2000012A CA2000012C CA 2000012 C CA2000012 C CA 2000012C CA 002000012 A CA002000012 A CA 002000012A CA 2000012 A CA2000012 A CA 2000012A CA 2000012 C CA2000012 C CA 2000012C
- Authority
- CA
- Canada
- Prior art keywords
- data
- lines
- masks
- array
- mask
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
- G06V30/41—Analysis of document content
- G06V30/412—Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/174—Form filling; Merging
Abstract
A computer-implemented method operable with conventional OCR
scanning equipment and software, extracts character data from printed forms.
A blank master form is scanned and its digital image stored.
Clusters of ON bits of the master form image are first recognized as part of a line and then connected to form lines. All of the lines in the master form image are then identified by row and column start position and column end position, thereby creating a master-form-description. The resulting image, which consists only of lines in the master form, can then be displayed. Regions or masks in the displayed image of master form lines are then created, each mask corresponding to a field where data would be located in a filled-in form. Each data mask is spaced from nearby lines by a predetermined data margin, referred to as D.
A filled-in or data form is then scanned and lines are also recognized and identified in a similar manner to create a data-form-description. The data-form-description is compared with the master-form-description by computing the horizontal and vertical offsets and skew of the two forms relative to one another. The created data masks, whose orientation with respect to the master form has been previously determined, are then transposed into the data form image using the computed values of horizontal and vertical offsets and skew. In this manner, the data masks are correctly located on the data form so that the actual data values in the data form reside within the corresponding data masks. Routines are then implemented for detecting extraneous data intruding into the data masks and for growing the masks, i.e. enlarging the masks to capture data which may extend beyond the perimeter of the masks. Thus, the data masks are adaptive in that they are grown if data does not lie entirely within the perimeter of the masks. During the mask growth routine, lines which are part of the background form are detected and removed by line removal algorithms.
Following the removal of extraneous data from the masks, the growth of the masks to capture data, and any subsequent line removal, the remaining data from the masks is extracted and transferred to a new file. The new file then contains only data comprising characters of the data values in the desired regions, which can then be operated on by conventional OCR software to identify the specific character values.
scanning equipment and software, extracts character data from printed forms.
A blank master form is scanned and its digital image stored.
Clusters of ON bits of the master form image are first recognized as part of a line and then connected to form lines. All of the lines in the master form image are then identified by row and column start position and column end position, thereby creating a master-form-description. The resulting image, which consists only of lines in the master form, can then be displayed. Regions or masks in the displayed image of master form lines are then created, each mask corresponding to a field where data would be located in a filled-in form. Each data mask is spaced from nearby lines by a predetermined data margin, referred to as D.
A filled-in or data form is then scanned and lines are also recognized and identified in a similar manner to create a data-form-description. The data-form-description is compared with the master-form-description by computing the horizontal and vertical offsets and skew of the two forms relative to one another. The created data masks, whose orientation with respect to the master form has been previously determined, are then transposed into the data form image using the computed values of horizontal and vertical offsets and skew. In this manner, the data masks are correctly located on the data form so that the actual data values in the data form reside within the corresponding data masks. Routines are then implemented for detecting extraneous data intruding into the data masks and for growing the masks, i.e. enlarging the masks to capture data which may extend beyond the perimeter of the masks. Thus, the data masks are adaptive in that they are grown if data does not lie entirely within the perimeter of the masks. During the mask growth routine, lines which are part of the background form are detected and removed by line removal algorithms.
Following the removal of extraneous data from the masks, the growth of the masks to capture data, and any subsequent line removal, the remaining data from the masks is extracted and transferred to a new file. The new file then contains only data comprising characters of the data values in the desired regions, which can then be operated on by conventional OCR software to identify the specific character values.
Description
ZOO~Ol:~
A ~ INPl~ ~u MRT~n~ ~OR AUTOMATqC
EXTRACTION OF DATA FROM ~kl~.~u FOgMS
Technical Field This invention relates to optical character recognition (OCR) devices and processes, and more particularly to a computer-implemented method for automatic extraction of data from printed forms.
~ ,,.
Background of the Invention Automatic computer-implemented reading of data from printed forms is typically done in a sequence of three steps. First a form is optically scanned to create an electronic image, which is then written in digital storage as a rectangular array of 0's and l's representing white and black subareas or pixels. Then the image is processed to extract regions or fields cont~in;ng the data to be read. Finally, the black and white subimage in each extracted region is interpreted and expressed as an alphanumeric code, such as ASCII or EBCDIC.
The data present in printed forms may be defined as having two aspects: a value and a significance. For example, the word "Yes" is a value that becomes data only when its siqnificance, i.e., the question it answers, is made clear. Printed forms provide a conventional means for recording data in which significance is predefined as a background of text and graphics, such as boxed areas. Since forms are printed mechanically, the background is identical over different instances of the same form. Thus the position of data values on the form is in correspondence with the data significance. Optical character recognition (OCR) devices take advantage of this fact to read data from credit card receipts, billing statements, etc. Such "OCR forms" are designed with data values entered in spaces well SA9-89-006 Page 1 ~
A ~ INPl~ ~u MRT~n~ ~OR AUTOMATqC
EXTRACTION OF DATA FROM ~kl~.~u FOgMS
Technical Field This invention relates to optical character recognition (OCR) devices and processes, and more particularly to a computer-implemented method for automatic extraction of data from printed forms.
~ ,,.
Background of the Invention Automatic computer-implemented reading of data from printed forms is typically done in a sequence of three steps. First a form is optically scanned to create an electronic image, which is then written in digital storage as a rectangular array of 0's and l's representing white and black subareas or pixels. Then the image is processed to extract regions or fields cont~in;ng the data to be read. Finally, the black and white subimage in each extracted region is interpreted and expressed as an alphanumeric code, such as ASCII or EBCDIC.
The data present in printed forms may be defined as having two aspects: a value and a significance. For example, the word "Yes" is a value that becomes data only when its siqnificance, i.e., the question it answers, is made clear. Printed forms provide a conventional means for recording data in which significance is predefined as a background of text and graphics, such as boxed areas. Since forms are printed mechanically, the background is identical over different instances of the same form. Thus the position of data values on the form is in correspondence with the data significance. Optical character recognition (OCR) devices take advantage of this fact to read data from credit card receipts, billing statements, etc. Such "OCR forms" are designed with data values entered in spaces well SA9-89-006 Page 1 ~
- 2~0(~01~
separated from background printing to assure that the latter are not erroneously interpreted as data values. Data significance does not appear explicitly! but is stored in the computer and associated with the data values on the basis of position in the image. In some cases, forms are printed in a color invisible to the scanner to avoid a possibility of confusion. Data values are carefully positioned during printing, and the form precisely registered during sc~nn; ng. All these steps serve to guarantee that the data values are exactly where the reading or scanning equipment performs its extraction process.
In recent years, demand has grown for a capability to capture data from printed forms that do not meet OCR constraints.
Forms routinely used in government and commercial operations, such as birth and marriage certificates, are designed to be intelligible to the human eye and brain. While people are sophisticated processors of visual images, they also require that both attributes of a data element, the significance and the value, be present on the document. Thus background printing is provided to supply the meaning of each data field, and lines and boxes are imposed to make clear the association of data value and data significance. The crowded appearance of these "people forms", compared with OCR forms, is a necessary outcome of a requirement to pack a great deal of information into a limited space.
It is likewise difficult to enforce controls in the preparation of people forms. A birth or marriage certificate filled out with a typewriter is registered by eye, often with errors in translation and skew compared to the ideal orientation.
Data values may superimpose on the form background as a result.
The printing process itself is subject to mechanical slippage that may give the same effect. Finally, mechanical slippage and electronic noise occurring during the optical scanning process i ~ . .
2(~0~)12 present a further source of registration error. This is particularly true if economical general-purpose scanners are used. The net result of all these factors is that printinq of a given data value on people forms may be skewed, may overlap boundary lines separating data regions, and even when ideally positioned does not consistently appear in a fixed, predictable region in scanned images of different instances of the form.
These difficulties pose severe problems for automatic computer-implemented data extraction, rendering inapplicable the sort of processing used for OCR forms.
Summary of the Invention The invention is a computer-implemented method operable with conventional OCR scAnning equipment and software for the automatic extraction of data from printed forms.
A blank master form is scanned and its digital image stored.
Clusters of ON bits of the master form image are first recognized as part of a line and then connected to form lines. All of the lines in the master form image are then identified by row and column start position and column end position, thereby creating a master-form-description. The resulting image, which consists only of lines in the master form, can then be displayed. Regions or masks in the displayed image of master form lines are then created, each mask corresponding to a field where data would be located in a filled-in form. Each data mask is spaced from nearby lines by a predetermined data margin, referred to as D.
A filled-in or data form is then scanned and lines are also recognized and identified in a similar manner to create a data-form-description. The data-form-description is compared with the master-form-description by computing the horizontal and vertical offsets and skew of the two forms relative to one another. The created data masks, whose orientation with respect i ZOOGO~
to the master form has been previously determined, are then transposed into the data form image using the computed values of horizontal and vertical offsets and skew. In this manner, the data masks are correctly located on the data form so that the actual data values in the data form reside within the corresponding data masks. Routines are then implemented for detecting extraneous data intruding into the data masks and for --;
growing the masks, i.e. enlarging the masks to capture data which may extend beyond the perimeter of the masks. Thus, the data masks are adaptive in that they are grown if data does not lie entirely within the perimeter of the masks. During the mask growth routine, lines which are part of the background form are detected and removed by line removal algorithms.
Following the removal of extraneous data from the masks, the growth of the masks to capture data, and any subsequent line removal, the remaining data from the masks is extracted and transferred to a new file. The new file then contains only data comprising characters of the data values in the desired regions, which can then be operated on by conventional OCR software to identify the specific character values.
For a fuller understanding of the nature and advantages of the present invention reference should be made to the following detailed description taken in conjunction with the accompanying drawings.
Brief Description of the Drawings Fig. 1 is a schematic illustrating the relationship of the invention to conventional hardware and software for scanning an image and converting the image to stored characters through the use of OCR software;
Fig. 2 is a schematic illustrating the two portions of the data extraction software designated Phase I and Phase II ;
2000~12 Fig. 3 is a schematic of the functions present in the Phase I portion of the data extraction software;
Fig. 4 is a schematic of the functions present in the Phase II portion of the data extraction software;
Fig. 5 is a schematic of the Copy Masks function of the Phase II portion of the data extraction software;
Fig. 6 is an illustration of the manner in which horizontal and vertical adjustments between the master form and the data form are used to register the data form following scanning;
Fig. 7 is a depiction of an alternative solution to the problem of skew in the location of a data mask;
Fig. 8 is a graphical illustration of the Grow Bottom function for growing a data mask to capture data which crosses a mask perimeter;
Fig. 9 is a graphical depiction of a line removal process to capture character data which is intersected by a linej the line having been detected during the Grow Bottom function;
Fig. 10 is a representation of ON pixels for data intersected by a line;
- Fig. 11 is a representation of the ON pixels depicted as dashes following detection of the line pixels through the use of the algorithm in Appendix C;
Fig. 12 is a typical bland birth certificate master form;
and Fig. 13 is a typical filled-in birth certificate data form.
Description of the Preferred Embodiments Referring first to Fig. 1, the conventional hardware and software for converting characters from a printed form into stored di~ital data comprises a scanner, such as any 300 pixel bi-level scanner, and a central processing unit (CPU). The CPU
is connected to conventional peripheral devices such as a ;
i 2CO(~Ol;~
display, a data storage (e.g. a hard disk file~ and a printer.
The scanner converts the printed image from the form into bi-level digital data which is transferred to the CPU and converted into an array of O's and l's, representing white and black pixels on the scanned form. Conventional OCR software is then utilized to recognize characters from the array of digital data. The CPU then stores and/or displays the results of the OCR
software operation to any of its peripheral devices. Thus, the output from operation of the OCR software is character data, represented for example in ASCII format, which can be easily manipulated by the CPU and stored or transmitted to peripheral devices.
As illustrated in Fig. 1, in the present invention a blank master form is first scanned, converted to bi-level data, received by the CPU and stored. An example of a master form is shown in Fig. 12. A completed or filled in form, is then scanned, converted to bi-level data and stored by the CPU in an array referred to as an "old-map". An example of a filled-in or data form is shown in Fig. 13. The present invention is the method of data extraction performed by the data extraction software which allows just the pixels in the data regions or fields to be extracted from the old-map so that this data can then be received and operated on by the OCR software in the conventional manner. The functions of the data extraction software will be described with reference to the schematics of Figs. 2-5 and to the "pseudo-code" included in Appendix A, which is incorporated herein by reference. A list of definitions to assist in underst~n~ing the pseudo-code is listed in Appendix B
and incorporated herein by reference.
Referring now to Fig. 2, the data extraction software includes two phases, referred to as Phase I and Phase II. The operation of Phase I is identical for both the master form and i Z~O~Ol;~
the data form, while Phase II operates only on the data form.
Thus, the first step in the data extraction process is the scanning of the blank master form and, through the use of the functions in Phase I to be described below, the creation of a master-form-description. The master-form-description includes the identification of all lines by location and length and the skew of the form, i.e. its angular orientation relative to a fixed coordinate system of the scanning device.
Referring to Fig. 3, the description of the operation of Phase I will be described relative to its operation on a scanned data form, i.e. the old-map, although as just indicated, the operation of Phase I is identical for the blank master form.
This is because the function of Phase I is to locate all horizontal lines on the form, to generate a line-list and to identify the location and length of all of the lines on the form, be it the master form or the data form The old-map is a two-dimensional array of digital data corresponding to the pixels of the scanned data form image. Each pixel is either 1 or 0, or "ON" or "OFF", depending upon the presence or absence of either background lines or character data at the corresponding X-Y location. This digital data is analyzed with a Build Table routine (Appendix A, 1.1), which is a search for vertical runs of ON bits having a height less than an predetermined number, e.g. 3 in the preferred embodiment. The Build Table routine identifies all such vertical runs and creates a cluster-list for identifying the location of these vertical runs. The Build Table routine outputs a list of all vertical clusters located in the old-map and a cluster-index which points to the start of each row for each 3-pixel-high vertical cluster.
This information is then passed to the Build Lines routine (Appendix A, 1.2) whose function is to connect horizontally related clusters to detect hori~ontal lines. Thus, the Build i Z~0~012 Lines routine searches through all clusters in the cluster-list, and with knowledge of their location from the cluster-index, identifies horizontal lines. If there is an additional cluster within a predetermined minimum length away from the portion of the line just built, then that cluster is added to the existing line. The output of the Build Lines routine is a line-list of all of the lines in the data form. Each line in the line-list is identified by its row-start, row-end, column-start, and column-end.
The lines are then normalized or de-skewed by the Normalize Lines (Appendix A, 1.3) routine. Because the data form may not have been perfectly aligned with the scanning device, or the background lines on the form not perfectly oriented on the form, the lines which have been identified from the Build Lines routine may be skewed. This means that the pixels comprising a line may not necessarily begin and end on the same row. The Normalize Lines routine calculates the average skew of the lines in the line-list. The skew is defined as the ratio of the difference between the start and stop row of a line to the difference between the start and stop column of the line. After the skew has been calculated in this manner, the lines are normalized or de-skewed so that each line in the line-list can now be identified by a row-start, column-start and column-end. As a necessary output of the Normalize Lines routine, the skew of the data form has been calculated and is available for later use.
The Connect Lines routine (Appendix A, 1.4) then takes the list of Normalize Lines and determines if any of these lines are within a predetermined line tolerance of one another. If so, the lines are connected to make a single line, thereby reducing the number of lines in the line-list. As a result of the Connect Lines routine, the output is a data-form-description which is a listing of all horizontal lines in the data form, where each of 2~!0~0~2 those lines is identified by its row-start, its column-start and its column-end. A similar procedure may be performed to detect and identify vertical lines. Thus, at the completion of Phase I
the data extraction software has generated in digital form an exact replica of all background lines present on the filled-in data form.
As previously described, the Phase I process is first performed on a master form, i.e. a form identical to a filled-in data form, but without any character data present in the data fields. This is the first step in the process in the present invention and results in a master-form-description which is a list of all horizontal lines on the master form identified by row-start, column-start, and column-end. Vertical lines would be identified by row-start, column-start, and row-end.
Referring now to Paragraph 1.5 of Appendix A, the next step is the creation of the data-field-masks from the master form, utilizing the master-form-description output by Phase I when the master form was scanned. The function of the Create Masks step is to define a series of rectangular masks separated by portions of the lines identified in the line-list of the master form, wherein each mask corresponds to a desired region where a data value is located in the data form. While there are numerous techniques for creating the masks, the preferred technique is to generate the output of Phase I from the master form on a visual display (See Fig. 1). A user, through the use of a light pen, then points to those regions bounded by lines where it is desired to extract data from the data forms. For example, the light pen could identify the upper left and the lower right corners of each desired data mask, which would result in identifying the location and dimensions of each rectangular data region. Each data mask has a perimeter which is preferably spaced from the lines by a predetermined data margin, D. As a result of the Create Mask Æ~0~012 ....
step, the CPU will have stored the location and dimensions of the data regions by use of a mask-list, which identifies each mask by row-start, column-start and row-end, column-end. Since the normalized lines in the master form are presented on the visual display during this step it is not necessary to compensate for skew when the data masks are created on the master form.
At this point in the description of the invention, a master form has been scanned, a master-form-description has been defined through the use of Phase I (Fig. 3 and Appendix A, 1.1-1.4), the data masks have been created, a data form has been scanned and a data-form-description has been generated through the use of Phase I.
Phase II LS now performed on the data form. Referring now to Fig. 4 and Paragraph 2 of Appendix A, it is noted that Phase II has several functions. The first function is the Prepare Form routine (Appendix A, 2.1) which compares the data-form-description with the master-form-description in order to calculate the differences in terms of field locations between the two descriptions. The function of the Prepare Form routine can be better understood by reference to Fig. 6. Because the data form cannot always be perfectly aligned with the scanner, there will usually be both vertical and horizontal offsets of the data form lines from the master farm lines form each time a new data form is scanned. Thus, the Prepare Form routine compares each line in the line-list in the master-form-description with its corresponding line in the line-list in the data-form-description, and using the differences between the row-start, column-start and column-end values, determines the offset of the data form from the master form. This offset is essentially the horizontal adjustment (horiz-adj) and the vertical adjustment ~vert-adj) necessary to transform the master form into the data form. The output of the Prepare Form routine 2C!0U~)1Z
are the values or horiz-adj, vert-adj, and skew, the latter term being the rotation of the data form relative to the master form ~See Fig. 6).
In the Get Data Masks function (Fig. 4 and Appendix A, 2.2), the data masks created from the master form and which are identified by location and ~;me~cion~ are retrieved and placed in corresponding locations on the data form so as to create data masks located in the data form. As part of the Get Data Masks function, the width of the output array is calculated as the width of the longest data mask created from the master form plus a constant ~such as two bytes). This number becomes the width of the output image cont~i n; ng the extracted data.
The next function in Phase II is the Read Group function (Appendix A, 2.3), which transfers the old-map image from data storage into main memory, creating an array of records, referred to as "current-group", where each record has a width corresponding to the width of the scanned image. Thus, the Read GrouP function essentially converts the bi-level raw data into an array having a width and height corresponding to that of the scanned image. The output of the Read Group function is the current-group.
The Copy Masks function of Phase II comprises the five routines shown in Fig. 5. The Copy Masks function locates the data masks in a current-group to be extracted, grows or enlarges the masks, if necessary, to capture information which lies outside of the masks, removes any lines which intrude into the data masks, and finally writes the extracted data to the extracted image. As part of the Copy Masks function, the mask parameters are calculated in the Calculate Mask Parameters routine. This process essentially adjusts the data masks to compensate for skew, horizontal offset and vertical offset relative to the master form. The operation of the Calculate Mask 2QO~(~12 Parameters can best illustrated by reference to Fig. 6. The input to the routine are the data mask locations and ~;mension, and vert-adj, horiz-adj and skew which have been previously calculated in the Prepare Form function. Fig. 6 lllustrates how each of these parameters is used to transpose the location of the data masks, whose relationships relative to the master form are known, to their correct locations on the data form. As shown in Fig. 6, the skew rotates the masks by the degree of the skew.
However, while this may be an optimal solution in that the size of the masks remain the same as in the master form, the CPU cost in performing the rotation step and in growing the masks may be inefficient if the skew remains within a reasonable range. In an alternative technique, as shown in Fig. 7, the skew is used to alter the height of the mask in the data form, so that the resulting mask in the data form retains its rectangular shape.
As the skew increases, the chance of the predefined rectangle not fitting inside the field increases. To compensate for this, as the skew becomes greater the heights of the masks are reduced.
The output of the Calculate Mask Parameters routine is the location and dimension of the data masks within the data form, each data mask being defined by its start or left column, its end or right column, and its top-row and bottom-row.
After the Calculate Masks Parameters routine has been performed, if the data is perfectly located within the masks, then no additional data extraction routines are needed and the data can be extracted directly by the Write Row routine (Appendix A, 2.4.5). Perfect data is considered to be data that resides entirely within a predefined data mask. If there are no ON pixels along the parameter of the mask, the data is considered to be wholly inside the mask and completely perfect. In the Write Row function, all of the data in the current-group which is contained in the rectangles defined by left, right, top-row and i ~J
2QOl~OlZ
bottom-row, i.e. all of the data in the masks, is copied to a new file. The result is that the new file contains only extracted data without any extraneous lines of the background form. The data located in the new file can then be operated on by conventional OCR software to identify the data as specific character data. In addition, because each data mask is predefined as corresponding to a certain significance, i.e. name, age, etc., the data which has been extracted from the form and converted to character data by the OCR software is immediately identifiable with a predetermined siqnificance.
If, on the other hand, the data present in the data masks is not perfectly located within the perimeter of the data masks, then it is necessary to perform one of the additional functions of the Copy Masks function (Fig. 5).
The Remove Vert Bars (Appendix A, 2.4.2) removes vertically oriented bars which may be intrudinq into a data field, such as the portion of the signature in the "Middle" data field shown in Fig. 13. The Remove Vert Bars routine checks the two right-most and left-most bytes for the entire height of each field to determine if there is a run of connected ON bits that occurs that forms a vertical line. If such a line is not included since it assumed that data occurring beyond the vertical bar belongs to another field.
Referring now to Fig. 8, the Grow Bottom routine (Appendix A, 2.4.4) enlarges the bottom of the mask until all of the data in the field has been captured or until the growth of the mask reaches a predetermined limit, defined as 2D, the spacing between neighboring data masks. The mask is grown and during the growth lines are detected in the manner similar to the line detection technique described in Phase I. The mask is grown Z~O~OlZ
by searching a perimeter row for ON bits. If a violation ~the presence of ON bits) is encountered, the column to the left and right of the violation on the next row (one row up if growing the top of the mask, one row down if growing the bottom of the mask) is checked for ON bits. If either column has ON bits, then the process is repeated. This process terminates if either both bytes become OFF, or the maximum growth limit of 2D is reached.
Once any portion of a line is detected during mask growth the complete line is removed. A similar technique is used in the Grow Top (~ppendix A, 2.4.3) to enlarge the bottom of the mask to capture data. For example, the word "Twin" in Fig. 13 would be located entirely within the grown data mask and the line removed.
The line removal process is activated whenever the Grow Top or Grow Bottom routine detects a line in the ambiguous 2D area (Fig. 8). Line removal begins by following the path of the line and erasing the line as it is followed. The erasing of the line also erases anything that lies either one pixel above or one pixel below the line. This is to account for edge noise that is generated during printing and scanning. As the line is being erased, a check is made to determine whether there is any data that may be either touching or going through the line. If such data is found, the areas where the pixels may be from the line or from the data are not erased (see Fig. 9). The line detection and erasing algorithm has also been implemented in APL and is listed in Appendix C together with descriptions of the respective lines and routines. Figs. 10 and 11 illustrate a pattern processed by the algorithm of Appendix C wherein Fig. 10 is the input pattern where all black or ON pixels are denoted with an asterisk, and Fig. 11 is the result of the line detection algorithm wherein the lines have been detected and the asterisks replaced with dashes.
2~0(~012 While the preferred embodiments of the present invention have been illustrated in detail, it should be apparent that modifications and adaptations to those embodiments may occur to one skilled in the art without departing from the scope of the present invention as set forth in the following claims.`
APPENDIX A
1.0 Phase I
Phase I is responsible for building a description of the old-map (incoming image) that can be compared to a master form.
1.1 Build Table 1.1.1 Description This process searches through the first 500 rows of the old-map, looking for vertical runs that are less than 3 bits high. A table called cluster-list is built that lists all of these vertical runs.
1.1.2 Inputs old-map 1.1.3 Process while entire block not read for each byte in a row if byte > 0 increment cluster-size-counter else if cluster-size-counter 3 add cluster to cluster-list read next row sort cluster-list index cluster-list to create cluster-index SA9-89-006 ¦ ~
, ~ ~ ., 2Q0~012 -1.1.4 Outputs cluster-list, cluster-index 1.2 Build Lines 1.2.1 Description This process converts cluster-list to line-list by connecting horizontally related clusters.
1.2.2 Inputs cluster-list, cluster-index 1.2.3 Process for each cluster-list-item while cluster-list-items connected search for the next connected cluster-list-item if the next cluster-list-item is less than min-length away add cluster-list-item to line list else cluster-list-items no longer connected 1.2.4 Outputs line-list 1.3 Normalize Lines 1.3.1 Description The process converts line-list from a ~row-start, ~ row-end, column-start, column-end) orientation to a ~row, column-start, column-end) orientation 1.3.2 Inputs line-list 1.3.3 Process calculate the average row-start - row-end variation (row-deviation) calculate the average col-start - col-end variation (col-deviation) if col-deviation >0 skew = row-deviation t col-deviation for each line-list-item line-list-item[row-start] =
line-list-item[row-start] -skew*line-list-item[col-start], 1.3.4 Outputs line-list (normalized),skew 1.4 Connect Lines 1.4.1 Description SA9-89-006 ¦
200C~0~2 All lines that are within a line-toleranCe distance of each other are connected to become one line. These lines then become the data-form-description ; 1.4.2 line-list (normalized) 1.4.3 Process for each line if the line-list-item is within line-toler of another line make the two line-list-items a single line-list-item move the line-list-item to data-form-description 1.4.4 Outputs data-form-description 1.5 Create Masks (only for master form) 2.0 Phase II
Phase II is responsible for locating each data field (mask) on the data form, extracting character data from the masks and placing it in a new file. Phase II also removes any lines that may be interfering with the data field.
2.1 Prepare Form 2.1.1 Description 200(~012 Prepare Form compares the data-form-description to the master-form-description and calculates the offset in terms of field locations between the two descriptions.
2.1.2 Output master-form-description, data-form-description 2.1.3 Process for each data-form-description item horiz-adj = horiz-adj +
master-form-description[col-start] -data-form-description[col-start] +
master-form-description[col-end] -data-form-description[col-end]
vert-adj = vert-adj + master-form-description[row] -data-form-description[row]
2.1.4 Outputs horiz-adj, skew, vert-adj 2.2 Get Data Masks 2.2.1 Description Get Data Masks gets the master-form-masks and places them in the data form and also records the maximum allowable width of a mask.
2.2.2 Inputs i SA9-89-006 ~ ~
2~!0C0~2 master-form-masks 2.2.3 Process Place all master-form-masks in data-mask max-width = max length (data-mask item) 2.2.4 Outputs max-width, data-masks 2.3 Read Group 2.3.1 Description ` This process converts the image into current-group (an array of records, each map-width in size) 2.3.2 Inputs old-map 2.3.3 Process while not eof old-map place map-width bytes in current-group-item 2.3.4 Outputs current-group 2~0G012 2.4 Copy Masks Description This process locates the mask in the current-group to be extracted, grows the mask, if necessary, to capture information that lies outside of the original mask, removes any intruding lines, and writes the extracted to extracted-image Inputs horiz-adj, skew, vert-adj, max-width, data-mask, current-group, skew Process for each data-mask item Calculate Mask Parameters Remove Vert Bars Grow Top Grow Bottom Write Row Outputs extracted-image 2.4.1 Calculate Mask Parameters 2.4.1.1 Description This process adjusts the data mask to compensate for skew, horizontal offset, and verticai offset.
21~0(~0~2 ~_ 2.4.1.2 Inputs data-mask, vert-adj, skew, horiz-adj 2.4.1.3 Process adj-row = skew * data-mask[col-start]
adj-col = skew * data-mask[row-start] / 200 top-row = min (data-mask[row-start],data-mask[row-end]) +
adj-row + vert-adj bottom-row = max (data-mask[row-start],data-mask[row-end]) +
adj-row + vert-adj left = data-mask[col-start] + horiz-adj - adj-col right = data-mask[col-end] + horiz-adj - adj-col 2.4.1.4 Outputs left, right, top-row, bottom-row 2.4.2 Remove Vert Bars 2.4.2.1 Description This process removes the vertical bars that may be passing through the data field.
2.4.2.2 Inputs left, right, top-row, bottom-row,max-width 200(~01 2 2.4.2.3 Process set found-bar to true set current-column to left while found bar true and (current-column (left+8)) calculate the number of columns with ON bytes from top-row to bottom-row if the number of ON bytes > bar-size current-column = current-column+1 else found-bar = false set found-bar to true ; set current-column to right current-column = current-column - 1 while found bar true and (current-column (right-8)) calculate the number of columns with ON bytes from top-row to bottom-row if the number of ON bytes > bar-size current-column = current-column-1 else found-bar = false right = current-column 2.4.2.4 Outputs current-group (with the vertical bars for the field removed) 2.4.3 Grow Top 2.4.3.1 Description a-~( 2C~0~12 - ~ This process will grow the top of the mask until all of the data in the field has been captured or the growth of the mask reaches the 2D limit.
2.4.3.2 Inputs left, right, top-row, max-width, current-group 2.4.3.3 Process found = true displace = top-row - cur-top + 1 if (top-row 2D) growth = top-row - 1 else growth = 2D
while found and (i growth) i = i + 1 start-left = left for each column from left to right if (current-group~displace-i][column] is ON) tot-line = tot-line + 4;
else if tot-line > (line-length*4) for each column from start-left to (i-1) current-group[displace-i+l][column] =
current-group[displace-i+2][column]
r current-group[displace-i][column] =
current-group[displace-i+l][column]
current-group[displace-i-l][column] =
; current-group[displace-i-2][column~
ZC!OG0~2 start-left = i tot-line = 0 if tot-line > 0 for each column from start-left to right current-group[displace-i][column] =
current-group[displace-i+l][column]
if current-group[displace-i+l] has any ON bits near current-group[displace-i] ON bits found = true else found = false 2.4.3.4 Outputs current-group (with the top portion of the adaptive mask adjusted for the field adjusted) 2.4.4 Grow Bottom 2.4.4.1 Description .. ..
- This process will grow the bottom of the mask until all of the data in the field has been captured or the growth of the mask reaches the 2D
limit.
2.4.4.2 Inputs left, right, bottom-row, max-width '~,0 .
2~0C0~2 -2.4.4.3 Process.
found = true displace = bottom-row - cur-top + 1 if (max-grow+bottom-row vert-max) growth = vert-max-bottom-row - 1 else growth = 2D
while found and (i growth) i = i + 1 start-left = left for each column from left to right if (current-group[displace+i][column] is ON) tot-line = tot-line + 4;
else if tot-line > (line-length*4) for each column from start-left to (i-l) current-group[displace+i-l][column] =
current-group[displace+i-2][column]
current-grouptdisplace+i][column] =
current-group[displace+i-l][column3 current-group[displace+i+l][column] =
current-group[displace+i+2][column]
start-left = i tot-line = 0 if tot-line > 0 for each column from start-left to right current-group[displace-i][column] =
current-group[displace-i+l][column]
2~0(~012 if current-group[displace+i-1] has any ON bits near current-group[displace+i] ON bits found = true else found = false 2.4.4.4 Outputs current-group (with the bottom portion of the adaptive mask adjusted for the field adjusted) 2.4.5 Write Row - 2.4.5.1 Description This process will copy the extracted data to extracted image 2.4.5.2 Inputs left, right, top-row, bottom-row, current-group 2.4.5.3 Process copy all data in current-group contained in the rectangle defined by left, right, top-row, bottom-row 2.4.5.4 Outputs extracted-image ~ Q
200(~01Z
APPENDIX B
Process I
cluster-index : index of cluster-list that points to the start of each row, array of integer cluster-list : list of vertical clusters located in old-map array of (col, row-start, row-end), integers data-form-description: normalized list of lines located in old-map file of (row, col-start, col-end), integers line-list : list of horizontal lines located in old-map array of ~row-start, row-end, col-start, col-end), integers old-map : bi-level uncompressed scanned image of a form, file of bytes same-line : maximum vertical variation between lines in order for them to be considered on same row skew : angle of lines located in data form, floating point Process II
bottom-row : the last row of the perfect data for a data-mask to be extracted from current-group, integers current-group : The formatted array from old-map array [height of image] [width of image], bytes Z(!()(~(112 data-form-descriPtiOn : normalized list of lines located in old-map file of (row, col-start, col-end), integers data-mask : master-form-mask extracted-image : The fields extracted from old-map file of [max-width], bytes horiz-adj : difference between the horizontal location of lines in the data-form-description and the master-form-description, integer left : the column to begin extracting the current data-mask from current-group, integer master-form-description : normalized list of lines located in the master form file of (row, col-start, col-end), integers master-form-masks : list of mask locations to be extracted from the data-form array of (row-start, row-end, col-start, col-end), integers max-width : the length of the longest mask, integer old-map : bi-level uncompressed scanned image of a form, file of bytes right : the column to end extracting of the current master-form-mask from current-group, integers skew : angle of lines located in data form, floating point 3~
200(~ 2 top-row : the first row of the perfect data for a master-form-mask to be extracted from current-group, integers vert-adj : difference between the vertical location of lines in the data-form-description and the master-form-description, integers 2~0(~0~2 A~ C
V Z'~TH ~RAC~'~HR~ X;S R:~UN RU~1 RUN2 EDGE ~CAR~Y;TCARRY;B;~
C1~ R
C2] St (S[3;]~L~)/S'STRINCS2D X
C3] RU~'(pX) UNS~RING2D S
C4] RU~1~(1 0 SHF~ RUN)V ( 1 0 SHF~ RUN) C5] RUN2'(~RUN1)~R~(1 0 SHF~ RUN1)V ( 1 0 SHF~ RU~1) C6] RUN1~(~RUN)ARUN1 C7] ~DCE' THICK(RUN2~X) C8] TCdRRY'RUN1AXA(1 0 SHF~ EDCE) C9] Z~TCARRYv~CARRY RUN1AX~( 1 0 SH~ EDCE) Clo~ BR&CL.~+/+~CARRY
tll] B:+/+/~CA~RY
~12] ~CARRY~(~CARRYv(1 0 SHF~ ~CARRY))A(X~RUNvRUN1) ~13] ~CARRY'(~CARRYV( 1 0 SH~ BCARRY))~(XARUNvRUN1) C14] )~RX~ +/+/~CARRY)V(Bc+/+/~CARRY) ~15] 2~2v (~RUN1VRUN)AX
C16] Z:ZV(TCAR~YATHIC~ BCARRY~V(BCARRYATHIC~ ~CAR~Y) Notes In this routine the input arguments are X, the input bitmap, and LTH, a parameter that is larger than the width of any character pattern. Thus a horizontal string of black pixels having length greater than LTH can be assumed to belong to a horizo`ntal line.
TRACKTHRU first creates a subpattern of X contA i n ing all black pixels within 1 pixel of a long run. Black pixels outside this subpattern are assumed to belong to the pattern to be preserved. Black pixels inside the subpattern are assumed to belong to the character if they are within the upward and downward "shadow" of known character pixels. The shadow consists of pixels that belong to vertical black runs terminating on character pixels. To accommodate slanting of character parts that pass through the line, the algorithm also includés as character pixels those that are in anly one shadow, but ad~acent ito the other.
Z~OG~2 Line 2. STRINGS2D is a function that returns the starting (row, column) and length of each horizontal black run in X.
After execution of this statement array S contains these values for all runs having length greater than LTH.
Line 3. Unstring2D creates bit map RUN containing the long runs described in S. RUN is called the "long run image."
Line 4. RUN1 is a thickened version of RUN obtained by shifting up and down by 1 pixel and ORing the two shifted bitmaps.
Line 5. RUN2 is a bitmap that is black at a distance of 2 pixels from the long run image.
Line 6. RUN1 is a bitmap that is black at a distance of 1 pixel from the long run image.
Line 7. THICK is a function that adds 1 pixel of thickness to the upper and lower edges of an input pattern. Thus EDGE is black at pixels that either belong to or are vertically adjacent to black pixels of X that are at distance 2 from the long run image.
Line 8. TCARRY is black at pixels of X that are distance 1 above the long run image and that are directly beneath black pixels at distance 2.
Line 9. BCARRY is defined similarly to TCARRY, but for pixels below the long run image. Output Z is initially assigned the union of TCARRY and BCARRY.
Line 10-14. (Repetitive loop) In this loop all black pixels at distance 1 or less from the long run image, and which lie directly beneath black pixels in TCARRY, are ORed into TCARRY.
The same is done for BCARRY, which accumulates black pixels above it.
Line 15. Pixels of X that are within distance 1 or less from the long run image are ORed to the output Z.
ZC!OG0~2 ~_, Line 16. Pixels that are black in either TCARRY or BCARRY, and are within 1 pixel of the intersection of these two subpatterns of X, are ORed to the output.
i
separated from background printing to assure that the latter are not erroneously interpreted as data values. Data significance does not appear explicitly! but is stored in the computer and associated with the data values on the basis of position in the image. In some cases, forms are printed in a color invisible to the scanner to avoid a possibility of confusion. Data values are carefully positioned during printing, and the form precisely registered during sc~nn; ng. All these steps serve to guarantee that the data values are exactly where the reading or scanning equipment performs its extraction process.
In recent years, demand has grown for a capability to capture data from printed forms that do not meet OCR constraints.
Forms routinely used in government and commercial operations, such as birth and marriage certificates, are designed to be intelligible to the human eye and brain. While people are sophisticated processors of visual images, they also require that both attributes of a data element, the significance and the value, be present on the document. Thus background printing is provided to supply the meaning of each data field, and lines and boxes are imposed to make clear the association of data value and data significance. The crowded appearance of these "people forms", compared with OCR forms, is a necessary outcome of a requirement to pack a great deal of information into a limited space.
It is likewise difficult to enforce controls in the preparation of people forms. A birth or marriage certificate filled out with a typewriter is registered by eye, often with errors in translation and skew compared to the ideal orientation.
Data values may superimpose on the form background as a result.
The printing process itself is subject to mechanical slippage that may give the same effect. Finally, mechanical slippage and electronic noise occurring during the optical scanning process i ~ . .
2(~0~)12 present a further source of registration error. This is particularly true if economical general-purpose scanners are used. The net result of all these factors is that printinq of a given data value on people forms may be skewed, may overlap boundary lines separating data regions, and even when ideally positioned does not consistently appear in a fixed, predictable region in scanned images of different instances of the form.
These difficulties pose severe problems for automatic computer-implemented data extraction, rendering inapplicable the sort of processing used for OCR forms.
Summary of the Invention The invention is a computer-implemented method operable with conventional OCR scAnning equipment and software for the automatic extraction of data from printed forms.
A blank master form is scanned and its digital image stored.
Clusters of ON bits of the master form image are first recognized as part of a line and then connected to form lines. All of the lines in the master form image are then identified by row and column start position and column end position, thereby creating a master-form-description. The resulting image, which consists only of lines in the master form, can then be displayed. Regions or masks in the displayed image of master form lines are then created, each mask corresponding to a field where data would be located in a filled-in form. Each data mask is spaced from nearby lines by a predetermined data margin, referred to as D.
A filled-in or data form is then scanned and lines are also recognized and identified in a similar manner to create a data-form-description. The data-form-description is compared with the master-form-description by computing the horizontal and vertical offsets and skew of the two forms relative to one another. The created data masks, whose orientation with respect i ZOOGO~
to the master form has been previously determined, are then transposed into the data form image using the computed values of horizontal and vertical offsets and skew. In this manner, the data masks are correctly located on the data form so that the actual data values in the data form reside within the corresponding data masks. Routines are then implemented for detecting extraneous data intruding into the data masks and for --;
growing the masks, i.e. enlarging the masks to capture data which may extend beyond the perimeter of the masks. Thus, the data masks are adaptive in that they are grown if data does not lie entirely within the perimeter of the masks. During the mask growth routine, lines which are part of the background form are detected and removed by line removal algorithms.
Following the removal of extraneous data from the masks, the growth of the masks to capture data, and any subsequent line removal, the remaining data from the masks is extracted and transferred to a new file. The new file then contains only data comprising characters of the data values in the desired regions, which can then be operated on by conventional OCR software to identify the specific character values.
For a fuller understanding of the nature and advantages of the present invention reference should be made to the following detailed description taken in conjunction with the accompanying drawings.
Brief Description of the Drawings Fig. 1 is a schematic illustrating the relationship of the invention to conventional hardware and software for scanning an image and converting the image to stored characters through the use of OCR software;
Fig. 2 is a schematic illustrating the two portions of the data extraction software designated Phase I and Phase II ;
2000~12 Fig. 3 is a schematic of the functions present in the Phase I portion of the data extraction software;
Fig. 4 is a schematic of the functions present in the Phase II portion of the data extraction software;
Fig. 5 is a schematic of the Copy Masks function of the Phase II portion of the data extraction software;
Fig. 6 is an illustration of the manner in which horizontal and vertical adjustments between the master form and the data form are used to register the data form following scanning;
Fig. 7 is a depiction of an alternative solution to the problem of skew in the location of a data mask;
Fig. 8 is a graphical illustration of the Grow Bottom function for growing a data mask to capture data which crosses a mask perimeter;
Fig. 9 is a graphical depiction of a line removal process to capture character data which is intersected by a linej the line having been detected during the Grow Bottom function;
Fig. 10 is a representation of ON pixels for data intersected by a line;
- Fig. 11 is a representation of the ON pixels depicted as dashes following detection of the line pixels through the use of the algorithm in Appendix C;
Fig. 12 is a typical bland birth certificate master form;
and Fig. 13 is a typical filled-in birth certificate data form.
Description of the Preferred Embodiments Referring first to Fig. 1, the conventional hardware and software for converting characters from a printed form into stored di~ital data comprises a scanner, such as any 300 pixel bi-level scanner, and a central processing unit (CPU). The CPU
is connected to conventional peripheral devices such as a ;
i 2CO(~Ol;~
display, a data storage (e.g. a hard disk file~ and a printer.
The scanner converts the printed image from the form into bi-level digital data which is transferred to the CPU and converted into an array of O's and l's, representing white and black pixels on the scanned form. Conventional OCR software is then utilized to recognize characters from the array of digital data. The CPU then stores and/or displays the results of the OCR
software operation to any of its peripheral devices. Thus, the output from operation of the OCR software is character data, represented for example in ASCII format, which can be easily manipulated by the CPU and stored or transmitted to peripheral devices.
As illustrated in Fig. 1, in the present invention a blank master form is first scanned, converted to bi-level data, received by the CPU and stored. An example of a master form is shown in Fig. 12. A completed or filled in form, is then scanned, converted to bi-level data and stored by the CPU in an array referred to as an "old-map". An example of a filled-in or data form is shown in Fig. 13. The present invention is the method of data extraction performed by the data extraction software which allows just the pixels in the data regions or fields to be extracted from the old-map so that this data can then be received and operated on by the OCR software in the conventional manner. The functions of the data extraction software will be described with reference to the schematics of Figs. 2-5 and to the "pseudo-code" included in Appendix A, which is incorporated herein by reference. A list of definitions to assist in underst~n~ing the pseudo-code is listed in Appendix B
and incorporated herein by reference.
Referring now to Fig. 2, the data extraction software includes two phases, referred to as Phase I and Phase II. The operation of Phase I is identical for both the master form and i Z~O~Ol;~
the data form, while Phase II operates only on the data form.
Thus, the first step in the data extraction process is the scanning of the blank master form and, through the use of the functions in Phase I to be described below, the creation of a master-form-description. The master-form-description includes the identification of all lines by location and length and the skew of the form, i.e. its angular orientation relative to a fixed coordinate system of the scanning device.
Referring to Fig. 3, the description of the operation of Phase I will be described relative to its operation on a scanned data form, i.e. the old-map, although as just indicated, the operation of Phase I is identical for the blank master form.
This is because the function of Phase I is to locate all horizontal lines on the form, to generate a line-list and to identify the location and length of all of the lines on the form, be it the master form or the data form The old-map is a two-dimensional array of digital data corresponding to the pixels of the scanned data form image. Each pixel is either 1 or 0, or "ON" or "OFF", depending upon the presence or absence of either background lines or character data at the corresponding X-Y location. This digital data is analyzed with a Build Table routine (Appendix A, 1.1), which is a search for vertical runs of ON bits having a height less than an predetermined number, e.g. 3 in the preferred embodiment. The Build Table routine identifies all such vertical runs and creates a cluster-list for identifying the location of these vertical runs. The Build Table routine outputs a list of all vertical clusters located in the old-map and a cluster-index which points to the start of each row for each 3-pixel-high vertical cluster.
This information is then passed to the Build Lines routine (Appendix A, 1.2) whose function is to connect horizontally related clusters to detect hori~ontal lines. Thus, the Build i Z~0~012 Lines routine searches through all clusters in the cluster-list, and with knowledge of their location from the cluster-index, identifies horizontal lines. If there is an additional cluster within a predetermined minimum length away from the portion of the line just built, then that cluster is added to the existing line. The output of the Build Lines routine is a line-list of all of the lines in the data form. Each line in the line-list is identified by its row-start, row-end, column-start, and column-end.
The lines are then normalized or de-skewed by the Normalize Lines (Appendix A, 1.3) routine. Because the data form may not have been perfectly aligned with the scanning device, or the background lines on the form not perfectly oriented on the form, the lines which have been identified from the Build Lines routine may be skewed. This means that the pixels comprising a line may not necessarily begin and end on the same row. The Normalize Lines routine calculates the average skew of the lines in the line-list. The skew is defined as the ratio of the difference between the start and stop row of a line to the difference between the start and stop column of the line. After the skew has been calculated in this manner, the lines are normalized or de-skewed so that each line in the line-list can now be identified by a row-start, column-start and column-end. As a necessary output of the Normalize Lines routine, the skew of the data form has been calculated and is available for later use.
The Connect Lines routine (Appendix A, 1.4) then takes the list of Normalize Lines and determines if any of these lines are within a predetermined line tolerance of one another. If so, the lines are connected to make a single line, thereby reducing the number of lines in the line-list. As a result of the Connect Lines routine, the output is a data-form-description which is a listing of all horizontal lines in the data form, where each of 2~!0~0~2 those lines is identified by its row-start, its column-start and its column-end. A similar procedure may be performed to detect and identify vertical lines. Thus, at the completion of Phase I
the data extraction software has generated in digital form an exact replica of all background lines present on the filled-in data form.
As previously described, the Phase I process is first performed on a master form, i.e. a form identical to a filled-in data form, but without any character data present in the data fields. This is the first step in the process in the present invention and results in a master-form-description which is a list of all horizontal lines on the master form identified by row-start, column-start, and column-end. Vertical lines would be identified by row-start, column-start, and row-end.
Referring now to Paragraph 1.5 of Appendix A, the next step is the creation of the data-field-masks from the master form, utilizing the master-form-description output by Phase I when the master form was scanned. The function of the Create Masks step is to define a series of rectangular masks separated by portions of the lines identified in the line-list of the master form, wherein each mask corresponds to a desired region where a data value is located in the data form. While there are numerous techniques for creating the masks, the preferred technique is to generate the output of Phase I from the master form on a visual display (See Fig. 1). A user, through the use of a light pen, then points to those regions bounded by lines where it is desired to extract data from the data forms. For example, the light pen could identify the upper left and the lower right corners of each desired data mask, which would result in identifying the location and dimensions of each rectangular data region. Each data mask has a perimeter which is preferably spaced from the lines by a predetermined data margin, D. As a result of the Create Mask Æ~0~012 ....
step, the CPU will have stored the location and dimensions of the data regions by use of a mask-list, which identifies each mask by row-start, column-start and row-end, column-end. Since the normalized lines in the master form are presented on the visual display during this step it is not necessary to compensate for skew when the data masks are created on the master form.
At this point in the description of the invention, a master form has been scanned, a master-form-description has been defined through the use of Phase I (Fig. 3 and Appendix A, 1.1-1.4), the data masks have been created, a data form has been scanned and a data-form-description has been generated through the use of Phase I.
Phase II LS now performed on the data form. Referring now to Fig. 4 and Paragraph 2 of Appendix A, it is noted that Phase II has several functions. The first function is the Prepare Form routine (Appendix A, 2.1) which compares the data-form-description with the master-form-description in order to calculate the differences in terms of field locations between the two descriptions. The function of the Prepare Form routine can be better understood by reference to Fig. 6. Because the data form cannot always be perfectly aligned with the scanner, there will usually be both vertical and horizontal offsets of the data form lines from the master farm lines form each time a new data form is scanned. Thus, the Prepare Form routine compares each line in the line-list in the master-form-description with its corresponding line in the line-list in the data-form-description, and using the differences between the row-start, column-start and column-end values, determines the offset of the data form from the master form. This offset is essentially the horizontal adjustment (horiz-adj) and the vertical adjustment ~vert-adj) necessary to transform the master form into the data form. The output of the Prepare Form routine 2C!0U~)1Z
are the values or horiz-adj, vert-adj, and skew, the latter term being the rotation of the data form relative to the master form ~See Fig. 6).
In the Get Data Masks function (Fig. 4 and Appendix A, 2.2), the data masks created from the master form and which are identified by location and ~;me~cion~ are retrieved and placed in corresponding locations on the data form so as to create data masks located in the data form. As part of the Get Data Masks function, the width of the output array is calculated as the width of the longest data mask created from the master form plus a constant ~such as two bytes). This number becomes the width of the output image cont~i n; ng the extracted data.
The next function in Phase II is the Read Group function (Appendix A, 2.3), which transfers the old-map image from data storage into main memory, creating an array of records, referred to as "current-group", where each record has a width corresponding to the width of the scanned image. Thus, the Read GrouP function essentially converts the bi-level raw data into an array having a width and height corresponding to that of the scanned image. The output of the Read Group function is the current-group.
The Copy Masks function of Phase II comprises the five routines shown in Fig. 5. The Copy Masks function locates the data masks in a current-group to be extracted, grows or enlarges the masks, if necessary, to capture information which lies outside of the masks, removes any lines which intrude into the data masks, and finally writes the extracted data to the extracted image. As part of the Copy Masks function, the mask parameters are calculated in the Calculate Mask Parameters routine. This process essentially adjusts the data masks to compensate for skew, horizontal offset and vertical offset relative to the master form. The operation of the Calculate Mask 2QO~(~12 Parameters can best illustrated by reference to Fig. 6. The input to the routine are the data mask locations and ~;mension, and vert-adj, horiz-adj and skew which have been previously calculated in the Prepare Form function. Fig. 6 lllustrates how each of these parameters is used to transpose the location of the data masks, whose relationships relative to the master form are known, to their correct locations on the data form. As shown in Fig. 6, the skew rotates the masks by the degree of the skew.
However, while this may be an optimal solution in that the size of the masks remain the same as in the master form, the CPU cost in performing the rotation step and in growing the masks may be inefficient if the skew remains within a reasonable range. In an alternative technique, as shown in Fig. 7, the skew is used to alter the height of the mask in the data form, so that the resulting mask in the data form retains its rectangular shape.
As the skew increases, the chance of the predefined rectangle not fitting inside the field increases. To compensate for this, as the skew becomes greater the heights of the masks are reduced.
The output of the Calculate Mask Parameters routine is the location and dimension of the data masks within the data form, each data mask being defined by its start or left column, its end or right column, and its top-row and bottom-row.
After the Calculate Masks Parameters routine has been performed, if the data is perfectly located within the masks, then no additional data extraction routines are needed and the data can be extracted directly by the Write Row routine (Appendix A, 2.4.5). Perfect data is considered to be data that resides entirely within a predefined data mask. If there are no ON pixels along the parameter of the mask, the data is considered to be wholly inside the mask and completely perfect. In the Write Row function, all of the data in the current-group which is contained in the rectangles defined by left, right, top-row and i ~J
2QOl~OlZ
bottom-row, i.e. all of the data in the masks, is copied to a new file. The result is that the new file contains only extracted data without any extraneous lines of the background form. The data located in the new file can then be operated on by conventional OCR software to identify the data as specific character data. In addition, because each data mask is predefined as corresponding to a certain significance, i.e. name, age, etc., the data which has been extracted from the form and converted to character data by the OCR software is immediately identifiable with a predetermined siqnificance.
If, on the other hand, the data present in the data masks is not perfectly located within the perimeter of the data masks, then it is necessary to perform one of the additional functions of the Copy Masks function (Fig. 5).
The Remove Vert Bars (Appendix A, 2.4.2) removes vertically oriented bars which may be intrudinq into a data field, such as the portion of the signature in the "Middle" data field shown in Fig. 13. The Remove Vert Bars routine checks the two right-most and left-most bytes for the entire height of each field to determine if there is a run of connected ON bits that occurs that forms a vertical line. If such a line is not included since it assumed that data occurring beyond the vertical bar belongs to another field.
Referring now to Fig. 8, the Grow Bottom routine (Appendix A, 2.4.4) enlarges the bottom of the mask until all of the data in the field has been captured or until the growth of the mask reaches a predetermined limit, defined as 2D, the spacing between neighboring data masks. The mask is grown and during the growth lines are detected in the manner similar to the line detection technique described in Phase I. The mask is grown Z~O~OlZ
by searching a perimeter row for ON bits. If a violation ~the presence of ON bits) is encountered, the column to the left and right of the violation on the next row (one row up if growing the top of the mask, one row down if growing the bottom of the mask) is checked for ON bits. If either column has ON bits, then the process is repeated. This process terminates if either both bytes become OFF, or the maximum growth limit of 2D is reached.
Once any portion of a line is detected during mask growth the complete line is removed. A similar technique is used in the Grow Top (~ppendix A, 2.4.3) to enlarge the bottom of the mask to capture data. For example, the word "Twin" in Fig. 13 would be located entirely within the grown data mask and the line removed.
The line removal process is activated whenever the Grow Top or Grow Bottom routine detects a line in the ambiguous 2D area (Fig. 8). Line removal begins by following the path of the line and erasing the line as it is followed. The erasing of the line also erases anything that lies either one pixel above or one pixel below the line. This is to account for edge noise that is generated during printing and scanning. As the line is being erased, a check is made to determine whether there is any data that may be either touching or going through the line. If such data is found, the areas where the pixels may be from the line or from the data are not erased (see Fig. 9). The line detection and erasing algorithm has also been implemented in APL and is listed in Appendix C together with descriptions of the respective lines and routines. Figs. 10 and 11 illustrate a pattern processed by the algorithm of Appendix C wherein Fig. 10 is the input pattern where all black or ON pixels are denoted with an asterisk, and Fig. 11 is the result of the line detection algorithm wherein the lines have been detected and the asterisks replaced with dashes.
2~0(~012 While the preferred embodiments of the present invention have been illustrated in detail, it should be apparent that modifications and adaptations to those embodiments may occur to one skilled in the art without departing from the scope of the present invention as set forth in the following claims.`
APPENDIX A
1.0 Phase I
Phase I is responsible for building a description of the old-map (incoming image) that can be compared to a master form.
1.1 Build Table 1.1.1 Description This process searches through the first 500 rows of the old-map, looking for vertical runs that are less than 3 bits high. A table called cluster-list is built that lists all of these vertical runs.
1.1.2 Inputs old-map 1.1.3 Process while entire block not read for each byte in a row if byte > 0 increment cluster-size-counter else if cluster-size-counter 3 add cluster to cluster-list read next row sort cluster-list index cluster-list to create cluster-index SA9-89-006 ¦ ~
, ~ ~ ., 2Q0~012 -1.1.4 Outputs cluster-list, cluster-index 1.2 Build Lines 1.2.1 Description This process converts cluster-list to line-list by connecting horizontally related clusters.
1.2.2 Inputs cluster-list, cluster-index 1.2.3 Process for each cluster-list-item while cluster-list-items connected search for the next connected cluster-list-item if the next cluster-list-item is less than min-length away add cluster-list-item to line list else cluster-list-items no longer connected 1.2.4 Outputs line-list 1.3 Normalize Lines 1.3.1 Description The process converts line-list from a ~row-start, ~ row-end, column-start, column-end) orientation to a ~row, column-start, column-end) orientation 1.3.2 Inputs line-list 1.3.3 Process calculate the average row-start - row-end variation (row-deviation) calculate the average col-start - col-end variation (col-deviation) if col-deviation >0 skew = row-deviation t col-deviation for each line-list-item line-list-item[row-start] =
line-list-item[row-start] -skew*line-list-item[col-start], 1.3.4 Outputs line-list (normalized),skew 1.4 Connect Lines 1.4.1 Description SA9-89-006 ¦
200C~0~2 All lines that are within a line-toleranCe distance of each other are connected to become one line. These lines then become the data-form-description ; 1.4.2 line-list (normalized) 1.4.3 Process for each line if the line-list-item is within line-toler of another line make the two line-list-items a single line-list-item move the line-list-item to data-form-description 1.4.4 Outputs data-form-description 1.5 Create Masks (only for master form) 2.0 Phase II
Phase II is responsible for locating each data field (mask) on the data form, extracting character data from the masks and placing it in a new file. Phase II also removes any lines that may be interfering with the data field.
2.1 Prepare Form 2.1.1 Description 200(~012 Prepare Form compares the data-form-description to the master-form-description and calculates the offset in terms of field locations between the two descriptions.
2.1.2 Output master-form-description, data-form-description 2.1.3 Process for each data-form-description item horiz-adj = horiz-adj +
master-form-description[col-start] -data-form-description[col-start] +
master-form-description[col-end] -data-form-description[col-end]
vert-adj = vert-adj + master-form-description[row] -data-form-description[row]
2.1.4 Outputs horiz-adj, skew, vert-adj 2.2 Get Data Masks 2.2.1 Description Get Data Masks gets the master-form-masks and places them in the data form and also records the maximum allowable width of a mask.
2.2.2 Inputs i SA9-89-006 ~ ~
2~!0C0~2 master-form-masks 2.2.3 Process Place all master-form-masks in data-mask max-width = max length (data-mask item) 2.2.4 Outputs max-width, data-masks 2.3 Read Group 2.3.1 Description ` This process converts the image into current-group (an array of records, each map-width in size) 2.3.2 Inputs old-map 2.3.3 Process while not eof old-map place map-width bytes in current-group-item 2.3.4 Outputs current-group 2~0G012 2.4 Copy Masks Description This process locates the mask in the current-group to be extracted, grows the mask, if necessary, to capture information that lies outside of the original mask, removes any intruding lines, and writes the extracted to extracted-image Inputs horiz-adj, skew, vert-adj, max-width, data-mask, current-group, skew Process for each data-mask item Calculate Mask Parameters Remove Vert Bars Grow Top Grow Bottom Write Row Outputs extracted-image 2.4.1 Calculate Mask Parameters 2.4.1.1 Description This process adjusts the data mask to compensate for skew, horizontal offset, and verticai offset.
21~0(~0~2 ~_ 2.4.1.2 Inputs data-mask, vert-adj, skew, horiz-adj 2.4.1.3 Process adj-row = skew * data-mask[col-start]
adj-col = skew * data-mask[row-start] / 200 top-row = min (data-mask[row-start],data-mask[row-end]) +
adj-row + vert-adj bottom-row = max (data-mask[row-start],data-mask[row-end]) +
adj-row + vert-adj left = data-mask[col-start] + horiz-adj - adj-col right = data-mask[col-end] + horiz-adj - adj-col 2.4.1.4 Outputs left, right, top-row, bottom-row 2.4.2 Remove Vert Bars 2.4.2.1 Description This process removes the vertical bars that may be passing through the data field.
2.4.2.2 Inputs left, right, top-row, bottom-row,max-width 200(~01 2 2.4.2.3 Process set found-bar to true set current-column to left while found bar true and (current-column (left+8)) calculate the number of columns with ON bytes from top-row to bottom-row if the number of ON bytes > bar-size current-column = current-column+1 else found-bar = false set found-bar to true ; set current-column to right current-column = current-column - 1 while found bar true and (current-column (right-8)) calculate the number of columns with ON bytes from top-row to bottom-row if the number of ON bytes > bar-size current-column = current-column-1 else found-bar = false right = current-column 2.4.2.4 Outputs current-group (with the vertical bars for the field removed) 2.4.3 Grow Top 2.4.3.1 Description a-~( 2C~0~12 - ~ This process will grow the top of the mask until all of the data in the field has been captured or the growth of the mask reaches the 2D limit.
2.4.3.2 Inputs left, right, top-row, max-width, current-group 2.4.3.3 Process found = true displace = top-row - cur-top + 1 if (top-row 2D) growth = top-row - 1 else growth = 2D
while found and (i growth) i = i + 1 start-left = left for each column from left to right if (current-group~displace-i][column] is ON) tot-line = tot-line + 4;
else if tot-line > (line-length*4) for each column from start-left to (i-1) current-group[displace-i+l][column] =
current-group[displace-i+2][column]
r current-group[displace-i][column] =
current-group[displace-i+l][column]
current-group[displace-i-l][column] =
; current-group[displace-i-2][column~
ZC!OG0~2 start-left = i tot-line = 0 if tot-line > 0 for each column from start-left to right current-group[displace-i][column] =
current-group[displace-i+l][column]
if current-group[displace-i+l] has any ON bits near current-group[displace-i] ON bits found = true else found = false 2.4.3.4 Outputs current-group (with the top portion of the adaptive mask adjusted for the field adjusted) 2.4.4 Grow Bottom 2.4.4.1 Description .. ..
- This process will grow the bottom of the mask until all of the data in the field has been captured or the growth of the mask reaches the 2D
limit.
2.4.4.2 Inputs left, right, bottom-row, max-width '~,0 .
2~0C0~2 -2.4.4.3 Process.
found = true displace = bottom-row - cur-top + 1 if (max-grow+bottom-row vert-max) growth = vert-max-bottom-row - 1 else growth = 2D
while found and (i growth) i = i + 1 start-left = left for each column from left to right if (current-group[displace+i][column] is ON) tot-line = tot-line + 4;
else if tot-line > (line-length*4) for each column from start-left to (i-l) current-group[displace+i-l][column] =
current-group[displace+i-2][column]
current-grouptdisplace+i][column] =
current-group[displace+i-l][column3 current-group[displace+i+l][column] =
current-group[displace+i+2][column]
start-left = i tot-line = 0 if tot-line > 0 for each column from start-left to right current-group[displace-i][column] =
current-group[displace-i+l][column]
2~0(~012 if current-group[displace+i-1] has any ON bits near current-group[displace+i] ON bits found = true else found = false 2.4.4.4 Outputs current-group (with the bottom portion of the adaptive mask adjusted for the field adjusted) 2.4.5 Write Row - 2.4.5.1 Description This process will copy the extracted data to extracted image 2.4.5.2 Inputs left, right, top-row, bottom-row, current-group 2.4.5.3 Process copy all data in current-group contained in the rectangle defined by left, right, top-row, bottom-row 2.4.5.4 Outputs extracted-image ~ Q
200(~01Z
APPENDIX B
Process I
cluster-index : index of cluster-list that points to the start of each row, array of integer cluster-list : list of vertical clusters located in old-map array of (col, row-start, row-end), integers data-form-description: normalized list of lines located in old-map file of (row, col-start, col-end), integers line-list : list of horizontal lines located in old-map array of ~row-start, row-end, col-start, col-end), integers old-map : bi-level uncompressed scanned image of a form, file of bytes same-line : maximum vertical variation between lines in order for them to be considered on same row skew : angle of lines located in data form, floating point Process II
bottom-row : the last row of the perfect data for a data-mask to be extracted from current-group, integers current-group : The formatted array from old-map array [height of image] [width of image], bytes Z(!()(~(112 data-form-descriPtiOn : normalized list of lines located in old-map file of (row, col-start, col-end), integers data-mask : master-form-mask extracted-image : The fields extracted from old-map file of [max-width], bytes horiz-adj : difference between the horizontal location of lines in the data-form-description and the master-form-description, integer left : the column to begin extracting the current data-mask from current-group, integer master-form-description : normalized list of lines located in the master form file of (row, col-start, col-end), integers master-form-masks : list of mask locations to be extracted from the data-form array of (row-start, row-end, col-start, col-end), integers max-width : the length of the longest mask, integer old-map : bi-level uncompressed scanned image of a form, file of bytes right : the column to end extracting of the current master-form-mask from current-group, integers skew : angle of lines located in data form, floating point 3~
200(~ 2 top-row : the first row of the perfect data for a master-form-mask to be extracted from current-group, integers vert-adj : difference between the vertical location of lines in the data-form-description and the master-form-description, integers 2~0(~0~2 A~ C
V Z'~TH ~RAC~'~HR~ X;S R:~UN RU~1 RUN2 EDGE ~CAR~Y;TCARRY;B;~
C1~ R
C2] St (S[3;]~L~)/S'STRINCS2D X
C3] RU~'(pX) UNS~RING2D S
C4] RU~1~(1 0 SHF~ RUN)V ( 1 0 SHF~ RUN) C5] RUN2'(~RUN1)~R~(1 0 SHF~ RUN1)V ( 1 0 SHF~ RU~1) C6] RUN1~(~RUN)ARUN1 C7] ~DCE' THICK(RUN2~X) C8] TCdRRY'RUN1AXA(1 0 SHF~ EDCE) C9] Z~TCARRYv~CARRY RUN1AX~( 1 0 SH~ EDCE) Clo~ BR&CL.~+/+~CARRY
tll] B:+/+/~CA~RY
~12] ~CARRY~(~CARRYv(1 0 SHF~ ~CARRY))A(X~RUNvRUN1) ~13] ~CARRY'(~CARRYV( 1 0 SH~ BCARRY))~(XARUNvRUN1) C14] )~RX~ +/+/~CARRY)V(Bc+/+/~CARRY) ~15] 2~2v (~RUN1VRUN)AX
C16] Z:ZV(TCAR~YATHIC~ BCARRY~V(BCARRYATHIC~ ~CAR~Y) Notes In this routine the input arguments are X, the input bitmap, and LTH, a parameter that is larger than the width of any character pattern. Thus a horizontal string of black pixels having length greater than LTH can be assumed to belong to a horizo`ntal line.
TRACKTHRU first creates a subpattern of X contA i n ing all black pixels within 1 pixel of a long run. Black pixels outside this subpattern are assumed to belong to the pattern to be preserved. Black pixels inside the subpattern are assumed to belong to the character if they are within the upward and downward "shadow" of known character pixels. The shadow consists of pixels that belong to vertical black runs terminating on character pixels. To accommodate slanting of character parts that pass through the line, the algorithm also includés as character pixels those that are in anly one shadow, but ad~acent ito the other.
Z~OG~2 Line 2. STRINGS2D is a function that returns the starting (row, column) and length of each horizontal black run in X.
After execution of this statement array S contains these values for all runs having length greater than LTH.
Line 3. Unstring2D creates bit map RUN containing the long runs described in S. RUN is called the "long run image."
Line 4. RUN1 is a thickened version of RUN obtained by shifting up and down by 1 pixel and ORing the two shifted bitmaps.
Line 5. RUN2 is a bitmap that is black at a distance of 2 pixels from the long run image.
Line 6. RUN1 is a bitmap that is black at a distance of 1 pixel from the long run image.
Line 7. THICK is a function that adds 1 pixel of thickness to the upper and lower edges of an input pattern. Thus EDGE is black at pixels that either belong to or are vertically adjacent to black pixels of X that are at distance 2 from the long run image.
Line 8. TCARRY is black at pixels of X that are distance 1 above the long run image and that are directly beneath black pixels at distance 2.
Line 9. BCARRY is defined similarly to TCARRY, but for pixels below the long run image. Output Z is initially assigned the union of TCARRY and BCARRY.
Line 10-14. (Repetitive loop) In this loop all black pixels at distance 1 or less from the long run image, and which lie directly beneath black pixels in TCARRY, are ORed into TCARRY.
The same is done for BCARRY, which accumulates black pixels above it.
Line 15. Pixels of X that are within distance 1 or less from the long run image are ORed to the output Z.
ZC!OG0~2 ~_, Line 16. Pixels that are black in either TCARRY or BCARRY, and are within 1 pixel of the intersection of these two subpatterns of X, are ORed to the output.
i
Claims (10)
1. A machine-implemented method for extracting character data from printed forms of the type having generally straight lines and data regions separated by the lines, the method utilizing a processor coupled to an image scanning device and a data storage means, the method comprising the steps of:
(a) scanning a blank form so as to create a digital image of a first array of pixels;
(b) identifying pixels in the array which form generally straight lines of connected line pixels;
(c) creating data masks in the array at locations separated by the identified lines, the data masks corresponding to the data regions in the printed form;
(d) scanning a filled-in form so as to create a digital image of a second array of pixels;
(e) repeating step (b) for the second array;
(f) calculating the offset of the lines in the second array from the lines in the first array;
(g) locating the data masks created in the first array into the second array by use of the calculated offset; and (h) extracting data corresponding to character pixels from the data masks in the second array.
(a) scanning a blank form so as to create a digital image of a first array of pixels;
(b) identifying pixels in the array which form generally straight lines of connected line pixels;
(c) creating data masks in the array at locations separated by the identified lines, the data masks corresponding to the data regions in the printed form;
(d) scanning a filled-in form so as to create a digital image of a second array of pixels;
(e) repeating step (b) for the second array;
(f) calculating the offset of the lines in the second array from the lines in the first array;
(g) locating the data masks created in the first array into the second array by use of the calculated offset; and (h) extracting data corresponding to character pixels from the data masks in the second array.
2. The method according to claim 1 wherein the step of identifying pixels connected to form lines further comprises the steps of:
(a) selecting clusters of vertically-connected pixels, the clusters having a vertical height less than a predetermined number of pixels; and (b) connecting horizontally adjacent clusters.
(a) selecting clusters of vertically-connected pixels, the clusters having a vertical height less than a predetermined number of pixels; and (b) connecting horizontally adjacent clusters.
3. The method according to claim 1 wherein the step of identifying pixels connected to form lines comprises the step of determining the rotational skew of the connected line pixels.
4. The method according to claim 1 wherein the method also utilizes a display coupled to the processor and wherein the step of creating data masks comprises the steps of:
(a) displaying the identified lines on the display, and:
(b) storing in the data storage means the array locations of data masks selected by a user in response to the displayed lines.
(a) displaying the identified lines on the display, and:
(b) storing in the data storage means the array locations of data masks selected by a user in response to the displayed lines.
5. The method according to claim 1 wherein the step of calculating the offset further comprises the steps of:
(a) calculating the horizontal adjustment of the lines in the second array relative to the lines in the first array;
(b) calculating the vertical adjustment of the lines in the second array relative to the lines in the first array; and (c) calculating the rotational skew of the lines in the second array.
(a) calculating the horizontal adjustment of the lines in the second array relative to the lines in the first array;
(b) calculating the vertical adjustment of the lines in the second array relative to the lines in the first array; and (c) calculating the rotational skew of the lines in the second array.
6. The method according to claim 5 wherein the step of locating the data masks in the second array comprises rotating the data masks by the degree of calculated skew.
7. The method according to claim 5 wherein the step of locating the data masks in the second array comprises adjusting the vertical height of the data masks by the degree of calculated skew.
8. The method according to claim 1 including the steps of, prior to extracting the data from the data masks:
(a) determining if any connected character pixels extend across the perimeter of a data mask; and (b) in response to said determination, enlarging the data mask until the perimeter-crossing connected character pixels are located within the enlarged data mask.
(a) determining if any connected character pixels extend across the perimeter of a data mask; and (b) in response to said determination, enlarging the data mask until the perimeter-crossing connected character pixels are located within the enlarged data mask.
9. The method according to claim 8 wherein the step of enlarging the data mask comprises the step of growing the mask only in the region of the mask perimeter crossed by the connected character pixels.
10. The method according to claim 8 wherein the step of enlarging the data mask further comprises the steps of:
(a) detecting if any connected character pixels extend across a line; and (b) selectively removing those line pixels which are not located in the portion of the line crossed by the connected character pixels.
(a) detecting if any connected character pixels extend across a line; and (b) selectively removing those line pixels which are not located in the portion of the line crossed by the connected character pixels.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US07/305,828 | 1989-02-02 | ||
| US07/305,828 US5140650A (en) | 1989-02-02 | 1989-02-02 | Computer-implemented method for automatic extraction of data from printed forms |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2000012A1 CA2000012A1 (en) | 1990-08-02 |
| CA2000012C true CA2000012C (en) | 1995-03-21 |
Family
ID=23182538
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002000012A Expired - Fee Related CA2000012C (en) | 1989-02-02 | 1989-10-02 | Computer-implemented method for automatic extraction of data from printed forms |
Country Status (3)
| Country | Link |
|---|---|
| US (1) | US5140650A (en) |
| JP (1) | JPH02231692A (en) |
| CA (1) | CA2000012C (en) |
Families Citing this family (101)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0767136B2 (en) * | 1989-09-21 | 1995-07-19 | 大日本スクリーン製造株式会社 | Image processing device |
| US5402504A (en) * | 1989-12-08 | 1995-03-28 | Xerox Corporation | Segmentation of text styles |
| CA2056865C (en) * | 1990-12-11 | 1999-01-05 | Barbara Clapp | Interactive computerized document assembly system |
| US5388167A (en) * | 1991-03-12 | 1995-02-07 | Hitachi, Ltd. | Document image processing system and document image processing method |
| US6683697B1 (en) | 1991-03-20 | 2004-01-27 | Millenium L.P. | Information processing methodology |
| US5258855A (en) * | 1991-03-20 | 1993-11-02 | System X, L. P. | Information processing methodology |
| JPH04343190A (en) * | 1991-05-21 | 1992-11-30 | Hitachi Ltd | Character data input system |
| US5991440A (en) * | 1991-06-21 | 1999-11-23 | Canon Kabushiki Kaisha | Method and apparatus for recognizing input handwritten characters associated with title information |
| US5293429A (en) * | 1991-08-06 | 1994-03-08 | Ricoh Company, Ltd. | System and method for automatically classifying heterogeneous business forms |
| US5568566A (en) * | 1991-11-25 | 1996-10-22 | Kabushiki Kaisha Toshiba | Drawing processing apparatus |
| US5666549A (en) * | 1992-03-10 | 1997-09-09 | Hitachi, Ltd. | Method and system for processing a document transmitted via facsimile in an initially input form stored in a knowledge base |
| US5418865A (en) * | 1992-03-20 | 1995-05-23 | Xerox Corporation | Mark sensing on a form |
| US5317646A (en) * | 1992-03-24 | 1994-05-31 | Xerox Corporation | Automated method for creating templates in a forms recognition and processing system |
| US5251273A (en) * | 1992-04-15 | 1993-10-05 | International Business Machines Corporation | Data processing system and method for sequentially repairing character recognition errors for scanned images of document forms |
| US5987149A (en) * | 1992-07-08 | 1999-11-16 | Uniscore Incorporated | Method for scoring and control of scoring open-ended assessments using scorers in diverse locations |
| US5672060A (en) * | 1992-07-08 | 1997-09-30 | Meadowbrook Industries, Ltd. | Apparatus and method for scoring nonobjective assessment materials through the application and use of captured images |
| JPH06110996A (en) * | 1992-08-21 | 1994-04-22 | Xerox Corp | Automatic image formation by combination of text image and format image |
| US5321767A (en) * | 1992-08-28 | 1994-06-14 | Kabushiki Kaisha Komatsu Seisakusho | Method of forming a mask in image processing operation |
| JPH06131495A (en) * | 1992-10-15 | 1994-05-13 | Fujitsu Ltd | Image information extraction system |
| US5416849A (en) * | 1992-10-21 | 1995-05-16 | International Business Machines Corporation | Data processing system and method for field extraction of scanned images of document forms |
| US5437554A (en) | 1993-02-05 | 1995-08-01 | National Computer Systems, Inc. | System for providing performance feedback to test resolvers |
| JP3258122B2 (en) * | 1993-03-31 | 2002-02-18 | 株式会社東芝 | Image processing device |
| US6411972B1 (en) | 1993-04-08 | 2002-06-25 | International Business Machines Corporation | Method and apparatus for filling in forms by segments using a scanner and a printer |
| JP3442466B2 (en) * | 1993-04-12 | 2003-09-02 | 株式会社リコー | Image processing apparatus and image processing method |
| US5428694A (en) * | 1993-10-14 | 1995-06-27 | International Business Machines Corporation | Data processing system and method for forms definition, recognition and verification of scanned images of document forms |
| US5542007A (en) * | 1993-10-27 | 1996-07-30 | International Business Machines Corporation | Form dropout compression method which handles form white-out and writing in shaded and white-out areas of the form |
| US5521991A (en) * | 1993-10-29 | 1996-05-28 | International Business Machines Corporation | Method and system for fast forms recognition of document form images |
| DE69432114T2 (en) * | 1993-11-24 | 2003-10-30 | Canon Kk | System for identifying and processing forms |
| CA2134255C (en) * | 1993-12-09 | 1999-07-13 | Hans Peter Graf | Dropped-form document image compression |
| US5668897A (en) * | 1994-03-15 | 1997-09-16 | Stolfo; Salvatore J. | Method and apparatus for imaging, image processing and data compression merge/purge techniques for document image databases |
| US5748780A (en) * | 1994-04-07 | 1998-05-05 | Stolfo; Salvatore J. | Method and apparatus for imaging, image processing and data compression |
| JP3547025B2 (en) * | 1994-08-23 | 2004-07-28 | 株式会社リコー | Information extraction method |
| US5568550A (en) * | 1994-10-05 | 1996-10-22 | Shmuel Ur | Method and system for identifying documents generated by an unauthorized software copy |
| US6072871A (en) * | 1994-10-05 | 2000-06-06 | Leon H. Charney | Method and system for identifying the source of documents generated by software and documents generated thereby |
| JPH08207380A (en) * | 1994-11-25 | 1996-08-13 | Xerox Corp | Method and device for automatic entry in original form |
| US5692206A (en) * | 1994-11-30 | 1997-11-25 | Taco Bell Corporation | Method and apparatus for automating the generation of a legal document |
| US5822454A (en) * | 1995-04-10 | 1998-10-13 | Rebus Technology, Inc. | System and method for automatic page registration and automatic zone detection during forms processing |
| JPH08305778A (en) * | 1995-04-21 | 1996-11-22 | Xerox Corp | Method for investigation of existence mark created by user |
| JP4114959B2 (en) * | 1995-06-20 | 2008-07-09 | キヤノン株式会社 | Image processing method and apparatus |
| US6811086B1 (en) * | 1995-07-20 | 2004-11-02 | Fujitsu Limited | Stand for pivotably mounting an optical reading device |
| US5815595A (en) * | 1995-12-29 | 1998-09-29 | Seiko Epson Corporation | Method and apparatus for identifying text fields and checkboxes in digitized images |
| US6201894B1 (en) * | 1996-01-23 | 2001-03-13 | Canon Kabushiki Kaisha | Method and apparatus for extracting ruled lines or region surrounding ruled lines |
| US5692073A (en) * | 1996-05-03 | 1997-11-25 | Xerox Corporation | Formless forms and paper web using a reference-based mark extraction technique |
| US6327387B1 (en) | 1996-12-27 | 2001-12-04 | Fujitsu Limited | Apparatus and method for extracting management information from image |
| SE511242C2 (en) * | 1997-04-01 | 1999-08-30 | Readsoft Ab | Method and apparatus for automatic data capture of forms |
| JP3558493B2 (en) * | 1997-06-10 | 2004-08-25 | 富士通株式会社 | Paper alignment device, paper alignment method, and computer-readable recording medium recording paper alignment program |
| US6023534A (en) * | 1997-08-04 | 2000-02-08 | Xerox Corporation | Method of extracting image data from an area generated with a halftone pattern |
| EP1013070B1 (en) * | 1997-08-29 | 2003-05-07 | Océ Printing Systems GmbH | Method and device for electronic archiving of a computer data flow |
| US6325420B1 (en) | 1998-08-17 | 2001-12-04 | Inspectron Corporation | Method for embedding non-intrusive encoded data in printed matter and system for reading same |
| US6507662B1 (en) * | 1998-09-11 | 2003-01-14 | Quid Technologies Llc | Method and system for biometric recognition based on electric and/or magnetic properties |
| US6741737B1 (en) | 1999-03-18 | 2004-05-25 | Choicepoint Asset Company | System and method for the secure data entry from document images |
| US6400845B1 (en) | 1999-04-23 | 2002-06-04 | Computer Services, Inc. | System and method for data extraction from digital images |
| JP4073120B2 (en) * | 1999-06-03 | 2008-04-09 | 富士通株式会社 | Information processing device |
| JP3708768B2 (en) | 1999-11-22 | 2005-10-19 | シャープ株式会社 | Reader and data processing system |
| US6845366B1 (en) * | 1999-11-22 | 2005-01-18 | Ncr Corporation | Method of processing a check and an apparatus therefor |
| JP4078009B2 (en) * | 2000-02-28 | 2008-04-23 | 東芝ソリューション株式会社 | CHARACTERISTIC RECORDING AREA DETECTION DEVICE FOR FORM, CHARACTER RECORDING AREA DETECTION METHOD FOR FORM, STORAGE MEDIUM, AND FORM FORMAT CREATION DEVICE |
| US7149347B1 (en) | 2000-03-02 | 2006-12-12 | Science Applications International Corporation | Machine learning of document templates for data extraction |
| US6961482B2 (en) * | 2001-03-05 | 2005-11-01 | Ncs Pearson, Inc. | System for archiving electronic images of test question responses |
| US6675133B2 (en) | 2001-03-05 | 2004-01-06 | Ncs Pearsons, Inc. | Pre-data-collection applications test processing system |
| US6751351B2 (en) | 2001-03-05 | 2004-06-15 | Nsc Pearson, Inc. | Test question response verification system |
| US6810232B2 (en) | 2001-03-05 | 2004-10-26 | Ncs Pearson, Inc. | Test processing workflow tracking system |
| US7818219B2 (en) | 2001-12-27 | 2010-10-19 | American Hungarian Technologies Inc. | Electronic realty and transaction system and method therein |
| US7561734B1 (en) | 2002-03-02 | 2009-07-14 | Science Applications International Corporation | Machine learning of document templates for data extraction |
| US20070198910A1 (en) * | 2002-03-26 | 2007-08-23 | Aatrix Software, Inc. | Method and apparatus for creating and filing forms |
| US7171615B2 (en) | 2002-03-26 | 2007-01-30 | Aatrix Software, Inc. | Method and apparatus for creating and filing forms |
| US7142728B2 (en) * | 2002-05-17 | 2006-11-28 | Science Applications International Corporation | Method and system for extracting information from a document |
| US8484551B2 (en) * | 2002-09-05 | 2013-07-09 | Adobe Systems Incorporated | Creating input fields in electronic documents |
| US20040064472A1 (en) * | 2002-09-27 | 2004-04-01 | Oetringer Eugen H. | Method and system for information management |
| US20040098664A1 (en) * | 2002-11-04 | 2004-05-20 | Adelman Derek A. | Document processing based on a digital document image input with a confirmatory receipt output |
| US6993818B2 (en) * | 2003-03-17 | 2006-02-07 | Memx, Inc. | Multi-fixture assembly of cutting tools |
| US7305612B2 (en) * | 2003-03-31 | 2007-12-04 | Siemens Corporate Research, Inc. | Systems and methods for automatic form segmentation for raster-based passive electronic documents |
| WO2004107257A1 (en) * | 2003-05-30 | 2004-12-09 | Fujitsu Limited | Forms processing program, method, and device |
| DE10342594B4 (en) * | 2003-09-15 | 2005-09-15 | Océ Document Technologies GmbH | Method and system for collecting data from a plurality of machine readable documents |
| US20050067482A1 (en) * | 2003-09-26 | 2005-03-31 | Wu Daniel Huong-Yu | System and method for data capture and management |
| US7870504B1 (en) * | 2003-10-01 | 2011-01-11 | TestPlant Inc. | Method for monitoring a graphical user interface on a second computer display from a first computer |
| US7487438B1 (en) * | 2005-03-08 | 2009-02-03 | Pegasus Imaging Corporation | Method and apparatus for recognizing a digitized form, extracting information from a filled-in form, and generating a corrected filled-in form |
| US7630519B2 (en) * | 2005-09-29 | 2009-12-08 | Xerox Corporation | Systems and methods for printing on pre-printed forms |
| US20070094296A1 (en) * | 2005-10-25 | 2007-04-26 | Peters Richard C Iii | Document management system for vehicle sales |
| JP2007241356A (en) * | 2006-03-06 | 2007-09-20 | Fuji Xerox Co Ltd | Image processor and image processing program |
| US7711191B2 (en) * | 2006-12-21 | 2010-05-04 | Michael John Kosek | Electronic transaction processing server with automated transaction evaluation |
| JP2008276487A (en) * | 2007-04-27 | 2008-11-13 | Fuji Xerox Co Ltd | Document processing program, document processor, and document processing system |
| US20100277772A1 (en) * | 2007-08-15 | 2010-11-04 | I. R. I. S. | Method for managing sets of digitally acquired images and method for separation and identification of digitally acquired documents |
| US8108764B2 (en) * | 2007-10-03 | 2012-01-31 | Esker, Inc. | Document recognition using static and variable strings to create a document signature |
| US8094976B2 (en) * | 2007-10-03 | 2012-01-10 | Esker, Inc. | One-screen reconciliation of business document image data, optical character recognition extracted data, and enterprise resource planning data |
| US20090097769A1 (en) * | 2007-10-16 | 2009-04-16 | Sytech Solutions, Inc. | Systems and methods for securely processing form data |
| US20090213404A1 (en) * | 2008-02-27 | 2009-08-27 | Ching-Feng Cheng | Method and apparatus for populating a form with information |
| US20100017742A1 (en) * | 2008-07-20 | 2010-01-21 | Rhodes Gary J | Form input skip-field software tool |
| JP2010033360A (en) * | 2008-07-29 | 2010-02-12 | Canon Inc | Information processor, job processing method, storage medium and program |
| JP5387193B2 (en) * | 2009-07-16 | 2014-01-15 | 富士ゼロックス株式会社 | Image processing system, image processing apparatus, and program |
| US8903201B2 (en) | 2011-12-12 | 2014-12-02 | Symbol Technologies, Inc. | Method and apparatus for enhanced document capture |
| US8850305B1 (en) * | 2011-12-20 | 2014-09-30 | Google Inc. | Automatic detection and manipulation of calls to action in web pages |
| US9536141B2 (en) * | 2012-06-29 | 2017-01-03 | Palo Alto Research Center Incorporated | System and method for forms recognition by synthesizing corrected localization of data fields |
| US20140304579A1 (en) * | 2013-03-15 | 2014-10-09 | SnapDoc | Understanding Interconnected Documents |
| US10049096B2 (en) | 2015-06-19 | 2018-08-14 | Infosys Limited | System and method of template creation for a data extraction tool |
| CN106572389A (en) * | 2015-10-08 | 2017-04-19 | 小米科技有限责任公司 | Method and apparatus for adjusting display image |
| US10068132B2 (en) | 2016-05-25 | 2018-09-04 | Ebay Inc. | Document optical character recognition |
| US10095925B1 (en) | 2017-12-18 | 2018-10-09 | Capital One Services, Llc | Recognizing text in image data |
| US20200152008A1 (en) * | 2018-11-08 | 2020-05-14 | Meter Image Capturing, LLC | Progressive jackpot monitoring systems and methods |
| US11257006B1 (en) * | 2018-11-20 | 2022-02-22 | Amazon Technologies, Inc. | Auto-annotation techniques for text localization |
| US11861302B2 (en) | 2019-02-04 | 2024-01-02 | Aatrix Software, Inc. | AUF XML specification compiler |
| CN112541332B (en) * | 2020-12-08 | 2023-06-23 | 北京百度网讯科技有限公司 | Form information extraction method and device, electronic equipment and storage medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE3113555A1 (en) * | 1981-04-03 | 1982-10-21 | Siemens AG, 1000 Berlin und 8000 München | METHOD FOR AUTOMATICALLY RECOGNIZING WHITE BLOCKS AND TEXT, GRAPHIC AND / OR GRAY IMAGE AREAS ON PRINT ORIGINALS |
| JPS5887652A (en) * | 1981-11-19 | 1983-05-25 | Ricoh Co Ltd | Picture processor |
| US4741045A (en) * | 1983-09-23 | 1988-04-26 | Dest Corporation | Optical character isolation system, apparatus and method |
| JPS60120485A (en) * | 1983-12-02 | 1985-06-27 | Hitachi Ltd | Pattern recognizing system |
| JPH07107694B2 (en) * | 1984-08-31 | 1995-11-15 | 株式会社日立製作所 | Document processor |
| JPH0750483B2 (en) * | 1985-05-22 | 1995-05-31 | 株式会社日立製作所 | How to store additional information about document images |
-
1989
- 1989-02-02 US US07/305,828 patent/US5140650A/en not_active Expired - Fee Related
- 1989-10-02 CA CA002000012A patent/CA2000012C/en not_active Expired - Fee Related
-
1990
- 1990-02-01 JP JP2020554A patent/JPH02231692A/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| US5140650A (en) | 1992-08-18 |
| CA2000012A1 (en) | 1990-08-02 |
| JPH02231692A (en) | 1990-09-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2000012C (en) | Computer-implemented method for automatic extraction of data from printed forms | |
| CN109657665B (en) | Invoice batch automatic identification system based on deep learning | |
| US5822454A (en) | System and method for automatic page registration and automatic zone detection during forms processing | |
| JP2930612B2 (en) | Image forming device | |
| JP3278471B2 (en) | Area division method | |
| US5418865A (en) | Mark sensing on a form | |
| US6400845B1 (en) | System and method for data extraction from digital images | |
| US5272764A (en) | Detection of highlighted regions | |
| US6778703B1 (en) | Form recognition using reference areas | |
| KR100251600B1 (en) | Image processing method and digital computer | |
| EP0629078B1 (en) | Apparatus for processing and reproducing image information | |
| US20020085243A1 (en) | Document processing apparatus and method | |
| JPH05250514A (en) | Method for detecting handwritten instruction image while using morpheme technique | |
| JP3594228B2 (en) | Frame erasing device, frame erasing method, and authoring device | |
| CN110598566A (en) | Image processing method, device, terminal and computer readable storage medium | |
| US7965293B2 (en) | Image processing device, image processing method, and image processing program for reconstructing data | |
| CN113901952A (en) | Print form and handwritten form separated character recognition method based on deep learning | |
| US6958755B1 (en) | Personalized computer fonts | |
| JPS6325391B2 (en) | ||
| JP3223878B2 (en) | Character string collating device, method and recording medium | |
| US20090097699A1 (en) | Information processing apparatus and method, computer program and computer-readable recording medium | |
| JPH1145307A (en) | Character extraction system and recording medium | |
| JP3484274B2 (en) | Area segmentation method for similar documents | |
| JP3199033B2 (en) | Optical character reading method and optical character reading device | |
| CN113780293A (en) | Interface character recognition method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |