CA2575859A1 - Systems and methods for automated diagnosis and grading of tissue images - Google Patents

Abstract

Systems and methods are provided for automated diagnosis and grading of tissue images based on morphometric data extracted from the images by a computer. The morphometric data may include image-level morphometric data such as fractal dimension data, fractal code data, wavelet data, and/or color channel histogram data. The morphometric data may also include object-level morphometric data such as color, structural, and/or textural properties of segmented image objects (e.g., stroma, nuclei, red blood cells, etc.).

Description

SYSTEMS AND METHODS FOR AUTOMATED
DIAGNOSIS AND GRADING OF TISSUE IMAGES
Field of the Invention [0001] Embodiments of the invention relate to systems and methods for automated diagnosis and grading of tissue images. For example, in one embodiment, the invention provides systems and methods for extracting morphometric data from an image of prostate tissue and evaluating the data to determine whether the tissue is cancerous or non-cancerous.
In another embodiment, the invention evaluates the morphometric data to determine a grade of cancer aggressiveness (e.g., a Gleason grade) for the prostate tissue.

1o Background of the Invention [0002] Prostate cancer is the most prevalent form of cancer and the second most common cause of death among men in the United States. One of the most reliable methods for prostate cancer diagnosis is the examination of the glandular architecture of a prostate tissue specimen under a microscope by a pathologist. For example, FIG. 1(a) shows an image of normal prostate tissue, which consists of gland units surrounded by fibromuscular tissue called "stroma" that holds the gland units together. Each gland unit is made of a row of epithelial cells located around a circularly shaped "hole" in the tissue, named the lumen.
When cancer occurs, epithelial cells replicate in an uncontrolled way, thereby disrupting the normal arrangement of the gland units. This causes the lumens to become filled with epithelial cells and the stroma to virtually disappear. FIGS. 1(b) and 1(c) show images of cancerous prostate tissue, where the cancer depicted in FIG. 1(c) is more aggressive than the cancer depicted in FIG. 1(b). The images in FIGS. 1(a)-(c) were originally color images of Hematoxylin-and-Eosin ("H&E") stained tissue cores from a tissue microarray (TMA), but have been depicted in FIG. 1 in grayscale for reproducibility. The inclusion of these images in this Background section is for informational purposes only, and is not an admission of prior art. On the contrary, these images were generated by an image processing system described in commonly-owned U.S. Patent Application No. 11/080,360, filed March 14, 2005.

[0003] Pathologists typically quantify cancer aggressiveness through tissue "grading."
Tissue grading involves having the pathologist look at an image of prostate tissue and, based on that pathologist's experience and expertise, assigning a grade to the tissue correlating to the aggressiveness of the cancer. Tissue grading is valuable to physicians in several ways.
First, it aids with identifying the extent of the disease. Second, cancer grade correlates well 1o with patient survival. Finally, knowledge of cancer grade helps with determining an appropriate treatment option for a patient (e.g., selecting a more aggressive treatment for a patient with a higher cancer grade).

[0004] The most common method for grading prostate tissue is the Gleason grading system, shown in Figure 2. In this system, prostate tissue is classified into five grades numbered 1 through 5. The grade increases with increasing malignancy level and cancer aggressiveness.
Particularly, the Gleason grade characterizes tumor differentiation, which is the degree of tumor resemblance to normal tissue. Grade 1 corresponds to well-differentiated tissue, which is tissue with the highest degree of resemblance to normal tissue. Thus, patients with Grade 1 prostate tissue typically have a high chance of survival. On the other hand, grade 5 corresponds to poorly differentiated tissue, and thus patients with grade 5 tissue typically have a lower chance of survival. For reference, a pathologist has determined that the prostate tissue shown in FIG. 1(b) has a Gleason grade of 2 and the prostate tissue in FIG. 1(c) has a Gleason grade of 5. In addition to a patient's Gleason grade, pathologists often also assign a Gleason score to a tissue section. A patient's overall Gleason Score is the sum of the two most predominant Gleason grades present in a prostate tissue section (which can come from analysis of several tissue portions corresponding to different parts of the tissue section). For example, if the most predominant Gleason grade is 3 and the second most common grade is 4, then the Gleason score is 3+ 4 = 7. Thus, the Gleason score can vary from 2 to 10.

[0005] Although the above-described cancer diagnosis and Gleason grading by a pathologist are widely considered to be reliable, these are subjective processes. Particularly, physicians rely heavily on their own expertise and training when viewing tissues samples in order to determine whether the tissue is cancerous or non-cancerous and/or to determine Gleason grade(s) for the tissue. Thus, different pathologists viewing the same tissue samples may come up with conflicting interpretations.

[0006] Various studies have focused on computer-assisted systems for cancer detection in prostate tissue and, more particularly, in images of prostate tissue. The following discussion is presented for informational purposes only and is not an admission of prior art. One machine vision system uses shape and Haralick texture features to identify stroma, normal, and cancerous regions in the image [1]. The Haralick texture features are calculated by first constructing a so-called co-occurrence matrix, and then calculating 13 second-order statistics.
The system is reported to have achieved an accuracy of 79.3% in classifying image regions.
lo Another system has been developed for classifying blocks of tissue images into the stroma, benign prostatic hyperplasia, prostatic intraepithelial neoplasia (PIN), and prostate cancer classes [2]. The images are captured in multiple spectral bands which refer to light wavelength ranges. For each image block, texture features and the area occupied by nuclei and lumens relative to the area of the image block are computed. These features are then used to classify the image block using principal component analysis and the linear Gaussian classifier. A classification accuracy of 94% using a cross-validation method has been reported. However, it should be noted that both of these systems involve iinage segmentation prior to extracting image features, which can introduce segmentation error.

[0007] Additionally, several methods have been proposed for computer-assisted Gleason grading of prostate cancer. In one method, statistical and structural features are extracted from the spatial distribution of epithelial nuclei over the image area [3]. A
hybrid neural network/Gaussian statistical classifier is used to distinguish moderately and poorly differentiated histological samples. An accuracy of 77% on a set of 130 independent test images was reported. Notably, no algorithm for segmenting the epithelial nuclei was described, and thus this stage of analysis was most likely performed manually.
In another method, the power spectrum is used to represent the texture characteristics of tissue images, and principal component analysis is applied to the power spectrum for feature space dimensionality reduction [4]. A nearest-neighbor (NN) classifier is used to assign the input image to Gleason grade 1, 2, 3, or combined grades of 4 and 5. An accuracy of 90% on a set of 20 independent test images was reported. Still another method has proposed the use of features derived from spanning trees connecting cell nuclei across the tumor image to represent tissue images belonging to each grade [5]. No quantitative results on the performance of this method have been reported. In another method, features based on co-occurrence matrices, wavelet packets, and multiwavelets are extracted and a k-NN classifier is used to classify each image into grade 2, 3, 4, or 5 [6]. An accuracy of 97% using the leave-one-out (LOO) procedure for error estimation was reported. However, the same leave-one-out procedure was used for both training and testing. This could potentially have introduced positive bias into the reported results.

[0008] Thus it is seen that traditional methods for computer-assisted cancer diagnosis and grading have achieved varying results. Accordingly, it would be desirable to provide improved systems and methods for automated cancer diagnosis and grading of tissue images.
1o Summary of the Invention [00091 Embodiments of the invention relate to systems and methods for automated diagnosis and grading of tissue images. For example, in one embodiment, the invention provides systems and methods for extracting image-level morphometric data from an image of prostate tissue and evaluating the image-level morphometric data alone to determine whether the tissue is cancerous or non-cancerous. In another embodiment, the invention determines a Gleason grade for the prostate tissue by evaluating image-level morphometric data from the tissue image. As used herein, morphometric data is any data extracted from the image by a computer that characterizes a color, structural, and/or textural property of the image. "Image-level" morphometric data is any morphometric data that can be extracted from unsegmented images. Contrast this to "object-level" morphometric data, which can only be extracted from a tissue image after the image has been segmented into histological object(s) (e.g., objects such as stroma, cytoplasm, epithelial nuclei, stroma nuclei, lumen, and red blood cells and/or sub-objects such as folded or unfolded chromatin texture). Examples of image-level morphometric data include fractal dimension data, fractal code data, wavelet data, and color channel histogram data.
[0010] In an aspect of the present invention, systems and methods are provided for extracting image-level morphometric data from a tissue image and using this data to diagnose and/or grade cancerous tissue. When the morphometric data is from a tissue image with a known classification with respect to a medical condition (e.g., cancer/non-cancer or Gleason grade), the morphometric data may be subject to feature selection and/or classifier training in order to build a model to predict the medical condition. When the morphometric data is for a tissue image with an unknown classification, the morphometric data may be evaluated by a model that classifies the medical condition in the tissue.
[0011] In one embodiment, systems and methods are provided for extracting fractal dimension data from a tissue image. The tissue image is thresholded at one or more Nb fixed, equally-spaced thresholds in one or more of its red, green, and blue (RGB) color channels to obtain one or more binary images. For example, thresholding the image at all Nb thresholds in each of the RGB channels results in 3Nb binary images. A fractal dimension value y k is computed through the use of the box counting algorithm for each binary image.
The resulting measurements of these fractal dimension values may be used as image-level morphometric features for classification.
[0012) In another embodiment, systems and methods are provided for extracting fractal code data from a tissue image. The tissue image is partitioned into non-overlapping blocks of two different sizes: range blocks B, and domain blocks A, . Generally, for each range block Bj, a search is performed for a domain block and a transformation T(described below) such that the transformed domain block T.A; is a close approximation of B.. All domain blocks are examined to find the closest approximation to the given range block.
Characteristics of the transformation may then be used as image-level morphometric features for classification.
[0013] More particularly, for each range block Bj, the construction of the fractal transformation (code) is performed. This involves averaging and downsampling the domain block A; with a transformation denoted as T,.d , and the resulting block is denoted as A. = Td A; . Then, a search is performed for a transformation T' such that a mean square error (MSE) between the domain block and range block is minimized. T,.' can be represented as a composition of two transforms:

T,=' =T,. oTs "
where Ts is a pixel shuffling transform (stage704) and T,. denotes an affine transform on the pixel gray level values (stage 706). The pixel shuffling transform can be one of the following: (1) identity; (2-5) reflections about the mid-vertical and mid-horizontal axes, and first and second diagonals; and (6-8) rotations about the block center by + 90 ,- 90 , and +180 .
[0014] T,. may be determined as follows: In one embodiment, for a fixed T,.s , the optimal T that minimizes the MSE is determined as follows. Let X and Y be image blocks A. and Bj reordered as:

X'4111 "i12 A'N] and Y=LB111 B.i12 ... BiNNJ.

where this reduces the problem of finding an optimal T" to a matrix calculus problem of finding a minimum. Particularly, the squared Euclidean distance DE between the transformed block T,. X and Y is considered as a matrix function of the transform T. :

DE(T X,Y)= T,. X-Y 2.

Differentiating this with respect to T and setting the derivative to zero, this becomes:
T"XXr -YX' = 0 where (.)' denotes the transpose. Assuming that (XX')-' exists, the solution to this latter equation is given by:

T = YX+ = YX' (XX' )-' where (.)+ is the Moore-Penrose pseudoinverse. The best combination of T,.s and T for the range block B, is found by repeating the above process for all possible T,.s and taking the pair of T;s and T,. that minimizes the MSE. In another embodiment, T may be selected from a plurality of predetermined T,. based on whether the range block under examination is a shade, edge, or midrange block (where the predetermined T,. is not necessarily the optimal T ).
[0015] Each range image block Bj is then characterized using one or more of the parameters of the fractal code constructed for it, and these parameter(s) may be used as image-level morphometric feature(s) for classification. Let I = argmin MSE, , where i i indexes domain blocks. The parameters may include MSE,j , shift and scaling parameters of the affine transform T,,' , shuffling transform T, , and the Euclidean distance between A, and B j in the image plane.

[0016] In still another embodiment, systems and methods are provided for extracting wavelet data from a tissue image. A wavelet representation of the tissue image may be generated using a 4-level, dyadic transform and the symlet 4 wavelet filter.
The variance a 2 of the coefficients is computed for each of the subbands, where a 2 is given by:

62 ZE x2 _ 'MN YExijlZ , MN i j i j 1o where x, , i=1,...,M , j=1,...,N , denotes a wavelet coefficient in a subband of size MN .
The resulting measurements of these variances may be used as image-level morphometric features for classification.
[0017] In another embodiment, systems and methods are provided for extracting color channel histogram data from a tissue image. The tissue image is separated into its red, green, and blue (RGB) color channels. One or more of the color channel images is separated into bins representing various levels of color intensity. A pixel count is computed for one or more of these bins. These pixel count(s) may be used as image-level morphometric features for classification.
[0018] In another aspect of the present invention, systems and methods are provided for generating a predictive model based on image-level morphometric data extracted from tissue images with known classification with respect to a medical condition (e.g., cancer/non-cancer or Gleason grade). Generating a predictive model may include using an analytical tool to train a neural network or other learning machine with image-level morphometric data from a plurality of tissue images with known classification. In one embodiment, the training data includes image-level morphometric data consisting of fractal dimension data, fractal code data, wavelet data, color channel histogram data, or a combination thereof.
The analytical tool may determine an affect of the features on the ability of an associated model to predict the medical condition. Features that increase the predictive power of the model may be included in the final model, whereas features that do not increase (e.g., maintain or decrease) the predictive power may be removed from consideration.
[0019] In another aspect of the present invention, systems and methods are provided that use a 2-stage procedure for tissue image analysis. In a first stage, a first set of morphometric data from a tissue image with unknown classification is evaluated by a model that predicts whether the corresponding tissue is cancerous or non-cancerous. In a second stage, if the model classifies the tissue as cancerous tissue, a second set of morphometric data from the tissue image is evaluated by a model that assigns a cancer grade to the tissue (e.g., a Gleason grade for prostate tissue). The second set of morphometric feature(s) may include at least 1o one morphometric feature not included in the first set of morphometric feature(s). For example, in the first stage, one or more image-level morphometric features from the tissue image (e.g., fractal dimension, fractal code, wavelet, and/or color channel histogram feature(s)) may be evaluated by the model that predicts whether the tissue is cancerous or non-cancerous. In the second stage, one or more object-level features, alone or in combination with image-level features, may be evaluated by the model that assigns a cancer grade to the tissue. In an embodiment, the second set of morphometric features may be extracted from the tissue image only if the tissue is classified by the model as being cancerous. In this way, the resources of physicians, other individuals and/or automated processing equipment (e.g., equipment for extracting morphometric data from tissue images) may be conserved. In another embodiment, the first and second sets of morphometric features may be extracted from the tissue image at the same time. This two-level procedure may be used to, for example, identify in a whole tissue section (or other suitable size tissue section), portions of the section that are cancerous, and then to further analyze those sections.
Brief Description of the Drawings [0020] For a better understanding of the present invention, reference is made to the following description, taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout, and in which:
[0021] FIGS. l(a)-l(c) show grayscale images of normal and cancerous prostate tissue;
[0022] FIG. 2 shows the Gleason grading system for characterizing the level of aggressiveness of cancer in prostate tissue;

[0023] FIG. 3 is a flowchart of illustrative stages involved in generating a model for automated cancer diagnosis or grading of tissue images based on morphometric data from one or more tissue images with known classiflcation;
[0024] FIG. 4 is a flowchart of illustrative stages involved in extracting fractal dimension data from a tissue image;
[0025] FIG. 5 is a diagram that illustrates the decomposition of a color image into red, green, and blue (RGB) images, with all color images depicted in grayscale for reproducibility, and from RGB images into binary images for extraction of fractal dimension data;
l0 [0026] FIG. 6 is a flowchart of illustrative stages involved in extracting fractal code data from a tissue image;
[0027] FIG. 7 is a diagram of illustrative substages involved in the stage of constructing a fractal transformation (FIG. 6);
[0028] FIG. 8 is a flowchart of illustrative stages involved in extracting wavelet transform data from a tissue image;
[0029] FIG. 9 is a flowchart of illustrative stages involved in extracting color channel histogram data from a tissue image;
[0030] FIG. 10(a) shows a histogram of the green channel for the tissue images depicted in FIG. l;
[0031] FIG. 10(b) shows histograms of the difference between the values in the red and blue channels for the images depicted in FIG. 1;
[0032] FIG. 11 is a flowchart of illustrative stages involved in segmenting a tissue image into objects for extraction of object-level morphometric data;
[0033] FIG. 12 is a flowchart of illustrative stages involved in screening tissue through the use of a two-level procedure;
[0034] FIGS. 13(a) and 13(b) are block diagrams of systems that use a predictive model to make a medical decision;
[0035] FIG. 13(c) is a block diagram of a system for generating a predictive model;
[0036] FIG. 14 shows the results of study in which models capable of automated prostate cancer diagnosis were generated based on morphometric data from tissue images with known classification;

[0037] FIG. 15 is a scatterplot of samples for the best-performing two features for the study of FIG. 14, where the solid line depicts the decision boundary;
[0038] FIG. 16 shows the results of study in which models capable of automated Gleason grading were generated based on morphometric data from tissue images with known classification; and [0039] FIG. 17 is a scatterplot of samples for the best-performing two features for the study of FIG. 16, where the solid line depicts the decision boundary.

Detailed Description of the Invention [0040] Embodiments of the invention relate to systems and methods for automated diagnosis and grading of tissue images. The diagnosis and/or grading may be based on any suitable image-level morphometric data extracted from the tissue images including fractal dimension data, fractal code data, wavelet data, and/or color channel histogram data. As used herein, "data" of a particular type (e.g., fractal dimension or wavelet) may include one or more features of that type. The diagnosis and/or grade may be used by physicians or other individuals to, for example, select an appropriate course of treatment for a patient. The following description focuses primarily on the application of the present invention to cancer diagnosis and Gleason grading of images of prostate tissue. However, the teachings provided herein are also applicable to, for example, the diagnosis, prognosis, and/or grading of other medical conditions in tissue images such as other types of disease (e.g., epithelial and mixed-neoplasms including breast, colon, lung, bladder, liver, pancreas, renal cell, and soft tissue).
[0041] In an aspect of the present invention, an analytical tool may be provided that determines correlations between morphometric data from one or more tissue images and a medical condition. The correlated features may form a model that can be used to predict the condition. For example, based on image-level morphometric data from tissue images for which it is known whether the corresponding tissue is cancerous or non-cancerous, an analytical tool may generate a model that can predict whether tissue depicted in a tissue image for a new patient is cancerous or non-cancerous. As another example, based on image-level morphometric data from tissue images for which cancer grades for the corresponding tissue are known (e.g., the grades being determined manually by a pathologist), an analytical tool may generate a model that can predict a cancer grade for tissue depicted in a tissue image for a new patient. In both examples, the correlated features may be extracted from the new tissue image and evaluated by the model. In these contexts, the predictive model may determine the current status of the tissue. In other contexts, predictive models may be generated that can make determinations about the future status of the associated patient (e.g., whether and/or when the patient is likely to experience, at some future point in time, disease such as cancer occurrence or recurrence).
[0042] FIG. 3 is a flowchart 300 of illustrative stages involved in generating a predictive model for automated diagnosis or grading of tissue images based on morphometric data from one or more tissue images. At stage 302, one or more electronic (digital) images may be generated from a corresponding one or more tissue specimens with known classification such lo as, for example, tissue microarray (TMA) whole tissue sections or cores stained with Hematoxylin and Eosin (H&E). For example, for automated cancer diagnosis, it may have been previously determined by a pathologist for each of the images whether the corresponding tissue is cancerous or non-cancerous. For automated cancer grading, cancer grades may have been previously determined by a pathologist for the tissue depicted in the images. An image processing tool may be provided that includes a light microscope for capturing tissue images at 20X magnification using a SPOT Insight QE Color Digital Camera (KAII2000) and for producing images with 1600 x 1200 pixels. The images may be stored as images with 24 bits per pixel in TIFF format. Such an image processing tool is described in above-incorporated U.S. Patent Application No. 11/080,360, and the images shown in FIGS.
1(a)-1(c) were generated by such an image processing tool. Such equipment is only illustrative and any other suitable hardware, software, or combination thereof may be used to capture the tissue images from tissue specimens without departing from the scope of the present invention.
[0043] At stage 304, the tissue image(s) may be pre-processed in order to remove background noise. Stage 304 may be optionally included when, for example, the tissue images are TMA cores. For example, referring to the images of TMA cores shown in FIGS.
1(a)-1(c), the transparent regions at the corners of the tissue images correspond to background noise in that these regions are not occupied by the tissue. Thus, these transparent regions may be identified and removed from further analysis by, for example, an intensity thresholding and convex hull operation.
[0044] At stage 306, a second pre-preprocessing stage may be performed in order to match histograms of the image(s) to that of a reference image [7]. This aims to alleviate color variations in the images due to varying staining and illumination conditions.
Stage 306 may be optionally included when, for example, the image is to be segmented into histological objects such as stroma, lumen, and red blood cells for extraction and analysis of object-level features (described below), because color variations between images potentially can affect segmentation accuracy. Segmenting of the image(s) may also optionally be performed at stage 306. Note that while histogram matching benefits image segmentation, it may distort image color information. Thus, morphometric color features are preferably extracted from the original (i.e., not segmented) images. For example, morphometric color features may be extracted from segmented images by superimposing the segmented objects on the original image.
[0045] At stage 308, morphometric data including one or more morphometric features is extracted from the tissue image(s). For example, the morphometric data may include image-level data extracted from the image such as fractal dimension data, fractal code data, statistical data computed from the wavelet transform of the image, and color channel histogram data. Image-level morphometric data is preferably obtained from the original (i.e., not segmented) image to, for example, prevent the influx of segmentation error into the measurements of the image-level morphometric features. As another example, the morphometric data may include-object level data, which must be extracted from a segmented version of the original image. Segmenting of tissue images into histological objects of various classes (e.g., lumen, stroma, nuclei) and extracting object-level data from the segmented images are described in above-incorporated U.S. Patent Application No.
11/080,360. For example, the object-level data may include statistics such as the mean intensity, area, and the standard deviation of the intensity for each of the segmented object classes. The object-level features may also include features describing the spatial relationships between histological objects. In one embodiment, extraction of object-level morphometric data may be performed by an image processing tool that includes the commercially-available Definiens Cellenger Developer Studio (v. 4.0) adapted to perform the segmenting and classifying of, for example, the various pathological objects described above and to measure various morphometric features of these objects. An example of such an image processing tool is Aureon Laboratories' MAGICTM system. Additional details regarding the Definiens Cellenger product are described in [8]. Any suitable combination of image-level data and/or object-level data may be extracted from the image at stage 308.

Additional details regarding extracting morphometric data from tissue images is described below in connection with FIGS. 4, 6-9 and 11.
[0046] At stage 310, feature selection and classification are performed on the one or more morphometric features extracted at stage 308. Feature selection is the process of choosing, from a set of available features, at least a subset of the features that are most relevant to the classification task. Classification is the process of assigning to a sample (e.g., a dataset of morphometric features for a patient) a label from a finite set of labels. For example, the morphometric features extracted at stage 308 may be selected for inclusion in a model that predicts whether tissue is cancerous or non-cancerous or that assigns a particular cancer grade (e.g., a Gleason grade) to the tissue. Features that increase the discriminative power of the model may be included in the final model, whereas features that do not increase (e.g., or decrease) the discriminative power may be removed from consideration. The final model may be used to evaluate morphometric data from a tissue image for a new patient in order to make a classification decision for the patient.
[0047] Feature selection has two benefits. It reduces the computational cost of recognition (e.g., reducing the number of features that must be computed for a new patient) and it usually improves classification accuracy by mitigating the effect of the "curse of dimensionality" [9]
[ 10] [ 11 ]. All feature selection algorithms are characterized by two traits [9]. The first is the optimality criterion J with respect to which the quality of a feature subset is evaluated. For example, in the illustrative Studies 1 and 2 described below, J was selected as the classification accuracy estimated using the "leave one out" (LOO) procedure.
The second characteristic is a search strategy used to find a locally or globally optimal feature combination. In "exhaustive search" (ES), the globally optimal subset of features is found by evaluating all possible feature combinations. ES is often computationally infeasible. Instead, greedy strategies are used that add or remove features from the feature subset in a stepwise fashion. For example, the "sequential forward search" (SFS) algorithm described in the below Studies 1 and 2 is a greedy strategy [9]. The use of LOO, ES, and SFS
for feature selection is only illustrative and any other procedure for selecting features for inclusion in a predictive model may be used without departing from the scope of the present invention. For example, in another embodiment, a sequential back search algorithm may be used for feature selection.
[0048] Statistical classifiers fall into two general categories of parametric and non-parametric methods. Parametric methods rely on the assumption that the functional form of class-conditional distributions are known (e.g., Gaussian), wliereas non-parametric methods make minimal assumptions about the form of the distributions. The choice of the classifier type depends on the sample size and optionally prior knowledge about the distribution of the features. In the illustrative Studies 1 and 2 described below, parametric methods were used.
Thus, by assuming that the class-conditional distributions are Gaussian, the Bayes decision rule f(X) is a quadratic function of the feature vector X described in [9]:

f(X)=(X-Ito)TEDI(X- o)-(X-FtITE'(X- i)+lnl~0 I-lnPO (1) IIiI I'i where P, ; , E; are the prior probability, mean, and covariance matrix of class i, i 0, 1, respectively, and 1. 1 denotes a matrix determinant. For identical covariance matrices E = E, = E,(1) simplifies to a linear function given by:

.f (X) = (Ftt - P-o )T y-1X+ ~ (!~o E-tllo - F~ i Z-I N-i ) - ln P (2) i The classifiers given by (1) and (2) are referred to as quadratic and linear classifiers, respectively. The Bayes decision rule is as follows: decide class 0 (e.g., non-cancerous image), if f(X) < 0; decide class 1(e.g., cancerous image), if f(X) > 0; and pick a class at random if f(X) = 0. The use of parametric methods for classification is only illustrative.

Non-parametric methods or other methods for classification may be used without departing from the scope of the present invention.
[0049] For a given classifier and a dataset to be classified (e.g., a dataset including morphometric data from images for a plurality of patients with known classification), the sample classification accuracy PQ is estimated as PQ = n, /n , where n, denotes the number of correctly classified samples and ra is the total number of samples in the dataset. Two related performance measures often used in detection tasks are sensitivity and specificity. Let class 0 and class 1 correspond to, for example, the absence and presence of disease (or a lower or higher degree of disease) and let n, be the number of patients in class i, i =
0, 1. Moreover, let nj , i = 0,1, j = 0,1, denote the number of patients in class i classified into class j.

Sample sensitivity PSE is defined as PSE = nõ / n, and sample specificity PSP
is defined as PsP = jzoo / jzo = Generally, predictive models with higher sensitivity and specificity make more accurate classification decisions such as, for example, whether tissue is cancerous or non-cancerous and/or whether the tissue should be assigned a particular grade.
[0050] FIGS. 4, 6-9 and 11 are flowcharts of illustrative stages involved in extracting morphometric data from tissue image(s). For example, one or more of the stages shown in FIGS. 4, 6-9 and 11 may be substages of stage 308 (FIG. 3), which relates to extracting morphometric data from one or more known tissue images in order to generate a predictive model. As another example, one or more of the stages shown in FIGS. 4, 6-9 and 11 may be used to extract morphometric data from a tissue image for a new patient with an unknown classification for a determination of the classification by a final predictive model. However, when morphometric data is extracted from an image for a new patient, preferably only the morphometric feature(s) evaluated by the predictive model (and therefore previously determined to correlate with a medical condition) are extracted from the image (although, in another embodiment, all features may be extracted and only the relevant features considered).
[0051] FIG. 4 is a flowchart 400 of illustrative stages involved in extracting fractal dimension data from a tissue image. Generally, fractal geometry provides a tool for quantitative description of complex, irregularly shaped objects in pathological images [11].
A common fractal property of an object is its fractal dimension. The fractal dimension provides a quantitative measure of the space-filling capacity of an object.
For example, the fractal dimension of a straight line is the same as its topological dimension (i.e., 1) since it can only fill a one-dimensional sub-space. For a more complex curve, the fractal dimension can be fractional and therefore different from its topological dimension. A
more detailed description of the fractal dimension and traditional methods of its calculation through the use of the box counting algorithm is given in [ 11 ][ 12]. Traditionally, the fractal dimension has been measured with respect to the outline of a histological object in a segmented image that has not been decomposed into its respective color channels.
[0052] In contrast, at stage 402, the tissue image is thresholded at Nb fixed, equally-spaced thresholds in each of its red, green, and blue (RGB) color channels, resulting in 3Nb binary images. For example, when each of the RGB channels has a fnll-range intensity of 0-250, each of these color channels may be separated into Nb bins equally-spaced apart in intensity level intervals of 250/ Nb intensity levels (e.g., bin 1 including all pixels of intensity 0 through 250/ Nb ). For each bin, the pixels from the original image that have been assigned to that bin are rendered white and all other pixels are rendered black. FIG. 5 is a diagram that illustrates the decomposition of an input color image into RGB images, with all color images depicted in grayscale for reproducibility, and from RGB images into binary images, in accordance with an embodiment of the present invention.

[0053] At stage 404, a fractal dimension value yk is computed through the use of the box counting algorithm for each binary image Bk obtained by applying the k-th threshold to the i-th channel. At stage 406, the values y k, i=1, 2, 3, k=1,..., Nb are put together to form a 3Nb -dimensional feature vector y=171 Y'Nb , Yi YNb ~Yi Nb ] This vector provides a fractal 1o representation of the original input image. When the fractal dimension vector is for a patient with a known classification, each of the entries in the vector may be tested by the above-described feature selection and classification procedures. Accordingly, it may be determined whether these fractal dimension features correlate with a medical condition.
For example, in the illustrative Studies 1 and 2 described below for automated cancer diagnosis and Gleason grading, Nb =12 , which yielded a feature vector y including 36 fractal dimension features.
In other embodiments of the present invention, only a subset of the 3Nb features may be computed and/or included in the feature vector used for feature selection and classification (e.g., based on prior knowledge that one or more of the 3Nb features are more important than others). Moreover, when the vector is for a new patient, the vector preferably includes only the fractal dimension feature(s) evaluated by the final predictive model.
[0054] FIG. 6 is a flowchart 600 of illustrative stages involved in extracting fractal code data from a tissue image. Fractal code features are motivated by the notions of fractal image compression [12] [13]. Fractal codes are discrete contractive affine transformations mapping image blocks to image blocks. They capture the self-similarity of image blocks at two different scales. Generally, the aim is to measure the extent to which, and manner in which, smaller portions of the image can be represented by larger portions of the image using affine transform. Visual examination of the images in FIG. 1 reveals that the texture of prostate tissue appears to be finer and more isotropic at a microscale (and therefore potentially more fractal in nature) with increasing level of cancer aggressiveness. In the illustrative Studies 1 3o and 2 described herein, the contractive constraint on the pixel gray level affine transformation was relaxed. Fractal codes have inspired descriptors for texture characterization and image retrieval [14][15], but fractal codes can also be for characterization of medical images.
Additionally, different methods of calculating fractal codes may be used.
While some embodiments of the invention are related to predicting a Gleason grade based on fractal code data, the inventors also envision that one or more fractal code measurements of self similarity could potentially replace Gleason grading altogether.
[0055] At stage 602, the image is partitioned into non-overlapping blocks of two different sizes: range blocks Bj of size N x N pixels and domain blocks A1 of size 2N x 2N. Here, the factor of 2 was chosen arbitrarily. Any other suitable one or more factors may be used without departing from the scope of the present invention. For every range block Bj , a lo search is performed for a domain block A; and a transformation T,.
(described below) such that the transformed domain block T,.A, is a close approximation of Bj. All domain blocks are examined to find the closest approximation to the given range block. The mean-square error (MSE) given by:

II
T A; - Bj 112 2 (3) MSE,~ I2 where T,.A. and Bj are the transformed domain and range blocks, respectively, and the MSE
is used as the measure of closeness between the transformed domain block and the range block.
[0056] Then for each range block Bj, the construction of the fractal transformation (code) is performed at stage 604. Stage 604 includes the three substages shown in FIG. 7. At stage 702, the domain block A; is averaged in 2 x 2 blocks and downsampled by a factor of 2.

This transformation is denoted as Td , and the resulting block is denoted as A, = Td 1 . It should be noted that A, is of the same size as Bj . Then, a search is performed for a transformation T,.' such that (3) is minimized. Following [13], T' can be represented as a composition of two transforms:

7.r _Ta 7,s where Ts is a pixel shuffling transform (stage704) and T denotes an affine transform on the pixel gray level values (stage 706). The pixel shuffling transform can be one of the following, as set forth in [13]: (1) identity; (2-5) reflections about the mid-vertical and mid-horizontal axes, and first and second diagonals; and (6-8) rotations about the block center by +90 , -90 , and +180 .
[0057] However, in accordance with an embodiment of the present invention, a different procedure is proposed for calculating T than the procedure described in [13].
Particularly, in [13] range blocks are classified into categories of shade, edge, and midrange blocks, and within each category a different, predetermined T,. is used (where the T is not necessarily the optimal T,. ). Additionally, [13] applies a quantization procedure to discretize the transform coefficients. In contrast, the following procedure according to an embodiment of the present invention does not require individual blocks to be classified into categories, and no predefined limitations are placed on the than can be used. Particularly, for a fixed Ts , the optimal T,. , which minimizes (3), is determined as follows. Let X and Y be image blocks A; and Bj reordered as:

X"i11 412 ... A1N] and Y=[Bjll B.112 ... BJNN].

where this reduces the problem of finding an optimal T,. to a matrix calculus problem of finding a minimum. Particularly, the squared Euclidean distance DE between the transformed block T,. X and Y is considered as a matrix function of the transform T :

DE (T= X,Y) = IT"X - YI12 . (4) Differentiating (4) with respect to T,. and setting the derivative to zero, (4) becomes:

T,=aXX' -YX' =0 (5) where (.)' denotes the transpose. Assuming that (XX' )-1 exists, the solution to (5) is given 2o by:

Ta = YX+ = YXr(Xxr)-1 where (.)+ is the Moore-Penrose pseudoinverse [9]. The best combination of T,.s and T,. for the range block Bi is found by repeating the above process for all possible Ts and taking the pair of T;S and T,. that minimizes (3). In another embodiment of the present invention, the procedure described in [13] may be used to calculate T,." .

[0058] Returning to FIG. 6, at stage 606 each range image block Bj is characterized using five parameters of the fractal code constructed for it. Let I = argmin MSEj , where i indexes i domain blocks. The parameters are MSE,~ , shift and scaling parameters of the affine transform Tja , shuffling transform Tj, and the Euclidean distance between A, and Bj in the image plane. At stage 608, the information in each of these parameters is summarized using lo the histogram of its values across all range blocks in the image. The histograms of the parameters are then put together to form a vector including the five fractal code features for that image at stage 610. When the fractal code vector is for a patient with known classification, each of the entries in the vector may be tested by the above-described feature selection and classification procedures. Accordingly, it may be determined whether these fractal code features correlate with a medical condition. For example, in the illustrative Studies 1 and 2 described below for automated cancer diagnosis and Gleason grading, N was set equal to 100 to obtain the fractal code features. Additionally, the number of bins for MSE,~ , shift, scaling, shuffling transform, and the Euclidean distance between the domain and range blocks were set to 50, 20, 20, 8, and 10, respectively, yielding 108 total values for the five fractal code features. In other embodiments of the present invention, only a subset of the above-described five fractal code features or other fractal code features may be computed and/or included in the feature vector used for feature selection and classification (e.g., based on prior knowledge that one or more fractal code features are more important than others).
Moreover, when the vector is for a new patient, the fractal code vector preferably includes only the fractal code feature(s) analyzed by the final predictive model.
[0059] FIG. 8 is a flowchart 800 of illustrative stages involved in extracting wavelet transform data from a tissue image. Wavelet transform features characterize texture at different scales and orientations [16]. At stage 802, a wavelet representation of the tissue image is obtained using a 4-level, dyadic transform and the symlet 4 wavelet filter [17]. At stage 804, the variance a Z of the coefficients is computed for each of the subbands, where 6 2 is given by:

62 ~,x~jz-'~~,,1YxJ~z r i j where x. , M , j=1,...,N , denotes a wavelet coefficient in a subband of size MN .
The measurements of the coefficients may then be used as features for classification.
[0060] FIG. 9 is a flowchart 900 of illustrative stages involved in extracting color channel histogram data from a tissue image. Generally, visual examination of the images in FIG. 1 reveals noticeable changes in colors of H&E-stained images (even in grayscale, as presented here for reproducibility) as the tissue transforms from benign to cancerous, and as the level of malignancy increases. This is due to the fact that the epithelial nuclei, characterized by a dark blue color, invade the pinkish stroma and whitish lumen regions. The relative area of lumen and stroma regions decreases with increasing cancer malignancy. This pattern of color changes can be captured by the color channel histograms of the tissue image.
Accordingly, at stage 902, the image is separated into each of its RGB color channels. Then, each color channel image is separated into bins (e.g., 15 bins in the illustrative Studies 1 and 2 described below) representing grayscale levels of intensity at stage 904. When the tissue image is for a patient with known classification, the pixel content of the image in at least a subset of these bins may be computed at stage 906 (e.g., computing only the pixel content(s) in the one or more bins based on prior knowledge that these bin(s) more highly correlate with the medical condition). These pixel content(s) may be used as features in feature selection and classification in order to determine whether the features correlate with a medical condition.
The correlated features may be included in a final model that predicts the medical condition.
When the tissue image is for a new patient, preferably only the pixel content of the bin(s) included in the final predictive model are computed at stage 906. In another embodiment, the feature(s) may be taken from a 3-dimensional color histogram instead of three 1-dimensional histograms.
[0061] FIG. 10(a) shows a histogram of the green channel, which is highly correlated with image brightness, for the images depicted in FIG. 1. It is observed that there are far more bright pixels in the normal image than in the cancerous images. Moreover, the number of bright pixels decreases as the Gleason grade increases from 2 to 5.
[0062] FIG. 10(b) shows histograms of the difference between the values in the red and blue channels for the images in FIG. 1. Epithelial (and other) nuclei are characterized by large negative values of the difference between red and blue channels. The histograms indicate that for the normal tissue image there are fewer pixels with large negative values than for the cancerous images. As the Gleason grade increases (i.e., cancer becomes more aggressive), the number of such pixels rises.
[0063] The "white" pixels in prostate tissue images represent not only lumens but also background and artifacts caused by broken tissue structure. The effect of such white pixels on histogram analysis was studied by considering the histograms of the tissue images both before and after removing white pixels. Initial experiments have indicated that removing 1o white pixels results in improved classification accuracy. White pixels were detected via simple thresholding using a fixed, global threshold. This was accomplished by first transforming the image from the RGB color space into the YCbCr space [7], and then applying the threshold to the luminance (Y) component.
[0064] As described above, object-level morphometric data may be extracted from tissue images for use in automated diagnosis and/or grading of tissue images. This involves segmenting the original image into object classes (e.g., stroma, nuclei, red blood cells, lumen, etc.) and measuring various morphometric features of the objects including spectral-based characteristics (red, green, blue (RGB) channel characteristics, such as mean values, standard deviations, etc.), position, size, perimeter, shape (asymmetry, compactness, elliptic fit, etc.) and/or relationships to neighboring objects (contrast). An image processing tool may measure these features for every instance of every identified pathological object in the image and may output these features for, for example, evaluation by a feature selection and classiflcation procedure or by a final predictive model.
[0065] FIG. 11 is a flowchart of illustrative stages involved in segmenting a tissue image into objects for extraction of object-level morphometric data. As described above, an image processing tool including the commercially-available Definiens Cellenger Developer Studio (v. 4.0) may be used adapted to perform the segmenting of tissue images and the measuring of various morphometric data from the segmented images.
[0066] Initial Segnaentation. In a first stage, the image processing tool may segment an image (e.g., an H&E stained tissue microarray (TMA) image or an H&E of a whole tissue section) into small groups of contiguous pixels known as objects. These objects may be obtained by a region-growing algorithm which finds contiguous regions based on color similarity and shape regularity. The size of the objects can be varied by adjusting a few parameters [18]. In this system, an object rather than a pixel is typically the smallest unit of processing. Thus, all morphometric feature calculations and operations may be performed with respect to objects. For example, when a threshold is applied to the image, the feature values of the object are subject to the threshold. As a result, all the pixels within an object are assigned to the same class. In one embodiment, the size of objects may be controlled to be 10-20 pixels at the fmest level. Based on this level, subsequent higher and coarser levels are built by forming larger objects from the smaller ones in the lower level.
[0067] Background Extraction. Subsequent to initial segmentation, the image processing 1o tool may segment the image tissue core from the background (transparent region of the slide) using intensity threshold and convex hull. The intensity threshold is an intensity value that separates image pixels in two classes: "tissue core" and "background". Any pixel with an intensity value greater than or equal the threshold is classified as a "tissue core" pixel, otherwise the pixel is classified as a"background" pixel. The convex hull of a geometric object is the smallest convex set (polygon) containing that object. A set S is convex if, whenever two points P and Q are inside S, then the whole line segment PQ is also in S.
[0068] Coarse Segmentation. In a next stage, the image processing tool may re-segment the foreground (e.g., TMA core) into rough regions corresponding to nuclei and white spaces.
For example, the main characterizing feature of nuclei in H&E stained images is that they are stained blue compared to the rest of the pathological objects. Therefore, the difference in the red and blue channels (R-B) intensity values may be used as a distinguishing feature.
Particularly, for every image object obtained in the initial segmentation step, the difference between average red and blue pixel intensity values may be determined. The length/width ratio may also be used to determine whether an object should be classified as nuclei area. For example, objects which fall below a (R-B) feature threshold and below a length/width threshold may be classified as nuclei area. Similarly, a green channel threshold can be used to classify objects in the tissue core as white spaces. Tissue stroma is dominated by the color red. The intensity difference d, "red ratio" r = R/(R + G + B) and the red channel standard deviation aR of image objects may be used to classify stroma objects.

[0069] White Space Classification. In the stage of coarse segmentation, the white space regions may correspond to both lumen (pathological object) and artifacts (broken tissue areas) in the image. The smaller white space objects (area less than 100 pixels) are usually artifacts. Thus, the image processing tool may apply an area filter to classify them as artifacts.
[0070] Nuclei De-fusion and Classification. In the stage of coarse segmentation, the nuclei area is often obtained as contiguous fused regions that encompass several real nuclei.
Moreover, the nuclei region might also include surrounding misclassified cytoplasm. Thus, these fused nuclei areas may need to be de-fused in order to obtain individual nuclei.
[0071] The image processing tool may use two different approaches to de-fuse the nuclei.
The first approach may be based on a region growing algorithm that fuses the image objects constituting nuclei area under shape constraints (roundness). This approach has been determined to work well when the fusion is not severe.
[0072] In the case of severe fusion, the image processing tool may use a different approach based on supervised learning. This approach involves manual labeling of the nuclei areas by an expert (pathologist). The features of image objects belonging to the labeled nuclei may be used to design statistical classifiers.
[0073] In one embodiment, in order to reduce the number of feature space dimensions, feature selection may be performed on the training set using two different classifiers: the Bayesian classifier and the k nearest neighbor classifier [3]. The leave-one-out method [4]
may be used for cross-validation, and the sequential forward search algorithm may be used to choose the best features. Finally, two Bayesian classifiers may be designed with number of features equal to 1 and 5, respectively. The class-conditional distributions may be assumed to be Gaussian with diagonal covariance matrices.
[0074] In some embodiments, the input image may include different kinds of nuclei:
epithelial nuclei, fibroblasts, basal nuclei, endothelial nuclei, apoptotic nuclei and red blood cells. Since the number of epithelial nuclei is typically regarded as an important feature in grading the extent of the tumor, it may be important to distinguish the epithelial nuclei from the others. The image processing tool may accomplish this by classifying the detected nuclei into two classes: epithelial nuclei and "the rest" based on shape (eccentricity) and size (area) features.
[0075] Additional details regarding image segmentation and classification in accordance with the present invention are described in above-incorporated U.S. Patent Application No.
11/080,360, and in U.S. Patent Application No. 10/991,897, filed November 17, 2004, and U.S. Provisional Patent Application Nos. 60/520,515, filed November 17, 2003 and 60/552,497, filed March 12, 2004.
[0076] In another aspect, systems and methods are provided for screening for screening tissue images through the use of a two-level procedure. Referring to FIG. 12, at stage 1202, a first set of morphometric data from a tissue image for a new patient is evaluated by a model that predicts whether the tissue is cancerous or non-cancerous. At stage 1204, if the model classifies the tissue as cancerous tissue, a second set of morphometric data from the tissue image is evaluated by a model that assigns a cancer grade to the tissue (e.g., a Gleason grade for prostate tissue). The second set of morphometric feature(s) may include at least one morphometric feature not included in the first set of morphometric feature(s).
For example, in the first stage, one or more image-level morphometric features from the tissue image (e.g., fractal dimension, fractal code, wavelet, and/or color channel histogram feature(s)) may be evaluated by the model that predicts whether the tissue is cancerous or non-cancerous. In the second stage, one or more object-level features, alone or in combination with image-level features, may be evaluated by the model that assigns a cancer grade to the tissue.
[0077] In one embodiment, the second set of morphometric features may be extracted from the tissue image only if the tissue is classified by the model as being cancerous (based on the first set of morphometric features). In this way, the resources of physicians, other individuals and/or automated processing equipment (e.g., equipment for extracting morphometric data from tissue images) may be conserved. For example, when object-level features are evaluated in the second stage, this may reduce the number of tissue images that require segmentation and for which the object-level features (which numbered 424 per image in the below illustrative Studies 1 and 2) must be extracted. In another embodiment, the first and second sets of morphometric features may be extracted from the tissue image at the same time.
[0078] In an embodiment, the two-level procedure shown in FIG. 12 may be applied to identify, in a whole tissue section (or other suitable size tissue section), portions of the section that are cancerous, and then further analyze those sections. For example, the tissue section may be subdivided (e.g., manually or automatically) into multiple individual portions, images may be taken of each of the portions, and each of these portions may be evaluated at stage 1202 by the model that predicts whether tissue is cancerous or non-cancerous. In another embodiment, an image of a tissue section may be subdivided into individual sub-images, each image corresponding to a particular tissue portion. Portions that are determined by the model to be cancerous may be assigned a first indicator (e.g., "1"), which may be stored in memory. Portions that are determined by the model to be non-cancerous may be assigned a second indicator (e.g., "0") for storage in memory. At stage 1204, only the portions having been assigned the first indicator may be analyzed by the predictive model that assigns a cancer grade to the tissue portion. Any suitable hardware, software, or both may be used to subdivide the tissue section (or image thereof) into individual portions, the design of which will be apparent to one of ordinary skill in the art.
[0079] FIGS. 13(a) and 13(b) show illustrative systems that use a predictive model to make 1o a decision as to a patient's status with respect to a medical condition (e.g., cancer/non-cancer or cancer grade). The arrangement in FIG. 13(a) may be used when, for example, a medical diagnostics lab provides support for a medical decision to a physician or other individual associated with a remote access device. The arrangement in FIG. 13(b) may be used when, for example, a test kit including the predictive model is provided for use in a facility such as a hospital, other medical facility, or other suitable location.
[0080] Referring to FIG. 13(a), predictive model 1302 is located in diagnostics facility 1304. Predictive model 1302 may include any suitable hardware, software, or combination thereof for receiving data for a patient, evaluating the data in order to make a decision as to the patient's status with respect to a medical condition, and outputting the 2o results of the evaluation. Diagnostics facility 1304 may receive morphometric data for a patient from remote access device 1306 via lnternet service provider (ISP) 1308 and communications networks 1310 and 1312, and may input the data to predictive model 1302 for evaluation. Other arrangements for receiving and evaluating data for a patient from a remote location are of course possible (e.g., via another connection such as a telephone line or through the physical mail). The remotely located physician or individual may acquire the data for the patient in any suitable manner and may use remote access device 1306 to transmit the data to diagnostics facility 1304. In some embodiments, the data for the patient may be at least partially generated by diagnostics facility 1304 or another facility.
For example, diagnostics facility 1304 may receive a digitized version of an H&E stained image from 3o remote access device 1306 or other device and may generate morphometric data including image-level morphometric data and/or object-level morphometric data for the patient based on the image. In another example, actual tissue samples may be received and processed by diagnostics facility 304 in order to generate the morphometric data. In other examples, a third party may receive an image or tissue for a new patient, generate the morphometric data based on the image or tissue, and provide the morphometric data to diagnostics facility 304.
Any suitable hardware, software, or combination thereof may be used to extract the morphometric data. For example, the image processing tool described in connection with FIG. 11 may be used to extract object-level morphometric data from tissue images. With respect to image-level data, the design of suitable hardware, software, or combination thereof capable of performing the functions of image-level feature extraction described in connection with FIGS. 4 and 6-9 will be apparent to one of ordinary skill in the art. Any suitable hardware, software, or both may be used to extract the image-level data. For example, software for extracting the image-level data may be developed through the use of a commercially-available software development tool such as MATLAB by MathWorks.
[0081] Diagnostics facility 1304 may provide the results of the evaluation to a physician or individual associated with remote access device 1306 through, for example, a transmission to remote access device 1306 via ISP 1308 and communications networks 1310 and 1312 or in another manner such as the physical mail or a telephone call. The results may include a diagnostic "score" (e.g., an indication of whether the tissue for the patient is cancerous or non-cancerous or a Gleason grade for the tissue), information indicating one or more features analyzed by predictive model 1302 as being correlated with the medical condition, information indicating the sensitivity and/or specificity of the predictive model, or other suitable diagnostic information or a combination thereof.
[0082] Remote access device 1306 may be any remote device capable of transmitting and/or receiving data from diagnostics facility 1304 such as, for example, a personal computer, a wireless device such as a laptop computer, a cell phone or a personal digital assistant (PDA), or any other suitable remote access device. Multiple remote access devices 1306 may be included in the system of FIG. 13(a) (e.g., to allow a plurality of physicians or other individuals at a corresponding plurality of remote locations to communicate data with diagnostics facility 1304), although only one remote access device 1306 has been included in FIG. 13(a) to avoid over-complicating the drawing. Diagnostics facility 1304 may include a server capable of receiving and processing communications to and/or from remote access device 1306. Such a server may include a distinct component of computing hardware and/or storage, but may also be a software application or a combination of hardware and software.
The server may be implemented using one or more computers.
[0083] Each of communications links 1310 and 1312 may be any suitable wired or wireless communications path or combination of paths such as, for example, a local area network, wide area network, telephone network, cable television network, intranet, or Internet. Some suitable wireless communications networks may be a global system for mobile communications (GSM) network, a time-division multiple access (TDMA) network, a code-division multiple access (CDMA) network, a Bluetooth network, or any other suitable wireless network.
l0 [0084] FIG. 13(b) shows a system in which test kit 1322 including the predictive model of the present invention is provided for use in facility 1324, which may be a hospital, a physician's office, or other suitable location. Test kit 1322 may include any suitable hardware, software, or combination thereof (e.g., a personal computer) that is adapted to receive morphometric data for a patient including image-level and/or object-level morphometric data from a tissue image for the patient, evaluate the patient's data with a predictive model (e.g., programmed in memory of the test kit), and output the results of the evaluation. For example, test kit 1322 may include a computer readable medium encoded with computer executable instructions for performing the functions of the predictive model.
The predictive model may be a predetermined model previously generated (e.g., by another system or application such as the system in FIG. 13(c)). In some embodiments, test kit 1322 may optionally include an image processing tool capable of generating data corresponding to fractal, wavelet, and/or other morphometric features from, for example, a tissue sample or image. Suitable image processing tools for object-level and image-level morphometric data are described above in connection with FIGS. 11 and 13(a). In other embodiments, test kit 1322 may receive pre-packaged data for the fractal, wavelet and/or other morphometric features as input from, for example, an input device (e.g., keyboard) or another device or location. Test kit 1322 may optionally include an input for receiving, for example, updates to the predictive model. The test kit may also optionally include an output for transmitting data, such as data useful for patient billing and/or tracking of usage, to a main facility or other suitable device or location. The billing data may include, for example, medical insurance information for a patient evaluated by the test kit (e.g., name, insurance provider, and account number). Such information may be useful when, for example, a provider of the test kit charges for the kit on a per-use basis and/or when the provider needs patients' insurance information to submit claims to insurance providers.
[0085] FIG. 13(c) shows an illustrative system for generating a predictive model. The system includes analytical tool 1332 and database 1334 of patients whose status with respect to the medical condition is known. Analytical tool 1332 may include any suitable hardware, software, or combination thereof for determining correlations between the data from database 1334 and a medical condition. The system in Figure 13(c) may also include image processing tool 1336 capable of generating fractal, wavelet, and/or other morphometric data based on, for example, a digitized version of an H&E stained tissue image, an actual tissue 1o sample, or both. Tool 1336 may generate the morphometric data for, for example, the known patients whose data is included in database 1334. Suitable image processing tools for object-level and image-level morphometric data are described above in connection with FIGS. 11 and 13 (a).
[0086] Database 1334 may include any suitable patient data such as tissue images of patients, and/or image-level and/or object-level morphometric data corresponding to the images. Database 1334 may also include data indicating the status of patients with respect to a medical condition such as whether tissue corresponding to the tissue image is cancerous or not and, if cancerous, the Gleason grade for that tissue.

[0087] ILLUSTRATIVE STUDIES
[0088] Two studies were performed in which the inventors have identified various morphometric features and their respective correlations with the classification of tissue images. Namely, in the Study 1, morphometric data from tissue images for which it was known whether the corresponding tissue was cancerous or non-cancerous was used to train models to predict whether tissue for a new patient was cancerous. In the Study 2, morphometric data from tissue images for which the corresponding Gleason grades were known was used to train models to predict a Gleason grade for tissue for a new patient. The results of these studies are presented below. The inventors expect that continued experimentation and/or the use of other suitable hardware, software, or combination thereof will yield various other sets of computer-generated morphometric features (e.g., a subset of the morphometric features identified by the inventors) that may correlate with these and other medical conditions. Additionally, while these studies focused on training models with computer-generated morphometric data from tissue images only, it will be understood that other patient data such as clinical data and/or molecular data may be used in conjunction with the morphometric data described herein without departing from the scope of the present invention.
[0089] In the following studies, the sequential forward search (SFS) algorithm was used for feature selection. The SFS algorithm begins with selecting the individual feature that maximizes J. Each consequent stage of the algorithm consists of augmenting the set of already selected features with a new feature such that the resulting feature subset maximizes J. The process of adding new features is continued until J reaches a maximum.
In the 1o studies, Gaussian class-conditional distributions were assumed, linear and quadratic Gaussian classifiers were utilized, and system performance measured as classification accuracy was estimated using cross-validation [2].
[0090] In the studies, the k -fold cross-validation (CV) method was used for classifier training and feature selection, as well as classifier evaluation. CV consists of splitting the data into k equal or almost equal sets. Classifier training and feature selection is done k times, each time using the samples in k - 1 sets. The resulting classifier is tested on the remaining set for classifier evaluation. The overall performance of the trained models is obtained as the mean of the accuracies over the k left-out sets. The special case k = n is referred to as the LOO method [2].
[0091] Two CV loops were used for classifier training and testing. The outer loop consisted of k -fold CV for classifier testing. For each outer CV run, an inner loop was executed for classifier training and feature selection on the training set for that CV run. The combination of the classifier and feature set which maximizes the LOO accuracy on the training set was selected as the optimal model in that CV run. The resulting classifier was tested on the left-out dataset in the outer CV loop. This process was repeated for all k left-out sets and the accuracies across the runs are averaged to obtain the overall performance of the feature set.
[0092] The tissue samples used in the studies were stained using Hematoxylin and Eosin (H&E). The images were captured and stored using the light microscope and digital camera described above in connection with FIG. 3. All images were captured under similar illumination conditions and labeled by an expert pathologist. Examples of the tissue images are shown in FIG. 1.

[0093] Study 1: Automated Cancer Diagnosis of Tissue Images [0094] The image set used for tumor/non-tumor classification consisted of 367 images obtained from TMA cores. Of these, 218 were tumors and 149 were non-tumors.
Images labeled as tumor had between 5% and 100% of their area covered by tumor.
Images labeled as non-tumor included samples of PIN (Prostatic Intraepithelial Neoplasia, a precursor to cancer), prostatitis (inflammation of prostate), and normal tissue.
[0095] The image set used for tumor/non-tumor classification was randomly split into five sets of almost equal sizes to obtain a 5-fold CV estimate of the classification performance.
Splitting was done such that the numbers of images per class was (almost) equal across sets.
1o Each run of the 5-fold CV algorithm consisted of using one image set for testing the classifier trained on the remaining images sets.
[0096] For each image in the tumor/non-tumor image set, all of the image-level features (i.e., fractal dimension, fractal code, wavelet, and color channel histogram features) described in connection with FIGS. 4, 6-9 were extracted. With respect to object-level features, 48 color, structure, and texture features for each of eight histological object classes (i.e., stroma, epithelial nuclei, non-epithelial nuclei, cytoplasm, red blood cells, apoptotic (dead) cells, lumens, and artifacts) were obtained. Moreover, epithelial nuclei were divided into six subclasses based on the number of their neighbors and four subclasses based on the number of nucleoli they encompassed, and four features were obtained for each subclass. The neighborhood subclasses were the nuclei with 0,..., 4, and more neighbors, and the nucleoli subclasses were nuclei with 0, 1, 2, and more nucleoli. The features were the number, area, mean area and the standard deviation of the mean area of the nuclei in the subclass. Overall, 8 x 48 + 4 x 6 + 4 x 4= 424 object-level features were obtained.
[0097] Feature selection using the SFS algorithm was then performed on individual feature sets as well as the combined feature set using both linear and quadratic Gaussian classifiers.
The optimal feature combination was determiried as the set with the highest LOO accuracy estimate on the training set. During the feature selection experiments, it was noted that the LOO accuracy is not always a unimodal function of the number of features selected. Thus, the first peak reached in the LOO accuracy does not necessarily correspond to the optimal feature combination. In order to address this problem, the SFS algorithm was continued until 15 features were selected, and the optimal feature combination was chosen as the one that maximized the accuracy within the range of feature combinations considered.
The accuracies of the feature sets obtained as the average accuracy taken over all CV test sets are shown in FIG. 14. As indicated, the object-level features (also referred to as the MAGICTM features) together with the linear Gaussian classifier achieved the highest accuracy of 96.7% with a 95% confidence interval (CI) of 1.8% using a median of seven features, where median was taken over the five CV runs.
[0098] To gain insight into the meaning and value of the features, the individually best features, resulting in the highest LOO accuracy on the entire dataset of 367 images, were obtained for each feature set using the linear Gaussian classifier. These features are also shown in FIG. 14. As indicated, the fractal dimension and wavelet features yielded the highest accuracy.
[0099] The best object-level (MAGICTM) and wavelet features have straightforward intuitive interpretations. The best MAGICTM feature is the number of epithelial nuclei objects with one neighbor. This feature captures information about the invasion of the stromal tissue by epithelial cells. Large numbers of relatively isolated epithelial nuclei indicate possible malignancy. The best wavelet feature differentiates tumor and non-tumor images based on their texture coarseness. Tumor images have finer texture compared to non-tumor images, as can be seen from FIG. 1. This leads to a higher variance of coefficients in the diagonal detail subbands of the tumor images compared to non-tumor images.
FIG. 14 also shows that color channel histograms are fairly effective in tumor/non-tumor classification.
[0100] The feature space arrangement of the samples was examined using a two-dimensional scatterplot. The scatterplot was obtained for the best two features that together maximize the LOO accuracy on the entire dataset of 367 images. These two features were obtained using the SFS algorithm. The scatterplot and the corresponding linear Gaussian classifier are shown in FIG. 15. The horizontal axis is the best fractal dimension feature from FIG. 14 and the vertical axis is the number of epithelial nuclei objects that have no epithelial nuclei neighbors, where the reason that the best fractal dimension feature was selected over the best wavelet feature is merely that the SFS algorithm encountered it before the best wavelet feature. Epithelial nuclei objects that have no epithelial nuclei neighbors suggest the possible invasion of stromal tissue by epithelial cells via counting the number of isolated epithelial nuclei. As the scatterplot confirms, larger numbers of epithelial nuclei with no neighboring epithelial nuclei correspond to cancerous tissue.

[0101] Study 2: Automated Gleason Grading of Tissue Images [0102] The image set used for Gleason grading consisted of 268 images obtained from whole tissue sections. Of these, 175 images belonged to grades 2 and 3, which were referred to as "low-grade", and 93 belonged to grades 4 and 5, referred to as "high-grade." All images in this set had at least 80% of their area covered by tumor.
[0103] Similar to the tumor/non-tumor classification problem, the Gleason grading image set was randomly split into five sets of (almost) equal sizes to conduct 5-fold CV. For each image, the image-level and object-level features described in above in connection with Study 1 were extracted. Feature selection was performed using the SFS procedure described above.
1o FIG. 16 shows the accuracy of the selected feature subsets. A subset of the combined feature set classified using the linear Gaussian classifier achieved the highest accuracy of 78.7% with a 95% CI of 4.9% using a median of seven features.
[0104] FIG. 16 also shows the individually best features for each feature set obtained on the dataset of 268 images using SFS, LOO, and the linear Gaussian classifier. The best overall feature is the object-level feature of standard deviation of the border lengths of lumen objects.
This feature can be interpreted in view of the larger variation in the size of lumens in low-grade tumor images compared to that in high-grade tumor images, as FIG. 1 suggests. The best wavelet feature can be viewed as a descriptor of texture coarseness. The coarser the texture is, the larger the variance of the approximation subband becomes. This feature captures the difference between the more coarsely textured low-grade tumor and the more finely textured high-grade tumor.
[0105] Although FIG. 16 also indicates that the color channel histograms conveyed little information about the Gleason grade (which is based moreso on structural patterns than colors), it is believed that color will continue to play an important role in tissue classification.
[0106] The scatterplot for the best two features, obtained on the entire dataset of 268 images using SFS, LOO, and the linear Gaussian classifier, is given in FIG.
17. The horizontal axis is the best object-level (MAGICTM) feature from FIG. 16 and the vertical axis is the mean density of lumens. Density d of an object is a measure of its compactness and is defined as the ratio of the square root of its area s to its radius, i.e., d=V-s-l(1 + var(X) + Jvar(Y) ), where var(.) denotes variance, and X and Y are the x - and y - coordinates of all pixels forming the object, respectively. Note that the closer an object resembles a square, the higher its d value is. Although not immediately obvious from FIG.

1, lumens in high-grade tumors have slightly higher d values than lumens in low-grade tumors.

[0107] SummM of Studies 1 and 2 [0108] Studies 1 and 2 focused on features for automated prostate cancer diagnosis and Gleason grading of histological images. An automated system brings objectivity and reproducibility to cancer diagnosis and grading and may contribute to improvement in patient's prognosis. An advantage of the studies presented herein over previous studies is that they addressed diagnosis and grading within a unified framework.
Moreover, features were considered that describe the color, morphometric, and texture characteristics of the image at the global and histological object levels, whereas previous studies only considered one or two of the above features types.
[0109] Global features describing the color and texture characteristics of the entire image were color channel histograms, fractal dimension features, fractal code features, and wavelet features.
[0110] Object features characterizing the color, structural, and texture characteristics of segmented histological objects were obtained using the MAGICTM system. The object-level features included shape features as well as features describing the spatial relationships between histological objects.
[0111] The linear and quadratic Gaussian classifiers were applied to a subset of features obtained via the SFS algorithm to classify images into the tumor/non-tumor classes and tumor images into the low-/high-grade classes. On cancer diagnosis, a 5-fold CV
classification accuracy of 96.7% was achieved with the object-level (MAGICTM) feature set and the linear Gaussian classifier. The most effective features for this task were those that characterize the spatial relationship between epithelial cells, which indicate the invasion of epithelial cells into the stroma tissue, and the tissue texture coarseness.
On Gleason grading, a 5-fold CV classification accuracy of 78.7% was achieved using the combined feature set and the linear Gaussian classifier. Among the most discriminative features in this problem were descriptors characterizing lumen shape and tissue texture coarseness.
[0112] An interesting commonality between the two classification tasks is that, according to Studies 1 and 2, both of these tasks can be addressed with fairly good accuracy using texture features alone. Features that characterize texture coarseness perform particularly well in both tasks.

[0113] Additional Embodiments [0114] Thus it is seen that systems and methods are provided for automated diagnosis and grading of tissue images. Although particular embodiments have been disclosed herein in detail, this has been done by way of example for purposes of illustration only, and is not intended to be limiting with respect to the scope of the appended claims, which follow. In particular, it is contemplated by the inventors that various substitutions, alterations, and modifications may be made without departing from the spirit and scope of the invention as defined by the claims. Other aspects, advantages, and modifications are considered to be within the scope of the following claims. The claims presented are representative of the inventions disclosed herein. Other, unclaimed inventions are also contemplated. The inventors reserve the right to pursue such inventions in later claims.
[0115] Insofar as embodiments of the invention described above are implementable, at least in part, using a computer system, it will be appreciated that a computer program for implementing at least part of the described methods and/or the described systems is envisaged as an aspect of the present invention. The computer system may be any suitable apparatus, system or device. For example, the computer system may be a programmable data processing apparatus, a general purpose computer, a Digital Signal Processor or a microprocessor. The computer program may be embodied as source code and undergo compilation for implementation on a computer, or may be embodied as object code, for example.
[0116] It is also conceivable that some or all of the functionality ascribed to the computer program or computer system aforementioned may be implemented in hardware, for example by means of one or more application specific integrated circuits.
Suitably, the computer program can be stored on a carrier medium in computer usable form, which is also envisaged as an aspect of the present invention. For example, the carrier medium may be solid-state memory, optical or magneto-optical memory such as a readable and/or writable disk for example a compact disk (CD) or a digital versatile disk (DVD), or magnetic memory such as disc or tape, and the computer system can utilize the program to configure it for operation. The computer program may also be supplied from a remote source embodied in a carrier medium such as an electronic signal, including a radio frequency carrier wave or an optical carrier wave.

[0117] REFERENCES
[0118] The following is a list of the references referred to in the foregoing description:
[1] J. Diamond, N. Anderson, P. Bartels, R. Montironi, and P. Hamilton, "The use of morphological characteristics and texture analysis in the identification of tissue composition in prostatic neoplasia," Human Pathology, vol. 35, pp. 1121-1131, 2004.

[2] M. A. Roula, J. Diamond, A. Bouridane, P. Miller, and A. Amira, "A
multispectral computer vision system for automatic grading of prostatic neoplasia," in Proc.
Proc.
IEEE Int. Symp. Biomed. Iinaging, Washington, DC, 2002, pp. 193- 196.

[3] R. Stotzka, R. Manner, P.H. Bartels, and D. Tompson, "A hybrid neural and statistical classifier system for histopathologic grading of prostate lesions," Anal.
Quant. Cytol.
Histol., vol. 17, pp. 204-218, 1995.

[4] Y. Smith, G. Zajieck, M. Werman, G. Pizov, and Y. Sherman, "Similarity measurement method for the classification of architecturally differentiated images,"
Conzp. Bionaed Res., vol. 32, pp. 1-12, 1999.

[5] A. W. Wetzel, R. Crowley, S. J. Kim, R. Dawson, L. Zheng, Y. M. Joo, Y.
Yagi, J.
Gilbertson, C. Gadd, D. W. Deerfield and M. J. Becich, "Evaluation of prostate tumor grades by content-based image retrieval," in Proc. SPIE AIPR Workshop on Advances in Computen-Assisted Recognition, vol. 3584, Washington, DC, 1999, pp. 244-252.
[6] K. Jafari-Khouzani and H. Soltanian-Zadeh, "Multiwavelet grading of pathological images of prostate," IEEE Trans. Biomed. Etag., vol. 50, pp. 697-704, 2003.

[7] R. C. Gonzales and R. E. Woods, Digital Image Processing. Addison-Wesley, New York, 1992.

[8] Definiens Cellenger Architecture: A Technical Review, April 2004.

[9] K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed.
Academic, New York, 1990.

[10] R. O. Duda, R. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed.
Wiley, New York, 2001.

[11] G. Landini "Applications of fractal geometry in pathology," in Fractal Geometry in Biological Systems: An Analytical Approach, P. M. Iannaccone and M. Khokha, Eds.
CRC Press, Boca Raton, FL, 1996, pp.205-246.

[12] N. Lu, Fi=actallmaging. Academic, San Diego, CA, 1997.

[13] A. E. Jacquin, "Fractal image coding: A review," Proc. IEEE, vol. 81, pp.

1465,1993.

[14] A. Sloan, "Retrieving database contents by image recognition: New fractal power," Advanced Imaging, vol. 5, pp. 26-30, 1994.

[15] B. A. M. Schouten and P. M. de Zeeuw, "Feature extraction using fractal codes," in Proc. Int. Conf. Visual Infonnation and Information Systems, Amsterdam, 1999, pp.
483-492.

[16] A. Laine and J. Fan, "Texture classification by wavelet packet signatures," IEEE
Trans. Pattern Anal. Machine Intell., vol. 15, pp. 1186-1191, 1993.

[17] I. Daubechies, Ten Lectures on Wavelets. SIAM, Philadelphia, PA, 1992.

[18] Baatz M. and Schape A., "Multiresolution Segmentation - An Optimization Approach for High Quality Multi-scale Image Segmentation," In Angewandte Geographische Informationsverarbeitung XII, Strobl, J., Blaschke, T., Griesebner, G. (eds.), Wichmann- Verlag, Heidelberg, 12-23, 2000.

Claims (44)

1. A method for evaluating a tissue image for a medical condition, the method comprising:
receiving morphometric data for the tissue image; and evaluating the morphometric data with a model predictive of the medical condition, wherein the model is based on one or more morphometric features from the group of morphometric features consisting of one or more fractal dimension features, one or more fractal code features, one or more wavelet features, and one or more color channel histogram features, thereby evaluating the tissue image for the medical condition.

2. The method of claim 1, wherein the model is based on one or more fractal dimension features from one or more binary images, each binary image corresponding to a particular color channel of the image.

3. The method of claim 2, further comprising extracting the one or more fractal dimension features from the tissue image, wherein said extracting comprises:
thresholding the tissue image at one or more N b fixed, equally-spaced thresholds in one or more of its red, green, and blue (RGB) color channels to obtain one or more binary images; and computing a fractal dimension value for each of the one or more binary images.

4. The method of claim 1, wherein the model is based on one or more fractal code features from the group of fractal code features consisting of a mean square error (MSE) between a domain block and a range block, a shift parameter of an affine transform, a scaling parameter of an affine transform, a shuffling transform, and a Euclidean distance between a domain block and a range block in an image plane.

5. The method of claim 4, further comprising extracting the one or more fractal code features from the tissue image, wherein said extracting comprises:
partitioning the tissue image into non-overlapping blocks of two different sizes: range blocks B j and domain blocks ; and searching for a domain block and a transformation T i such that the transformed domain block T i~i is an approximation of B j , wherein one or more characteristics of the transformation comprise the one or more fractal code features.

6. The method of claim 5, wherein said searching comprises searching for a domain block ~i and a transformation T i such that a mean square error (MSE) between the domain block and range block is minimized.

7. The method of claim 1, wherein the model is based on one or more wavelet features comprising one or more measurements of variance of wavelet coefficients for one or more wavelet representation subbands.

8. The method of claim 7, further comprising extracting the one or more wavelet features from the tissue image, wherein said extracting comprises:
generating a wavelet representation of the tissue image; and computing the variance of the wavelet subband coefficients for at least one of the wavelet representation subbands.

9. The method of claim 1, wherein the model is based on one or more color channel histogram features comprising one or more pixel counts at one or more intensities of one or more color channels.

10. The method of claim 9, further comprising extracting the one or more channel histogram features from the tissue image, wherein said extracting comprises:
separating the tissue image into one or more of its red, green, and blue (RGB) color channels to produce one or more color channel images;
separating one or more of the color channel images into bins representing levels of color intensity; and computing a pixel count for one or more of the bins.

11. The method of claim 1, wherein the model predicts whether the tissue in the tissue image is cancerous or non-cancerous.

12. The method of claim 1, wherein the model predicts a cancer grade for the tissue in the tissue image.

13. The method of claim 12, wherein the cancer grade comprises a Gleason grade.

14. A method for evaluating a tissue image, the method comprising:
evaluating a first set of morpometric data from the tissue image with a model that predicts whether the tissue in the tissue image is cancerous or non-cancerous; and when the model classifies the tissue as cancerous tissue, evaluating a second set of morphometric data from the tissue image with a model that predicts a cancer grade for the tissue.

15. The method of claim 14, wherein said first set of morphometric data comprises image-level morphometric data from the group of image-level morphometric data consisting of fractal dimension data, fractal code data, wavelet data, and color channel histogram data.

16. The method of claim 15, wherein said second set of morphometric data comprises object-level data comprising one or more color, structural, and/or textural properties of one or more segmented image objects.

17. The method of claim 14, wherein the tissue image comprises a tissue image of a portion of a tissue section, and wherein said method comprises:
performing, for each portion of said tissue section, said evaluating of said first set of morpometric data from a tissue image for that portion with said model that predicts whether tissue in the tissue image is cancerous or non-cancerous; and performing, only for portion(s) of the tissue section classified as cancerous, said evaluating a second set of morphometric data from the tissue image(s) with a model that predicts a cancer grade for the tissue.

18. A method for generating a model predictive of a medical condition, the method comprising:

receiving morphometric data from two or more tissue images with known classification with respect to the medical condition, wherein the morphometric data comprises morphometric data from the group of morphometric data consisting of fractal dimension data, fractal code data, wavelet data, and color channel histogram data; and performing feature selection and/or classification on the morphometric data, thereby generating the model predictive of the medical condition.

19. The method of claim 18, wherein the model predicts whether the tissue in the tissue image is cancerous or non-cancerous.

20. The method of claim 18, wherein the model predicts a cancer grade for the tissue in the tissue image.

21. The method of claim 20, wherein the cancer grade is a Gleason grade.

22. An apparatus for evaluating a tissue image for a medical condition, the apparatus comprising:
a model predictive of the medical condition, wherein the model is based on one or more morphometric features from the group of morphometric features consisting of one or more fractal dimension features, one or more fractal code features, one or more wavelet features, and one or more color channel histogram features, wherein the model is configured to:
receive morphometric data for the tissue image; and evaluate the morphometric data for the tissue image according to the model, thereby evaluating the tissue image for the medical condition.

23. The apparatus of claim 22, wherein the model is based on one or more fractal dimension features from one or more binary images, each binary image corresponding to a particular color channel of the image.

24. The apparatus of claim 23, further comprising an image processing tool configured to extract the one or more fractal dimension features from tissue image, wherein said image processing tool is configured to:

threshold the tissue image at one or more N b fixed, equally-spaced thresholds in one or more of its red, green, and blue (RGB) color channels to obtain one or more binary images; and compute a fractal dimension value .gamma. ~ for each of the one or more binary images.

25. The apparatus of claim 22, wherein the model is based on one or more fractal code features from the group of fractal code features consisting of a mean square error (MSE) between a domain block and a range block, a shift parameter of an affine transform, a scaling parameter of an affine transform, a shuffling transform, and a Euclidean distance between a domain block and a range block in an image plane.

26. The apparatus of claim 25, further comprising an image processing tool configured to extract the one or more fractal code features from the tissue image, wherein the image processing tool is configured to:
partition the tissue image into non-overlapping blocks of two different sizes:

range blocks B j and domain blocks ~i; and search for a domain block ~i and a transformation T i such that the transformed domain block T i~i is an approximation of B j, wherein one or more characteristics of the transformation comprise the one or more fractal code features.

27. The apparatus of claim 26, wherein said image processing tool is configured to search for a domain block ~i and a transformation T i such that a mean square error (MSE) between the domain block and range block is minimized.

28. The apparatus of claim 22, wherein the model is based on one or more wavelet features comprising one or more measurements of variance of wavelet coefficients for one or more wavelet representation subbands.

29. The apparatus of claim 28, further comprising an image processing tool configured to extract the one or more wavelet features from the tissue image, wherein the image processing tool is configured to:
generate a wavelet representation of the tissue image; and compute the variance of the wavelet subband coefficients for at least one of the wavelet representation subbands.

30. The apparatus of claim 22, wherein the model is based on one or more color channel histogram features comprising one or more pixel counts at one or more intensities of one or more color channels.

31. The apparatus of claim 30, further comprising an image processing tool configured to extract the one or more channel histogram features from the tissue image, wherein said image processing tool is configured to:
separate the tissue image into one or more of its red, green, and blue (RGB) color channels to produce one or more color channel images;
separate one or more of the color channel images into bins representing levels of color intensity; and compute a pixel count for one or more of the bins.

32. The apparatus of claim 22, wherein the model predicts whether the tissue in the tissue image is cancerous or non-cancerous.

33. The apparatus of claim 22, wherein the model predicts a cancer grade for the tissue in the tissue image.

34. The apparatus of claim 33, wherein the cancer grade comprises a Gleason grade.

35. An apparatus for evaluating a tissue image, the apparatus comprising:
a first model configured to evaluate a first set of morpometric data from the tissue image in order to predict whether the tissue in the tissue image is cancerous or non-cancerous; and a second model configured to evaluate, subsequent to the first model classifying the tissue as cancerous tissue, a second set of morphometric data from the tissue image in order to predict a cancer grade for the tissue.

36. The apparatus of claim 35, wherein said first set of morphometric data comprises image-level morphometric data from the group of image-level morphometric data consisting of fractal dimension data, fractal code data, wavelet data, and color channel histogram data.

37. The apparatus of claim 36, wherein said second set of morphometric data comprises object-level data comprising one or more color, structural, and/or textural properties of one or more segmented image objects.

38. The apparatus of claim 35, wherein the tissue image comprises a tissue image of a portion of a tissue section, and wherein:
said first model is configured to perform, for each portion of said tissue section, said evaluating of said first set of morpometric data from a tissue image for that portion; and said second model is configured to perform, only for portion(s) of the tissue section classified as cancerous by said first model, said evaluating of a second set of morphometric data from the tissue image(s).

39. An apparatus for generating a model predictive of a medical condition, the apparatus comprising:
an analytical tool configured to:
receive morphometric data from two or more tissue images with known classification with respect to the medical condition, wherein the morphometric data comprises morphometric data from the group of morphometric data consisting of fractal dimension data, fractal code data, wavelet data, and color channel histogram data; and perform feature selection and/or classification on the morphometric data, thereby generating the model predictive of the medical condition.

40. The apparatus of claim 39, wherein the model predicts whether the tissue in the tissue image is cancerous or non-cancerous.

41. The apparatus of claim 39, wherein the model predicts a cancer grade for the tissue in the tissue image.

42. The apparatus of claim 20, wherein the cancer grade is a Gleason grade.

43. A computer readable medium comprising computer executable instructions recorded thereon for performing the method comprising:
receiving morphometric data for a tissue image; and evaluating the morphometric data with a model predictive of a medical condition, wherein the model is based on one or more morphometric features from the group of morphometric features consisting of one or more fractal dimension features, one or more fractal code features, one or more wavelet features, and one or more color channel histogram features, thereby evaluating the tissue image for the medical condition.

44