WO2011087778A1 - Automated quantitative multidimensional volumetric analysis and visualization - Google Patents

Automated quantitative multidimensional volumetric analysis and visualization Download PDF

Info

Publication number
WO2011087778A1
WO2011087778A1 PCT/US2010/061487 US2010061487W WO2011087778A1 WO 2011087778 A1 WO2011087778 A1 WO 2011087778A1 US 2010061487 W US2010061487 W US 2010061487W WO 2011087778 A1 WO2011087778 A1 WO 2011087778A1
Authority
WO
WIPO (PCT)
Prior art keywords
cell
light source
image
cells
profile
Prior art date
Application number
PCT/US2010/061487
Other languages
French (fr)
Inventor
Ali Zahalka
Lynnae Schwartz
Original Assignee
The Children's Hospital Of Philadelphia
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by The Children's Hospital Of Philadelphia filed Critical The Children's Hospital Of Philadelphia
Publication of WO2011087778A1 publication Critical patent/WO2011087778A1/en

Links

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/36Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365Control or image processing arrangements for digital or video microscopes
    • G02B21/367Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/60Type of objects
    • G06V20/69Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/693Acquisition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00Indexing scheme relating to image or video recognition or understanding
    • G06V2201/12Acquisition of 3D measurements of objects

Definitions

  • the present invention relates generally to the field of pathology. More specifical ly, the present i nvention relates to systems and methods for characterizing cells in samples using brightfield microscopy.
  • Fluorescence i mmunohistochemistry is a valuable tool in this endeavor, but this technique is effectively impractical in clinical setti ngs where more rapid read-outs are required in a high volume and cost-constrai ned environment, and where chromogenic labeling is the standard.
  • Digital images of chromogenical ly labeled tissue may be acqui red using red/green/blue (RGB) camera filters such as the Bayer filter used i n the majority of color cameras to produce a "color" image.
  • RGB red/green/blue
  • satisfactory resolution can be achieved to allow quantification of the expression of a single-antibody labeled protein (though satisfactory results are several ly limited to a smal l subset of labels; furthermore, tissues cannot be counterstained with the standard hematoxylin or eosin stai ns), the poor spatial resolution and spectral fidelity inherent in RGB camera tech nology significantly limits the abi lity to confidently distinguish multiple, spatially overlappi ng
  • Spectral microscopy addresses such limitations by taki ng advantage of the increased sensitivity provided by a process of un-mixing the acquired spectral data , and analyzing the spectral profiles of each pixel in the acquired image (multiplexing spectroscopy) . Low expression and overlappi ng elements are thereby rendered disti nguishable within the tissue and cells bei ng exam ined .
  • Spectral platforms are l imited in their investigational and commercial usefulness because they are based on predefi ned spectral profiles or un-standardized user-biased man ually-selected representative spectra .
  • Site- and appl ication-specific optimization is needed to account for sample variations due to differences in staining protocols, tissues, and interactions between distinct biomarkers, and stati stical methods and automatization are needed to ensure accurate spectral -representation.
  • Current methodologies image a single, two-dimensional optical plane of a region of interest a nd perform all calculations of biomarker expression, often on a scale of 0-4+, on the slice.
  • the sensitivity of this approach is inherently limited by h uman factors, including distracti ng intensity fluctuations from out of focus regions, and because analysis of a three-di mensional object within a two- dimensional space constrai ns efforts to accurately determ ine volumes of distribution.
  • the tissues and biological samples under observation are
  • Three-di mensional images, formed from sampl ing of multi ple focal planes are invaluable in efforts to distinguish contiguous and overlapping elements that m ight otherwise be counted as a single unit i n a two-dimensional projection .
  • the invention features methods for characterizing cells in vitro.
  • the methods generally comprise labeling at least one biomarker on the cells with a chromogenic label, illuminating the cells with a spectrally-stable light sou rce configured to evenly illuminate the cells, detecti ng a plurality of optical planes of the cells with a brightfield microscope, acquiring a z-stack image of the optical planes, reconstructing the optical planes in three dimensions to form an image of the cells, determi ning a plurality of voxels in the image of the cells, and, calculating a volumetric profile for each voxel, wherein the volumetric profile indicates one or more conditions of the biomarker and/or the cells.
  • the invention also features methods for calibrati ng selection criteria of cells.
  • the methods generally comprise analyzing a chromogenically labeled cell in three
  • scoring at least one of the identity, intensity, or distribution of at least one chromogen on the cell optionally, scoring at least one of the cell volume, nuclear volume, cel l shape, cell texture, or spatial relationship between cells, inputting the score of the identity, intensity, or distribution of at least one chromogen on the cell into a computer programm ed to stati stically analyze the scores, optionally inputti ng the score of the cell volume, nuclear volume, cell shape, cell texture, or spatial relationship between cells into the computer, analyzing the scores on the computer to determi ne the scoring preference for each variable, optionally, repeating each of these steps at least one time, and, cal ibrati ng selection criteria for the chromogenic analysis of cells with the scoring preferences, wherein the calibrated selection criteria are appl ied to subsequent chromogenic analyses of cells.
  • one or both of the scoring steps is automated.
  • the invention also features systems for characterizing cel ls in vitro.
  • the systems comprise a spectral ly-stable light source configured to evenly illuminate a cell, a brightfield microscope, an imaging spectrometer, and a processor configured to calculate a volumetric profile for each voxel determined from a digital image of the cell .
  • the invention also features methods for determining the prognosis of a su bject having a tumor.
  • the methods comprise analyzing chromogenically labeled cells from a tissue sample isolated from the subject i n three di mensions, scoring at least one of the identity, intensity, or distribution of at least one chromogen on the cells, scoring at least one of the cell volume, nuclear volume, cell shape, cell texture, or spatial relationship between cells, serum tumor-marker concentration, and hormone level in the tissue sample, optionally, scoring the tumor type as normal , benign, or malignant, inputting the score of the identity, intensity, or distribution of at least one chromogen on the cells, and the cell volume, nuclear volume, cell shape, cell texture, or spatial relationship between cells, serum tumor-marker concentration, and hormone level in the tissue sample, and optionally inputti ng the tumor type score, i nto a computer programmed to statistical ly analyze the scores, analyzing the scores on the computer and generati ng a profile for the subject, and determi
  • the methods can further comprise compari ng the generated profi le to a profile previously generated from the subject, to a reference profile of a healthy population of the subject, to a reference profile of a population of the subject havi ng a benign tumor, or to a reference profi le of a popu lation of the subject having a mal ignant tumor, and determi ning a prognosis of the subject from the comparison .
  • Fig. 1 is a block diagram i llustrati ng an exemplary system for characterizing cells in vitro, according to a n embodi ment of the present invention ;
  • Fig . 2 is a block diagram i llustrati ng an exemplary spectrally stable light source used in the system shown in Fig. 1, accordi ng to a n embodi ment of the present invention
  • Fig. 3 is a block diagram i llustrati ng an exemplary image captu re section of the system shown in Fig. 1, accordi ng to an embodi ment of the present invention
  • Fig. 4 is a flow chart illustrati ng an exemplary method for characterizing cel ls in vitro, according to an embodi ment of the present invention
  • Fig. 5 is a flow chart illustrati ng an exemplary method for developing cellular selection criteria, according to an embodi ment of the present invention ;
  • Figs. 6A, 6B and 6C are flow charts illustrati ng exemplary methods for monitoring and adjusting the light source to maintai n a stable spectrum of the light source, accordi ng to embodiments of the present i nvention;
  • Fig. 7 is a flow chart illustrati ng an exemplary method for acquiring a Z-stack image, according to an embodi ment of the present invention
  • Fig. 8 is a flow chart illustrati ng an exemplary method for cal ibrati ng cellular selection criteria, according to a n embodi ment of the present invention
  • Fig. 9 is a flow chart illustrati ng an exemplary method for determining a plurality of voxels representing cells, accordi ng to an embodi ment of the present invention.
  • Fig. 10 is flow chart illustrating a n exemplary method for calculati ng a volumetric profile, according to an embodi ment of the present invention
  • Fig . 11 is a flow chart illustrati ng an exemplary method of generating a library of spectral profiles, according to an embodiment of the present invention
  • Fig . 12 is a flow chart illustrati ng an exemplary method of determining a prognosis of a subject having a tumor, according to an embodiment of the present invention
  • Fig. 13 is an example multispectral graph of absorption as a function of wavelength for a DAB-labeled nucleus (transcription factor positive) and a hematoxyl in- stained nucleus (transcription factor negative), according to a cell characterization method of the present invention ;
  • Fig. 14 is an example multispectral graph of absorption as a function of wavelength for a random sam ple of parenchymal nuclei (a mixture of transcription factor positive and negative) from a single parathyroid specimen, according to a cell characteri zation method of the present invention ;
  • Fig. 15 is an example multispectral graph of absorption as a function of wavelength for a random sample of parenchymal nuclei classified as grade zero
  • Fig. 16 is an example multispectral graph of absorption as a function of wavelength for a random sample of parenchymal nuclei classified as grade one + (transcription factor positive) from multiple parathyroid specimen, according to a cell characterization method of the present invention ;
  • Fig. 17 is an multispectral graph of absorption as a function of wavelength for a random sampl e of parenchymal nuclei classified as grade two + (transcription factor positive) from multiple parathyroid specimen, according to a cell characterization method of the present invention ;
  • Fig. 18 is an example multispectral graph of absorption as a function of wavelength for a random sam ple of parenchymal nuclei classified as grade three + (transcription factor positive) from multiple parathyroid specimen, according to a cell characteri zation method of the present invention ;
  • Fig. 19 is an example graph of classification results and overall score (indicating overall level of transcription factor expression) for a single parathyroid specimen, according to a cell characterization method of the present invention.
  • Fig. 20 is an example graph of classification results and overall score for normal tissue and carcinoma tissue of a training data set, according to a cel l characterization method of the present invention ;
  • Fig. 21 is an example graph of classification results and overall score for normal tissue of the trai ning data set and a test data set, according to a cell characterization method of the present invention
  • Fig. 22 is an example graph of classification results and overall score for carcinoma tissue of the training data set an d a test data set, according to a cel l characterization method of the present invention
  • Fig. 21 is an example graph of classification results and overall score for normal tissue of the trai ning data set and a test data set, according to a cell characterization method of the present invention
  • Fig. 22 is an example graph of classification results and overall score for carcinoma tissue of the training data set an d a test data set, according to a cel l characterization method of the present invention
  • Fig. 23 is an example graph of classification results and overall score for normal tissue and carcinoma tissues of the test data set, accordi ng to a cell characteri zation method of the present invention.
  • a "biological sampl e” includes any cell, tissue, fluid, and the like isolated or otherwise obtai ned from an organism. Any organism can be used, with mam mals being highly preferred and humans bei ng most preferred.
  • the invention featu res methods for imagi ng and characterizing cells.
  • the imagi ng and analysis of multiple chromogenical ly labeled biomarkers through proper sampl e illumination, z-stack acquisition, and multiplex spectroscopic imagi ng, followed by th ree-dimensional volumetric quantification of biomarker expression significantly enhances the resolution of images of pathology sampl es viewed
  • FIG. 1 a block diagram of an exemplary system 100 for
  • System 100 i ncludes image captu re section 102 for acquiring a z-stack image of speci men 108 and image analyzer 104, coupled to image captu re section 102, to determi ne a volumetric profile from the z-stack i mage.
  • the volumetric profile may indicate one or more conditions of a biomarker on cells of speci men 108 and/or the cells of specimen 108.
  • System 100 may also include memory 124, user interface 126 an d display device 128. Suitable components for use within system 100 will be understood by one of skill in the art from the description herein .
  • Memory 124 may store two dimensional images for various optical planes, as well as an acquired z-stack i mage from image capture section 102. Memory 124 m ay also store digital images, vol umetric profiles and/or cell analysis results from image analyzer 104. Mem ory 124 may be a memory, a magnetic disk, a database or essentially any local or remote device capabl e of stori ng data .
  • User interface 126 may be coupled to image captu re section 102 and/or image analyzer 104.
  • User i nterface 126 m ay be used to control image captu re in image capture 102 and image a nalysis in image analyzer 104.
  • user interface 126 may be used to select positions of sample stage 110, adjust performance parameters of light source device 106, control a focus of brightfield microscope 112 and select optical planes for z-stack i mage captu re.
  • User interface may also be used to select image processing parameters reducing image distortion in image processor 118, provide cell selection in cell detector 120, a nd for cel l analysis i n cell analyzer 122.
  • User interface 126 may also be capable of selecting i mages to be displayed and/or stored, and may include a text interface for entering information .
  • Display device 128 may be coupled to image captu re section 102 and/or image analyzer 104.
  • Display device 128 m ay present, for exam ple, two di mensional and z- stack i mages to a user (from image capture section 102) , cells selected by cell detector 120 and analysis of cells from cell analyzer 122.
  • display device 128 may include any display capable of presenting information including textual and/or graph ical information .
  • each of image captu re section 102 a nd image analyzer 104 may include one or more of memory 124, user interface 126 an d display device 128. According to another embodiment, one or more of memory 124, user interface 126 and display device 128 may be remote from image captu re section 102 and/or image analyzer 104.
  • system 100 may be configured to connect to a global information network, e.g., the Internet, (not shown) such that the z-stack image and/or the volumetric profile may also be transmitted to a remote location for further processing and/or storage.
  • a global information network e.g., the Internet
  • Image captu re section 102 i ncludes light source device 106, sampl e stage 110 for positioning specimen 108 in an imaging light path 312 ( Fig. 3), brightfield microscope 112, i magi ng spectromete r 114 and controller 116. Controller 116 may be configured to control brightfield microscope 112, sample stage 110 and light source device 106.
  • Controller 116 may be a digital signal processor.
  • Light source device 106 represents a spectral ly-stable light source that is configured to evenly illuminate cells of specimen 108. As described fu rther below with respect to Figs. 2, 4 and 6A-6C, light source device 106 may be monitored th roughout the acqu isition of the z-stack i mage. Various performance paramete rs of light source device 106 may be adjusted, for exam ple by controller 116, if light sou rce device 106 deviates from predetermi ned performance parameters. For example, a spectru m, a temperatu re and a current of light source device 106 may be monitored, to thermally and spectral ly regulate light source device 106.
  • Speci men 108 may be provided on sample stage 110.
  • Sample stage 110 i configured to position specimen 108 to receive illumination from light source device 106 along one or more of the x, y and z directions.
  • Brightfield microscope 112 i s configured to receive light modified by interaction with specimen 108.
  • specimen 108 may absorb some of the received light, depending upon the density of regions of the sample.
  • Light modified by the sample may be obtained by brightfield microscope, to detect a pl urality of optical planes.
  • Imaging spectrometer 114 receives the modified light from brightfield m icroscope 112, and captu res an image of the modified light. Imagi ng spectrometer 114 may also obtai n a spectral profile of the image. Imaging spectrometer 114 may acqu ire a z-stack image of the plurality of optical planes. For example, a vertical stack of images can be captu red at a range of optical planes, by vertically adjusting sample stage 110 at predefi ned intervals, and fully sampling a region of interest (for example, according to the Nyquist-Shannon sampl ing theorem) . Imagi ng spectrometer 114 may capture a spectral profile of each pixel at each focal plane during z-stack acquisition. Z-stack images may be acquired using structured illumination .
  • Image analyzer 104 includes image processor 118, cell detector 120 an d cell analyzer 122. It is understood that one or more of these components may include a digital signal processor.
  • Image processor 118 is configured to receive the z-stack image and to reconstruct the optical planes i n three-dimensions, to provide a volumetric image of the cell and its anatomic surroundings. In addition, image processor 118 ma y process the image, for exampl e, to reduce image di stortion . A description of methods for reducing image distorti on is provided further below, with respect to Fig. 4. Furthermore, image processor 118 may calibrate selection criteria of cells in the volumetric image, accordi ng to user-specific scoring preferences. A further description of the calibration is provided below with respect to Fig. 4, 5 and 8.
  • Cell detector 120 is configured to receive the volumetric i mage from image processor 118 and determ ine a plurality of voxels in the image of the cell .
  • the plurality of voxels may represent the cell and/or specific structu res in the cell .
  • Cell analyzer 122 is configured to calculate a volumetric profile for each voxel determi ned from selected voxels received from cel l detector 120.
  • Cell analyzer 122 may analyze the selected voxels and provide a nuclear classification and/or provide a score (such as for a particular biomarker) .
  • the cell analysis may be used to i ndicate one or more conditions of the biomarker and/or the cell.
  • microdissection may be used i n image captu re section 102 along with quantitative Reverse Transcription Polymerase Chain Reaction (qRT-PCR) of previously imaged cells (having stored spectral curves), to classify the imaged cells into grades (e.g., 0-3+), to create a standard cu rve correlating values of absorbance maxima to molecularly- determi ned levels of gene expression.
  • qRT-PCR quantitative Reverse Transcription Polymerase Chain Reaction
  • system 100 may be carried out according to any means suitable in the art, for example, using a computer programmed to carry out one or more of these functions.
  • th e spectral profile at each focal plane of a non-stai ned, control cell may be used.
  • Light source device 106 m ay be configured to evenly illuminate specimen 108 (Fig. 1) .
  • Light source device 106 is also configured to be thermally-regulated, and to produce spectral ly-stable light (i .e. , with a spectrum that does not shift over time or vary in luminah intensity).
  • Light source includes i llumination tube 201, ventilation slits 202, opaque sheet 203 to block ambient light, illumination source 204, diffuser lens 205, collimating lens 206, Peltier element 209, heat si nk 210, high velocity fan 211, temperatu re-monitori ng probe 212, and illumination port adapter 213.
  • Suitable components for use within light source device 106 wi ll be understood by one of skill in the art from the description herein .
  • Diffuser lens 205, col limati ng lens 206 and illumination source 204 are located along optical axis 207, to provide light along illumination beam path 208.
  • Diffuser lens 205 may combi ne the light from illumination source 204 (for example, from m ultiple light emitting diodes (LEDs) of illumination source 204) into a si ngle spectral ly homogeneous beam , and may project the light maxi mal ly into col limati ng lens 206.
  • Collimating lens 206 may collimate and concentrate the illumi nation light into a tighter beam and di rect it out of through illumination tube 201 and out of illumination port adapter 213 towards brightfield microscope 112 (Fig. 1) . It is understood th at light source device 106 may include additional optical components to provide a uniform light beam to specimen 108 ( Fig . 1) .
  • sheet 203 is a sheet of aluminum and heat sink 210 is formed of copper. It is understood th at sheet 203 may be formed of any material suitable for substantially blocking ambient l ight from entering light source device 106 and bei ng added to illumination beam path 208. Heat sink 210 may be any heat sink 210 capable of transferring heat from light source device 106.
  • Illumination port adapter 213 m ay be configured to be attach ed to a brightfield illumination port (not shown) on existing conventional brightfield microscopes.
  • light source device 106 as described herein may be varied by addi ng a motorized dichroic-mirror/filter switching mechanism between illumination port adapter 213 and brightfield microscope 112 ( Fig. 1) .
  • the user can not only easily customize dichroic-mirror and filter combinations, but may also rapidly switch between these combinations for multiple imaging techniques.
  • the motorized mirror/filter wheel may have one setting for visible spectrum imagi ng, another setting corresponding to an infrared-reflecting dichroic-mirror for infrared or IR-DIC imagi ng, and another setti ng corresponding to an ultraviolet (UV)-reflecting dichroic-mirror for UV illum ination and/or excitation .
  • UV ultraviolet
  • Illumination source 204 may i nclude any spectral ly stable light source, defined herein as exhibiting a predi cta ble change i n intensity amplitude and spectral shift when a temperatu re is varied.
  • Illumi nation source 204 may comprise a solid-state light source, such as a l ight emitting diode.
  • the light emitti ng diode may be a broadband source that emits white light.
  • the light emitting diode may be phosphorous coated, for example, a phosphorous-coated LED, to provide warm white light.
  • illumination source 204 includes an LED array . Although examples of white light are illustrated, it is understood that illumination source 204 m ay be configured to provide any suitable range of wavelengths, including single wavelengths. Illumination source 204 ma y be coupled with further optics to provide collimated light. In some aspects, a cl uster of multi-layer phosphorus-coated di odes may be used. In some aspects, combi nations of single-wavelength LEDs or laser diodes that produce a broadband spectrum may be used. Other broadband light sources such as halogen and incandescent bulbs, or solid state l ighting such as amorphous silicon can be used.
  • narrow-ba nd light sources can be used for specific chromogen combi nations or for specialized microscopic techniques such as Infrared Microscopy and Differential Interference Contrast microscopy (DIC) .
  • DIC Differential Interference Contrast microscopy
  • Illumination source 204 may be selected to exhibit a predictable (i .e., capable of being mathematically modeled) change in intensity ampl itude and spectral shift when a junction temperatu re or current is varied. Because illumination source 204 exh ibits a predictable change in intensity-amplitude and spectral -shift with varyi ng temperature, these variables can be considered during i magi ng and analysis of speci mens, to mai ntain an inter-specimen and intra-specimen image acqu isition comparabi lity.
  • aspects of the present invention include addressing major causes of illumination spectral variation in illumination source 204, including the effects of temperatu re, current, aging, and variations in individual LED performance.
  • LEDs are semiconductors, their energy band gaps may vary with changes in current and temperatu re, which in turn changes their luminous output and spectra.
  • the luminous intensity may decrease, causing the output spectrum to shift towards longer wavelengths.
  • the current increases, the luminous intensity increases and the output spectru m shifts towards shorter wavelengths.
  • LED efficiency may decrease over time through a complex mechanism, thus decreasing luminous output (for example, by about 50% of the original output after 50,000 h ours of operation).
  • cooling illumination source 204 may bel ow ambient temperatu re, by maintai ning a stable junction temperatu re along with a stabl e current and by monitori ng the light source's luminous output and spectra using imagi ng spectrometer 114 ( Fig. 1) .
  • imagi ng spectrometer 114 Fig. 1
  • the junction temperatu re of illumination source 204 may be controlled th rough a combination of free convection through angled ventilation slits 202 (that l imit entrance of ambient light) and active cooling th rough Peltier element 209 (which may dissipate excess thermal energy through a combi nation of heat-pipes and high surface-area radiating fins of heat si nk 210) coupled to high velocity fan 211. All of these elements may be regulated by controller 116 ( Fig . 1) . Cooling illumination source 204 and protecting illumination source 204 from large temperature variations may slow down its degradati on and thus increase its operational lifetime. Current may be controlled using current-regulating circuitry (not shown).
  • a reference image may be acquired (which may be repeated every time a new specimen is imaged).
  • Controller 116 (Fig. 1) may compare the spectral profile of the current reference image to a predetermined reference spectrum and may adjust the current to illumination source 204 and temperatu re, to recalibrate i llumination source 204 to match the predetermined reference spectrum and intensity.
  • FIG. 3 a block diagram of image capture section 102 is shown, illustrati ng a relationship between components of image captu re section 102 along optical axis 207 and i magi ng light path 312.
  • Image captu re section 102 also illustrates UV filter 302, dichroic mi rror 304, field diaphragm 306, apertu re diaphragm 308 an d condenser 310, wh ich are positioned between light source device 106 and objective 314 along imaging light path 312.
  • Fig 3 also illustrates controller 116 cou pled to light source device 106. Suitable components for use within light source device 106 wi ll be understood by one of skill in the art from the description herein .
  • Field diaphragm 306, a perture diaphragm 308, condenser 310 and objective 314 may be components of brightfield microscope 112.
  • Field diaphragm 306 m ay focus light from illumination tube 201 and adjust the amou nt of light passed to apertu re diaphragm 308.
  • Apertu re diaphragm 308 may adjust the amount of light which illuminates specimen 108.
  • UV filter 302 may be positioned paral lel to collimating lens 206 an d in front of dichroic mirror 304.
  • UV filter 302 may filter out UV radiation from an illumination beam (along illumination light path 208).
  • i n cludes an infrared (IR)-transmitting mirror angled at 45°.
  • Dichroic mirror 304 may remove IR radiation and reflect visible-spectrum light i nto field diaph ragm 306, pass through aperture diaphragm 308 and condenser 310 and then illuminate specimen 108.
  • UV-radiation i .e., heat
  • UV filter 302 and dichroic mirror 304 may remove potentially damagi ng IR and UV light with minimal visible-spectrum intensity loss.
  • UV filter 302 an d dichroic mi rror 304 may provide advantages over conventional light sources.
  • dichroic mirror 304 may provide a higher efficiency (reflecting greater than about 90% of incident light i ntensity across the visible spectrum) and cost-effectiveness.
  • dichroic mirror 304 may generate much less heat (because the unwanted electromagnetic wavelengths, in this case infrared light, may pass through dichroic mirror 304 rather than being absorbed by it) than a conventional IR-blocking filter.
  • Cooling and mai ntai ning a stable temperatu re may increases the output intensity of illumination source 206, may prolong its operating lifetime, and may mi nimize spectra l fluctuations.
  • Blocking IR light may decrease heating of specimen 108.
  • Blocking UV light may reduce image haze, damage to an observer's eyes, and damage to speci men 108.
  • Using the same imagi ng spectrometer 114 as th at used to analyze specimen 108 to measure spectral output accounts for any optical aberrations in the illumination and imagi ng pathway (illumination beam path 208 and imaging light path 312) .
  • a scoring preference of a user may be determined to generate a selection criteria of cells of interest.
  • the scoring preference is described further below, with respect to Fig. 5.
  • At step 402 at least one biomarker is labeled on the surface or the interior of a cell with a chromogenic label .
  • a biomarker on or in the cell can be labeled, and the biomarkers can be chosen accordi ng to the investigator analyzing the cells.
  • the biomarker can, for example, be any biomolecule, including any polypeptide, nucleic acid, lipid, phospholipid, glycoprotei n, polysaccharide, and the like, or can be an organelle of the cell .
  • Any label capable of detection using a brightfield microscope can be used in the methods, i ncluding chemi luminescent, metal colloid, and chromogenic labels, with chromogenic labels being highly preferred.
  • chromogenic labels to visualize antibody-labeled proteins of interest can be broadly divided i nto three categories: enzymatic, metal lic and biological ampl ifiers.
  • Two types of enzymatic chromogenic labels include the peroxidase-based and al kaline phosphatases substrates.
  • Metal lic chromogenic labels i n clude, but a re not limited to gold-colloid, nanogold products and silver stai n reagents.
  • Biological ampl ifiers and visualization enhancers include, but are not limited to, digoxigenin and tyrami de, among others. Multiple biomarkers can be labeled on a given cell, and multi ple labels can be used, for a particular biomarker, or for the cell on the whole.
  • the methods are preferably performed in vitro, using cells isolated from a subject, for example, from a pathology biopsy or sample.
  • the isolated cells can be connected together as part of a tissue (for example, connected via connective tissue, polypeptides, and the like), or can be further separated from the tissue isolated from the subject (for example, by enzymatic digestion) into groups of individual cells.
  • the cell is illuminated with broadband light, for example, from light source device 106 ( Fig . 1) .
  • th e light source and the spectrum of the broadband light are monitored, for example, by controller 116.
  • the light source is adjusted to maintai n a spectral ly stabl e light sou rce, for example, by controller 116 ( Fig. 1). Steps 406 and 408 a re described further below, with respect to Figs. 6A- 6C.
  • a z-stack image of a plurality of optical planes is acquired, for example, by imagi ng spectrometer 114 ( Fig. 1) .
  • Step 410 is described further below with respect to Fig. 7.
  • Step 410 may be performed simultaneously with steps 406 and 408.
  • optical planes in the z-stack image may be reconstructed in three dimensions, for example, by image processor 118 ( Fig. 1) , to form an image of the cells in the specimen .
  • the z-stack data may a lso be visualized using, for example, maximum intensity projection, a three-dimensional methodology to further distinguish the relative and unique positions of labeled biomarker.
  • Each labeled biomarker can be assigned a unique color that clearly contrasts with the other chromogens and background.
  • Concentration gradients can be used to increase image resolution because these gradients are defined by color intensity, such that higher concentrations of markers appear brighter.
  • a three-dimensional model of the labeled cells can be rendered using minimum intensity projection, so as to maintai n the stai n color.
  • Such techniques can allow for high-resolution image visualization of chromogenically-labeled biomarkers in both three-dimensional and x,y,z modes.
  • the image may be processed, for example, by image processor 118 (Fig . 1), to reduce image distorti on and enhance resol ution .
  • deconvolution may be used to su bstantially reverse optical distortion produced, for example, by brightfield microscope 112 ( Fig. 1) .
  • deconvolution may be used to improve signal-to-noise ratios and effectively mi nimize the effects of image distortion inherent i n optical microscopy.
  • the deconvolution may include use of a predetermi ned point spread function (PSF) or blind deconvolution (where the PSF is unknown) techniques.
  • PSF predetermi ned point spread function
  • blind deconvolution techniques are described in Holmes et a l ., "Blind deconvolution of 3D transmitted l ight brightfield micrographs," Journal of Microscopy, 2000, Vol. 200, pages 114 - 127, the contents of which are incorporated herein .
  • Another example of blind deconvolution is described i n Tadrous, "A method of PSF generation for 3D bri ghtfield deconvolution," Journal of Microscopy, 2010, Vol . 237, pages 192-199, the contents of which are incorporated herein .
  • the deconvolution may be appl ied on a wavelength-by-wavelength basis across the imaged spectrum at a each focal plane for the z-stack before cell analysis. It is understood that any suitable deconvolution and/or blind deconvolution techniques, including iterative and non-iterative blind deconvolution techniques, may be used wh ich are capable of reducing image distortion .
  • SI structured illumination
  • acqu isition e.g., during step 410
  • SI techniques may also be appl ied during acqu isition (e.g., during step 410) , such as illumi nati ng the sample with multiple orientations of patterned light, a nd measu ring and analyzing the resulting moire fringes (i .e., the interference pattern ) .
  • SI techniques have been used to improve image resolution of fluorescently labeled objects, and thus may a lso be utilized during acquisition of images from chromogenically labeled material to account for the out-of-focus regions of each image.
  • the images may also be normal ized so that they may be accurately compared regardless of disease state, specimen preparation, or amount of stai ning.
  • a histogram of the optical-density-converted image intensity may be generated.
  • the z-stack represents a series of aligned images taken along the z-axis of the specimen .
  • Each one of the optical slices may be imaged at a number of different wavelengths (so that each optical slice is composed of the same number of images as th e number of wavelengths) .
  • the broadest peak and the highest sharpest peak may detected, and their positions i n the histogram may be determined .
  • a difference in intensity betwee n these two peaks may then be measured as a calibration value.
  • the calibration value may be subtracted from a ll pixels in the image .
  • the same calibration may be performed for each image captu red at one or more desired wavelengths for the regions of interest.
  • the cal ibration value may reduce backgrou nd noise, thus increasi ng the signal-to-noise ratio.
  • selection criteria of the cells may be calibrated according to the user's scoring preference (determined in step 400) , for example, by image processor 118 (Fig . 1) . Step 416 is described further below with respect to Fig. 8.
  • a pl urality of voxels may be determ ined, for example, by cell detector 120 ( Fig. 1) .
  • Individual voxels can be determi ned, for example, by three- dimensional reconstruction of the pixel data from each plane. A spectral profile for each voxel can then be determi ned. Step 418 is described further below with respect to Fig. 9.
  • a volumetric profile may be calculated for each voxel, for example, by cell analyzer 122.
  • the volumetric profile may be calculated using a computer programm ed to ca lculate vol umetric profi les of voxels.
  • the volumetric profile indicates one or more conditions of the biomarker and/or the cell.
  • the conditions can be, but are not l imited to, normal conditions, path ological conditions, pre-pathological conditions, one or more disease states or stages, eel I death, infection, neoplasm, tumor stage, tumor or disease progression, and response to treatment, among others. Step 420 is described further below with respect to Fig. 10.
  • the exemplary method shown in Fig . 4 may be used to address questions concerning the cellular localization of various structures in chromogen ically labeled material, e.g., membrane associated receptors, n uclear transcription factors, specific versus nonspecific staining, the distribution of phenotypically-disti nct cel ls within densely popu lated tissue, and morphometric differences between cells.
  • chromogen ically labeled material e.g., membrane associated receptors, n uclear transcription factors, specific versus nonspecific staining, the distribution of phenotypically-disti nct cel ls within densely popu lated tissue, and morphometric differences between cells.
  • step 400 i n Fig . 4 is illustrated as being performed prior to step 402, step 400 may be performed after step 414 (prior to step 416) .
  • Fig . 4 illustrates the use of a z-stack image and a volumetric profile
  • one optical plane may be detected (for example, by brightfield microscope 112 of Fig. 1) and imaged (for example, using i magi ng
  • Pixels in the image may be used to determi ne at least one region of interest of the cell .
  • the at least one region of interest may be used to calculate a profile, which may indicate one or more conditions of the biomarker and/or the cell .
  • aspects of the present invention include detecting at least one optical plane of the cell with brightfield microscope 112 ( Fig . 1), forming an image of the cell from the detected at least one optical plane (for example, using imagi ng spectrometer 114), determi ning at least one region of interest in the image of the cell and calculating a profile for the at least one region of interest.
  • steps 402-412 are repeated for a predetermi ned specimen, such that ch romogenically labeled
  • predetermi ned cel ls are a nalyzed in three dimensions (into a volumetric image) .
  • the volumetric image is presented to the user, for example, by display device 128 (Fig. 1) .
  • a selection is received for cells of interest from the presented image, for example, by user interface 126 (Fig. 1).
  • a score is received for a feature of at least one chromogen on the cell, for example, by user interface 126 ( Fig. 1) .
  • the feature of the chromogen may include at least one of the identity, intensity, or distribution of the at least one chromogen .
  • a score may optionally be received for an additional feature regardi ng at least one cell, for example, by user interface 126 ( Fig. 1) .
  • the additional feature may include at least one of the cell volume, nuclear volume, cell shape, cel l textu re, or spatial relationship between cells.
  • step 510 cells in the volumetric i mage a re detected, for example, by cel l detector 120 ( Fig. 1) . Cell detection is described further below with respect to Fig . 9.
  • the score for the feature of the chromogen is statically analyzed, for example, by cell analyzer 122 (Fig. 1) .
  • a scoring preference for the feature is determined, based on the statistical analysis, for example, by cell analyzer 122 (Fig. 1).
  • a user's selection criteria may be incorporated into the cell detection, such that the cell detection may be calibrated according to the user's expertise.
  • the statistical analysis may be performed using a learning algorithm, such as a K-means based clustering algorithm.
  • a K-means clusteri ng algorithm is described in Hartigan et al ., "A K-means clustering algorithm," Appl ied statistics, 1979, Vol . 28, pages 100- 108, th e contents of which are incorporated herein.
  • the cell detection may include excluding cells marked as background and recalibrati ng the cell detection to i nclude any cells of interest that were initially missed.
  • the K-means based semi-supervised learning algorithm was used, in which a shape of the spectral curve, the nuclear size and shape are provided as vari ables, and al l cells (including cells of interest) are clustered into two clusters. Cells marked as background may be assigned to one cluster. A centroid of the backgrou nd cell cluster may be included as a parameter in the algorithm . A range of nuclear size and shape may be selected and provided to the cell detecting algorithm, to include potential missed cells.
  • the scoring preference for the featu re may be stored, for example, in memory 124 ( Fig. 1) . Accordi ngly, scoring preferences may be appl ied to the Z-stack image.
  • the score of the additional feature may be statistically analyzed, to determ ine and store a scoring preference for the additional featu re, similarly to steps 512-516 above.
  • selection criteria for the chromogenic analysis of cells may then be calibrated with the scoring preferences (step 416 of Fig. 4) .
  • the calibrated selection criteria may also be appl ied to subsequent chromogenic analyses of cells.
  • the method steps shown in Fig . 5 may be repeated at least one time, and may be repeated mu lti ple times, for example, to increase the statistical confidence.
  • one or more of the method steps may be automated.
  • a user may select each cell of interest in a three-dimensional manipulatible-volume (which concurrently displays the x,y,z planes), and may assign a score reflecting their impression of the quantity of chromogenically labeled elements (and other relevant diagnostic criteria) present in the cell .
  • the present invention provides continuous mon itoring and feedback to a user's input, by tracking, among other things, the identity, spectral label intensity and spatial distribution of chromogens in cells in a sample. Additional data captu red include cell and nuclear volumes, shape, texture and spatial relationships between cells.
  • the monitoring and feedback can be provided using various software, including multiple programs or aspects of multiple programs con currently, by integrati ng the interaction of the software.
  • Selection criteria are cal ibrated by statistical analysis of one or more of the variables, and the calibration is appl ied to additional subsequent sam ples.
  • This th ree- dimensional methodology analyzes and can automate scoring of samples based on much more sophisticated criteria (including those described and exempl ified herein, such as the preceding paragraph ) than conventional methodology, while retaining the scoring preferences of the interpreting practitioner.
  • Figs. 6A-6C exemplary methods for monitoring and adjusting the light source (steps 406 an d 408 of Fig. 4) are shown.
  • Fig . 6A is a flowchart illustrati ng an exemplary method of monitoring and regulating the spectrum of light source device 106 ( Fig . 1 )
  • Fig . 6B is a flowchart i llustrating an exemplary method of monitoring and regulati ng the junction temperatu re of illumination source 204 (Fig. 2);
  • Fig . 6C is a flowchart illustrati ng an exemplary method of monitoring and regulating the a current to illumi nation source 204 ( Fig . 2) .
  • Figs. 6A-6C may be used to thermally- and spectral ly-regulate light source device 106 (Fig. 1), to maintain a stable spectrum of light source device 106.
  • the spectrum of the broadban d light used to illuminate specimen 108 (Fig. 1) is acquired, for example, as a multispectral image by imaging spectrometer 114 (Fig. 1) .
  • the spectrum is compared to a reference spectrum, for example, by controller 116 ( Fig. 1) .
  • the reference spectrum may be determ ined, for example, based on man ufactu rer operation specifications for a particular illumination source 204 (Fig . 2) .
  • the reference spectrum may also be determi ned from an multispectral image of illumination source 204 ( Fig. 2) acqu ired upon initialization of system 100.
  • step 604 it is determ ined whether a compari son of the luminous intensity between the spectrum and the reference spectru m is less than an intensity threshold, for example, by controller 116 ( Fig. 1) . If it is determ ined that the luminous intensity is less than the intensity threshold, step 604 proceeds to step 6 06. At step 606, th e current to illumination source 204 (Fig. 2) is increased and step 606 proceeds to step 600.
  • step 604 proceeds to step 608.
  • step 608 it is determ ined whether a compari son of the spectru m and the reference spectrum indicates a sh ift in the spectrum, for example, by control ler 116 ( Fig . 1) .
  • a shift of about 10 nm, more generally between about 5-15 nm, may represent a th reshold for indicating a spectrum shift.
  • step 608 proceeds to step 610, and a z-stack image of specimen 108 ( Fig . 1) is acquired, for exampl e, by brightfield microscope 112 ( Fig . 1) and imagi ng spectrometer 114.
  • step 608 proceeds to step 612, At step 612, it is determi ned whether the spectru m is shifted to longer wavelengths, for example, by controller 116 ( Fig. 1) .
  • step 612 proceeds to step 614.
  • step 614 the temperatu re to illumination source 204 (Fig. 2) is decreased and step 614 proceeds to step 600.
  • step 612 proceeds to step 616.
  • step 616 the temperatu re to i llumination source 204 ( Fig . 2) is increased and step 616 proceeds to step 600.
  • the junction temperature of illumination source 204 ( Fig. 2) is measured, for example, by temperature-monitoring probe 212 (Fig . 2).
  • step 622 proceeds to step 624.
  • step 624 th e current to Peltier element 209 (Fig. 2) is increased and step 624 proceeds to step 620.
  • step 622 proceeds to step 626.
  • i t is determi ned whether the junction temperatu re is less than the temperatu re threshold, for example, by controller 116 ( Fig. 1) . If it is determ ined that the junction temperatu re is less than the temperatu re threshold, step 626 proceeds to step 628.
  • step 628 the current to Peltier element 209 ( Fig. 2) is decreased and step 628 proceeds to step 620. Otherwise, step 626 proceeds to step 620.
  • the current to i llumination source 204 ( Fig. 2) is measured .
  • step 632 proceeds to step 634.
  • step 634 the current to illumination source 204 ( Fig. 2) is decreased and step 634 proceeds to step 630.
  • step 632 proceeds to step 636.
  • step 636 it is determ ined whether the current is less than the current threshold, for example, by controller 116 ( Fig. 1) . If it is determi ned that the current is less than the current threshold, step 636 proceeds to step 638. At step 638, the current to illumination source 204 (Fig. 2) is increased and step 638 proceeds to step 630. Otherwise, step 636 proceeds to step 630.
  • Figs. 6B and 6C may represent continuous feedback loops performed th roughout z-stack acquisition (step 410 in Fig. 4) that may rescue spectral information through changes i n their respective temperatu re and current set poi nts.
  • a flow chart illustrating an exemplary method for acquiring a Z-stack image (step 410 i n Fig . 4) is shown .
  • sampl e stage 110 ( Fig. 1) is positioned to a n optimal plane of focus for at least one desired region of interest (ROI) for sampl e 108.
  • ROI region of interest
  • specimen 108 (Fig . 1) is moved out of the light path .
  • a spectrum of light source device 106 is measured, for example, by imaging spectrometer 114 (Fig. 1)
  • i t is determi ned whether the spectrum is different from a reference spectrum . If it is determi ned that the spectrum is different from the reference spectru m, step 706 proceeds to step 708.
  • steps 406 an d 408 are repeated, as described above, until a spectral ly-stabl e spectrum is obtai ned . Step 708 proceeds to step 710.
  • step 706 proceeds to step 710.
  • step 710 a pl urality of optical planes are detected with brightfield microscope 112 ( Fig. 1) and step 710 proceeds to step 712.
  • step 712 a z-stack i mage of the optical planes is acquired.
  • a flow chart illustrati ng an exemplary method for calibrating cellular selection criteria (step 416 i n Fig. 4) is shown .
  • Various steps shown in Fig . 8 may be performed, for example, by i mage processor 118 (Fig . 1) .
  • the selection criteria specific to a user is appl ied to the volumetric image of the cells.
  • the cal ibrated image of the cells are presented to the user, for exam ple, by display device 128 (Fig. 1) .
  • the presented image may include highlighti ng of the calculated cel ls of interest (based on the selection criteria) .
  • the user may review the highlighted cells to determi ne whether the selection criteria accurately reflects the user's scoring
  • an indication is received from the user indicating whether all of the desired cel ls of interest are highlighted, for example, by user interface 126 ( Fig . 1 ).
  • step 806 it is determ ined whether the received i ndication indicates that al l of the cells of interest are highlighted. If all of the cells are highlighted, step 806 proceeds to step 808. If all of the cells are not highlighted, step 806 proceeds to step 810.
  • step 808 it is determ ined whether cells that are not of interest are also highlighted . If cells not of interest are not highlighted, step 808 proceeds to step 814 and the process continues to step 418 ( Fig. 4) . If cells not of interested are also highlighted, step 808 proceeds to step 810.
  • step 810 an indication is received from the user which indicates any missing cells, for example, via user interface 126 ( Fig . 1) .
  • step 812 it is determi ned whether a predetermined percentage of cells of interest are highlighted . For example, whether about 95% of cells are highlighted. If it is determined that the predetermi ned percentage of cells are highlighted, step 812 proceeds to step 814 an d the process continues to step 418 ( Fig. 4) .
  • step 812 proceeds to step 816.
  • the scoring preferences are modified responsive to the indicated missing cells.
  • step 818 th e modified scoring preferences are stored, for example in memory 124 ( Fig . 1) . Step 818 proceeds to step 800.
  • a flow chart i llustrati ng an exemplary method for determi ning a pl urality of voxels (step 418 of Fig. 4) representing cells of interest is shown .
  • Various steps of Fig. 9 may be performed by cel l detector 120 (Fig. 1) .
  • the volumetric image is processed with a multi-step algorithm that locates cells (nuclei) of interest.
  • the volumetric image is received .
  • al l objects i .e. , potential cells of interest
  • all objects having an intensity greater than zero may be selected.
  • th resholds for sub- algorithms are set, based on statistical analysis (e.g., k-means clustering) of the selected cells.
  • the statistical analysis may include, for example, cell density, distribution, volume, a nd number of nuclei per cell .
  • each object that is interpreted may be visually displayed as a nucleus by highlighti ng it in the 3D volume.
  • step 904 objects outside of a predetermined range of volume are excluded, to remove unwanted objects such as i nflammatory cells and staining artifacts.
  • objects that have not been excluded are subject to expansion and dilation processes (also referred to herein as growi ng and su btraction processes) to exclude background and stromal cells.
  • the remaining objects are combi ned, by averaging the spectra at each voxel to obtain a single spectrum representing each cell .
  • objects that are touching are separated .
  • Step 912 objects are excluded based on volume, whether they are located on the edge of the image and based on object shape.
  • Step 912 may include a curve- analysis process that identifies and excludes aberrant objects by th eir spectra and a morphometric sub-algorithm that excludes any remai ning aberrant objects that are similar in size a nd spectra to objects of interest, but may not be cells.
  • the morphometric sub-algorithm may compare a shape formed by an outline of the nucleus under analysis to an oval of a perimeter that would fit within the nucleus.
  • a least squares approach may be used to determine whether a difference between the nucleus of interest and the oval is greater than a standard deviation than an average measu red nucleus. If the difference between the nucleus and the oval is greater than a standard deviation from a mean nucleus-oval difference, the object may be excluded as an artifact. At step 914, the remaining objects may be extracted a s cells of interest.
  • the calibrated i mage taken at one of the wavelengths may be selected (for each region of interest) .
  • the log of every pixel's intensity value may be obtai ned and may be used to convert the image i nto bi nary values, where any pixel with a value greater than 1 is assigned an intensity value of 100%.
  • This image may be used as a starti ng template for the cell detection .
  • a flow cha rt illustrating an exemplary method for calculating a volumetric profile (step 420 in Fig. 4) is shown.
  • Various steps of Fig. 10 may be performed, for example, by cel l analyzer 122 ( Fig . 1) .
  • the spectral profile of voxels (i .e., cells) of interest are analyzed.
  • the absorban ce spectrum of cells of interest may be determ ined .
  • step 1002 it is determi ned whether there is more than one chromogen .
  • cells of interest may include multiple spatial ly overlapping chromogens. If it is determi ned that there is more than one chromogen, step 1002 proceeds to step 1004. If it is determ ined that there are no overlappi ng chromogens, step 1002 proceeds to step 1006
  • the component signal from each component chromogen is unmixed (i .e. separated).
  • unmixing techniques There are multiple types of unmixing techniques, the most accurate of which take a multispectral approach . Even a multispectral approach may not accurately perform both unmixing of each component spectral curve (one for each chromogen) and scaling of the component curves so that the absorbance value at the wavelength of maxi mal absorption for each component curve is accurately calculated for each cell of interest.
  • the challenge of signal unmixing can be addressed in several ways, depending on the number of chromogens used and the spectral simi larity of multiplexed chromogens.
  • the spectral profi le of each voxel can be analyzed, and the chromogens can be unmixed based on their individual spectra. This may not be necessary in samples labeled with chromogens having distinct spectra . In such situations, statistical analyses of the spectral histogram for each channel in each focal plane may be used . Unmixed chromogens may be given an individual , unique pseudocolor look up table (LUT) that assigns brighter colors (higher pixel val ue) to pixels with lower intensity val ues. Pixel values may include both chrominance and luminance values.
  • LUT unique pseudocolor look up table
  • the nucleus is classified into grades. Based on this classification procedure, the original image may be visually spl it into, for example, four (one for each grade 0 to 3+, and another for background) multi-spectra i mages.
  • Classification may be performed by compari ng spectral curves at each voxel (or pixel) in an image to pre-selected spectral library curves (both pixel based and average- spectra based) .
  • This may produce, for example, four different spectral curves (one for each grade i n the library) and on a pixel-by-pixel basis to determ ine which curve the spectra at that pixel most closely resembles, or to classify it as backgrou nd if its unlike any of the library curves.
  • the original image may be visual ly split into five (one for each grade and another for backgrou nd) multi- spectra i mages.
  • object-based average spectra may be used for comparison to the pre-selected curves.
  • the classification may be performed using a learning algorithm such as k-nearest neighbor or Parzen window, which generally function by comparing any new data sets to an already classified training set (such as nuclei already classified into one of four different groups each representing a grade from 0 to 3+) and assigns the new data set to the group to which it is most similar.
  • K-nearest neighbor and Parzen window learning algorithms are described i n Bottou et al ., "Local Learning Algorithms," Neu ral Computation, 1992, Vol. 4, pages 888 -900, the contents of which are incorporated herein .
  • a score is computed for the specimen based on the classification of step 1006.
  • the score may be used to quantify a relative amount of analyte in each nucleus.
  • a number of nuclei from each grade may be divided by the total number of nuclei counted in the speci men, yielding a percentage for each grade.
  • the amount of analyte may provide an indication of a disease.
  • lo morphometry as calibrated by a practitioner, who may or may not be usi ng software.
  • a flow chart illustrati ng an exemplary method of generating a library of spectral profiles is shown .
  • the spectral profile library may be used for classification of nuclei .
  • Various steps of Fig . 11 may be performed by image analyzer 104.
  • a volumetric image is generated for each slide in a trai ning set.
  • a cal ibrated selection criteria is appl ied to the volumetric images. As discussed above, the selection criteria may take into account a specific user's scoring preferences.
  • cells of interest are determ ined for each slide.
  • a spectral profile is extracted from each nucleus from all slides of the training set.
  • step 1110 it is determi ned whether there is more than one chromogen . If it is determi ned that there is more than one chromogen, step 1110 proceeds to step 1112. At step 1112, overlappi ng chromogens are unmixed . Step 1112 proceeds to step 1114.
  • step 1110 proceeds to step 1114.5
  • clustering a nalysis e.g., k-means clustering
  • ea ch nucleus is classified i nto grades, based on the clustering analysis.
  • the curve data that is classified into grades is stored, for example, in memory 124 ( Fig. 1) .
  • a prognosis of a0 subject having a tumor is shown .
  • the method may be performed, for example, using system 100 (Fig. 1) .
  • analyzing chromogenically labeled cells are analyzed from a tissue sampl e isolated from the subject i n three dimensions from a volumetric image.
  • the volumetric image may be obtai ned for example, by performing steps 400- 418 ( Fig . 1) .
  • At step 1202 at least one chromogen on the cells of interest may be scored.
  • the chromogen may be scored, for example, for at least one of identity, intensity, or distribution on the cells.
  • a t least one characteristic of the cells may be scored. The characteristic may include, for example, at least one of cell volume, nuclear volume, cell shape, cell texture, for a spatial relationship between cells, serum tumor- marker concentration, and hormone level in the tissue sample.
  • a tumor type may be scored, for example, as normal, benign, or malignant.
  • the score determined in steps 1202 an d 1204 an d, optionally step 1206 may be statistical ly analyzed, for example, by k-means clusteri ng (as described above) to determi ne commonalities in the scores.
  • a prognosis profile may be determined for the subject, based on the statistical analysis (step 1208) .
  • a prognosis of the subject may be determined from the prognosis profi le.
  • th e prognosis profile may be compared to a reference profi le.
  • the reference profile may include, for example, a profi le previously generated from the subject, a reference profile of a healthy population of the subject, a reference profi le of a population of the subject having a benign tumor, or a reference profile of a population of the subject having a malignant tu mor.
  • a prognosis of the subject may be determi ned based on the comparison .
  • the invention improves on the shortcomings of current methodologies attem pting to determine analyte concentrations, through analysis of the absorption cross-section (across a number of wavelengths of light) at each optical plane.
  • the spatial resolution is significantly increased, and analyte localization, identity, and distri bution can be accurately determi ned.
  • a pseudo-colored 3D map of the analyte concentrations and distributions is rendered to spati ally represent the results.
  • Anonymized human parathyroid gland tissue was used to determine proof of principle for aspects of the present invention .
  • a primary antibody targeti ng a nuclear transcription factor was applied and visualized using 3,3' di ami nobenzidine (DAB) chromogenic immunohistochemistry. Nuclei were counterstai ned with hematoxylin . All slides were stained using the BenchMark XT IHC/ISH Stai ning Modu le (Ventana Medical Systems, Arlington, AZ), thus standardizing the proceedu re.
  • DAB 3,3' di ami nobenzidine
  • An automated i mage acquisition station includes an automated xyz stage, a n automated objective turret equipped with Plan APO objectives (Olympus America, Center Valley, PA) and a Nuance multi-spectral imagi ng device (CRi, Woburn, MA).
  • Plan APO objectives Olympus America, Center Valley, PA
  • Nuance multi-spectral imagi ng device CRi, Woburn, MA
  • An exemplary light source device having a spectral ly stable and homogenous illumi nation source was used, as shown in Fig. 2.
  • Regions of interest were imaged under high-power field.
  • the automated z-stage was positioned at the optimal plane of focus, and the optimal exposure time for each wavelength was calculated and stored.
  • the slide was then moved out of the light path and a multi-spectral reference z-stack (the thickness and sampl ing rate identical to those of the ROIs) was acqu ired.
  • the automated system then proceeded to captu re z-stacks of the three ROIs with multi-spectral images captu red at each optical plane and sampl ed (the number of optical planes imaged and the distance between the imaged optical planes) as calculated by the Nyquist-Shannon sam pling theorem .
  • each pixel on the CCD chip i .e. , imagi ng spectrometer 114 ( Fig . 1)
  • the transmittance value of each pixel was converted to its absorban ce value by using the reference multi-spectral reference cube to perform the calculation :
  • Each multi-spectral z-stack was processed with a multi-step algorithm that located nuclei of interest (in this embodi ment those of parenchymal cells) and performed calculations and measu rements, such as cel l density, distribution, volume, and number of nuclei per cell .
  • the initial segmentation was performed using a k- means clustering based al gorithm (a spectral ly-independent way of isolating nuclei) .
  • the program then visually displayed each object that it interpreted as a nucleus by highlighting it in the 3D volume (this feature could be toggled on and off either globally or on a cell by cell basis) .
  • the algorithm then set the thresholds for the sub-algorithms based on statistical analysis of the selected cells.
  • These sub-algorithms included : size exclusion processes to exclude unwanted objects such as inflammatory cel ls and staining artifacts; growing and subtraction processes to exclude background and stromal cells (any objects on the edges of the z-stack were also excluded) ; spectral processes which analyzed each isolated object, averagi ng the spectra at each voxel to obtai n a si ngle spectrum representing each cell; a curve-analysis process that identified and excluded aberrant objects by their spectra ; and a
  • the average absorban ce value across al l voxels of the nucleus was calculated for each wavelength (e.g. , 420n m to 720nm), thereby obtaining a spectral absorbance-curve representing each cell . Since absorbance val ues are directly correlated with chromogen concentration (Beer-Lambert Law) , which in turn is di rectly correlated to transcription factor expression (since the transcription factor is tagged with the chromogen), analysis and classification of the spectral curves could be used to measure relative transcription factor concentration per cell .
  • Fig. 13 is an example multispectral graph for a DAB- labeled nucleus (transcription factor positive) and a hematoxylin-stai ned nucleus (transcription factor negative) .
  • Each curve represents the spectral absorption values (measured across the imaged spectru m) of a single nucleus.
  • Each chromogen has a distinct spectral profile (curve).
  • the DAB chromogen was characterized by an early-rising curve in the 400n m region that peaked in the 440-470nm region, whereas the hematoxylin curve was characterized by a late-rising curve in the 600nm region that peaked i n the 610-640n m region .
  • absorban ce maxi ma and spectral curve shape analysis not only enables accurate nuclear classification, but is the basis of determi ning component chromogen contribution in a mixed-expression nucleus.
  • Fig. 14 is a multispectral graph for a random sample of parenchymal nuclei (a mixture of transcription factor positive and negative nuclei) from a single parathyroid specimen . This graph shows the large range of nuclear labeling and the subtle gradati on in shape between curves. Each curve represents the spectral absorption values (measu red across the imaged spectru m) of a si ngle nucleus.
  • K-means clustering was used to group simi lar-shaped nuclear spectral curves i nto one of four clusters ( Figs. 15-18) , thus creating cluster centroids for classification of nuclei from new specimen .
  • Fig . 15 is a multispectral graph for a random sample of parenchymal nuclei classified as grade zero (transcription factor negative) from multiple parathyroid speci men .
  • Each curve represents the spectral absorption values (measured across the imaged spectrum) of a single hematoxyl in-stai ning nucleus. Although the displayed curves have slight variations in shape, they are all distinctly hematoxylin-shaped (thus are transcription factor negative), so all are classified together in a single class.
  • Fig. 16 is a multispectral graph for a random sample of parenchymal nuclei classified as grade one + (transcription factor positive) from m ultiple parathyroid speci men .
  • Each curve represents the spectral absorption values (measured across the imaged spectrum) of a single DAB-stai ning nucleus.
  • Fig. 17 is a multispectral graph for a random sampl e of parenchymal nuclei classified as grade two + (transcription factor positive) from multiple parathyroid specimen .
  • Each curve represents the spectral absorption values (measu red across the imaged spectrum ) of a single DAB-staining nucleus.
  • Fig. 18 is a multispectral graph for a random sample of parenchymal nuclei classified as grade th ree +
  • Each curve represents the spectral absorption values (measured across the imaged spectru m) of a single DAB-stai ning nucleus.
  • the spectral curve of each nucleus was compi led for all cells from all slides in the trai ning set a nd provided to the algorithm for analysis. Then as new specimens were obtai ned, labeled, imaged, and analyzed, the spectral curve of each of the speci men's nuclei was analyzed by a learning algorithm (e.g., a k- nearest neighbors algorithm) that classifies each nucleus into one of the four classes using the centroid values determi ned from analyses of the training set.
  • a learning algorithm e.g., a k- nearest neighbors algorithm
  • Figures 15- 18 graphically show the results of the clustering analysis, with the spectral curves in each graph representi ng nuclei from all slides i rrespective of disease condition .
  • the identity of each classified nucleus was retai ned so as to en able tabulation of classification results on a slide per slide basis. From all the nuclei classified, only those stai ned by hematoxylin were externally known as to wh ich grade they belonged (grade zero) since they are all transcription factor negative.
  • the clustering algorithm recognized that al l hematoxyl in-stai ning nuclei were different enough from all other spectral curves to be classified separately, yet similar enough to each other to be classified into a single group, as can be seen in Fig . 13.
  • the spectral curves classified as grades 1 -3+ were all DAB-stai ning, and thus transcription factor positive. Within each grade, the nuclear spectral curves showed a great level of similarity, and between grades the spectral curve shape changed in a noticeable way such that they rose more steeply to their maxima and then fell more steeply across the rest of the spectrum as the grade increased (going from 1 + to 3+) .
  • each cluster corresponded to a grade (0-3+) and the number of nuclei from each slide i n each cluster was tabu lated and divided by the total number of nuclei counted on that sl ide, yielding a percentage for each grade.
  • the percentages were multiplied by their corresponding grade (0-3+) and then added togeth er to obtai n scores from 0-300.
  • Figure 19 shows the tabu lated values and overall score of the specimen from Fig. 14.
  • Fig. 19 is a graph of the classification results and overall score (indicati ng overall level of tra nscription factor expression) for a single parathyroid speci men.
  • the dark-grey columns represent the total tally of nuclei classified as each of the four grades (0-3+) for the specimen, and the light-grey columns express the nuclei tally for each grade as a percentage of all nuclei analyzed in the specimen .
  • the white col umn represents the speci men's overall analyte expression .
  • transcription factor level of expression Once transcription factor level of expression has been calculated, it can then be inputted into a learning algorithm (in this aspect, a Bayesian classifier, see, e.g., Egorov V. er a/. (2009) Breast Cancer Research Treatment 118(1) 67-80) along with other computed and collected criteria such as nuclear vol umes, cell density distribution, serum tumor-marker concentrations, and hormone levels, for each tissue specimen from each slide in the training set, along with a class label of the disease di agnosis (e.g. , normal, benign, mal ignant) .
  • a learning algorithm in this aspect, a Bayesian classifier, see, e.g., Egorov V. er a/. (2009) Breast Cancer Research Treatment 118(1) 67-80
  • other computed and collected criteria such as nuclear vol umes, cell density distribution, serum tumor-marker concentrations, and hormone levels
  • the technique can be employed as a prognostic tool (such as in the prognosis of breast cancer), aiding physician in determi ning course of treatment.
  • Figs. 20-23 example graphs of classification results for the cell analysis validation are shown .
  • Fig . 20 is an example graph of classification results and overal l score for normal tissue and carcinoma tissue of the training data set
  • Fig. 21 is an exam ple graph of classification results and overall score for normal tissue of the trai ning data set an d the test data set
  • Fig. 22 is an example graph of classification results and overal l score for carcinoma tissue of the training data set a nd a test data set
  • Fig. 23 is an example graph of classification results and overall score for norma l tissue and carcinoma tissues of the test data set.
  • clustering analysis was performed on all nuclei from the test sets (normal and carcinoma tissue images that were excluded from the initial clustering) through a cl usteri ng algorithm usi ng the cluster centroid val ues obtai ned from the training set, and assigned the nuclei into three clusters labeled 1 +, 2+, and 3+ .
  • a score was calculated to represent each tissue specimen .
  • the values from the test sets and those obtained from the trai ning set indicate that the training and test set values were similar.
  • the calculated test set scores for the normal and carcinoma tissues indicate that the statistically significant difference between the two disease state test sets was ma intai ned.
  • one or more components may be implemented i n software on microprocessors/general purpose computers (not shown) .
  • one or more of the functions of the various components may be implemented i n software that controls a general purpose computer.
  • This software may be embodied in a non-transitory tangible computer readabl e medium, for example, a magnetic or optical disk, or a memory-card.

Abstract

Methods and systems for characterizing a cell in vitro and calibrating selection criteria in samples are provided. The methods and systems utilize brightfield microscopy with a thermally-regulated, spectrally-stable light source. The method includes la beling at least one biomarker on the cell with a chromogenic label, illuminating the cell with a spectrally-stable light source configured to evenly illuminate the cell, detecting a plurality of optical planes of the cell with a brightfield microscope, acq uiring a z-stack image of the optical planes, reconstructing the optical planes in three dimensions to form an image of the cell, determining a plurality of voxels in the image of the cell, and calculating a volumetric profile for each voxel. The volumetric profile indicates one or more conditions of the biomarker and/or the cell.

Description

AUTOMATED QUANTITATIVE MULTIDIMENSIONAL VOLUMETRIC ANALYSIS AND
VISUALIZATION
RELATED APPLICATIONS
This appl ication claims priority on U .S. Provisional Patent Application 61/289,020,
"Automated Quantitative Multidimensional Volumetric Analysis and Visualization," filed December 22, 2009, the disclosure of which is incorporated herein by reference.
FIELD OF THE INVENTION
The present invention relates generally to the field of pathology. More specifical ly, the present i nvention relates to systems and methods for characterizing cells in samples using brightfield microscopy.
BACKGROUND OF TH E INVENTION
Various publ ications, including patents, published appl icati ons, technical articles and scholarly articles are cited throughout the specification . Each of these cited publications is incorporated by reference herein, in its entirety and for all purposes.
Recent and escalati ng progress in the identification and understanding of molecular mechanisms of human disease has brought forth the need to visualize and quantify in tissue the extracellular a nd intracel lular distribution of biomarkers that are both diagnostic and prognostic. Much of the promise of translational biomedicine is contingent upon the abi lity to accompl ish this in a timely and accurate man ner.
Fluorescence i mmunohistochemistry is a valuable tool in this endeavor, but this technique is effectively impractical in clinical setti ngs where more rapid read-outs are required in a high volume and cost-constrai ned environment, and where chromogenic labeling is the standard.
Digital images of chromogenical ly labeled tissue may be acqui red using red/green/blue (RGB) camera filters such as the Bayer filter used i n the majority of color cameras to produce a "color" image. Although satisfactory resolution can be achieved to allow quantification of the expression of a single-antibody labeled protein (though satisfactory results are several ly limited to a smal l subset of labels; furthermore, tissues cannot be counterstained with the standard hematoxylin or eosin stai ns), the poor spatial resolution and spectral fidelity inherent in RGB camera tech nology significantly limits the abi lity to confidently distinguish multiple, spatially overlappi ng
chromogenically labeled biomarkers of interest expressed i n the individual cells of the tissue bei ng examined. Additionally, since cameras using RGB filters or si milarly spectral ly-limited imaging methodologies do not acquire images at a chromogen's wavelength of maxi mal absorption (a requisite of the Beer-Lambert Law) , the accuracy of analysis of such images may be severely limited. Furthermore, the almost non-existent spectral resolution of such cameras cannot identify, and thus cannot account for, any bathochromic shifts of the chromogen absorption maxi mum (a very common occurrence due to the heterogeneity of tissue).
Spectral microscopy addresses such limitations by taki ng advantage of the increased sensitivity provided by a process of un-mixing the acquired spectral data , and analyzing the spectral profiles of each pixel in the acquired image (multiplexing spectroscopy) . Low expression and overlappi ng elements are thereby rendered disti nguishable within the tissue and cells bei ng exam ined . Spectral platforms, however, are l imited in their investigational and commercial usefulness because they are based on predefi ned spectral profiles or un-standardized user-biased man ually-selected representative spectra . Site- and appl ication-specific optimization, therefore, is needed to account for sample variations due to differences in staining protocols, tissues, and interactions between distinct biomarkers, and stati stical methods and automatization are needed to ensure accurate spectral -representation.
A subopti mal ability to confidently distinguish multiple, spatially and spectral ly overlappi ng chromogenically labeled biomarkers of interest expressed i n the individual cells decreases the accuracy of estimates of biomarker expression within a given volume of cellular or tissue material . Current methodologies image a single, two-dimensional optical plane of a region of interest a nd perform all calculations of biomarker expression, often on a scale of 0-4+, on the slice. The sensitivity of this approach , however, is inherently limited by h uman factors, including distracti ng intensity fluctuations from out of focus regions, and because analysis of a three-di mensional object within a two- dimensional space constrai ns efforts to accurately determ ine volumes of distribution. Most importantly, the tissues and biological samples under observation are
heterogeneous in nature, and thus accurate determination of expression and distribution of markers cannot be based on the analysis of a single optical slice (focal plane) .
Three-di mensional images, formed from sampl ing of multi ple focal planes are invaluable in efforts to distinguish contiguous and overlapping elements that m ight otherwise be counted as a single unit i n a two-dimensional projection . Multiple platforms exist to accompl ish three-dimensional image rendering, but are limited to fluorescence immunohistochemistry, where a high resolution of distinct elements is inherent to the technique.
Current methodologies attem pt to quantify the concentration of analytes in a region of interest (i.e. , nuclear compartme nt of a cell) by employing variations of the Beer-Lambert law (BLL), which posits that the absorbance of light in a region is linearly correlated with its concentration. As discussed above, current techniques perform calculations on a si ngle two-dimensional focal plane. These current methodologies incorrectly assume that the absorbance calculated at a pixel of a two-dimensional image is representative of the absorbances of all pixels in focal planes in that volumetric column (they do not take into account the specimen thickness, a key part of the BLL equation) . Because of these limitations, their quantification techniques can only make relative approxi mations at best. Furthermore, they do not have the ability to distinguish between two co-localized analytes and two su perimposed analytes {i.e., two localized receptors as compared to an extra- and an inter-cellular receptor) .
A need for high-resolution visualization and volumetric expression analysis exists, whereby the limitations of prior techniques can be overcome and human error can be minimized .
SUMMARY OF THE INVENTION
The invention features methods for characterizing cells in vitro. The methods generally comprise labeling at least one biomarker on the cells with a chromogenic label, illuminating the cells with a spectrally-stable light sou rce configured to evenly illuminate the cells, detecti ng a plurality of optical planes of the cells with a brightfield microscope, acquiring a z-stack image of the optical planes, reconstructing the optical planes in three dimensions to form an image of the cells, determi ning a plurality of voxels in the image of the cells, and, calculating a volumetric profile for each voxel, wherein the volumetric profile indicates one or more conditions of the biomarker and/or the cells.
The invention also features methods for calibrati ng selection criteria of cells. The methods generally comprise analyzing a chromogenically labeled cell in three
dimensions, scoring at least one of the identity, intensity, or distribution of at least one chromogen on the cell, optionally, scoring at least one of the cell volume, nuclear volume, cel l shape, cell texture, or spatial relationship between cells, inputting the score of the identity, intensity, or distribution of at least one chromogen on the cell into a computer programm ed to stati stically analyze the scores, optionally inputti ng the score of the cell volume, nuclear volume, cell shape, cell texture, or spatial relationship between cells into the computer, analyzing the scores on the computer to determi ne the scoring preference for each variable, optionally, repeating each of these steps at least one time, and, cal ibrati ng selection criteria for the chromogenic analysis of cells with the scoring preferences, wherein the calibrated selection criteria are appl ied to subsequent chromogenic analyses of cells. In some preferred aspects, one or both of the scoring steps is automated.
The invention also features systems for characterizing cel ls in vitro. The systems comprise a spectral ly-stable light source configured to evenly illuminate a cell, a brightfield microscope, an imaging spectrometer, and a processor configured to calculate a volumetric profile for each voxel determined from a digital image of the cell .
The invention also features methods for determining the prognosis of a su bject having a tumor. In general, the methods comprise analyzing chromogenically labeled cells from a tissue sample isolated from the subject i n three di mensions, scoring at least one of the identity, intensity, or distribution of at least one chromogen on the cells, scoring at least one of the cell volume, nuclear volume, cell shape, cell texture, or spatial relationship between cells, serum tumor-marker concentration, and hormone level in the tissue sample, optionally, scoring the tumor type as normal , benign, or malignant, inputting the score of the identity, intensity, or distribution of at least one chromogen on the cells, and the cell volume, nuclear volume, cell shape, cell texture, or spatial relationship between cells, serum tumor-marker concentration, and hormone level in the tissue sample, and optionally inputti ng the tumor type score, i nto a computer programmed to statistical ly analyze the scores, analyzing the scores on the computer and generati ng a profile for the subject, and determi ning a prognosis of the subject from the profile. The methods can further comprise compari ng the generated profi le to a profile previously generated from the subject, to a reference profile of a healthy population of the subject, to a reference profile of a population of the subject havi ng a benign tumor, or to a reference profi le of a popu lation of the subject having a mal ignant tumor, and determi ning a prognosis of the subject from the comparison .
BRIEF DESCRIPTION OF THE DRAWINGS
Fig. 1 is a block diagram i llustrati ng an exemplary system for characterizing cells in vitro, according to a n embodi ment of the present invention ;
Fig . 2 is a block diagram i llustrati ng an exemplary spectrally stable light source used in the system shown in Fig. 1, accordi ng to a n embodi ment of the present invention;
Fig. 3 is a block diagram i llustrati ng an exemplary image captu re section of the system shown in Fig. 1, accordi ng to an embodi ment of the present invention;
Fig. 4 is a flow chart illustrati ng an exemplary method for characterizing cel ls in vitro, according to an embodi ment of the present invention;
Fig. 5 is a flow chart illustrati ng an exemplary method for developing cellular selection criteria, according to an embodi ment of the present invention ;
Figs. 6A, 6B and 6C are flow charts illustrati ng exemplary methods for monitoring and adjusting the light source to maintai n a stable spectrum of the light source, accordi ng to embodiments of the present i nvention;
Fig. 7 is a flow chart illustrati ng an exemplary method for acquiring a Z-stack image, according to an embodi ment of the present invention;
Fig. 8 is a flow chart illustrati ng an exemplary method for cal ibrati ng cellular selection criteria, according to a n embodi ment of the present invention;
Fig. 9 is a flow chart illustrati ng an exemplary method for determining a plurality of voxels representing cells, accordi ng to an embodi ment of the present invention;
Fig. 10 is flow chart illustrating a n exemplary method for calculati ng a volumetric profile, according to an embodi ment of the present invention; Fig . 11 is a flow chart illustrati ng an exemplary method of generating a library of spectral profiles, according to an embodiment of the present invention;
Fig . 12 is a flow chart illustrati ng an exemplary method of determining a prognosis of a subject having a tumor, according to an embodiment of the present invention;
Fig. 13 is an example multispectral graph of absorption as a function of wavelength for a DAB-labeled nucleus (transcription factor positive) and a hematoxyl in- stained nucleus (transcription factor negative), according to a cell characterization method of the present invention ;
Fig. 14 is an example multispectral graph of absorption as a function of wavelength for a random sam ple of parenchymal nuclei (a mixture of transcription factor positive and negative) from a single parathyroid specimen, according to a cell characteri zation method of the present invention ;
Fig. 15 is an example multispectral graph of absorption as a function of wavelength for a random sample of parenchymal nuclei classified as grade zero
(transcription factor negative) from multiple parathyroid specimen, according to a cel l characterization method of the present invention ;
Fig. 16 is an example multispectral graph of absorption as a function of wavelength for a random sample of parenchymal nuclei classified as grade one + (transcription factor positive) from multiple parathyroid specimen, according to a cell characterization method of the present invention ;
Fig. 17 is an multispectral graph of absorption as a function of wavelength for a random sampl e of parenchymal nuclei classified as grade two + (transcription factor positive) from multiple parathyroid specimen, according to a cell characterization method of the present invention ;
Fig. 18 is an example multispectral graph of absorption as a function of wavelength for a random sam ple of parenchymal nuclei classified as grade three + (transcription factor positive) from multiple parathyroid specimen, according to a cell characteri zation method of the present invention ;
Fig. 19 is an example graph of classification results and overall score (indicating overall level of transcription factor expression) for a single parathyroid specimen, according to a cell characterization method of the present invention;
Fig. 20 is an example graph of classification results and overall score for normal tissue and carcinoma tissue of a training data set, according to a cel l characterization method of the present invention ;
Fig. 21 is an example graph of classification results and overall score for normal tissue of the trai ning data set and a test data set, according to a cell characterization method of the present invention; Fig. 22 is an example graph of classification results and overall score for carcinoma tissue of the training data set an d a test data set, according to a cel l characterization method of the present invention ; and
Fig. 23 is an example graph of classification results and overall score for normal tissue and carcinoma tissues of the test data set, accordi ng to a cell characteri zation method of the present invention.
DETAILED DESCRIPTION OF THE INVENTION
Various terms relati ng to the methods and other aspects of the present invention are used throughout the specification and claims. Such terms are to be given their ordinary mean ing in the art unless otherwise indicated. Other specifically defined terms are to be construed in a manner consistent with the definition provided herein.
As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the content clearly dictates otherwise. Thus, for example, reference to "a cel l" includes a combi nation of two or more cells, and the like.
A "biological sampl e" includes any cell, tissue, fluid, and the like isolated or otherwise obtai ned from an organism. Any organism can be used, with mam mals being highly preferred and humans bei ng most preferred.
It has been observed in accordance with the present invention that using a spectral ly stable light source to illuminate a chromogenically-labeled biological sample results in a more consistent i mage acquisition and analysis of the sam ple, thereby producing a more consistent and more accurate characterization of conditions of cells in the sample. Accordingly, the invention featu res methods for imagi ng and characterizing cells. The imagi ng and analysis of multiple chromogenical ly labeled biomarkers through proper sampl e illumination, z-stack acquisition, and multiplex spectroscopic imagi ng, followed by th ree-dimensional volumetric quantification of biomarker expression significantly enhances the resolution of images of pathology sampl es viewed
microscopically.
Referring to Fig. 1, a block diagram of an exemplary system 100 for
characteri zing cells in vitro is shown . System 100 i ncludes image captu re section 102 for acquiring a z-stack image of speci men 108 and image analyzer 104, coupled to image captu re section 102, to determi ne a volumetric profile from the z-stack i mage. The volumetric profile may indicate one or more conditions of a biomarker on cells of speci men 108 and/or the cells of specimen 108. System 100 may also include memory 124, user interface 126 an d display device 128. Suitable components for use within system 100 will be understood by one of skill in the art from the description herein .
Memory 124 may store two dimensional images for various optical planes, as well as an acquired z-stack i mage from image capture section 102. Memory 124 m ay also store digital images, vol umetric profiles and/or cell analysis results from image analyzer 104. Mem ory 124 may be a memory, a magnetic disk, a database or essentially any local or remote device capabl e of stori ng data .
User interface 126 may be coupled to image captu re section 102 and/or image analyzer 104. User i nterface 126 m ay be used to control image captu re in image capture 102 and image a nalysis in image analyzer 104. For exa mple, user interface 126 may be used to select positions of sample stage 110, adjust performance parameters of light source device 106, control a focus of brightfield microscope 112 and select optical planes for z-stack i mage captu re. User interface may also be used to select image processing parameters reducing image distortion in image processor 118, provide cell selection in cell detector 120, a nd for cel l analysis i n cell analyzer 122. User interface 126 may also be capable of selecting i mages to be displayed and/or stored, and may include a text interface for entering information .
Display device 128 may be coupled to image captu re section 102 and/or image analyzer 104. Display device 128 m ay present, for exam ple, two di mensional and z- stack i mages to a user (from image capture section 102) , cells selected by cell detector 120 and analysis of cells from cell analyzer 122.. It is contemplated that display device 128 may include any display capable of presenting information including textual and/or graph ical information .
It is understood that each of image captu re section 102 a nd image analyzer 104 may include one or more of memory 124, user interface 126 an d display device 128. According to another embodiment, one or more of memory 124, user interface 126 and display device 128 may be remote from image captu re section 102 and/or image analyzer 104.
It is contemplated that system 100 may be configured to connect to a global information network, e.g., the Internet, (not shown) such that the z-stack image and/or the volumetric profile may also be transmitted to a remote location for further processing and/or storage.
Image captu re section 102 i ncludes light source device 106, sampl e stage 110 for positioning specimen 108 in an imaging light path 312 ( Fig. 3), brightfield microscope 112, i magi ng spectromete r 114 and controller 116. Controller 116 may be configured to control brightfield microscope 112, sample stage 110 and light source device 106.
Controller 116 may be a digital signal processor.
Light source device 106 represents a spectral ly-stable light source that is configured to evenly illuminate cells of specimen 108. As described fu rther below with respect to Figs. 2, 4 and 6A-6C, light source device 106 may be monitored th roughout the acqu isition of the z-stack i mage. Various performance paramete rs of light source device 106 may be adjusted, for exam ple by controller 116, if light sou rce device 106 deviates from predetermi ned performance parameters. For example, a spectru m, a temperatu re and a current of light source device 106 may be monitored, to thermally and spectral ly regulate light source device 106.
Speci men 108 may be provided on sample stage 110. Sample stage 110 i s configured to position specimen 108 to receive illumination from light source device 106 along one or more of the x, y and z directions.
Brightfield microscope 112 i s configured to receive light modified by interaction with specimen 108. For example, specimen 108 may absorb some of the received light, depending upon the density of regions of the sample. Light modified by the sample may be obtained by brightfield microscope, to detect a pl urality of optical planes.
Imaging spectrometer 114 receives the modified light from brightfield m icroscope 112, and captu res an image of the modified light. Imagi ng spectrometer 114 may also obtai n a spectral profile of the image. Imaging spectrometer 114 may acqu ire a z-stack image of the plurality of optical planes. For example, a vertical stack of images can be captu red at a range of optical planes, by vertically adjusting sample stage 110 at predefi ned intervals, and fully sampling a region of interest (for example, according to the Nyquist-Shannon sampl ing theorem) . Imagi ng spectrometer 114 may capture a spectral profile of each pixel at each focal plane during z-stack acquisition. Z-stack images may be acquired using structured illumination .
Image analyzer 104 includes image processor 118, cell detector 120 an d cell analyzer 122. It is understood that one or more of these components may include a digital signal processor.
Image processor 118 is configured to receive the z-stack image and to reconstruct the optical planes i n three-dimensions, to provide a volumetric image of the cell and its anatomic surroundings. In addition, image processor 118 ma y process the image, for exampl e, to reduce image di stortion . A description of methods for reducing image distorti on is provided further below, with respect to Fig. 4. Furthermore, image processor 118 may calibrate selection criteria of cells in the volumetric image, accordi ng to user-specific scoring preferences. A further description of the calibration is provided below with respect to Fig. 4, 5 and 8.
Cell detector 120 is configured to receive the volumetric i mage from image processor 118 and determ ine a plurality of voxels in the image of the cell . For example, the plurality of voxels may represent the cell and/or specific structu res in the cell .
Cell analyzer 122 is configured to calculate a volumetric profile for each voxel determi ned from selected voxels received from cel l detector 120. Cell analyzer 122 may analyze the selected voxels and provide a nuclear classification and/or provide a score (such as for a particular biomarker) . The cell analysis may be used to i ndicate one or more conditions of the biomarker and/or the cell. Accordi ng to an embodi ment of the present invention, laser captu re
microdissection (LCM) may be used i n image captu re section 102 along with quantitative Reverse Transcription Polymerase Chain Reaction (qRT-PCR) of previously imaged cells (having stored spectral curves), to classify the imaged cells into grades (e.g., 0-3+), to create a standard cu rve correlating values of absorbance maxima to molecularly- determi ned levels of gene expression.
It is understood that the detection, acquisition, and reconstruction provided by system 100 may be carried out according to any means suitable in the art, for example, using a computer programmed to carry out one or more of these functions. For calibration purposes, th e spectral profile at each focal plane of a non-stai ned, control cell may be used.
Referring next to Fig. 2, a block diagram of exemplary light source device 106 i s shown . Light source device 106 m ay be configured to evenly illuminate specimen 108 (Fig. 1) . Light source device 106 is also configured to be thermally-regulated, and to produce spectral ly-stable light (i .e. , with a spectrum that does not shift over time or vary in lumi nous intensity). Light source includes i llumination tube 201, ventilation slits 202, opaque sheet 203 to block ambient light, illumination source 204, diffuser lens 205, collimating lens 206, Peltier element 209, heat si nk 210, high velocity fan 211, temperatu re-monitori ng probe 212, and illumination port adapter 213. Suitable components for use within light source device 106 wi ll be understood by one of skill in the art from the description herein .
Diffuser lens 205, col limati ng lens 206 and illumination source 204 are located along optical axis 207, to provide light along illumination beam path 208. Diffuser lens 205 may combi ne the light from illumination source 204 (for example, from m ultiple light emitting diodes (LEDs) of illumination source 204) into a si ngle spectral ly homogeneous beam , and may project the light maxi mal ly into col limati ng lens 206. Collimating lens 206 may collimate and concentrate the illumi nation light into a tighter beam and di rect it out of through illumination tube 201 and out of illumination port adapter 213 towards brightfield microscope 112 (Fig. 1) . It is understood th at light source device 106 may include additional optical components to provide a uniform light beam to specimen 108 ( Fig . 1) .
In an exemplary embodi ment, sheet 203 is a sheet of aluminum and heat sink 210 is formed of copper. It is understood th at sheet 203 may be formed of any material suitable for substantially blocking ambient l ight from entering light source device 106 and bei ng added to illumination beam path 208. Heat sink 210 may be any heat sink 210 capable of transferring heat from light source device 106.
Illumination port adapter 213 m ay be configured to be attach ed to a brightfield illumination port (not shown) on existing conventional brightfield microscopes. In some alternative aspects, light source device 106 as described herein may be varied by addi ng a motorized dichroic-mirror/filter switching mechanism between illumination port adapter 213 and brightfield microscope 112 ( Fig. 1) . Thus, the user can not only easily customize dichroic-mirror and filter combinations, but may also rapidly switch between these combinations for multiple imaging techniques. For example, the motorized mirror/filter wheel may have one setting for visible spectrum imagi ng, another setting corresponding to an infrared-reflecting dichroic-mirror for infrared or IR-DIC imagi ng, and another setti ng corresponding to an ultraviolet (UV)-reflecting dichroic-mirror for UV illum ination and/or excitation .
Illumination source 204 may i nclude any spectral ly stable light source, defined herein as exhibiting a predi cta ble change i n intensity amplitude and spectral shift when a temperatu re is varied. Illumi nation source 204 may comprise a solid-state light source, such as a l ight emitting diode. The light emitti ng diode may be a broadband source that emits white light. The light emitting diode may be phosphorous coated, for example, a phosphorous-coated LED, to provide warm white light. Accordi ng to another
embodi ment, illumination source 204 includes an LED array . Although examples of white light are illustrated, it is understood that illumination source 204 m ay be configured to provide any suitable range of wavelengths, including single wavelengths. Illumination source 204 ma y be coupled with further optics to provide collimated light. In some aspects, a cl uster of multi-layer phosphorus-coated di odes may be used. In some aspects, combi nations of single-wavelength LEDs or laser diodes that produce a broadband spectrum may be used. Other broadband light sources such as halogen and incandescent bulbs, or solid state l ighting such as amorphous silicon can be used.
Additionally, narrow-ba nd light sources can be used for specific chromogen combi nations or for specialized microscopic techniques such as Infrared Microscopy and Differential Interference Contrast microscopy (DIC) .
Illumination source 204 may be selected to exhibit a predictable (i .e., capable of being mathematically modeled) change in intensity ampl itude and spectral shift when a junction temperatu re or current is varied. Because illumination source 204 exh ibits a predictable change in intensity-amplitude and spectral -shift with varyi ng temperature, these variables can be considered during i magi ng and analysis of speci mens, to mai ntain an inter-specimen and intra-specimen image acqu isition comparabi lity.
Aspects of the present invention include addressing major causes of illumination spectral variation in illumination source 204, including the effects of temperatu re, current, aging, and variations in individual LED performance. Because LEDs are semiconductors, their energy band gaps may vary with changes in current and temperatu re, which in turn changes their luminous output and spectra. As the junction temperatu re increases, for example, the luminous intensity may decrease, causing the output spectrum to shift towards longer wavelengths. As another example, when the current increases, the luminous intensity increases and the output spectru m shifts towards shorter wavelengths. Additionally, LED efficiency may decrease over time through a complex mechanism, thus decreasing luminous output (for example, by about 50% of the original output after 50,000 h ours of operation).
These performance variables may be addressed by cooling illumination source 204 to bel ow ambient temperatu re, by maintai ning a stable junction temperatu re along with a stabl e current and by monitori ng the light source's luminous output and spectra using imagi ng spectrometer 114 ( Fig. 1) . For example, as much as about 60-90% of a high-powered LED's input power is lost as heat. Accordingly, it is desirable to provide thermal management of illumination source 204 to m aintai n a spectral ly stable output.
The junction temperatu re of illumination source 204 may be controlled th rough a combination of free convection through angled ventilation slits 202 (that l imit entrance of ambient light) and active cooling th rough Peltier element 209 (which may dissipate excess thermal energy through a combi nation of heat-pipes and high surface-area radiating fins of heat si nk 210) coupled to high velocity fan 211. All of these elements may be regulated by controller 116 ( Fig . 1) . Cooling illumination source 204 and protecting illumination source 204 from large temperature variations may slow down its degradati on and thus increase its operational lifetime. Current may be controlled using current-regulating circuitry (not shown).
To maintain a stable spectral output, a reference image may be acquired (which may be repeated every time a new specimen is imaged). Controller 116 (Fig. 1) may compare the spectral profile of the current reference image to a predetermined reference spectrum and may adjust the current to illumination source 204 and temperatu re, to recalibrate i llumination source 204 to match the predetermined reference spectrum and intensity.
Referring to Fig. 3, a block diagram of image capture section 102 is shown, illustrati ng a relationship between components of image captu re section 102 along optical axis 207 and i magi ng light path 312. Image captu re section 102 also illustrates UV filter 302, dichroic mi rror 304, field diaphragm 306, apertu re diaphragm 308 an d condenser 310, wh ich are positioned between light source device 106 and objective 314 along imaging light path 312. Fig 3 also illustrates controller 116 cou pled to light source device 106. Suitable components for use within light source device 106 wi ll be understood by one of skill in the art from the description herein .
Field diaphragm 306, a perture diaphragm 308, condenser 310 and objective 314 may be components of brightfield microscope 112. Field diaphragm 306 m ay focus light from illumination tube 201 and adjust the amou nt of light passed to apertu re diaphragm 308. Apertu re diaphragm 308 may adjust the amount of light which illuminates specimen 108.
UV filter 302 may be positioned paral lel to collimating lens 206 an d in front of dichroic mirror 304. UV filter 302 may filter out UV radiation from an illumination beam (along illumination light path 208). Accordi ng to an exemplary embodi ment dichroic mirror 304 i ncludes an infrared (IR)-transmitting mirror angled at 45°. Dichroic mirror 304 may remove IR radiation and reflect visible-spectrum light i nto field diaph ragm 306, pass through aperture diaphragm 308 and condenser 310 and then illuminate specimen 108. IR-radiation (i .e., heat) generated by UV filter 302, after absorbi ng the incident UV radiation, may pass through dichroic mirror 304 and thus out of imaging light path 312. Accordi ngly, UV filter 302 and dichroic mirror 304 may remove potentially damagi ng IR and UV light with minimal visible-spectrum intensity loss.
Spectral ly stabl e light source device 106, UV filter 302 an d dichroic mi rror 304 may provide advantages over conventional light sources. Using dichroic mirror 304 may provide a higher efficiency (reflecting greater than about 90% of incident light i ntensity across the visible spectrum) and cost-effectiveness. In addition, dichroic mirror 304 may generate much less heat (because the unwanted electromagnetic wavelengths, in this case infrared light, may pass through dichroic mirror 304 rather than being absorbed by it) than a conventional IR-blocking filter. Cooling and mai ntai ning a stable temperatu re may increases the output intensity of illumination source 206, may prolong its operating lifetime, and may mi nimize spectra l fluctuations. Blocking IR light may decrease heating of specimen 108. Blocking UV light may reduce image haze, damage to an observer's eyes, and damage to speci men 108. Using the same imagi ng spectrometer 114 as th at used to analyze specimen 108 to measure spectral output accounts for any optical aberrations in the illumination and imagi ng pathway (illumination beam path 208 and imaging light path 312) .
Referring next to Fig. 4, an exemplary method for characterizing cells in vitro is shown. At step 400, a scoring preference of a user may be determined to generate a selection criteria of cells of interest. The scoring preference is described further below, with respect to Fig. 5.
At step 402, at least one biomarker is labeled on the surface or the interior of a cell with a chromogenic label . Any biomarker on or in the cell can be labeled, and the biomarkers can be chosen accordi ng to the investigator analyzing the cells. The biomarker can, for example, be any biomolecule, including any polypeptide, nucleic acid, lipid, phospholipid, glycoprotei n, polysaccharide, and the like, or can be an organelle of the cell . Any label capable of detection using a brightfield microscope can be used in the methods, i ncluding chemi luminescent, metal colloid, and chromogenic labels, with chromogenic labels being highly preferred. Commonly used chromogenic labels to visualize antibody-labeled proteins of interest can be broadly divided i nto three categories: enzymatic, metal lic and biological ampl ifiers. Two types of enzymatic chromogenic labels include the peroxidase-based and al kaline phosphatases substrates. Metal lic chromogenic labels i nclude, but a re not limited to gold-colloid, nanogold products and silver stai n reagents. Biological ampl ifiers and visualization enhancers include, but are not limited to, digoxigenin and tyrami de, among others. Multiple biomarkers can be labeled on a given cell, and multi ple labels can be used, for a particular biomarker, or for the cell on the whole.
The methods are preferably performed in vitro, using cells isolated from a subject, for example, from a pathology biopsy or sample. The isolated cells can be connected together as part of a tissue (for example, connected via connective tissue, polypeptides, and the like), or can be further separated from the tissue isolated from the subject (for example, by enzymatic digestion) into groups of individual cells.
At step 404, the cell is illuminated with broadband light, for example, from light source device 106 ( Fig . 1) . At step 406, th e light source and the spectrum of the broadband light are monitored, for example, by controller 116. At step 408, the light source is adjusted to maintai n a spectral ly stabl e light sou rce, for example, by controller 116 ( Fig. 1). Steps 406 and 408 a re described further below, with respect to Figs. 6A- 6C.
At step 410, a z-stack image of a plurality of optical planes is acquired, for example, by imagi ng spectrometer 114 ( Fig. 1) . Step 410 is described further below with respect to Fig. 7. Step 410 may be performed simultaneously with steps 406 and 408.
At step 412, optical planes in the z-stack image may be reconstructed in three dimensions, for example, by image processor 118 ( Fig. 1) , to form an image of the cells in the specimen . The z-stack data may a lso be visualized using, for example, maximum intensity projection, a three-dimensional methodology to further distinguish the relative and unique positions of labeled biomarker. Each labeled biomarker can be assigned a unique color that clearly contrasts with the other chromogens and background.
Concentration gradients can be used to increase image resolution because these gradients are defined by color intensity, such that higher concentrations of markers appear brighter. Alternatively, a three-dimensional model of the labeled cells can be rendered using minimum intensity projection, so as to maintai n the stai n color. Such techniques can allow for high-resolution image visualization of chromogenically-labeled biomarkers in both three-dimensional and x,y,z modes.
At step 414, the image may be processed, for example, by image processor 118 (Fig . 1), to reduce image distorti on and enhance resol ution . For example, deconvolution may be used to su bstantially reverse optical distortion produced, for example, by brightfield microscope 112 ( Fig. 1) . Accordingly, deconvolution may be used to improve signal-to-noise ratios and effectively mi nimize the effects of image distortion inherent i n optical microscopy. The deconvolution may include use of a predetermi ned point spread function (PSF) or blind deconvolution (where the PSF is unknown) techniques. For example, blind deconvolution techniques are described in Holmes et a l ., "Blind deconvolution of 3D transmitted l ight brightfield micrographs," Journal of Microscopy, 2000, Vol. 200, pages 114 - 127, the contents of which are incorporated herein . Another example of blind deconvolution is described i n Tadrous, "A method of PSF generation for 3D bri ghtfield deconvolution," Journal of Microscopy, 2010, Vol . 237, pages 192-199, the contents of which are incorporated herein . The deconvolution may be appl ied on a wavelength-by-wavelength basis across the imaged spectrum at a each focal plane for the z-stack before cell analysis. It is understood that any suitable deconvolution and/or blind deconvolution techniques, including iterative and non-iterative blind deconvolution techniques, may be used wh ich are capable of reducing image distortion .
Because the amount of out-of-focus light increases as optical planes further from the plane of greatest focus are imaged (both above a nd below), such distortion may be significant enough (especially at higher magn ifications, where the loss of contrast is apparent) to include the appl ication of resolution-enhancing techniques. For example, structured illumination (SI) techniques may also be appl ied during acqu isition (e.g., during step 410) , such as illumi nati ng the sample with multiple orientations of patterned light, a nd measu ring and analyzing the resulting moire fringes (i .e., the interference pattern ) . SI techniques have been used to improve image resolution of fluorescently labeled objects, and thus may a lso be utilized during acquisition of images from chromogenically labeled material to account for the out-of-focus regions of each image.
In addition to reducing image distortion, the images may also be normal ized so that they may be accurately compared regardless of disease state, specimen preparation, or amount of stai ning. Accordi ng to an exemplary embodi ment, a histogram of the optical-density-converted image intensity may be generated. The z-stack represents a series of aligned images taken along the z-axis of the specimen . Each one of the optical slices may be imaged at a number of different wavelengths (so that each optical slice is composed of the same number of images as th e number of wavelengths) . For each image taken at each wavelength at each z-dimension, there is a corresponding optical- density converted image i ntensity histogram . The broadest peak and the highest sharpest peak may detected, and their positions i n the histogram may be determined . A difference in intensity betwee n these two peaks may then be measured as a calibration value. The calibration value may be subtracted from a ll pixels in the image . The same calibration may be performed for each image captu red at one or more desired wavelengths for the regions of interest. The cal ibration value may reduce backgrou nd noise, thus increasi ng the signal-to-noise ratio.
At step 416, selection criteria of the cells may be calibrated according to the user's scoring preference (determined in step 400) , for example, by image processor 118 (Fig . 1) . Step 416 is described further below with respect to Fig. 8.
At step 418, a pl urality of voxels may be determ ined, for example, by cell detector 120 ( Fig. 1) . Individual voxels can be determi ned, for example, by three- dimensional reconstruction of the pixel data from each plane. A spectral profile for each voxel can then be determi ned. Step 418 is described further below with respect to Fig. 9.
At step 420, a volumetric profile may be calculated for each voxel, for example, by cell analyzer 122. In some aspects, the volumetric profile may be calculated using a computer programm ed to ca lculate vol umetric profi les of voxels. The volumetric profile indicates one or more conditions of the biomarker and/or the cell. The conditions can be, but are not l imited to, normal conditions, path ological conditions, pre-pathological conditions, one or more disease states or stages, eel I death, infection, neoplasm, tumor stage, tumor or disease progression, and response to treatment, among others. Step 420 is described further below with respect to Fig. 10.
The exemplary method shown in Fig . 4 may be used to address questions concerning the cellular localization of various structures in chromogen ically labeled material, e.g., membrane associated receptors, n uclear transcription factors, specific versus nonspecific staining, the distribution of phenotypically-disti nct cel ls within densely popu lated tissue, and morphometric differences between cells.
The steps i llustrated in the drawi ng figures represent exemplary embodi ments of the present invention . It is understood that certai n steps shown in the figures may be performed in an order different from what is shown . It is also understood that certain steps may be elimi nated. For example, although step 400 i n Fig . 4 is illustrated as being performed prior to step 402, step 400 may be performed after step 414 (prior to step 416) .
Although Fig . 4 illustrates the use of a z-stack image and a volumetric profile, according to another embodiment, one optical plane may be detected (for example, by brightfield microscope 112 of Fig. 1) and imaged (for example, using i magi ng
spectrometer 114) . Pixels in the image may be used to determi ne at least one region of interest of the cell . The at least one region of interest may be used to calculate a profile, which may indicate one or more conditions of the biomarker and/or the cell .
Accordi ngly, aspects of the present invention include detecting at least one optical plane of the cell with brightfield microscope 112 ( Fig . 1), forming an image of the cell from the detected at least one optical plane (for example, using imagi ng spectrometer 114), determi ning at least one region of interest in the image of the cell and calculating a profile for the at least one region of interest.
Referring to Fig. 5, an exemplary method for developing cellular selection criteria (step 400 of Fig. 4) is shown . At step 500, steps 402-412 (and optionally step 414) are repeated for a predetermi ned specimen, such that ch romogenically labeled
predetermi ned cel ls are a nalyzed in three dimensions (into a volumetric image) .
At step 502, the volumetric image is presented to the user, for example, by display device 128 (Fig. 1) . At step 504, a selection is received for cells of interest from the presented image, for example, by user interface 126 (Fig. 1).
At step 506, a score is received for a feature of at least one chromogen on the cell, for example, by user interface 126 ( Fig. 1) . The feature of the chromogen may include at least one of the identity, intensity, or distribution of the at least one chromogen . At step 508, a score may optionally be received for an additional feature regardi ng at least one cell, for example, by user interface 126 ( Fig. 1) . The additional feature may include at least one of the cell volume, nuclear volume, cell shape, cel l textu re, or spatial relationship between cells.
At step 510, cells in the volumetric i mage a re detected, for example, by cel l detector 120 ( Fig. 1) . Cell detection is described further below with respect to Fig . 9.
At step 512, the score for the feature of the chromogen is statically analyzed, for example, by cell analyzer 122 (Fig. 1) . At step 514, a scoring preference for the feature is determined, based on the statistical analysis, for example, by cell analyzer 122 (Fig. 1). In steps 512 an d 514, a user's selection criteria may be incorporated into the cell detection, such that the cell detection may be calibrated according to the user's expertise.
Accordi ng to an exemplary embodiment, the statistical analysis may be performed using a learning algorithm, such as a K-means based clustering algorithm. An example of a K-means clusteri ng algorithm is described in Hartigan et al ., "A K-means clustering algorithm," Appl ied statistics, 1979, Vol . 28, pages 100- 108, th e contents of which are incorporated herein. The cell detection may include excluding cells marked as background and recalibrati ng the cell detection to i nclude any cells of interest that were initially missed.
To exclude cells marked as background, the K-means based semi-supervised learning algorithm was used, in which a shape of the spectral curve, the nuclear size and shape are provided as vari ables, and al l cells (including cells of interest) are clustered into two clusters. Cells marked as background may be assigned to one cluster. A centroid of the backgrou nd cell cluster may be included as a parameter in the algorithm . A range of nuclear size and shape may be selected and provided to the cell detecting algorithm, to include potential missed cells. At step 516, the scoring preference for the featu re may be stored, for example, in memory 124 ( Fig. 1) . Accordi ngly, scoring preferences may be appl ied to the Z-stack image.
At optional steps 518-522, the score of the additional feature may be statistically analyzed, to determ ine and store a scoring preference for the additional featu re, similarly to steps 512-516 above.
Accordi ngly, selection criteria for the chromogenic analysis of cells may then be calibrated with the scoring preferences (step 416 of Fig. 4) . The calibrated selection criteria may also be appl ied to subsequent chromogenic analyses of cells.
Optionally, the method steps shown in Fig . 5 may be repeated at least one time, and may be repeated mu lti ple times, for example, to increase the statistical confidence. In some aspects, one or more of the method steps may be automated.
Accordi ngly, a user may select each cell of interest in a three-dimensional manipulatible-volume (which concurrently displays the x,y,z planes), and may assign a score reflecting their impression of the quantity of chromogenically labeled elements (and other relevant diagnostic criteria) present in the cell . As may be appreciated based on the description herein, the present invention provides continuous mon itoring and feedback to a user's input, by tracking, among other things, the identity, spectral label intensity and spatial distribution of chromogens in cells in a sample. Additional data captu red include cell and nuclear volumes, shape, texture and spatial relationships between cells. The monitoring and feedback can be provided using various software, including multiple programs or aspects of multiple programs con currently, by integrati ng the interaction of the software.
Selection criteria are cal ibrated by statistical analysis of one or more of the variables, and the calibration is appl ied to additional subsequent sam ples. This th ree- dimensional methodology analyzes and can automate scoring of samples based on much more sophisticated criteria (including those described and exempl ified herein, such as the preceding paragraph ) than conventional methodology, while retaining the scoring preferences of the interpreting practitioner.
Referring to Figs. 6A-6C, exemplary methods for monitoring and adjusting the light source (steps 406 an d 408 of Fig. 4) are shown. In particular, Fig . 6A is a flowchart illustrati ng an exemplary method of monitoring and regulating the spectrum of light source device 106 ( Fig . 1 ) ; Fig . 6B is a flowchart i llustrating an exemplary method of monitoring and regulati ng the junction temperatu re of illumination source 204 (Fig. 2); and Fig . 6C is a flowchart illustrati ng an exemplary method of monitoring and regulating the a current to illumi nation source 204 ( Fig . 2) . The methods shown in Figs. 6A-6C may be used to thermally- and spectral ly-regulate light source device 106 (Fig. 1), to maintain a stable spectrum of light source device 106. Referring to Fig. 6A, at step 600, the spectrum of the broadban d light used to illuminate specimen 108 (Fig. 1) is acquired, for example, as a multispectral image by imaging spectrometer 114 (Fig. 1) . At step 602, the spectrum is compared to a reference spectrum, for example, by controller 116 ( Fig. 1) . The reference spectrum, may be determ ined, for example, based on man ufactu rer operation specifications for a particular illumination source 204 (Fig . 2) . The reference spectrum may also be determi ned from an multispectral image of illumination source 204 ( Fig. 2) acqu ired upon initialization of system 100.
At step 604, it is determ ined whether a compari son of the luminous intensity between the spectrum and the reference spectru m is less than an intensity threshold, for example, by controller 116 ( Fig. 1) . If it is determ ined that the luminous intensity is less than the intensity threshold, step 604 proceeds to step 6 06. At step 606, th e current to illumination source 204 (Fig. 2) is increased and step 606 proceeds to step 600.
If the luminous intensity is greater than or equal to the intensity threshold, step 604 proceeds to step 608. At step 608, it is determ ined whether a compari son of the spectru m and the reference spectrum indicates a sh ift in the spectrum, for example, by control ler 116 ( Fig . 1) . For example, a shift of about 10 nm, more generally between about 5-15 nm, may represent a th reshold for indicating a spectrum shift.
If it is determined that the spectru m is not shifted, step 608 proceeds to step 610, and a z-stack image of specimen 108 ( Fig . 1) is acquired, for exampl e, by brightfield microscope 112 ( Fig . 1) and imagi ng spectrometer 114.
If it is determi ned that the spectrum is sh ifted, step 608 proceeds to step 612, At step 612, it is determi ned whether the spectru m is shifted to longer wavelengths, for example, by controller 116 ( Fig. 1) .
If it is determined that the spectru m is shifted to longer wavelengths, step 612 proceeds to step 614. At step 614, the temperatu re to illumination source 204 (Fig. 2) is decreased and step 614 proceeds to step 600.
If it is determined that the spectru m is not shifted to longer wavelengths (i .e. shifted to shorter wavelengths), step 612 proceeds to step 616. At step 616, the temperatu re to i llumination source 204 ( Fig . 2) is increased and step 616 proceeds to step 600.
Referring to Fig. 6B, at step 620, the junction temperature of illumination source 204 ( Fig. 2) is measured, for example, by temperature-monitoring probe 212 (Fig . 2). At step 622, it is determi ned whether the junction temperatu re is greater than a temperatu re threshold, for example, by controller 116 (Fig. 1) .
If it is determined that the junction temperatu re is greater th an the temperature threshold, step 622 proceeds to step 624. At step 624, th e current to Peltier element 209 (Fig. 2) is increased and step 624 proceeds to step 620. If it is determined that the junction temperature is less than or equal to the temperatu re threshold, step 622 proceeds to step 626. At step 626, i t is determi ned whether the junction temperatu re is less than the temperatu re threshold, for example, by controller 116 ( Fig. 1) . If it is determ ined that the junction temperatu re is less than the temperatu re threshold, step 626 proceeds to step 628. At step 628, the current to Peltier element 209 ( Fig. 2) is decreased and step 628 proceeds to step 620. Otherwise, step 626 proceeds to step 620.
Referring to Fig. 6C, at step 630, the current to i llumination source 204 ( Fig. 2) is measured . At step 632, it is determined whether the current is greater than a current th reshold, for exam ple, by controller 116 (Fig. 1 ) .
If it is determined that the current is greater than the current th reshold, step 632 proceeds to step 634. At step 634, the current to illumination source 204 ( Fig. 2) is decreased and step 634 proceeds to step 630.
If it is determined that the current is less than or equal to the current threshold, step 632 proceeds to step 636. At step 636, it is determ ined whether the current is less than the current threshold, for example, by controller 116 ( Fig. 1) . If it is determi ned that the current is less than the current threshold, step 636 proceeds to step 638. At step 638, the current to illumination source 204 (Fig. 2) is increased and step 638 proceeds to step 630. Otherwise, step 636 proceeds to step 630.
Figs. 6B and 6C may represent continuous feedback loops performed th roughout z-stack acquisition (step 410 in Fig. 4) that may rescue spectral information through changes i n their respective temperatu re and current set poi nts.
Referring to Fig. 7, a flow chart illustrating an exemplary method for acquiring a Z-stack image (step 410 i n Fig . 4) is shown . At step 700, sampl e stage 110 ( Fig. 1) is positioned to a n optimal plane of focus for at least one desired region of interest (ROI) for sampl e 108. At step 702, specimen 108 (Fig . 1) is moved out of the light path .
At step 704, a spectrum of light source device 106 is measured, for example, by imaging spectrometer 114 (Fig. 1) At step 706, i t is determi ned whether the spectrum is different from a reference spectrum . If it is determi ned that the spectrum is different from the reference spectru m, step 706 proceeds to step 708. At step 708, steps 406 an d 408 are repeated, as described above, until a spectral ly-stabl e spectrum is obtai ned . Step 708 proceeds to step 710.
If it is determined that the spectru m is substantially the same as the reference spectru m, step 706 proceeds to step 710. At step 710, a pl urality of optical planes are detected with brightfield microscope 112 ( Fig. 1) and step 710 proceeds to step 712. At step 712, a z-stack i mage of the optical planes is acquired.
Referring to Fig . 8, a flow chart illustrati ng an exemplary method for calibrating cellular selection criteria (step 416 i n Fig. 4) is shown . Various steps shown in Fig . 8 may be performed, for example, by i mage processor 118 (Fig . 1) . At step 800, the selection criteria specific to a user is appl ied to the volumetric image of the cells. At step 802, the cal ibrated image of the cells are presented to the user, for exam ple, by display device 128 (Fig. 1) . The presented image may include highlighti ng of the calculated cel ls of interest (based on the selection criteria) . The user may review the highlighted cells to determi ne whether the selection criteria accurately reflects the user's scoring
preferences. At step 804, an indication is received from the user indicating whether all of the desired cel ls of interest are highlighted, for example, by user interface 126 ( Fig . 1 ).
At step 806, it is determ ined whether the received i ndication indicates that al l of the cells of interest are highlighted. If all of the cells are highlighted, step 806 proceeds to step 808. If all of the cells are not highlighted, step 806 proceeds to step 810.
At step 808, it is determ ined whether cells that are not of interest are also highlighted . If cells not of interest are not highlighted, step 808 proceeds to step 814 and the process continues to step 418 ( Fig. 4) . If cells not of interested are also highlighted, step 808 proceeds to step 810.
At step 810, an indication is received from the user which indicates any missing cells, for example, via user interface 126 ( Fig . 1) . At step 812, it is determi ned whether a predetermined percentage of cells of interest are highlighted . For example, whether about 95% of cells are highlighted. If it is determined that the predetermi ned percentage of cells are highlighted, step 812 proceeds to step 814 an d the process continues to step 418 ( Fig. 4) .
If it is determined that the predetermi ned percentage of cells are not highlighted, step 812 proceeds to step 816. At step 816, the scoring preferences are modified responsive to the indicated missing cells. At step 818, th e modified scoring preferences are stored, for example in memory 124 ( Fig . 1) . Step 818 proceeds to step 800.
Referring to Fig . 9, a flow chart i llustrati ng an exemplary method for determi ning a pl urality of voxels (step 418 of Fig. 4) representing cells of interest is shown . Various steps of Fig. 9 may be performed by cel l detector 120 (Fig. 1) . In general, the volumetric image is processed with a multi-step algorithm that locates cells (nuclei) of interest.
At step 900, the volumetric image is received . At step 902, al l objects (i .e. , potential cells of interest) in the volume are selected. For example, all objects having an intensity greater than zero may be selected. At steps 904-912, th resholds for sub- algorithms are set, based on statistical analysis (e.g., k-means clustering) of the selected cells. The statistical analysis may include, for example, cell density, distribution, volume, a nd number of nuclei per cell . During steps 904-914, each object that is interpreted may be visually displayed as a nucleus by highlighti ng it in the 3D volume. At step 904, objects outside of a predetermined range of volume are excluded, to remove unwanted objects such as i nflammatory cells and staining artifacts. At step 906, objects that have not been excluded (selected objects) are subject to expansion and dilation processes (also referred to herein as growi ng and su btraction processes) to exclude background and stromal cells. At step 908, the remaining objects are combi ned, by averaging the spectra at each voxel to obtain a single spectrum representing each cell . At step 910, objects that are touching are separated .
At step 912, objects are excluded based on volume, whether they are located on the edge of the image and based on object shape. Step 912 may include a curve- analysis process that identifies and excludes aberrant objects by th eir spectra and a morphometric sub-algorithm that excludes any remai ning aberrant objects that are similar in size a nd spectra to objects of interest, but may not be cells. The morphometric sub-algorithm may compare a shape formed by an outline of the nucleus under analysis to an oval of a perimeter that would fit within the nucleus. A least squares approach may be used to determine whether a difference between the nucleus of interest and the oval is greater than a standard deviation than an average measu red nucleus. If the difference between the nucleus and the oval is greater than a standard deviation from a mean nucleus-oval difference, the object may be excluded as an artifact. At step 914, the remaining objects may be extracted a s cells of interest.
According to an exemplary embodiment, to increase the ability to accurately detect objects of interest wh ile disregardi ng background artifacts, the calibrated i mage taken at one of the wavelengths (for example, at 580n m) may be selected (for each region of interest) . The log of every pixel's intensity value may be obtai ned and may be used to convert the image i nto bi nary values, where any pixel with a value greater than 1 is assigned an intensity value of 100%. This image may be used as a starti ng template for the cell detection .
Referring to Fig. 10, a flow cha rt illustrating an exemplary method for calculating a volumetric profile (step 420 in Fig. 4) is shown. Various steps of Fig. 10 may be performed, for example, by cel l analyzer 122 ( Fig . 1) . At step 1000, the spectral profile of voxels (i .e., cells) of interest are analyzed. For example, the absorban ce spectrum of cells of interest may be determ ined .
At step 1002, it is determi ned whether there is more than one chromogen . For example, cells of interest may include multiple spatial ly overlapping chromogens. If it is determi ned that there is more than one chromogen, step 1002 proceeds to step 1004. If it is determ ined that there are no overlappi ng chromogens, step 1002 proceeds to step 1006
At step 1004, to distinguish between multiple spatially overlapping chromogens, the component signal from each component chromogen is unmixed (i .e. separated). There are multiple types of unmixing techniques, the most accurate of which take a multispectral approach . Even a multispectral approach may not accurately perform both unmixing of each component spectral curve (one for each chromogen) and scaling of the component curves so that the absorbance value at the wavelength of maxi mal absorption for each component curve is accurately calculated for each cell of interest. The challenge of signal unmixing can be addressed in several ways, depending on the number of chromogens used and the spectral simi larity of multiplexed chromogens.
In some aspects, the spectral profi le of each voxel can be analyzed, and the chromogens can be unmixed based on their individual spectra. This may not be necessary in samples labeled with chromogens having distinct spectra . In such situations, statistical analyses of the spectral histogram for each channel in each focal plane may be used . Unmixed chromogens may be given an individual , unique pseudocolor look up table (LUT) that assigns brighter colors (higher pixel val ue) to pixels with lower intensity val ues. Pixel values may include both chrominance and luminance values.
At step 1006, the nucleus is classified into grades. Based on this classification procedure, the original image may be visually spl it into, for example, four (one for each grade 0 to 3+, and another for background) multi-spectra i mages.
Classification may be performed by compari ng spectral curves at each voxel (or pixel) in an image to pre-selected spectral library curves (both pixel based and average- spectra based) . This may produce, for example, four different spectral curves (one for each grade i n the library) and on a pixel-by-pixel basis to determ ine which curve the spectra at that pixel most closely resembles, or to classify it as backgrou nd if its unlike any of the library curves. Then, based on this classification scheme, the original image may be visual ly split into five (one for each grade and another for backgrou nd) multi- spectra i mages. According to another embodiment, object-based average spectra may be used for comparison to the pre-selected curves.
In another embodi ment, the classification may be performed using a learning algorithm such as k-nearest neighbor or Parzen window, which generally function by comparing any new data sets to an already classified training set (such as nuclei already classified into one of four different groups each representing a grade from 0 to 3+) and assigns the new data set to the group to which it is most similar. Examples of K-nearest neighbor and Parzen window learning algorithms are described i n Bottou et al ., "Local Learning Algorithms," Neu ral Computation, 1992, Vol. 4, pages 888 -900, the contents of which are incorporated herein .
At step 1008, a score is computed for the specimen based on the classification of step 1006. The score may be used to quantify a relative amount of analyte in each nucleus. A number of nuclei from each grade may be divided by the total number of nuclei counted in the speci men, yielding a percentage for each grade. The amount of analyte may provide an indication of a disease.
Conventional scoring methodologies i n two-dimensional image a nalysis captu re only a smal l subset of information provided by a sample, limiti ng the accuracy of 5 analysis. For example, in clinical practice, specimens are typical ly scored by the
interpreting pathologist on an ordi nal scale (0-4+) . Although current platforms are avai lable to perform more automated image analysis, computations are nevertheless limited to a two-dimensional view of modestly resolved acquisitions, using very basic criteria such as pixel intensity, color (captured from RGB cameras), and cel l
lo morphometry as calibrated by a practitioner, who may or may not be usi ng software.
Referring to Fig. 11, a flow chart illustrati ng an exemplary method of generating a library of spectral profiles is shown . As discussed above with respect to Fig. 10, the spectral profile library may be used for classification of nuclei . Various steps of Fig . 11 may be performed by image analyzer 104.
i s At step 1100, a volumetric image is generated for each slide in a trai ning set. At step 1102, a cal ibrated selection criteria is appl ied to the volumetric images. As discussed above, the selection criteria may take into account a specific user's scoring preferences. At step 1104, cells of interest are determ ined for each slide. At step 1106, a spectral profile is extracted from each nucleus from all slides of the training set. At0 step 1108, spectra a re compi led from all nuclei from all the slides of the training set
At step 1110, it is determi ned whether there is more than one chromogen . If it is determi ned that there is more than one chromogen, step 1110 proceeds to step 1112. At step 1112, overlappi ng chromogens are unmixed . Step 1112 proceeds to step 1114.
If it is determined that there is one chromogen, step 1110 proceeds to step 1114.5 At step 1114, clustering a nalysis (e.g., k-means clustering) is performed on the spectral curve of each nucleus for each slide in the training set. At step 1116, ea ch nucleus is classified i nto grades, based on the clustering analysis. At step 1118, the curve data that is classified into grades is stored, for example, in memory 124 ( Fig. 1) .
Referring next to Fig . 12, an exemplary method of determining a prognosis of a0 subject having a tumor is shown . The method may be performed, for example, using system 100 (Fig. 1) . At step 1200, analyzing chromogenically labeled cells are analyzed from a tissue sampl e isolated from the subject i n three dimensions from a volumetric image. The volumetric image may be obtai ned for example, by performing steps 400- 418 ( Fig . 1) .
5 At step 1202, at least one chromogen on the cells of interest may be scored. The chromogen may be scored, for example, for at least one of identity, intensity, or distribution on the cells. At step 1204, a t least one characteristic of the cells may be scored. The characteristic may include, for example, at least one of cell volume, nuclear volume, cell shape, cell texture, for a spatial relationship between cells, serum tumor- marker concentration, and hormone level in the tissue sample. At optional step 1206, a tumor type may be scored, for example, as normal, benign, or malignant.
At step 1208, the score determined in steps 1202 an d 1204 an d, optionally step 1206 may be statistical ly analyzed, for example, by k-means clusteri ng (as described above) to determi ne commonalities in the scores. At step 1210, a prognosis profile may be determined for the subject, based on the statistical analysis (step 1208) . At step 1212, a prognosis of the subject may be determined from the prognosis profi le.
At optional step 1214, th e prognosis profile may be compared to a reference profi le. The reference profile may include, for example, a profi le previously generated from the subject, a reference profile of a healthy population of the subject, a reference profi le of a population of the subject having a benign tumor, or a reference profile of a population of the subject having a malignant tu mor. At optional step 1216, a prognosis of the subject may be determi ned based on the comparison .
The invention improves on the shortcomings of current methodologies attem pting to determine analyte concentrations, through analysis of the absorption cross-section (across a number of wavelengths of light) at each optical plane. By analyzing the atten uation of the spectral absorpti on profile in the z-direction in addition to its attenuation in the x and y directions, the spatial resolution is significantly increased, and analyte localization, identity, and distri bution can be accurately determi ned. A pseudo- colored 3D map of the analyte concentrations and distributions is rendered to spati ally represent the results.
The following examples are provided to describe exemplary aspects of the invention in greater detai l . They are intended to illustrate, not to limit, the invention.
EXAMPLE 1
Anonymized human parathyroid gland tissue was used to determine proof of principle for aspects of the present invention . A primary antibody targeti ng a nuclear transcription factor was applied and visualized using 3,3' di ami nobenzidine (DAB) chromogenic immunohistochemistry. Nuclei were counterstai ned with hematoxylin . All slides were stained using the BenchMark XT IHC/ISH Stai ning Modu le (Ventana Medical Systems, Tucson, AZ), thus standardizing the procedu re.
An automated i mage acquisition station includes an automated xyz stage, a n automated objective turret equipped with Plan APO objectives (Olympus America, Center Valley, PA) and a Nuance multi-spectral imagi ng device (CRi, Woburn, MA). An exemplary light source device having a spectral ly stable and homogenous illumi nation source was used, as shown in Fig. 2. Multiple software platforms were utilized to acquire and a nalyze data, specifically Volocity® (Improvision PerkinElmer, Waltham MA), Nuance™ (CRi, Woburn, MA), ImageJ (National Institutes of Health, Bethesda, MD) , Microsoft Excel®, R-language and environment for statistical computing ( R Foundati on for Statistical Computing, Vienna, Austria) .
Slides were first scanned and imaged at low power magnification in order to identify nuclei and regions of interest ( ROI) positive for immunohistochemical labeling of the substance of interest. From this positive-labeled area, three ROI were selected at random to exami ne and image at higher magnification .
Regions of interest were imaged under high-power field. The automated z-stage was positioned at the optimal plane of focus, and the optimal exposure time for each wavelength was calculated and stored. The slide was then moved out of the light path and a multi-spectral reference z-stack (the thickness and sampl ing rate identical to those of the ROIs) was acqu ired. The automated system then proceeded to captu re z-stacks of the three ROIs with multi-spectral images captu red at each optical plane and sampl ed (the number of optical planes imaged and the distance between the imaged optical planes) as calculated by the Nyquist-Shannon sam pling theorem . The transmittance value of each pixel (each pixel on the CCD chip (i .e. , imagi ng spectrometer 114 ( Fig . 1)) of the camera recorded th e intensity of the incident transmitted l ight) at each wavelength for each optical plane in the z-stack was converted to its absorban ce value by using the reference multi-spectral reference cube to perform the calculation :
-log(%TransmitanceROI-image / %TransmitanceReference-image) * 10000
Performi ng this transformation accounted for any optical and i llumi nation heterogeneity, and also linearly correlated absorbance val ues and transcription factor concentration . Additionally, by taki ng the opposite of the log (using the formula above) , objects of interest (chromogenically stai ned nuclei) were left with positive intensity values: a scaling factor of 10000 was u sed.
Each multi-spectral z-stack was processed with a multi-step algorithm that located nuclei of interest (in this embodi ment those of parenchymal cells) and performed calculations and measu rements, such as cel l density, distribution, volume, and number of nuclei per cell . In this embodiment, the initial segmentation was performed using a k- means clustering based al gorithm (a spectral ly-independent way of isolating nuclei) . The program then visually displayed each object that it interpreted as a nucleus by highlighting it in the 3D volume (this feature could be toggled on and off either globally or on a cell by cell basis) .
All cells of interest in the volume were selected. The algorithm then set the thresholds for the sub-algorithms based on statistical analysis of the selected cells. These sub-algorithms included : size exclusion processes to exclude unwanted objects such as inflammatory cel ls and staining artifacts; growing and subtraction processes to exclude background and stromal cells (any objects on the edges of the z-stack were also excluded) ; spectral processes which analyzed each isolated object, averagi ng the spectra at each voxel to obtai n a si ngle spectrum representing each cell; a curve-analysis process that identified and excluded aberrant objects by their spectra ; and a
morphometric sub-algorithm that excluded any remaining aberrant objects that were simi lar in size and spectra to objects of interest, but were not cells.
For each object of interest (nucleus), the average absorban ce value across al l voxels of the nucleus was calculated for each wavelength (e.g. , 420n m to 720nm), thereby obtaining a spectral absorbance-curve representing each cell . Since absorbance val ues are directly correlated with chromogen concentration (Beer-Lambert Law) , which in turn is di rectly correlated to transcription factor expression (since the transcription factor is tagged with the chromogen), analysis and classification of the spectral curves could be used to measure relative transcription factor concentration per cell .
Each chromogen had a unique spectral curve with a characteristic absorbance maximum, as shown in Fig. 13. Fig . 13 is an example multispectral graph for a DAB- labeled nucleus (transcription factor positive) and a hematoxylin-stai ned nucleus (transcription factor negative) . Each curve represents the spectral absorption values (measured across the imaged spectru m) of a single nucleus. Each chromogen has a distinct spectral profile (curve).
The DAB chromogen was characterized by an early-rising curve in the 400n m region that peaked in the 440-470nm region, whereas the hematoxylin curve was characterized by a late-rising curve in the 600nm region that peaked i n the 610-640n m region . Together, absorban ce maxi ma and spectral curve shape analysis not only enables accurate nuclear classification, but is the basis of determi ning component chromogen contribution in a mixed-expression nucleus.
Within each nuclear classification (DAB stai ning indicati ng transcription factor- expression versus hematoxyl in stai ning indicating lack of expression), the value of the absorbance maximum correlated with the relative nuclear-transcription factor expression level, as can be seen from the variety of similar-shaped spectral curves (each representing a physical nucleus from a single slide) mainly differing in the value of maximal absorbance, as shown in Fig. 14. Fig. 14 is a multispectral graph for a random sample of parenchymal nuclei (a mixture of transcription factor positive and negative nuclei) from a single parathyroid specimen . This graph shows the large range of nuclear labeling and the subtle gradati on in shape between curves. Each curve represents the spectral absorption values (measu red across the imaged spectru m) of a si ngle nucleus.
It should be noted that wh ile the hematoxyl in curves varied in their peak absorbance values, all nuclei stai ning with this marker were not further sub-classified into degrees of expression because hematoxyl in is a counter-stai n that simply indicates lack of transcription factor expression . Additionally, as the staining of nuclei became more intense, the absorbance maxima of the spectral curves not only became greater in val ue (along y-axis), but the shape of the curve changed slightly. These observations were corroborated by comparison of spectral curves from visually similar-intensity stai ning nuclei from several different slides, which showed that they were al l very similar in shape.
K-means clustering was used to group simi lar-shaped nuclear spectral curves i nto one of four clusters ( Figs. 15-18) , thus creating cluster centroids for classification of nuclei from new specimen .
Fig . 15 is a multispectral graph for a random sample of parenchymal nuclei classified as grade zero (transcription factor negative) from multiple parathyroid speci men . Each curve represents the spectral absorption values (measured across the imaged spectrum) of a single hematoxyl in-stai ning nucleus. Although the displayed curves have slight variations in shape, they are all distinctly hematoxylin-shaped (thus are transcription factor negative), so all are classified together in a single class. Fig. 16 is a multispectral graph for a random sample of parenchymal nuclei classified as grade one + (transcription factor positive) from m ultiple parathyroid speci men . Each curve represents the spectral absorption values (measured across the imaged spectrum) of a single DAB-stai ning nucleus. Fig. 17 is a multispectral graph for a random sampl e of parenchymal nuclei classified as grade two + (transcription factor positive) from multiple parathyroid specimen . Each curve represents the spectral absorption values (measu red across the imaged spectrum ) of a single DAB-staining nucleus. Fig. 18 is a multispectral graph for a random sample of parenchymal nuclei classified as grade th ree +
(transcription factor positive) from multiple parathyroid specimen . Each curve represents the spectral absorption values (measured across the imaged spectru m) of a single DAB-stai ning nucleus.
To create the cluster identities, the spectral curve of each nucleus was compi led for all cells from all slides in the trai ning set a nd provided to the algorithm for analysis. Then as new specimens were obtai ned, labeled, imaged, and analyzed, the spectral curve of each of the speci men's nuclei was analyzed by a learning algorithm (e.g., a k- nearest neighbors algorithm) that classifies each nucleus into one of the four classes using the centroid values determi ned from analyses of the training set.
Figures 15- 18 graphically show the results of the clustering analysis, with the spectral curves in each graph representi ng nuclei from all slides i rrespective of disease condition . The identity of each classified nucleus was retai ned so as to en able tabulation of classification results on a slide per slide basis. From all the nuclei classified, only those stai ned by hematoxylin were externally known as to wh ich grade they belonged (grade zero) since they are all transcription factor negative. Thus it was desirable that the clustering algorithm recognized that al l hematoxyl in-stai ning nuclei were different enough from all other spectral curves to be classified separately, yet similar enough to each other to be classified into a single group, as can be seen in Fig . 13.
The spectral curves classified as grades 1 -3+ (Figs. 16- 18) were all DAB-stai ning, and thus transcription factor positive. Within each grade, the nuclear spectral curves showed a great level of similarity, and between grades the spectral curve shape changed in a noticeable way such that they rose more steeply to their maxima and then fell more steeply across the rest of the spectrum as the grade increased (going from 1 + to 3+) .
There were several spectral curves that showed a bathochromic shift of about lOnm (Fig . 14), but on visual inspection of the nuclei under question (how a pathologist would normally analyze the cells), and after comparison of cell characteristics such as shape and size, there were no observed significant differences.
To quantify the relative amount of analyte in each nucleus, a four grade (0-3+) scoring system was used . Each cluster corresponded to a grade (0-3+) and the number of nuclei from each slide i n each cluster was tabu lated and divided by the total number of nuclei counted on that sl ide, yielding a percentage for each grade. To quantify the relative expression of transcription factor in each specimen the percentages were multiplied by their corresponding grade (0-3+) and then added togeth er to obtai n scores from 0-300.
Figure 19 shows the tabu lated values and overall score of the specimen from Fig. 14. Fig. 19 is a graph of the classification results and overall score (indicati ng overall level of tra nscription factor expression) for a single parathyroid speci men. The dark-grey columns represent the total tally of nuclei classified as each of the four grades (0-3+) for the specimen, and the light-grey columns express the nuclei tally for each grade as a percentage of all nuclei analyzed in the specimen . The white col umn represents the speci men's overall analyte expression .
On visual inspection of the tissue section, the investigator would see many positively staining nuclei, and if a simple positive/negative classification was used, it would incorrectly give too much weight to these nuclei, and thus over-esti mate the level of transcription factor expression. When expression is analyzed on the cellular level, however, as shown by the weighted score (Fig. 19), the majority of these positive nuclei showed low to moderate expression, and correspondi ngly, the specimen received a moderate score of 113 (out of 300).
Since th is quantification system is relative, and is based u pon the difference in shape of the DAB spectral -curve at different concentrations, it renders i rrelevant the fact that DAB is not an ideal absorber and is thus not perfectly modeled by the Beer-Lambert Law. EXAMPLE 2
Prediction of Disease Severity (Prognosis)
This is a prophetic exam ple. Once transcription factor level of expression has been calculated, it can then be inputted into a learning algorithm ( in this aspect, a Bayesian classifier, see, e.g., Egorov V. er a/. (2009) Breast Cancer Research Treatment 118(1) 67-80) along with other computed and collected criteria such as nuclear vol umes, cell density distribution, serum tumor-marker concentrations, and hormone levels, for each tissue specimen from each slide in the training set, along with a class label of the disease di agnosis (e.g. , normal, benign, mal ignant) . Then, when new specimens are analyzed, these same parameters can be inputted into the algorithm, which in turn classifies the speci men as one of the disease states, along with how closely it fits this classification . Furthermore, if samples from early stage disease are obtai ned along with accompanying pati ent history and known disease end-state (e.g. , normal, benign, malignant) and analyzed as previously described, the technique can be employed as a prognostic tool (such as in the prognosis of breast cancer), aiding physician in determi ning course of treatment.
EXAMPLE 3
Validation of Cell Analysis between Training and Testing Data Sets
Referring to Figs. 20-23, example graphs of classification results for the cell analysis validation are shown . In particular, Fig . 20 is an example graph of classification results and overal l score for normal tissue and carcinoma tissue of the training data set; Fig. 21 is an exam ple graph of classification results and overall score for normal tissue of the trai ning data set an d the test data set; Fig. 22 is an example graph of classification results and overal l score for carcinoma tissue of the training data set a nd a test data set; and Fig. 23 is an example graph of classification results and overall score for norma l tissue and carcinoma tissues of the test data set.
Multiple regions from a normal parathyroid speci men and one with carci noma were i maged in three di mensions. A cal ibrated multispectral profile of each nucleus from each specimen was measu red. Clustering analysis was perform ed on the multispectral profiles to group the nuclei into three clusters labeled 1 + , 2+ , and 3+ (and to calculate the centroid value of each cluster) . A score was cal culated to represent each tissue specimen, by summing the product of the percentage of nuclei counted in each cluster by the correspondi ng cluster label (to get a score between 0-300) . As shown in Fig . 20, the calculated scores between the normal and carcinoma tissues i ndicate that the difference was statistical ly significant (P-value < 0.05) .
Next, clustering analysis was performed on all nuclei from the test sets (normal and carcinoma tissue images that were excluded from the initial clustering) through a cl usteri ng algorithm usi ng the cluster centroid val ues obtai ned from the training set, and assigned the nuclei into three clusters labeled 1 +, 2+, and 3+ . A score was calculated to represent each tissue specimen . As shown in Figs. 21 and 22, the values from the test sets and those obtained from the trai ning set indicate that the training and test set values were similar. As shown in Fig. 23, the calculated test set scores for the normal and carcinoma tissues indicate that the statistically significant difference between the two disease state test sets was ma intai ned.
Although the invention has been described in terms of systems and methods for characterizing cells in vitro, it is contemplated that one or more components may be implemented i n software on microprocessors/general purpose computers (not shown) . In this embodiment, one or more of the functions of the various components may be implemented i n software that controls a general purpose computer. This software may be embodied in a non-transitory tangible computer readabl e medium, for example, a magnetic or optical disk, or a memory-card.
Although the invention is illustrated and described herein with reference to specific embodiments, the invention is not intended to be l imited to the detai ls shown . Rather, various modifications may be made in the detai ls within the scope a nd range of equivalents of the clai ms and without depa rting from the invention .

Claims

Claims What is Claimed :
1. A method for characterizing a cell in vitro, comprising the steps of
(a) labeling at least one biomarker on the cell with a chromogenic label;
(b) illuminating the cell with a spectrally -stable light source configured to evenly illuminate the cell;
(c) detecting a plurality of optical planes of the cell with a brightfield microscope;
(d) acquiring a z-stack image of the optical planes;
(e) reconstructing the optical planes in three dimensions to form an image of the cell;
(f) determining a plurality of voxels in the image of the cell; and
(g) calculating a volumetric profile for each voxel, wherein the volumetric profile
indicates one or more conditions of the biomarker and/or the cell.
2. The method of claim 1, further comprising :
receiving a score for at least one of an identity, an intensity, or a distribution of at least one chromogen on the cell;
statistically analyzing the received score to determine a scoring preference; and calibrating selection criteria for chromogenic analysis of the cell with the scoring preference, such that the volumetric profile is calculated based on the scoring preference.
3. The method of claim 1, further comprising analyzing a spectral profile of each voxel and unmixing chromogens in each voxel based on their individual spectra.
4. The method of claim 1, further comprising monitoring a temperature of the
spectrally stable light source and adjusting the temperature if the temperature is different from a temperature threshold value.
5. The method of claim 1, further comprising monitoring a current to the spectrally stable light source and adjusting the current if the current is different from a current threshold value.
6. The method of claim 1, further comprising monitoring a spectrum of the spectrally stable light source and adjusting a temperature of the spectrally stable light source responsive to a shift in the spectrum.
7. The method of claim 1, further comprising monitoring a light intensity from the spectrally stable light source and adjusting the light intensity if the light intensity is different form a light intensity threshold value.
8. The method of claim 1, wherein the spectrally stable light source is at least one light emitting diode.
9. The method of claim 8, wherein the at least one light emitting diode emits white light.
10. The method of claim 8, wherein the at least one light emitting diode is phosphorous coated.
11. The method of claim 1, wherein the spectrally-stable light source exhibits a
predictable change in intensity amplitude and spectral shift when temperature is varied.
12. The method of claim 1, wherein the spectral profile of each pixel at each focal plane is captured during step (d).
13. The method of claim 1, wherein individual voxels are determined according to step (f) by three-dimensionally reconstructing the pixel data from each plane.
14. The method of claim 1, wherein the z-stack image is acquired using structured illumination.
15. A method for calibrating selection criteria of cells, comprising the steps of:
(a) analyzing a chromogenically labeled cell in three dimensions;
(b) scoring at least one of an identity, an intensity, or a distribution of at least one chromogen on the cell;
(c) optionally, scoring at least one of a cell volume, a nuclear volume, a cell shape, a cell texture, or a spatial relationship between cells;
(d) inputting the score from step (b) into a computer programmed to statistically analyze the scores;
(e) optionally, inputting the score from step (c) into the computer;
(f) analyzing the scores on the computer to determine a scoring preference for each variable;
(g) optionally, repeating steps (a)-(f) at least one time; and,
(h) calibrating selection criteria for chromogenic analysis of the cell w ith the scoring preference, wherein the calibrated selection criteria are applied to subsequent chromogenic analyses of cells.
16. The method of claim 15, wherein the scoring according to step (b) is automated.
17. The method of claim 15, wherein the scoring according to step (c) is automated.
18. A system for characterizing cells in vitro, comprising :
a spectrally-stable light source configured to evenly illuminate a cell with light; a brightfield microscope configured to receive light modified by the cell;
an imaging spectrometer coupled to the brightfield microscope and configured to produce a z-stack image of a plurality of optical planes, the z-stack image associated with the cell; and
an image analyzer configured to calculate a volumetric profile for each voxel determined from the z-stack image.
19. The system of claim 18, wherein the spectrally stable light source includes at least one light emitting diode.
20. The system of claim 19, wherein the at least one light emitting diode emits white light.
21. The system of claim 19, wherein the at least one light emitting diode is
phosphorous coated.
22. The system of claim 18, wherein the light source is a broadband light source.
23. The system of claim 18, further comprising a thermal regulation system for
adjusting the temperature of the light source.
24. The system of claim 18, wherein the light source comprises a collimating lens.
25. The system of claim 18, further comprising a dichroic mirror between the light source and the brightfield microscope.
26. The system of claim 18, further comprising an ultraviolet (UV) light filter between the light source and the brightfield microscope.
27. The system of claim 18, wherein the system comprises a current regulator for adjusting a current provided to the light source.
28. The system of claim 18, further comprising a controller configured to monitor a spectrum of the light source and to control adjustment of at least one of a temperature of the light source or a current to the light source.
29. A method for determining the prognosis of a subject having a tumor, comprising the steps of:
(a) analyzing chromogenically labeled cells from a tissue sample isolated from the subject in three dimensions;
(b) scoring at least one of the identity, intensity, or distribution of at least one
chromogen on the cells;
(c) scoring at least one of the cell volume, nuclear volume, cell shape, cell texture, or spatial relationship between cells, serum tumor-marker concentration, and hormone level in the tissue sample;
(d) optionally, scoring the tumor type as normal, benign, or mal ignant;
(d) inputting the score from steps (b) and (c), and optionally inputting the score from step (d) into a computer programmed to statistically analyze the scores;
(e) analyzing the scores on the computer and generating a profile for the subject; and,
(f) determining a prognosis of the subject from the profile.
30. The method of claim 29, further comprising comparing the generated profile to a profile previously generated from the subject, to a reference profile of a healthy population of the subject, to a reference profile of a population of the subject having a benign tumor, or to a reference profile of a population of the subject having a malignant tumor, and determining a prognosis of the subject from the comparison.
31. A method for characterizing a cell in vitro, comprising the steps of
(a) labeling at least one biomarker on the cell with a chromogenic label;
(b) illuminating the cell with a spectrally-stable light source configured to evenly illuminate the cell;
(c) detecting at least one optical plane of the cell with a brightfield microscope;
(d) forming an image of the cell from the detected at least one optical plane;
(e) determining at least one region of interest in the image of the cell; and
(f) calculating a profile for the at least one region of interest, whe rein the profile indicates one or more conditions of the biomarker and/or the cell.
32. The method of claim 31, further comprising :
receiving a score for at least one of an identity, an intensity, or a distribution of at least one chromogen on the cell;
statistically analyzing the received score to determine a scoring preference; and calibrating selection criteria for chromogenic analysis of the cell with the scoring preference, such that the profile is calculated based on the scoring preference.
PCT/US2010/061487 2009-12-22 2010-12-21 Automated quantitative multidimensional volumetric analysis and visualization WO2011087778A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US28902009P 2009-12-22 2009-12-22
US61/289,020 2009-12-22

Publications (1)

Publication Number Publication Date
WO2011087778A1 true WO2011087778A1 (en) 2011-07-21

Family

ID=44304558

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2010/061487 WO2011087778A1 (en) 2009-12-22 2010-12-21 Automated quantitative multidimensional volumetric analysis and visualization

Country Status (1)

Country Link
WO (1) WO2011087778A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2700036A2 (en) * 2011-04-22 2014-02-26 General Electric Company Analyzing the expression of biomarkers in cells with clusters

Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5022744A (en) * 1988-04-26 1991-06-11 Wild Leitz Gmbh Microscope with a camera and automatic color temperature balance
US6453060B1 (en) * 1999-06-29 2002-09-17 Tri Path Imaging, Inc. Method and apparatus for deriving separate images from multiple chromogens in a branched image analysis system
US20020145728A1 (en) * 2001-04-06 2002-10-10 Speedfam-Ipec Corporation Method and apparatus for a spectrally stable light source using white light LEDs
US20030086067A1 (en) * 2000-08-08 2003-05-08 Volker Gerstner Assembly for increasing the depth discrimination of an optical imaging system
US6690817B1 (en) * 1993-08-18 2004-02-10 Applied Spectral Imaging Ltd. Spectral bio-imaging data for cell classification using internal reference
US20040190132A1 (en) * 2003-03-19 2004-09-30 Axel Laschke Control unit for mixed light illumination, especially for microscopy
US20060050146A1 (en) * 1997-04-09 2006-03-09 Richardson Technologies Inc. Color translating UV microscope
US20060072335A1 (en) * 2004-10-01 2006-04-06 Cytyc Corporation Isotropic illumination
US20070097702A1 (en) * 2005-10-27 2007-05-03 Austin Crowder Surgical headlight
US20070258122A1 (en) * 2004-10-06 2007-11-08 Bc Cancer Agency Computer-Tomography Microscope and Computer-Tomography Image Reconstruction Methods
US20080032325A1 (en) * 2006-08-07 2008-02-07 Northeastern University Phase subtraction cell counting method
US20080094631A1 (en) * 1996-01-02 2008-04-24 Jung Wayne D Apparatus and method for measuring optical characteristics of an object
US20080260637A1 (en) * 2004-11-17 2008-10-23 Dalia Dickman Methods of Detecting Prostate Cancer
US20090001262A1 (en) * 2003-10-22 2009-01-01 Erik Visser System and Method for Spectral Analysis
US20090004204A1 (en) * 2005-08-16 2009-01-01 Genentech, Inc. Assays and methods using biomarkers
US20090190822A1 (en) * 1999-01-25 2009-07-30 Amnis Corporation Blood and cell analysis using an imaging flow cytometer
US20090238457A1 (en) * 2008-03-21 2009-09-24 General Electric Company Methods and systems for automated segmentation of dense cell populations

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5022744A (en) * 1988-04-26 1991-06-11 Wild Leitz Gmbh Microscope with a camera and automatic color temperature balance
US6690817B1 (en) * 1993-08-18 2004-02-10 Applied Spectral Imaging Ltd. Spectral bio-imaging data for cell classification using internal reference
US20080094631A1 (en) * 1996-01-02 2008-04-24 Jung Wayne D Apparatus and method for measuring optical characteristics of an object
US20060050146A1 (en) * 1997-04-09 2006-03-09 Richardson Technologies Inc. Color translating UV microscope
US20090190822A1 (en) * 1999-01-25 2009-07-30 Amnis Corporation Blood and cell analysis using an imaging flow cytometer
US6453060B1 (en) * 1999-06-29 2002-09-17 Tri Path Imaging, Inc. Method and apparatus for deriving separate images from multiple chromogens in a branched image analysis system
US20030086067A1 (en) * 2000-08-08 2003-05-08 Volker Gerstner Assembly for increasing the depth discrimination of an optical imaging system
US20020145728A1 (en) * 2001-04-06 2002-10-10 Speedfam-Ipec Corporation Method and apparatus for a spectrally stable light source using white light LEDs
US20040190132A1 (en) * 2003-03-19 2004-09-30 Axel Laschke Control unit for mixed light illumination, especially for microscopy
US20090001262A1 (en) * 2003-10-22 2009-01-01 Erik Visser System and Method for Spectral Analysis
US20060072335A1 (en) * 2004-10-01 2006-04-06 Cytyc Corporation Isotropic illumination
US20070258122A1 (en) * 2004-10-06 2007-11-08 Bc Cancer Agency Computer-Tomography Microscope and Computer-Tomography Image Reconstruction Methods
US20080260637A1 (en) * 2004-11-17 2008-10-23 Dalia Dickman Methods of Detecting Prostate Cancer
US20090004204A1 (en) * 2005-08-16 2009-01-01 Genentech, Inc. Assays and methods using biomarkers
US20070097702A1 (en) * 2005-10-27 2007-05-03 Austin Crowder Surgical headlight
US20080032325A1 (en) * 2006-08-07 2008-02-07 Northeastern University Phase subtraction cell counting method
US20090238457A1 (en) * 2008-03-21 2009-09-24 General Electric Company Methods and systems for automated segmentation of dense cell populations

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
PILATI ET AL.: "A Rapid Method Combining Golgi and Nissl Staining to Study Neuronal Morphology and Cytoarchitecture", December 2008 (2008-12-01), pages 540 - 542, Retrieved from the Internet <URL:http://jhc.sagepub.com/content/56/61539.full.pdf> [retrieved on 20110220] *
SARKAR: "Alterations of Centromere Positions in Nuclei of Immortalized and Malignant Mouse Lymphocytes", December 2007 (2007-12-01), Retrieved from the Internet <URL:http://onlinelibrary.wiley.com/doi110.1002Jcyto.a.20395/pdf> [retrieved on 20110220] *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2700036A2 (en) * 2011-04-22 2014-02-26 General Electric Company Analyzing the expression of biomarkers in cells with clusters

Similar Documents

Publication Publication Date Title
JP7047059B2 (en) Automated Assay Evaluation and Normalization for Image Processing
US10025271B2 (en) Method and system for detecting and/or classifying cancerous cells in a cell sample
EP3005293B1 (en) Image adaptive physiologically plausible color separation
US20170309021A1 (en) Systems and methods for co-expression analysis in immunoscore computation
DE60316113T2 (en) METHOD FOR QUANTITATIVE VIDEO MICROSCOPY AND DEVICE AND COMPUTER PROGRAM FOR IMPLEMENTING THE PROCESS
EP1428016B1 (en) Method of quantitative video-microscopy and associated system and computer software program product
US20190087953A1 (en) System and method for calculating focus variation for a digital microscope
JP2023030033A (en) Method for storing and reading out digital pathology analysis result
US20220351860A1 (en) Federated learning system for training machine learning algorithms and maintaining patient privacy
JP2013092540A5 (en)
JP5469070B2 (en) Method and system using multiple wavelengths for processing biological specimens
US10929972B2 (en) Devices, systems and methods for automated quantitative scoring of digitized tissue images
US9607372B2 (en) Automated bone marrow cellularity determination
US20220351379A1 (en) Non-tumor segmentation to support tumor detection and analysis
JP5154844B2 (en) Image processing apparatus and image processing program
CN115032196B (en) Full-scribing high-flux color pathological imaging analysis instrument and method
WO2016189469A1 (en) A method for medical screening and a system therefor
CN110363734B (en) Thick sample microscopic fluorescence image reconstruction method and system
WO2011087778A1 (en) Automated quantitative multidimensional volumetric analysis and visualization
US20210174147A1 (en) Operating method of image processing apparatus, image processing apparatus, and computer-readable recording medium
JP2022535798A (en) Hyperspectral quantitative imaging cytometry system
CN117546007A (en) Information processing device, biological sample observation system, and image generation method
CN114326075B (en) Digital microscopic imaging system and microscopic detection method for biological sample
WO2023157756A1 (en) Information processing device, biological sample analysis system, and biological sample analysis method
WO2023248954A1 (en) Biological specimen observation system, biological specimen observation method, and dataset creation method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10843533

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 10843533

Country of ref document: EP

Kind code of ref document: A1