WO2002093494A2 - A system for modeling static and dynamic three dimensional anatomical structures by 3-d models - Google Patents

A system for modeling static and dynamic three dimensional anatomical structures by 3-d models Download PDF

Info

Publication number
WO2002093494A2
WO2002093494A2 PCT/US2002/014634 US0214634W WO02093494A2 WO 2002093494 A2 WO2002093494 A2 WO 2002093494A2 US 0214634 W US0214634 W US 0214634W WO 02093494 A2 WO02093494 A2 WO 02093494A2
Authority
WO
WIPO (PCT)
Prior art keywords
target object
contour
model
initial model
shape
Prior art date
Application number
PCT/US2002/014634
Other languages
French (fr)
Other versions
WO2002093494A3 (en
Inventor
Alok Gupta
Marie-Pierre Jolly
Thomas O'donnell
Original Assignee
Siemens Corporate Research, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens Corporate Research, Inc. filed Critical Siemens Corporate Research, Inc.
Priority to DE10296836T priority Critical patent/DE10296836T5/en
Priority to JP2002590093A priority patent/JP2004529714A/en
Publication of WO2002093494A2 publication Critical patent/WO2002093494A2/en
Publication of WO2002093494A3 publication Critical patent/WO2002093494A3/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/12Edge-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/149Segmentation; Edge detection involving deformable models, e.g. active contour models
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10072Tomographic images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20112Image segmentation details
    • G06T2207/20116Active contour; Active surface; Snakes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30048Heart; Cardiac
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00Indexing scheme for image generation or computer graphics
    • G06T2210/41Medical

Definitions

  • the present invention relates to the computer modeling of a potentially dynamic anatomical structure in a given image volume(s). This allows for the measurement of the geometry of the underlying object at a specific point in time or over a set of time periods.
  • Temporal variation in organ morphology is frequently of interest to physicians and surgeons.
  • the change in blood pool volume of the left ventricle over the cardiac cycle is used to diagnose dysfunction and determine a course of treatment.
  • differential lung capacity is an effective determinant for assessing pulmonary disease.
  • swelling in response to injury may be tracked over a period of time to provide an indicator of severity of trauma.
  • disease progression such as cirrhosis is reflected in the shape evolution of the liver. Indeed, the history of tumor volume is a value scrutinized closely by doctors.
  • a method for segmenting objects from 3D images using a 3D deformable surface which was made up of a series of 2D planar curves is proposed.
  • this model is not cohesive in terms of being 3-D and is more a propagation of 2-D contours in space followed by a stitching together of the contours.
  • the 2D planar curves are not recovered via optimal active contours ("optimal" meaning that the energy function describing the contour is globally minimized).
  • the proposed method employs snakes and relies on balloon forces to explore crevices.
  • the problem with balloon forces is that the snake might leak where the image boundaries are not well defined.
  • 2D deformable surfaces have also been applied to segmentation but the approaches have not been "optimal". For ID contours, optimality is a well understood concept. How this concept might be extended to 2D surfaces still presents difficulty.
  • a voxel is a unit of graphic information that defines a point in three dimensional space (in other words, a volume pixel).
  • Segmentation via the propagation of 2D active contours is problematic.
  • a change in an object's circumference in a slice might be due to a change in the radius of the object under recovery, or it might be due to a change in direction of the path taken by the object in space. Determining if a change in circumference or direction has occurred is essential for selecting an appropriate starting point for segmentation in the slices to follow.
  • Two-dimensional active contours lack the global properties necessary to account for these instances.
  • 3-D models are powerful tools. They can provide detailed description of an object. It is difficult, however, to directly employ 3-D models in the segmentation process since they are not guaranteed to locate the optimal boundaries in cross-sectional slices. Propagating 2D active contours from slice to slice, on the other hand, to delineate an object's boundaries is effective but encounters problems when an object's shape dramatically changes such as in areas of high curvature.
  • a cooperative framework to exploit the positive aspects of both 3D model and 2D active contour approaches for segmentation and recovery is advantageous.
  • a default model shape, positioned in the data would provide starting points for a set of 2D segmentations (refinements) performed by active contours.
  • the same model is fitted to the results of the segmentation.
  • the present invention relates to a system of modeling a three dimensional target object which is represented by a plurality of cross-sectional images in order to provide a representative corresponding three dimensional model.
  • the invention selects an initial model from a plurality of available initial models. This selection involves identifying an initial model based on physical similarity to the target object and then superimposing an initial model upon the target object, for each of the plurality of cross-sectional images. A determination is then made of an intersection contour of the initial model and a cross- sectional image of the target object and the determined intersection contour is refined in order to more closely delineate the target object. By sub-sampling points which represent the refined determined intersection contour, the invention obtains a sub-sampled contour dataset. The initial model is then adjusted towards the sub-samples contour to obtain a representative three dimensional model of the target object. BRIEF DESCRIPTION OF DRAWINGS
  • FIG. 1 is a flow diagram showing method steps and three stages of the present invention
  • FIG. 2 is a flow diagram showing in further detail the steps of Stage 1 of FIG. 1;
  • FIG. 3 is a flow diagram showing in further detail the steps of Stage 2 of FIG. 1;
  • FIG. 4 is a flow diagram showing in further detail the steps of Stage 3 of FIG. 1;
  • FIG. 5 is a perspective view of a portion of a hybrid model showing a parametric component and a local component
  • FIG 6 shows the user selecting points on two orthogonal images to initially position the parametric component of the model
  • FIG 7 illustrates the technique for refining the contours segmentation
  • FIG 8 shows the intersection of one image axis with the parametric component of the hybrid model. This intersection forms the starting point for a 2-D segmentation. The result of the segmentation is also shown;
  • FIG 9 shows the same process as FIG 9 but for a different (orthogonal) axis
  • FIG 10 shows the contours resulting from the 2-D segmentations in space and the hybrid model fit to them.
  • a method comprises quantifying the change in organ morphology using a semi-automatic segmentation method.
  • a user approximately positions a model of the organ (or organs) in an image volume. After this, the process is fully automatic. Steps include intersecting the user-positioned model with the individual image planes of the volume to create a series of contours approximating the position of the underlying organ. Then, optimal 2-D segmentation algorithms are used to refine the boundary of the organ using the approximate contours as starting points. From these optimal contours a new model is applied. Volume and other morphometric measurements are then taken on the model. It is to be noted that in an alternative embodiment a model may be automatically positioned in an image volume.
  • the approximate position of the LV is specified by the user.
  • the approximate inner and outer boundaries are outlined with a series mouse clicks on both a short axis image as well as a long axis image, the model is automatically fit to these points.
  • the resulting model is referred to as the initial model. Note that this model is not required to sit exactly on the boundary of the organ. It is enough that it is close to the boundary of the organ.
  • the initial model is intersected with each of the image planes in the image volume. The result is a series of contours (inner and outer) in each of the cross-sectional images of the volume. These individual contours form starting points for optimal 2-D segmentations of the LV. A new model (or the initial one) is automatically fit to these recovered contours. This procedure is then repeated for every temporal instance.
  • a cooperative framework includes a method for combining three dimensional (3D) models with two dimensional (2D) segmentations.
  • a user initializes a parametric component of a hybrid model.
  • a hybrid model is a parametric model with a spline-like mesh on its surface which is capable of deforming to describe fine detail in an image volume. Intersections of image slices with the parametric component serve as starting points for a segmentation of the object boundaries using active contours. The resulting segmentations are sampled to create a data set of 3D points to which the original hybrid model is fit.
  • the hybrid model is free to deform both parametrically and locally in order to represent the object of interest.
  • the user initializes the hybrid model, the hybrid model initializes the 2D active contours, and the active contours constrain the fit of the hybrid model. This fit is referred to as a physically motivated fit. The process may then be repeated for different time instances.
  • block 100 of FIG. 1 represents an initialization of the model .
  • segmentation using active contours is performed on the object in the image. The segmented contours are used to create a data set of 3D points in block 120.
  • the model is fit to the data points of block 120 to represent the object. This process is repeated for subsequent time instances as shown in block 131.
  • a detailed flow chart is provided for block 100 initializing the model.
  • This step includes identifying approximate positions on the surface from at least two views which bounds the extremities of the object. In the case of the left ventricle of the heart both inner and outer walls (endocardial and epicardial) must be delineated.
  • the approximate positions of the points on the surface of the object are identified in blocks 102 and 106.
  • the orthogonal cross section is identified in block 104. If necessary this step is repeated after the step in block 106.
  • a parametric component of the model is then fitted to the points as shown in block 108.
  • FIG. 3 of the drawings illustrates, block 105 of FIG. 1 in a detailed flowchart, in a preferred embodiment a two dimensional image is represented as a graph in block 112. These intersections are used as the starting points for 2D segmentation.
  • the edge cost is found between graph nodes using image gradients in block 114.
  • a search space is defined around the approximate input contour in block 116.
  • a path of lowest energy between pixels/nodes is determined using Dijkstra's algorithm in block 118. Dijkstra's algorithm is a standard graph algorithm and can be found in any algorithm's book such as CG. Brassard and P.
  • FIG. 4 of the drawings is a detailed flow chart of block 110 of FIG.1 illustrating fitting the model of block 130.
  • This fitting of the model includes determining data and smoothing forces for the points of the data set in block 132.
  • the data and smoothing forces are scaled in block 134.
  • Shape functions that reduce the number of iterations needed to fit the model to the data are applied to finite element model (FEM) elements in block 136.
  • FEM finite element model
  • An (FEM) is a finite element model which is made up of a finite number of elements on the surface of the model and serve to describe the model.
  • Data and smoothing forces are combined in block 138. Applying the adjusted forces to the hybrid model to deform the hybrid model is performed in block 140.
  • Segmentation is among the most widely researched topics in computer vision.
  • a comparison of the cooperative framework described in this disclosure is compared with the most closely related work.
  • the method as described is applicable to other techniques and systems.
  • a preferred embodiment of a hybrid 3-D model for the heart left ventricle is described in the following. It is understood that this invention is not restricted to this model only. The invention applies to any model of any object. The approach may make use of any 3D model form.
  • the hybrid model is chosen. Referring to FIG. 5, for the use of a hybrid model of the LV, the model formulation is hybrid in that it is made up of a parametric component 20 and a local, spline-like mesh component 22 over the parametric component. This local component may deform away from the parametric surface to describe fine detail.
  • the parametric component of the model is based on two ellipsoids, one describing the inner wall of the LV and one describing the outer wall.
  • the model has an underlying parametric component which resembles a double hulled ellipsoid (one for the inner wall of the left ventricle and one for the outer wall).
  • the model is broken up into octants which are described by their own parametric formulations.
  • the quadrants are called PosPosPos, NegPosPos, NegNegPos, PosNegPos, PosPosNeg, NegPosNeg, NegNegNeg, PosNegNeg with representations for the inner and outer walls.
  • This parametric representation is swept out by two intrinsic spherical coordinates, u and v.
  • the extrinsic parameters of alPosInner alPosOuter rPosPosInner rPosPosOuter a2PosInner a2PosOuter rNegPosInner rNegPosOuter alNeglnner alNegOuter rNegNeglnner rNegNegOuter a2NegInner a2NegOuter rPosNeglnner rPosNegOuter a3PosInner describe the general shape of the double hulled ellipsoid.
  • Y alpha Cos[u] (1 + alpha tapyNeglnner Sin[u]) Sin[v] (a2PosInner + rNegPosInner Sin[v] + rNegPosInner Sin[3 v]);
  • Z a3NegInner alpha Sin[u];
  • the user provides an approximation to the object of interest by approximately fitting the model to the data.
  • the method for doing this depends on the topology of the object and the form of the parametric component.
  • a method for a left ventricle is described. Note that the approximation may be quite rough as the segmentation routine (Stage 2) will discover the exact object boundaries.
  • the initial model In order for the segmentation routine to be effective, the initial model must be reasonably close, within 10 pixels for example.
  • the user defines points on the approximate border of the both the inner and outer (endocardial and epicardial) walls of the left ventricle for a cross-section. Then, in order to specify the general bounds of the shape, the user repeats this action on an orthogonal image. For the LV, these would be the short axis and long axis images. (FIG 6).
  • the task of refining the contours is also known as segmentation.
  • segmentation module is described. It is given two approximate contours (endocardium and epicardium) from the previous step. These contours, however, do not exactly delineate the features in the image.
  • the present invention seeks to refine the contours so that the endocardium exactly delineates the border between the myocardium and the blood pool, and the epicardium exactly delineates the border between the myocardium and the outside. See FIG. 7.
  • FIGS. 8 and 9 also describe refining the contours following intersection of the model with the image from the approximate contour (150) to the recovered contour (160) as illustrated in FIG. 9.
  • the present invention uses an algorithm similar to Geiger et al. (D. Geiger, A. Gupta, L. A. Costa, and J. Vlontzos, "Dynamic programming for detecting, tracking, and matching deformable contours", IEEE Trans. Pattern Analysis and Machine Intelligence, 17(3):294- 302, 1995), but using Dijktra's graph search algorithm as in Mortensen and Barrett (E. N. Mortensen and W. A. Barrett, "Interactive segmentation with intelligent scissors", Graphical Models and Image Processing, 60:349-384, 1998).
  • Dijkstra's algorithm is a standard graph theory algorithm and can be found in any algorithm's book such as (G. Brassard and P. Bratley, Algorithmics: Theory & Practice, Prentice Hall, 1988).
  • the present system involves creating a graph by defining a node for every pixel in the image. Nodes that correspond to neighboring pixels are connected by a link whose weight is inversely proportional to the gradient in the image between the corresponding pixels. Then, given a source node, Dijkstra's algorithm is able to find the minimum path between the source node and all nodes in a set of sink nodes. Since the cost of a path is the sum of the cost of the links it goes through, the minimum path corresponds to the contour with maximum gradient. If one is not sure on the starting point, then one can define a set of source nodes and connect all of them to a "pseudo" source node with a zero cost link.
  • Block 200 shows an image of the left ventricle in a short axis cross section and an approximate contour for the endocardium.
  • the method of the present system defines a search space of a given width (say 10 pixels) around the approximate contour 202(a). Then, a contiguous set of nodes is defined as the source nodes. Since the contour is closed, it does not really matter which nodes are defined as source nodes as long as they cut the search space like a sector. Their neighbors on one side are also defined as the sink nodes (see block 202).
  • the present system runs Dijkstra's algorithm to find the minimum cost path between the "pseudo" source node and the sink nodes. This generates a contour, but nothing in the algorithm guarantees that it is closed, and most of the time, it is not (see block 204).
  • the source node is uniquely defined as the mid-point of the previously recovered contour.
  • the sink nodes are its neighbors on one side (see block 206) and Dijkstra's algorithm is called again.
  • the recovered contour is guaranteed to be closed since the source and sink nodes are defined to be connected (see block 208).
  • the contour is built clockwise by the Dijkstra process and the image gradient points from bright to dark.
  • the contour is forced to separate a bright region inside from a dark region outside.
  • the initial approximation is fit to the object structure provided by the user in Stage 1 to the sampled segmented data garnered from Stage 2 (FIG. 10, 170).
  • the method employed is an iterative closest point method based on the physically-motivated recovery paradigm. In this paradigm, data points are thought of as exerting forces attracting the model towards them.
  • a hybrid model is capable of rigid, parametric and local (spline-like) deformations. Local deformations may include a deformation penalty which serves to smooth the data.
  • Finite element model (FEM) shape functions are used to distribute the data forces from element surfaces to element nodes. Both the data forces and smoothing forces are scaled, and the data forces are combined with the smoothing forces.
  • the rigid and parametric fitting is discontinued once the local deformations are initiated. The result is shown in FIG. 10, 180.
  • the technique was tested on a volunteer data set with two phases of the cardiac cycle, En-diastole (ED) and en-systole (ES).
  • the blood pool volume for ED was 271ml with a myocardial volume of 275 ml.
  • the blood volume was 231ml with a myocardial volume of 285 ml.
  • the ejection fraction for the volunteer was 14%.

Abstract

The present invention relates to a system of modeling a three dimensional target object which is represented by a plurality of cross-sectional images in order to provide a representative corresponding three dimensional model. The invention selects an initial model from a plurality of available initial models. This selection involves identifying an initial model based on physical similarity to the target object and then superimposing an initial model upon the target object, for each of the plurality of cross-sectional images. A determination is then made of an intersection contour of the initial model and a cross-sectional image of the target object and the determined intersection contour is refined in order to more closely delineate the target object. By sub-sampling points which represent the refined determined intersection contour, the invention obtains a sub-sampled contour dataset. The initial model is then adjusted towards the sub-samples contour to obtain a representative three dimensional model of the target object.

Description

A SYSTEM FOR MODELING STATIC AND DYNAMIC THREE DIMENSIONAL ANATOMICAL STRUCTURES BY 3-D MODELS
BACKGROUND OF THE INVENTION:
1. Field of the Invention
The present invention relates to the computer modeling of a potentially dynamic anatomical structure in a given image volume(s). This allows for the measurement of the geometry of the underlying object at a specific point in time or over a set of time periods.
2. Description of Related Art:
Temporal variation in organ morphology is frequently of interest to physicians and surgeons. In the heart, for example, the change in blood pool volume of the left ventricle over the cardiac cycle is used to diagnose dysfunction and determine a course of treatment. Similarly, differential lung capacity is an effective determinant for assessing pulmonary disease. In the brain, swelling in response to injury may be tracked over a period of time to provide an indicator of severity of trauma. In another example, disease progression such as cirrhosis is reflected in the shape evolution of the liver. Indeed, the history of tumor volume is a value scrutinized closely by doctors.
A method for segmenting objects from 3D images using a 3D deformable surface which was made up of a series of 2D planar curves is proposed. However, this model is not cohesive in terms of being 3-D and is more a propagation of 2-D contours in space followed by a stitching together of the contours. In addition, the 2D planar curves are not recovered via optimal active contours ("optimal" meaning that the energy function describing the contour is globally minimized). Instead the proposed method employs snakes and relies on balloon forces to explore crevices. The problem with balloon forces is that the snake might leak where the image boundaries are not well defined. 2D deformable surfaces have also been applied to segmentation but the approaches have not been "optimal". For ID contours, optimality is a well understood concept. How this concept might be extended to 2D surfaces still presents difficulty.
Direct application of 3D models to 3D image volumes has also met with mixed success. Again, describing deep crevices becomes a problem. Some proposed methods fit parallel sets of 2D contours to recover a 3D object. Once the fit is settled the methods repeat the process from an orthogonal direction using the results of the previous iteration as a starting point. However, these methods employ balloon forces to fit the 2Dcontours and their result is not a coherent 3-D surface model. In addition, these methods are applied on relatively simple synthetic shapes. Region growing techniques are also used for segmentation. However, while these techniques are often effective they suffer from bleeding in areas where the object boundary is not well defined. In addition, these techniques do not result in a geometric description of the object, rather they result in a collection of voxels. A voxel is a unit of graphic information that defines a point in three dimensional space (in other words, a volume pixel).
Segmentation via the propagation of 2D active contours (i.e., using the result from a previous slice as the starting point for a segmentation of the current slice) is problematic. A change in an object's circumference in a slice might be due to a change in the radius of the object under recovery, or it might be due to a change in direction of the path taken by the object in space. Determining if a change in circumference or direction has occurred is essential for selecting an appropriate starting point for segmentation in the slices to follow. Two-dimensional active contours lack the global properties necessary to account for these instances.
3-D models are powerful tools. They can provide detailed description of an object. It is difficult, however, to directly employ 3-D models in the segmentation process since they are not guaranteed to locate the optimal boundaries in cross-sectional slices. Propagating 2D active contours from slice to slice, on the other hand, to delineate an object's boundaries is effective but encounters problems when an object's shape dramatically changes such as in areas of high curvature.
A cooperative framework to exploit the positive aspects of both 3D model and 2D active contour approaches for segmentation and recovery is advantageous. In this framework, a default model shape, positioned in the data would provide starting points for a set of 2D segmentations (refinements) performed by active contours. The same model is fitted to the results of the segmentation.
Therefore, a need exists for a general cooperative approach for segmenting objects from 3D image volumes which exploits the positive aspects of both 2D and 3D traditional approaches.
SUMMARY OF THE INVENTION
The present invention relates to a system of modeling a three dimensional target object which is represented by a plurality of cross-sectional images in order to provide a representative corresponding three dimensional model. The invention selects an initial model from a plurality of available initial models. This selection involves identifying an initial model based on physical similarity to the target object and then superimposing an initial model upon the target object, for each of the plurality of cross-sectional images. A determination is then made of an intersection contour of the initial model and a cross- sectional image of the target object and the determined intersection contour is refined in order to more closely delineate the target object. By sub-sampling points which represent the refined determined intersection contour, the invention obtains a sub-sampled contour dataset. The initial model is then adjusted towards the sub-samples contour to obtain a representative three dimensional model of the target object. BRIEF DESCRIPTION OF DRAWINGS
The invention is described in detail in the following description of preferred embodiments with reference to the following figures wherein:
FIG. 1 is a flow diagram showing method steps and three stages of the present invention;
FIG. 2 is a flow diagram showing in further detail the steps of Stage 1 of FIG. 1;
FIG. 3 is a flow diagram showing in further detail the steps of Stage 2 of FIG. 1;
FIG. 4 is a flow diagram showing in further detail the steps of Stage 3 of FIG. 1;
FIG. 5 is a perspective view of a portion of a hybrid model showing a parametric component and a local component;
FIG 6 shows the user selecting points on two orthogonal images to initially position the parametric component of the model;
FIG 7 illustrates the technique for refining the contours segmentation;
FIG 8 shows the intersection of one image axis with the parametric component of the hybrid model. This intersection forms the starting point for a 2-D segmentation. The result of the segmentation is also shown;
FIG 9 shows the same process as FIG 9 but for a different (orthogonal) axis; and
FIG 10 shows the contours resulting from the 2-D segmentations in space and the hybrid model fit to them.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
In the present system a method comprises quantifying the change in organ morphology using a semi-automatic segmentation method. A user approximately positions a model of the organ (or organs) in an image volume. After this, the process is fully automatic. Steps include intersecting the user-positioned model with the individual image planes of the volume to create a series of contours approximating the position of the underlying organ. Then, optimal 2-D segmentation algorithms are used to refine the boundary of the organ using the approximate contours as starting points. From these optimal contours a new model is applied. Volume and other morphometric measurements are then taken on the model. It is to be noted that in an alternative embodiment a model may be automatically positioned in an image volume.
For the case of recovering the left ventricle (LV) of the heart, for each stage of the cardiac cycle the following steps are taken: First, the approximate position of the LV is specified by the user. The approximate inner and outer boundaries are outlined with a series mouse clicks on both a short axis image as well as a long axis image, the model is automatically fit to these points. The resulting model is referred to as the initial model. Note that this model is not required to sit exactly on the boundary of the organ. It is enough that it is close to the boundary of the organ. Further, the initial model is intersected with each of the image planes in the image volume. The result is a series of contours (inner and outer) in each of the cross-sectional images of the volume. These individual contours form starting points for optimal 2-D segmentations of the LV. A new model (or the initial one) is automatically fit to these recovered contours. This procedure is then repeated for every temporal instance.
A cooperative framework includes a method for combining three dimensional (3D) models with two dimensional (2D) segmentations. In one example, a user initializes a parametric component of a hybrid model. A hybrid model is a parametric model with a spline-like mesh on its surface which is capable of deforming to describe fine detail in an image volume. Intersections of image slices with the parametric component serve as starting points for a segmentation of the object boundaries using active contours. The resulting segmentations are sampled to create a data set of 3D points to which the original hybrid model is fit. The hybrid model is free to deform both parametrically and locally in order to represent the object of interest. In this way, the user initializes the hybrid model, the hybrid model initializes the 2D active contours, and the active contours constrain the fit of the hybrid model. This fit is referred to as a physically motivated fit. The process may then be repeated for different time instances.
Referring to FIGS. 1-10 of the present invention and in particular to FIG. 1 which provides a flow chart of the method of the present invention. As described above, block 100 of FIG. 1 represents an initialization of the model . This includes taking a volumetric image of an object to be modeled. The intersection of the models with the imaging planes results in an approximation to the cross-section of the object in the image. This approximation serves as a starting point for active contour segmentations in the image, in block 105. In block 110, segmentation using active contours is performed on the object in the image. The segmented contours are used to create a data set of 3D points in block 120. In block 130, the model is fit to the data points of block 120 to represent the object. This process is repeated for subsequent time instances as shown in block 131.
Referring to FIG. 2 of the drawings, a detailed flow chart is provided for block 100 initializing the model. This step includes identifying approximate positions on the surface from at least two views which bounds the extremities of the object. In the case of the left ventricle of the heart both inner and outer walls (endocardial and epicardial) must be delineated. The approximate positions of the points on the surface of the object are identified in blocks 102 and 106. The orthogonal cross section is identified in block 104. If necessary this step is repeated after the step in block 106. A parametric component of the model is then fitted to the points as shown in block 108.
Referring to FIG. 3 of the drawings, in block 111, the initial model image and, the image planes intersect. FIG. 3 of the drawings illustrates, block 105 of FIG. 1 in a detailed flowchart, in a preferred embodiment a two dimensional image is represented as a graph in block 112. These intersections are used as the starting points for 2D segmentation. The edge cost is found between graph nodes using image gradients in block 114. A search space is defined around the approximate input contour in block 116. A path of lowest energy between pixels/nodes is determined using Dijkstra's algorithm in block 118. Dijkstra's algorithm is a standard graph algorithm and can be found in any algorithm's book such as CG. Brassard and P. Bratley, Algorithmics: Theory and Practice, Prentice Hall, 1988. This step is repeated for a different search space if the resulting contour is not closed in block 122. By determining the lowest energy path a contour (path) is defined in block 124.
FIG. 4 of the drawings is a detailed flow chart of block 110 of FIG.1 illustrating fitting the model of block 130. This fitting of the model includes determining data and smoothing forces for the points of the data set in block 132. The data and smoothing forces are scaled in block 134. Shape functions that reduce the number of iterations needed to fit the model to the data are applied to finite element model (FEM) elements in block 136. An (FEM) is a finite element model which is made up of a finite number of elements on the surface of the model and serve to describe the model. Data and smoothing forces are combined in block 138. Applying the adjusted forces to the hybrid model to deform the hybrid model is performed in block 140.
Segmentation is among the most widely researched topics in computer vision. A comparison of the cooperative framework described in this disclosure is compared with the most closely related work. However, the method as described is applicable to other techniques and systems.
A preferred embodiment of a hybrid 3-D model for the heart left ventricle is described in the following. It is understood that this invention is not restricted to this model only. The invention applies to any model of any object. The approach may make use of any 3D model form. In the present example the hybrid model is chosen. Referring to FIG. 5, for the use of a hybrid model of the LV, the model formulation is hybrid in that it is made up of a parametric component 20 and a local, spline-like mesh component 22 over the parametric component. This local component may deform away from the parametric surface to describe fine detail. The parametric component of the model is based on two ellipsoids, one describing the inner wall of the LV and one describing the outer wall. The model has an underlying parametric component which resembles a double hulled ellipsoid (one for the inner wall of the left ventricle and one for the outer wall). The model is broken up into octants which are described by their own parametric formulations. The quadrants are called PosPosPos, NegPosPos, NegNegPos, PosNegPos, PosPosNeg, NegPosNeg, NegNegNeg, PosNegNeg with representations for the inner and outer walls. This parametric representation is swept out by two intrinsic spherical coordinates, u and v.
The extrinsic parameters of alPosInner alPosOuter rPosPosInner rPosPosOuter a2PosInner a2PosOuter rNegPosInner rNegPosOuter alNeglnner alNegOuter rNegNeglnner rNegNegOuter a2NegInner a2NegOuter rPosNeglnner rPosNegOuter a3PosInner describe the general shape of the double hulled ellipsoid.
PosPosPosInner: X = alpha
Cos[u] (alPosInner Cos[v] + rPosPosInner Sin[2 v] );
Y = alpha Cos[u] Sin[v] (a2PosInner + rPosPosInner Sin[v] + rPosPosInner Sin[3 v]);
Z = a3PosInner alpha Sin[u];
NegPosPosInner:
X = alpha Cos[u] Cos[v] (alNeglnner - rNegPosInner Cos[v] + rNegPosInner Cos[3 v]);
Y = alpha Cos[u] Sin[v] (a2PosInner + rNegPosInner Sin[v] + rNegPosInner Sin[3 v]);
Z = a3PosInner alpha Sin[u]; NegNegPosInner:
X = alpha Cos[u] Cos[v] (alNeglnner - rNegNeglnner Cos[v] + rNegNeglnner
Cos[3 v]);
Y = -(alpha Cos[u] Sin[v] (-a2NegInner + rNegNeglnner Sin[v] + rNegNeglnner Sin[3 v]));
Z = a3PosInner alpha Sin[u];
PosNegPosInner:
X = -(alpha Cos[u] Cos[v] (-alPosInner - rPosNeglnner Cos[v] + rPosNeglnner
Cos[3 v]));
Y = -(alpha Cos[u] Sin[v] (-a2NegInner + rPosNeglnner Sin[v] + rPosNeglnner Sin[3 v]));
Z = a3PosInner alpha Sin[u];
PosPosNeglnner:
X = alpha Cos[u] (1 + alpha tapxPosInner Sin[u])
(alPosInner Cos[v] + rPosPosInner Sin[2 v] );
Y = alpha Cos[u] (1 + alpha tapyPosInner Sin[u]) Sin[v] (a2PosInner + rPosPosInner Sin[v] + rPosPosInner Sin[3 v]);
Z = a3NegInner alpha Sin[u];
NegPosNeglnner:
X = alpha Cos[u] Cos[v] (alNeglnner - rNegPosInner Cos[v] + rNegPosInner
Cos[3 v]) (1 + alpha tapxNeglnner Sin[u]);
Y = alpha Cos[u] (1 + alpha tapyNeglnner Sin[u]) Sin[v] (a2PosInner + rNegPosInner Sin[v] + rNegPosInner Sin[3 v]); Z = a3NegInner alpha Sin[u];
NegNegNeglnner:
X = alpha Cos[u] Cos[v] (alNeglnner - rNegNeglnner Cos[v] + rNegNeglnner
Cos[3 v]) (1 + alpha tapxNeglnner Sin[u]);
Y = -(alpha Cos[u] (1 + alpha tapyNeglnner Sin[u]) Sin[v] (-a2NegInner + rNegNeglnner Sin[v] + rNegNeglnner Sin[3 v]));
Z = a3NegInner alpha Sin[u];
PosNegNeglnner:
X = -(alpha Cos[u] Cos[v] (-alPosInner - rPosNeglnner Cos[v] + rPosNeglnner
Cos[3 v]) (1 + alpha tapxPosInner Sin[u]));
Y = -(alpha Cos[u] (1 + alpha tapyPosInner Sin[u]) Sin[v] (-a2NegInner + rPosNeglnner Sin[v] + rPosNeglnner Sin[3 v]));
Z = a3NegInner alpha Sin[u];
PosPosPosOuter:
2n X = alpha
Cos[u] (alPosOuter Cos[v] + rPosPosOuter Sin[2 v] );
Y = alpha Cos[u] Sin[v] (a2PosOuter + rPosPosOuter Sin[v] + rPosPosOuter Sin[3 v]);
Z = a3PosOuter alpha Sin[u];
NegPosPosOuter:
X = alpha Cos[u] Cos[v] (alNegOuter - rNegPosOuter Cos[v] + rNegPosOuter Cos[3 v]);
Y = alpha Cos[u] Sin[v] (a2PosOuter + rNegPosOuter Sin[v] + rNegPosOuter Sin[3 v]);
Z = a3PosOuter alpha Sin[u];
NegNegPosOuter:
X = alpha Cos[u] Cos[v] (alNegOuter - rNegNegOuter Cos[v] + rNegNegOuter
Cos[3 v]);
Y = -(alpha Cos[u] Sin[v] (-a2NegOuter + rNegNegOuter Sin[v] + rNegNegOuter Sin[3 v]));
Z = a3PosOuter alpha Sin[u];
PosNegPosOuter:
X = -(alpha Cos[u] Cos[v] (-alPosOuter - rPosNegOuter Cos[v] + rPosNegOuter
Cos[3 v]));
Y = -(alpha Cos[u] Sin[v] (-a2NegOuter + rPosNegOuter Sin[v] + rPosNegOuter Sin[3 v]));
Z = a3PosOuter alpha Sin[u];
PosPosNegOuter:
X = alpha Cos[u] (1 + alpha tapxPosOuter Sin[uJ)
(alPosOuter Cos[v] + rPosPosOuter Sin[2 v] );
Y = alpha Cos[u] (1 + alpha tapyPosOuter Sin[u]) Sin[v] (a2PosOuter + rPosPosOuter Sin[v] + rPosPosOuter Sin[3 v]);
Z = a3NegOuter alpha Sin[u]; NegPosNegOuter:
X = alpha Cos[u] Cos[v] (alNegOuter - rNegPosOuter Cos[v] + rNegPosOuter
Cos[3 v]) (1 + alpha tapxNegOuter Sin[u]);
Y = alpha Cos[u] (1 + alpha tapyNegOuter Sin[u]) Sin[v] (a2PosOuter + rNegPosOuter Sin[v] + rNegPosOuter Sin[3 v]);
Z = a3NegOuter alpha Sin[u];
NegNegNegOuter:
X = alpha Cos[u] Cos[v] (alNegOuter - rNegNegOuter Cos[v] + rNegNegOuter
Cos[3 v]) (1 + alpha tapxNegOuter Sin[u]);
Y = -(alpha Cos[u] (1 + alpha tapyNegOuter Sin[u]) Sin[v] (-a2NegOuter + rNegNegOuter Sin[v] + rNegNegOuter Sin[3 v]));
Z = a3NegOuter alpha Sin[u];
PosNegNegOuter:
X = -(alpha Cos[u] Cos[v] (-alPosOuter - rPosNegOuter Cos[v] + rPosNegOuter
Cos[3 v]) (1 + alpha tapxPosOuter Sin[u]));
Y = -(alpha Cos[u] (1 + alpha tapyPosOuter Sin[u]) Sin[v] (-a2NegOuter + rPosNegOuter Sin[v] + rPosNegOuter Sin[3 v]));
Z = a3NegOuter alpha Sin[u];
Stage 1 : User Initialization
In this stage, the user provides an approximation to the object of interest by approximately fitting the model to the data. The method for doing this depends on the topology of the object and the form of the parametric component. Here, a method for a left ventricle is described. Note that the approximation may be quite rough as the segmentation routine (Stage 2) will discover the exact object boundaries. In order for the segmentation routine to be effective, the initial model must be reasonably close, within 10 pixels for example.
The user defines points on the approximate border of the both the inner and outer (endocardial and epicardial) walls of the left ventricle for a cross-section. Then, in order to specify the general bounds of the shape, the user repeats this action on an orthogonal image. For the LV, these would be the short axis and long axis images. (FIG 6).
Stage 2: 2D Optimal Active Contour Segmentation
The task of refining the contours is also known as segmentation. Here the segmentation module is described. It is given two approximate contours (endocardium and epicardium) from the previous step. These contours, however, do not exactly delineate the features in the image. The present invention seeks to refine the contours so that the endocardium exactly delineates the border between the myocardium and the blood pool, and the epicardium exactly delineates the border between the myocardium and the outside. See FIG. 7.
FIGS. 8 and 9 also describe refining the contours following intersection of the model with the image from the approximate contour (150) to the recovered contour (160) as illustrated in FIG. 9.
The way to do this is to use active contours (snakes). Snakes were first proposed by Kass et al. (M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active contour models", International Journal of Computer Vision, 2:321-331, 1988) are defined as contours that are pushed or pulled towards image features by constraining forces. An energy function based on the smoothness and curvature of the contours and the edge strength in the image along the contour. This energy function is to be minimized. Kass et al. Proposed to minimize the energy function using gradient descent. Amini et al. (A. A. Amini, T. E. Wemouth, and R. C. Jain, "Using dynamic programming for solving variational problems in vision", IEEE Trans. Pattern Analysis and Machine Intelligence, 12(9):855-867, 1990) used dynamic programming.
The present invention uses an algorithm similar to Geiger et al. (D. Geiger, A. Gupta, L. A. Costa, and J. Vlontzos, "Dynamic programming for detecting, tracking, and matching deformable contours", IEEE Trans. Pattern Analysis and Machine Intelligence, 17(3):294- 302, 1995), but using Dijktra's graph search algorithm as in Mortensen and Barrett (E. N. Mortensen and W. A. Barrett, "Interactive segmentation with intelligent scissors", Graphical Models and Image Processing, 60:349-384, 1998).
Dijkstra's algorithm is a standard graph theory algorithm and can be found in any algorithm's book such as (G. Brassard and P. Bratley, Algorithmics: Theory & Practice, Prentice Hall, 1988). The present system involves creating a graph by defining a node for every pixel in the image. Nodes that correspond to neighboring pixels are connected by a link whose weight is inversely proportional to the gradient in the image between the corresponding pixels. Then, given a source node, Dijkstra's algorithm is able to find the minimum path between the source node and all nodes in a set of sink nodes. Since the cost of a path is the sum of the cost of the links it goes through, the minimum path corresponds to the contour with maximum gradient. If one is not sure on the starting point, then one can define a set of source nodes and connect all of them to a "pseudo" source node with a zero cost link.
Reference is made to the illustrative steps of the algorithm with an example (see FIG. 7). Block 200 shows an image of the left ventricle in a short axis cross section and an approximate contour for the endocardium.
In the first step the method of the present system defines a search space of a given width (say 10 pixels) around the approximate contour 202(a). Then, a contiguous set of nodes is defined as the source nodes. Since the contour is closed, it does not really matter which nodes are defined as source nodes as long as they cut the search space like a sector. Their neighbors on one side are also defined as the sink nodes (see block 202). The present system runs Dijkstra's algorithm to find the minimum cost path between the "pseudo" source node and the sink nodes. This generates a contour, but nothing in the algorithm guarantees that it is closed, and most of the time, it is not (see block 204).
So another set of source and sink points in the same search space is defined. The source node is uniquely defined as the mid-point of the previously recovered contour. The sink nodes are its neighbors on one side (see block 206) and Dijkstra's algorithm is called again. The recovered contour is guaranteed to be closed since the source and sink nodes are defined to be connected (see block 208).
In the case of the heart left ventricle, both contours are segmented independently. For the endocardium, the cost of a link between two nodes is defined as:
e (p ,P2) = |VY(/>2)| + ε
1 / ε
Figure imgf000016_0001
For the epicardium, it is defined as:
e2 (Pι,P2) = :. ~ 2
|V7(Λ)|2 + *
Here, |V/| is the gradient magnitude and is the gradient direction in the image, ε
is a small constant to bound the energy function. In this case, the contour is built clockwise by the Dijkstra process and the image gradient points from bright to dark. By testing whether the sign of the z component of the cross product between the image gradient and the contour direction is positive, and setting the energy to a large number otherwise, the contour is forced to separate a bright region inside from a dark region outside.
Stage 3: 3D Hybrid Model Fit
The initial approximation is fit to the object structure provided by the user in Stage 1 to the sampled segmented data garnered from Stage 2 (FIG. 10, 170). The method employed is an iterative closest point method based on the physically-motivated recovery paradigm. In this paradigm, data points are thought of as exerting forces attracting the model towards them. A hybrid model is capable of rigid, parametric and local (spline-like) deformations. Local deformations may include a deformation penalty which serves to smooth the data. Finite element model (FEM) shape functions are used to distribute the data forces from element surfaces to element nodes. Both the data forces and smoothing forces are scaled, and the data forces are combined with the smoothing forces. The rigid and parametric fitting is discontinued once the local deformations are initiated. The result is shown in FIG. 10, 180.
Typically, in hybrid model recovery rigid body characteristics (translation and rotation) are adjusted until they settle. Then, rigid body characteristics and the parametric component are fit. And, once they settle, the rigid body characteristics, the parametric component and the local component are fit. The present method is different from this standard scheme by freezing the rigid body characteristics and parametric component during the local model fit. The rationale is that if the parametric component is to provide a gross description of the data it should arrive at the same result irrespective of the presence of a local component. In addition, freezing the rigid body characteristics and parametric component during the local model fit, speeds the fitting process significantly.
Experimental Results:
The technique was tested on a volunteer data set with two phases of the cardiac cycle, En-diastole (ED) and en-systole (ES). The blood pool volume for ED was 271ml with a myocardial volume of 275 ml. For ES, the blood volume was 231ml with a myocardial volume of 285 ml. The ejection fraction for the volunteer was 14%.
While certain embodiments have been shown and described, it is distinctly understood that the invention is not limited thereto but may be otherwise embodied within the scope of the appended claims.

Claims

1. A method of modeling a three dimensional target object represented by a plurality of cross- sectional images to provide a representative corresponding three dimensional model, comprising the steps of: selecting an initial model from a plurality of available initial models by identifying an initial model based on physical similarity to said target object; superimposing an initial model upon said target object; for each of said plurality of cross-sectional images, determining an intersection contour of said initial model and a cross-sectional image of said target object; refining said determined intersection contour to more closely delineate said target object; sub-sampling points representing said refined determined intersection contour to provide a sub-sampled contour dataset; and adjusting said initial model towards said sub-sampled contour to provide a representative three dimensional model of said target object.
2. A method according to claim 1 wherein said step of refining said determined intersection contour includes the step of processing image data points representing said determined intersection contour to provide points representing a contour more closely delineating said target object.
3. A method according to claim 1 for providing a representative corresponding three dimensional model for a three dimensional target object varying in shape between different operational phases, including the step of repeating the steps of claim 1 for different shapes of said target object for corresponding different operational phases.
4. A method according to claim 3 wherein said different operational phases comprise at least one of (a) different phases of heart operation, (b) different movement phases of a skeletal structure, (c) different states of an anatomical organ.
5. A method according to claim 1 wherein said data set represents three dimensions.
6. A method according to claim 1 including the step of employing a scale factor to weight data locations from said sub-sampled contour dataset in refining said determined intersection contour to more closely delineate said target object.
7. A method according to claim 1 including the step of selecting an initial model from a plurality of available initial models by identifying an initial model based on physical similarity to said target object.
8. A method according to claim 1 wherein said method of claim 1 is performed automatically for a target object without User entry of data.
9. In an image processing system, a method of modeling a three dimensional target object, varying in shape between different operational phases and represented by a plurality of cross-sectional images, to provide a representative corresponding three dimensional model, comprising the steps of: for a first shape of said target object, superimposing an initial model upon said target object; for each of said plurality of cross-sectional images, determining an intersection contour of said initial model and a cross-sectional image of said target object; refining said determined intersection contour to more closely delineate said target object; and adjusting said initial model towards said refined determined intersection contour to provide a representative three dimensional model of said target object; and for a second shape of said target object, repeating the previous steps.
10. A method according to claim 8 including the step of selecting said initial model from a plurality of available initial models by identifying an initial model most closely fitting said target object.
11. A method according to claim 10, including the step of identifying said initial model most closely fitting said target object by positioning and rotating an image representing said initial model to most closely align with an image representing said target object.
12. A method according to claim 9 wherein said step of refining said determined intersection contour includes the step of applying image processing functions to image data points representing said determined intersection contour to provide points representing a contour more closely delineating said target object.
13. A method according to claim 9 including the step of employing said representative three dimensional model of (a) said target object first shape and (b) said target object second shape, in determining at least one of, (i) volume difference between said first and second shapes, (ii) strain on said target object and (iii) a suitable replacement for said target object.
14. A method according to claim 9 wherein said three dimensional target object varies in shape, between said first shape and said second shape associated with corresponding operational phases, on a substantially periodic basis.
15. In an image processing system, a method of modeling a three dimensional target object, varying in shape between different operational phases and represented by a plurality of cross-sectional images, to provide a representative corresponding three dimensional model, comprising the steps of: for a first shape of said target object, superimposing an initial model upon said target object; for each of said plurality of cross-sectional images, determining an intersection contour of said initial model and a cross-sectional image of said target object; refining said determined intersection contour to more closely delineate said target object; sub-sampling points representing said refined determined intersection contour to provide a sub-sampled contour dataset; and adjusting said initial model towards said sub-sampled contour to provide a representative three dimensional model of said target object; and for a second shape of said target object, repeating the previous steps.
16. A method according to claim 15 including the step of employing said representative three dimensional model of (a) said target object first shape and (b) said target object second shape, in determining at least one of, (i) volume difference between said first and second shapes, (ii) strain on said target object and (iii) a suitable replacement for said target object.
17. A method according to claim 15 including the step of selecting an initial model from a plurality of available initial models by identifying an initial model based on physical similarity to said target object.
PCT/US2002/014634 2001-05-16 2002-05-09 A system for modeling static and dynamic three dimensional anatomical structures by 3-d models WO2002093494A2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
DE10296836T DE10296836T5 (en) 2001-05-16 2002-05-09 System for modeling static and dynamic three-dimensional anatomical structures using 3D models
JP2002590093A JP2004529714A (en) 2001-05-16 2002-05-09 System for modeling static and dynamic 3D anatomical structures using 3D models

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US09/858,368 2001-05-16
US09/858,368 US6816607B2 (en) 2001-05-16 2001-05-16 System for modeling static and dynamic three dimensional anatomical structures by 3-D models

Publications (2)

Publication Number Publication Date
WO2002093494A2 true WO2002093494A2 (en) 2002-11-21
WO2002093494A3 WO2002093494A3 (en) 2003-08-07

Family

ID=25328139

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2002/014634 WO2002093494A2 (en) 2001-05-16 2002-05-09 A system for modeling static and dynamic three dimensional anatomical structures by 3-d models

Country Status (4)

Country Link
US (1) US6816607B2 (en)
JP (1) JP2004529714A (en)
DE (1) DE10296836T5 (en)
WO (1) WO2002093494A2 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10311172A1 (en) * 2003-03-12 2004-09-30 MedCom Gesellschaft für medizinische Bildverarbeitung mbH Image segmenting of objects, especially of human/animal organs and/or results of pathological changes, involves generating object/part profiles from three-dimensional image information in image planes
DE10254907B4 (en) * 2002-11-25 2008-01-03 Siemens Ag Process for surface contouring of a three-dimensional image
WO2018077747A1 (en) * 2016-10-25 2018-05-03 Koninklijke Philips N.V. A method and apparatus for segmenting a two-dimensional image of an anatomical structure
GB2557915A (en) * 2016-12-16 2018-07-04 Calderon Agudo Oscar Method of and apparatus for non invasive medical imaging using waveform inversion

Families Citing this family (64)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8882847B2 (en) 2001-05-25 2014-11-11 Conformis, Inc. Patient selectable knee joint arthroplasty devices
US8617242B2 (en) 2001-05-25 2013-12-31 Conformis, Inc. Implant device and method for manufacture
US8556983B2 (en) 2001-05-25 2013-10-15 Conformis, Inc. Patient-adapted and improved orthopedic implants, designs and related tools
US8735773B2 (en) 2007-02-14 2014-05-27 Conformis, Inc. Implant device and method for manufacture
US8771365B2 (en) 2009-02-25 2014-07-08 Conformis, Inc. Patient-adapted and improved orthopedic implants, designs, and related tools
US9603711B2 (en) 2001-05-25 2017-03-28 Conformis, Inc. Patient-adapted and improved articular implants, designs and related guide tools
US8480754B2 (en) 2001-05-25 2013-07-09 Conformis, Inc. Patient-adapted and improved articular implants, designs and related guide tools
US8545569B2 (en) 2001-05-25 2013-10-01 Conformis, Inc. Patient selectable knee arthroplasty devices
US7239908B1 (en) 1998-09-14 2007-07-03 The Board Of Trustees Of The Leland Stanford Junior University Assessing the condition of a joint and devising treatment
US7184814B2 (en) 1998-09-14 2007-02-27 The Board Of Trustees Of The Leland Stanford Junior University Assessing the condition of a joint and assessing cartilage loss
US9289153B2 (en) 1998-09-14 2016-03-22 The Board Of Trustees Of The Leland Stanford Junior University Joint and cartilage diagnosis, assessment and modeling
JP2005504563A (en) 2001-05-25 2005-02-17 イメージング セラピューティクス,インコーポレーテッド Methods and compositions for resurfacing joints
US20030038802A1 (en) * 2001-08-23 2003-02-27 Johnson Richard K. Automatic delineation of heart borders and surfaces from images
DE10144004A1 (en) * 2001-09-07 2003-03-27 Philips Corp Intellectual Pty Method of taking three-dimensional measurements for use in measuring physical properties in medical applications, by overlaying three dimensional images
US20030095696A1 (en) * 2001-09-14 2003-05-22 Reeves Anthony P. System, method and apparatus for small pulmonary nodule computer aided diagnosis from computed tomography scans
EP1555962B1 (en) 2002-10-07 2011-02-09 Conformis, Inc. Minimally invasive joint implant with 3-dimensional geometry matching the articular surfaces
JP2006505366A (en) 2002-11-07 2006-02-16 コンフォーミス・インコーポレイテッド Method of determining meniscus size and shape and devised treatment
US7211046B2 (en) * 2002-12-19 2007-05-01 Ge Medical Systems Global Technology Company Llc Systems and methods for creating reformatted computed tomography images
US7304644B2 (en) * 2003-03-12 2007-12-04 Siemens Medical Solutions Usa, Inc. System and method for performing a virtual endoscopy
JP4136771B2 (en) * 2003-04-23 2008-08-20 キヤノン株式会社 COMMUNICATION SYSTEM, COMMUNICATION DEVICE, ITS CONTROL METHOD, AND COMPUTER PROGRAM
US7092749B2 (en) * 2003-06-11 2006-08-15 Siemens Medical Solutions Usa, Inc. System and method for adapting the behavior of a diagnostic medical ultrasound system based on anatomic features present in ultrasound images
WO2005004065A1 (en) * 2003-06-25 2005-01-13 Siemens Corporate Research, Inc. Model assisted planning of medical imaging
US7321676B2 (en) * 2003-07-30 2008-01-22 Koninklijke Philips Electronics N.V. Automatic determination of the long axis of the left ventricle in 3D cardiac imaging
US7574247B2 (en) * 2003-11-17 2009-08-11 Siemens Medical Solutions Usa, Inc. Automatic coronary isolation using a n-MIP ray casting technique
ES2239531B2 (en) * 2004-01-29 2007-02-16 Rafael Ibañez Sanchez PROCEDURE FOR THE GENERATION OF REDIMENSIONED ORGANIC VOLUMES.
WO2005076186A1 (en) * 2004-02-09 2005-08-18 Institut De Cardiologie De Montreal Computation of a geometric parameter of a cardiac chamber from a cardiac tomography data set
US20050254708A1 (en) * 2004-04-09 2005-11-17 Marie-Pierre Jolly Segmentation of the left ventricle in apical echocardiographic views using a composite time-consistent active shape model
US7894647B2 (en) * 2004-06-21 2011-02-22 Siemens Medical Solutions Usa, Inc. System and method for 3D contour tracking of anatomical structures
US8064670B2 (en) * 2004-09-29 2011-11-22 Northrop Grumman Systems Corporation Analysis of multidimensional data
CN101189639B (en) * 2005-04-13 2010-09-29 皇家飞利浦电子股份有限公司 A method and a system for segmenting a surface in a multi-dimensional dataset
US7756563B2 (en) * 2005-05-23 2010-07-13 The Penn State Research Foundation Guidance method based on 3D-2D pose estimation and 3D-CT registration with application to live bronchoscopy
US7889905B2 (en) 2005-05-23 2011-02-15 The Penn State Research Foundation Fast 3D-2D image registration method with application to continuously guided endoscopy
US7352370B2 (en) * 2005-06-02 2008-04-01 Accuray Incorporated Four-dimensional volume of interest
ATE534098T1 (en) * 2005-09-23 2011-12-15 Koninkl Philips Electronics Nv METHOD AND SYSTEM FOR ADJUSTING A GEOMETRIC MODEL WITH SEVERAL PARTIAL TRANSFORMATIONS
US7773786B2 (en) * 2005-11-22 2010-08-10 General Electric Company Method and apparatus for three-dimensional interactive tools for semi-automatic segmentation and editing of image objects
EP1892671A3 (en) * 2006-08-23 2009-07-29 Medison Co., Ltd. System and method for determining the volume of an object by image processing
US9037215B2 (en) 2007-01-31 2015-05-19 The Penn State Research Foundation Methods and apparatus for 3D route planning through hollow organs
US8672836B2 (en) * 2007-01-31 2014-03-18 The Penn State Research Foundation Method and apparatus for continuous guidance of endoscopy
US20090156895A1 (en) * 2007-01-31 2009-06-18 The Penn State Research Foundation Precise endoscopic planning and visualization
US9275190B2 (en) * 2007-04-23 2016-03-01 Siemens Aktiengesellschaft Method and system for generating a four-chamber heart model
US20080281182A1 (en) * 2007-05-07 2008-11-13 General Electric Company Method and apparatus for improving and/or validating 3D segmentations
WO2009015295A1 (en) * 2007-07-24 2009-01-29 University Of Pittsburgh-Of The Commonwealth System Of Higher Education System and method for visualizing a structure of interest
US8023734B2 (en) * 2007-10-10 2011-09-20 Siemens Aktiengesellschaft 3D general lesion segmentation in CT
US8682052B2 (en) 2008-03-05 2014-03-25 Conformis, Inc. Implants for altering wear patterns of articular surfaces
JP5288447B2 (en) * 2008-03-28 2013-09-11 学校法人早稲田大学 Surgery support system, approach state detection device and program thereof
EP2303193A4 (en) 2008-05-12 2012-03-21 Conformis Inc Devices and methods for treatment of facet and other joints
WO2010099231A2 (en) 2009-02-24 2010-09-02 Conformis, Inc. Automated systems for manufacturing patient-specific orthopedic implants and instrumentation
AU2010295229B2 (en) * 2009-09-16 2015-07-16 4DMedical Limited Particle image velocimetry suitable for X-ray projection imaging
EP2509539B1 (en) 2009-12-11 2020-07-01 ConforMIS, Inc. Patient-specific and patient-engineered orthopedic implants
US8787521B2 (en) * 2009-12-23 2014-07-22 General Electric Company System and method of iterative image reconstruction for computed tomography
CN102711626B (en) * 2010-01-07 2014-12-10 株式会社日立医疗器械 Medical image diagnosis device, and method for extracting and processing contour of medical image
EP2559003B1 (en) * 2010-04-16 2017-11-29 Koninklijke Philips N.V. Image data segmentation
US8922547B2 (en) * 2010-12-22 2014-12-30 Electronics And Telecommunications Research Institute 3D model shape transformation method and apparatus
EP2754419B1 (en) 2011-02-15 2024-02-07 ConforMIS, Inc. Patient-adapted and improved orthopedic implants
DE102012209984B4 (en) * 2012-06-14 2018-12-27 Siemens Healthcare Gmbh Investigation of the position of the diaphragm of a living being
US9508140B2 (en) * 2012-08-27 2016-11-29 Agency For Science, Technology And Research Quantifying curvature of biological structures from imaging data
US9454643B2 (en) 2013-05-02 2016-09-27 Smith & Nephew, Inc. Surface and image integration for model evaluation and landmark determination
ES2841414T3 (en) 2014-04-15 2021-07-08 4Dmedical Ltd Imaging method
WO2015168792A1 (en) * 2014-05-06 2015-11-12 Circle Cardiovascular Imaging Inc. Method and system for analysis of myocardial wall dynamics
US11723617B2 (en) 2016-02-03 2023-08-15 4DMedical Limited Method and system for imaging
US10545077B2 (en) 2016-03-02 2020-01-28 Jp Scientific Limited Solid phase microextraction coating
US11278256B2 (en) 2016-03-04 2022-03-22 4DMedical Limited Method and system for imaging
CA3019256A1 (en) 2016-05-10 2017-11-16 Jp Scientific Limited System and method for desorbing and detecting an analyte sorbed on a solid phase microextraction device
DE102020213337A1 (en) * 2020-10-22 2022-04-28 Robert Bosch Gesellschaft mit beschränkter Haftung Method for autonomous navigation of a moveable robotic unit and robotic system with the robotic unit

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4873707A (en) 1987-09-11 1989-10-10 Brigham & Women's Hospital X-ray tomography phantoms, method and system
US5360006A (en) 1990-06-12 1994-11-01 University Of Florida Research Foundation, Inc. Automated method for digital image quantitation
US5517602A (en) 1992-12-03 1996-05-14 Hewlett-Packard Company Method and apparatus for generating a topologically consistent visual representation of a three dimensional surface
DE69332042T2 (en) * 1992-12-18 2003-01-02 Koninkl Philips Electronics Nv Delayed positioning of relatively elastically deformed spatial images by matching surfaces
US5435310A (en) 1993-06-23 1995-07-25 University Of Washington Determining cardiac wall thickness and motion by imaging and three-dimensional modeling
US5570430A (en) 1994-05-31 1996-10-29 University Of Washington Method for determining the contour of an in vivo organ using multiple image frames of the organ
US5689577A (en) 1994-10-14 1997-11-18 Picker International, Inc. Procedure for the simplification of triangular surface meshes for more efficient processing
US5782762A (en) 1994-10-27 1998-07-21 Wake Forest University Method and system for producing interactive, three-dimensional renderings of selected body organs having hollow lumens to enable simulated movement through the lumen
US5920319A (en) 1994-10-27 1999-07-06 Wake Forest University Automatic analysis in virtual endoscopy
AU6178696A (en) * 1995-06-16 1997-01-15 Trustees Of The University Of Pennsylvania, The Apparatus and method for dynamic modeling of an object
US5889524A (en) * 1995-09-11 1999-03-30 University Of Washington Reconstruction of three-dimensional objects using labeled piecewise smooth subdivision surfaces
US5682886A (en) * 1995-12-26 1997-11-04 Musculographics Inc Computer-assisted surgical system
US5795296A (en) 1996-03-29 1998-08-18 University Of Washington Pipeline process for automatically measuring object boundary from ultrasound image samples
US5988862A (en) 1996-04-24 1999-11-23 Cyra Technologies, Inc. Integrated system for quickly and accurately imaging and modeling three dimensional objects
US5923770A (en) 1996-09-30 1999-07-13 Siemens Corporate Research, Inc. 3D cardiac motion recovery system using tagged MR images
US6084590A (en) 1997-04-07 2000-07-04 Synapix, Inc. Media production with correlation of image stream and abstract objects in a three-dimensional virtual stage
US6106466A (en) 1997-04-24 2000-08-22 University Of Washington Automated delineation of heart contours from images using reconstruction-based modeling
US6201543B1 (en) 1997-12-17 2001-03-13 Siemens Corporate Research, Inc. Framework for segmentation of cylindrical structures using two dimensional hybrid models
US6563941B1 (en) * 1999-12-14 2003-05-13 Siemens Corporate Research, Inc. Model-based registration of cardiac CTA and MR acquisitions

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
GUPTA A ET AL: "Segmentation and tracking of cine cardiac MR and CT images using a 3-D deformation model" COMPUTERS IN CARDIOLOGY 1994 BETHESDA, MD, USA 25-28 SEPT. 1994, LOS ALAMITOS, CA, USA,IEEE COMPUT. SOC, US, 25 September 1994 (1994-09-25), pages 661-664, XP010147770 ISBN: 0-8186-6570-X *
MCINERNEY T ET AL: "A DYNAMIC FINITE ELEMENT SURFACE MODEL FOR SEGMENTATION AND TRACKING IN MULTIDIMENSIONAL MEDICAL IMAGES WITH APPLICATION TO TO CARDIAC 4D IMAGE ANALYSIS" COMPUTERIZED MEDICAL IMAGING AND GRAPHICS, PERGAMON PRESS, NEW YORK, NY, US, vol. 19, no. 1, 1995, pages 69-83, XP000934040 ISSN: 0895-6111 *
O'DONNELL T ET AL: "A cooperative framework for segmentation using 2D active contours and 3D hybrid models as applied to branching cylindrical structures" PROCEEDINGS OF IEEE 6TH INTERNATIONAL CONFERENCE ON COMPUTER VISION (IEEE CAT. NO.98CH36271), BOMBAY, INDIA, 4-7 JAN. 1998, pages 454-459, XP002237179 1998, New Delhi, India, Narosa Publishing House, India ISBN: 81-7319-221-9 *
O'DONNELL T ET AL: "Multi-modality model-based registration in the cardiac domain" PROCEEDINGS IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION. CVPR 2000 (CAT. NO.PR00662), HILTON HEAD ISLAND, SC, USA, 13-15 JUNE 2000, pages 790-791 vol.2, XP002237180 2000, Los Alamitos, CA, USA, IEEE Comput. Soc, USA ISBN: 0-7695-0662-3 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10254907B4 (en) * 2002-11-25 2008-01-03 Siemens Ag Process for surface contouring of a three-dimensional image
US7376254B2 (en) 2002-11-25 2008-05-20 Siemens Aktiengesellschaft Method for surface-contouring of a three-dimensional image
DE10311172A1 (en) * 2003-03-12 2004-09-30 MedCom Gesellschaft für medizinische Bildverarbeitung mbH Image segmenting of objects, especially of human/animal organs and/or results of pathological changes, involves generating object/part profiles from three-dimensional image information in image planes
WO2018077747A1 (en) * 2016-10-25 2018-05-03 Koninklijke Philips N.V. A method and apparatus for segmenting a two-dimensional image of an anatomical structure
CN109863534A (en) * 2016-10-25 2019-06-07 皇家飞利浦有限公司 Method and apparatus for dividing the two dimensional image of anatomical structure
US10984533B2 (en) 2016-10-25 2021-04-20 Koninklijke Philips N.V. Method and apparatus for segmenting a two-dimensional image of an anatomical structure
CN109863534B (en) * 2016-10-25 2023-08-01 皇家飞利浦有限公司 Method and device for segmenting a two-dimensional image of an anatomical structure
GB2557915A (en) * 2016-12-16 2018-07-04 Calderon Agudo Oscar Method of and apparatus for non invasive medical imaging using waveform inversion
GB2557915B (en) * 2016-12-16 2020-06-10 Calderon Agudo Oscar Method of and apparatus for non invasive medical imaging using waveform inversion

Also Published As

Publication number Publication date
US6816607B2 (en) 2004-11-09
JP2004529714A (en) 2004-09-30
DE10296836T5 (en) 2004-04-29
WO2002093494A3 (en) 2003-08-07
US20020172407A1 (en) 2002-11-21

Similar Documents

Publication Publication Date Title
US6816607B2 (en) System for modeling static and dynamic three dimensional anatomical structures by 3-D models
Bühler et al. Geometric methods for vessel visualization and quantification—a survey
Jolly Automatic segmentation of the left ventricle in cardiac MR and CT images
US7079674B2 (en) Variational approach for the segmentation of the left ventricle in MR cardiac images
Elnakib et al. Medical image segmentation: a brief survey
US7672493B2 (en) Method for analyzing medical image data using level set
EP2338141B1 (en) Segmentation of the long-axis late-enhancement cardiac mri
US20080170791A1 (en) Computerised Cortex Boundary Extraction From Mr Images
US20090202150A1 (en) Variable resolution model based image segmentation
van Assen et al. Cardiac LV segmentation using a 3D active shape model driven by fuzzy inference
WO2003090173A2 (en) Segmentation of 3d medical structures using robust ray propagation
JP2009517163A (en) Method, system and computer program for segmenting structure associated with reference structure in image
Shang et al. Region competition based active contour for medical object extraction
Fritz et al. Segmentation of the left and right cardiac ventricle using a combined bi-temporal statistical model
Archip et al. Ultrasound image segmentation using spectral clustering
Wei et al. Medical image segmentation and its application in cardiac MRI
Metaxas et al. Cardiac segmentation from MRI-tagged and CT images
Lotjonen Segmentation of MR images using deformable models: Application to cardiac images
Qian et al. Robust segmentation of 4D cardiac MRI-tagged images via spatio-temporal propagation
Imielinska et al. Hybrid segmentation methods
Metaxas et al. Hybrid deformable models for medical segmentation and registration
Suri et al. Parametric contour model in medical image segmentation
Metaxas et al. A segmentation and tracking system for 4D cardiac tagged MR images
Mille et al. Segmentation and tracking of the left ventricle in 3D MRI sequences using an active surface model
Dharanibai et al. Automatic localization and segmentation of left ventricle from short axis cine MR images: an image driven method

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): DE JP

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2002590093

Country of ref document: JP

RET De translation (de og part 6b)

Ref document number: 10296836

Country of ref document: DE

Date of ref document: 20040429

Kind code of ref document: P

WWE Wipo information: entry into national phase

Ref document number: 10296836

Country of ref document: DE

REG Reference to national code

Ref country code: DE

Ref legal event code: 8607