U.S. Patent Dec. 6, 1994 Sheet 5 of 5 5,371,778
CONCURRENT DISPLAY AND ADJUSTMENT OF 3D PROJECTION, CORONAL SLICE, SAGITTAL SLICE, AND TRANSVERSE SLICE IMAGES
5
BACKGROUND OF THE INVENTION
The present invention pertains to the image display art. It finds particular application in conjunction with the display of CT medical diagnostic images on video monitors and will be described with particular refer- 10 ence thereto. However, it is to be appreciated that the invention is also applicable to medical diagnostic images from magnetic resonance, nuclear, and other imaging modalities, to quality assurance and other three-dimensional, non-medical images, and the like. The invention 15 is also applicable to hard copy displays, film image displays, and other display formats.
Heretofore, CT scanners have irradiated a planar region of a subject from various angles and detected the intensity of radiation passing therethrough. From the 20 angle and radiation intensity information, two-dimensional image representations of the plane were reconstructed. A typical image representation included a 512x512 pixel array, although coarser and finer arrays are also known. 25
For three-dimensional imaging, the patient was moved along a longitudinal axis of the CT scanner either continuously for spiral scanning or incrementally, to generate a multiplicity of slices. The image data was reconstructed, extrapolating or interpolating as neces- 30 sary, to generate CT numbers corresponding to each of a three-dimensional array of voxels. For simplicity of illustration, each of the CT numbers can be conceptualized as being addressable by its coordinate location along three orthogonal axes, e.g. x, y, and z-axes of the 35 examined volume.
Typically, the volume data was displayed on the planar surface of a video monitor. Various planar representations of the volume data are now commonly available. Most commonly, the examined volume was a six 40 sided prism with square or rectangular faces. The operator could select a display depicting any one of the six faces of the prism or any one of the slices through an interior of the prism along one of the (x,y), (x,z) or (y,z) planes. Some display formats also permitted oblique 45 planes to be selected. Display formats were also available which permitted two or three sides of the prism to be displayed concurrently on a two-dimensional (i,j) image plane with appropriate visual cues to give the impression of a perspective view in three dimensions. 50 That is, the visible faces were foreshortened (or extended) and transformed from rectangles to parallelograms by a sine or cosine value of an angle by which the viewing direction was changed. In this manner, each face of the prism was transformed into its projection 55 along the viewing direction onto the viewing plane. This gives the faces the appearance of extending either parallel to the viewing plane or video monitor screen or extending away from the screen at an oblique angle. Some routines added shading to the view to give further 60 visual cues of depth.
More specifically, the operator could typically cause a selected surface, such as a transverse (x,y) plane on the face (z=0) of the examined volume to be displayed. The operator could then cause a selected number of trans- 65 verse planar slices to be peeled away or deleted by indexing along the z-axis (z= 1,2,3, . . . ,n) to view the nth interior transverse planes. The operator could then
2
position the cursor on the (x,y) or transverse plane to select a coronal or (x,z) plane. The selected coronal plane would then be displayed. The operator would then position the cursor on the displayed coronal plane to select a sagittal or (y,z) plane. Prior art medical image workstations commonly permitted the transverse, coronal, or sagittal planes or views to be displayed concurrently on the same screen. Some also permitted the three-dimensional projection image to be displayed concurrently as well.
One of the disadvantages of these prior art systems is that they did not permit simultaneous, interactive adjustment of the selected transverse, coronal, and sagittal planes. These prior art adjustments were commonly based on a two-dimensional reference plane which was always co-planar with the transverse, sagittal, or coronal planes, therefore restricting the sectioning cursor to two-dimensional movements. In the display format in which all three planes were displayed concurrently, the operator moved the cursor to one of the views, which then became the "active" view. By moving the cursor on the active view, the next planar slice could be reflected. By moving the cursor to the readjusted planar slice, the next slice could be readjusted. Thus, readjusting the displayed transverse, coronal, and sagittal views was sequential and, therefore, relatively slow and time consuming.
The present invention contemplates a new and improved method and apparatus for displaying images which permits concurrent, real-time readjustment of the transverse, coronal, and sagittal view displays by using a rotatable 3D object (or volume) and its projection view as a three-dimensional reference surface which allows the sectioning cursor to move in three dimensions.
SUMMARY OF THE INVENTION
In accordance with one aspect of the present invention, a volume object memory means is provided for holding data values indicative of each voxel of a volumetric region of the object. An affine transform means rotates, scales, and translates points, lines, and surfaces of the volumetric region (object space) into transformed points, lines, and surfaces of a 3D projection view when displayed on the pixels of a two-dimensional image plane or video display (image space). The transform means also supplies a reverse of the selected transform to transform the display pixels into corresponding locations of the object volumetric region. A video processor generates a video display of the data values that correspond to the reverse transformed locations in the volumetric region. An operator uses a cursor control means to move a cursor on the video display. The transform means also reversely transforms coordinates of the cursor from the image plane to a corresponding location in the volumetric region. A plane defining means defines orthogonal planes, preferably, transverse, coronal, and sagittal planes, which intersect the reversely transformed location in the volumetric region. The video processor means receives data values from the object memory lying along each of the planes and converts them into a corresponding video image. Preferably, the video processor converts the two-dimensional projection image representation and the planar images into images which are displayed concurrently in a common video display.
3
In accordance with another aspect of the present invention, a third image space coordinate is determined in accordance with a relative distance along the viewing direction from the screen pixel at the cursor to a point of intersection with a displayed voxel of the ob- 5 ject.
One advantage of the present invention is that the relationship between the volume projection view and the transverse, coronal, and sagittal section (re-sliced) planes is maintained when the volume view is rotated 10 for better visualization. These planes intersect at the cursor in both object and image space. The reverse transform between these spaces enables the planes to be updated correctly in object space regardless of the rotating (or view direction or orientation) of the volume 15 projection view.
Another advantage of the present invention is that it permits interactive and simultaneous adjustment of the transverse, coronal, and sagittal planes.
Another advantage of the present invention is that it 20 assists the operator in relating the position of the displayed transverse, sagittal, and coronal planes with their locations through a perspective type view of the volume.
Another advantage of the present invention is that it 25 permits the operator to select the intersection point of the transverse, coronal, and sagittal planes in three dimensions in object space by using a cursor on a two-dimensional screen.
Still further advantages of the present invention will 30 become apparent to those of ordinary skill in the art upon reading and understanding the following detailed description of the preferred embodiments.
BRIEF DESCRIPTION OF THE DRAWINGS 35
The invention may take form in various components and arrangements of components, and in various steps and arrangements of steps. The drawings are only for purposes of illustrating a preferred embodiment and are not to be construed as limiting the invention. 40
FIG. 1 is a diagrammatic illustration of an image data display system in accordance with the present invention;
FIG. 2 is a diagrammatic illustration of a preferred video display generated by the present invention; 45
FIG. 2A illustrates a transverse plane through the volumetric region;
FIG. 2B illustrates a coronal plane through the volumetric region;
FIG. 2C illustrates a sagittal plane through the volu- 50 metric region;
FIG. 3 is a diagrammatic explanation of the transverse, sagittal, and coronal planes relative to a human subject;
FIG. 4 is analogous to FIG. 2 but illustrates a projec- 55 tion view that has at least one obliquely cut surface;
FIG. 5 is analogous to FIG. 4 but with the perspective view rotated to another viewing orientation.
DETAILED DESCRIPTION OF THE „
PREFERRED EMBODIMENTS
With reference to FIG. 1, a diagnostic imaging device A non-invasively examines a polyhedral volumetric region of a subject and generates a data value indicative of each voxel within the volumetric region. The 65 data values corresponding to voxels of the polyhedron are stored in a three-dimensional object memory means B. The shape and size of the volumetric region is gener
4
ally defined by the diagnostic imaging device. In the embodiment illustrated in FIG. 2, the region is illustrated as a rectangular prism, i.e. a six-sided volume having rectangular or square orthogonal faces. With continuing reference to FIG. 2 and further reference to FIG. 3, the volumetric region is defined by x, y, and z-coordinates which are defined in terms of a transverse plane 10, coronal plane 12, and sagittal plane 14 of a patient or other examined object. For each voxel within the polyhedral examined volumetric region, the imaging device A generates a data value, e.g. a CT number, which, for simplicity of illustration, is retrievable from the object memory B by addressing the object memory with the (x,y,z) coordinates of the voxel. A data processing system C processes the three-dimensional object data to generate a video display D in accordance with instructions input by the operator on an operator control console or system E.
With reference to FIG. 2, the video display D includes a video display screen 20 having a plurality of, e.g. four, view ports. Each view port displays an independently changeable video image. In the preferred embodiment, a first view port 22 displays a projection image depicting a projection of the imaged volume onto the video screen or viewing plane 20. The video screen or viewing plane includes a two-dimensional array of pixels defined by coordinates (i,j). A third coordinate k is defined in a direction orthogonal to the i, j-coordinates of the viewing plane. Faces 24, 26, 28 of the 3D projection image are "distorted" to give visual cues indicative of the depth or distance along the k-axis between the viewing screen and each point on the surface. The rectangular faces in the illustrated projection image are displayed as parallelograms with the angles at the corners changed from orthogonal in proportion to the relative angular orientation or rotation of the viewing plane relative to the examined object region. The dimensions of the parallelograms are likewise foreshortened in accordance with the angular orientation or rotation. Note that if a face is orthogonal to the viewing plane, it is displayed full size with 90° corners. However, as the faces appear to become obliquely oriented toward the viewing screen, the faces are foreshortened and the change in the angles at the corners of the parallelograms becomes more pronounced.
With reference to FIGS. 4 and 5, the operator may conveniently position or rotate the 3D projection image with a selected apparent orientation when viewed from the viewing plane; conversely, the viewer may re-orient or rotate the viewing plane around the polyhedral imaged volume. The volume may be rotated about a selected axis to bring previously hidden faces into view.
With continuing reference to FIG. 2 and further reference to FIG. 3, the operator positions a cursor 30 at a selectable location on the first view port or portion 22 of the video display D. A second view port 32 displays the data along the transverse plane 10 through the position of the cursor. In the coordinate system of FIG. 2, the transverse plane is also the (x,y) plane. In a CT scanner in which a human patient is disposed in a prone position, the transverse plane is also known as an axial plane. Because the x, y, and z-coordinates in object space are fixed, the displayed (x,y) plane is selected by adjusting the selected distance along the z-axis. A third view port 34 displays an image of the coronal plane 12, i.e. the (x,z) plane. A fourth view port 36 displays the (y,z) or sagittal plane 14 through the imaged volume which intersects the (x,y,z) position of the cursor 30.
5 6
To index through the available coronal planes, the coronal and sagittal planes. The rotation can expose
operator moves the cursor 30 across face 24 along track surfaces that were not previously visible.
38c. By moving the cursor along track 38s, the sagittal with reference again to FIG. 1, the non-invasive plane is re-positioned left and right in the illustration of examination means A, in the illustrated embodiment, is
FIG. 2C. To index the transverse planes with the coro- 5 a CT scanner. However, other sources of three dimen
nal and sagittal planes, the operator uses either a trans- sional image data both outside the medical imaging field
verse slice selection means other than a cursor or tracks and in the medical imaging fieldj such ^ magnetic reso
along one of paths 38f and 38r. The examined volumet- nance i rSj ^ contemplated. The non-invasive
nc region illustrated at FIG 2 is through the pelvic medical diagnostic apparatus A includes an examination
region of the patient The pelvic bone 40 and lumbar 10 re^Qn 5Q for the subjec{ rted on a
oftt 7 t ... ... ° ^ ... tient couch or support 52. An irradiating means 54, such image of the first view port 22. The operator's view of * * j-? i
the pelvic bone, lumbar vertebrae, and other associated 38 "? ^ tube' maf ^ *'
tissue is adjusted by moving the cursor 30 until the £adiatfthe Pat,ent-A radlant energy receiving means
transverse, coronal, and sagittal images are optimized 15 56' such » rad»atl0n detectors, radio frequency receiy
for the selected diagnostic procedure. mS colls> or the hke' receive medical diagnostically
Of course, the examined volume may not coincide encoded radiant energy. In the illustrated CT scanner precisely with the region that the operator wants to example, the source of radiant energy is an x-ray tube examine. Other tissues and structures such as air and the which generates a fan-shaped beam of x-rays. The fanpatient couch, are commonly examined and imaged 20 shaped beam of x-rays passes through the subject in the along with the patient. An editing means 44 enables the examination region 50 impinging upon a ring of x-ray operator to make an effective removal of unwanted detectors of the radiant energy detection means 56. The voxels from the examination region. Although remov- x-ray tube is mounted for rotation by a motor or other ing a single selected voxel is conceptually simplest, the rotating means about the examination region such that operator more typically removes or edits larger groups 25 the patient is irradiated from a multiplicity of directions, of voxels. As is conventional in the art, the operator The radiation detectors are positioned either in a stamay define cutting planes, either parallel to one of the tionary ring surrounding the examination ring or in an transverse, coronal, or sagittal planes, or oblique cutting arc which rotates with the x-ray tube to receive the" planes. The operator may also define curved cutting radiation that has traversed the patient, surfaces. A volumetric region edited into a polygon 30 An image reconstruction means 58 reconstructs an with at least one oblique surface is illustrated in FIGS. image representation from the received radiation. For 4 and 5. Rather than editing voxels based on spatial example, the image reconstruction means may reconlocation, the operator can also edit voxels based on struct a 512x512 array of data values, each data value other criteria. For example, air, soft tissue, bone, and ^nig repreSentative of a radiation transmissive propother types of imaged subject matter Save CT numbers 35 ert of a ... voxei of the one plane or slice in distinct ranges. The operator can delete all voxels of the volumetric region. ^ atient couch is indexed with CT numbers corresponding to air for example. As ^ ^ h the examination region hetweea ^ another example, the operator may choose to edit all tQ ate a luraJi of ... q{ ^ data Q ion. voxels except those with CT numbers corresponding to ^ * ^ ^ CQUch ^ transMed continuously bone. This provides a skeletal display in the projection 40 , 5, „ . t. „t- t • A * *u n. <. _r such that the x-ray beam passes through the patient image. As yet another option, the operator may perform , . . „ . , K , . ° , r „. * J-*- *t. - *• • Jjc *u along a spiral path. If spiral data is generated, a convena separate editing for the projection image and the three ■ , - , j • • ^-i- J ^ orthogonal slice images. For example, the projection t,ona1' ^ data construction means is utilrzed to image may be a tissue specific depth Image with shading the sfral data lnto data ^ues eorrespondmg and the three orthogonal images can be interpolated CT 45 «"* of a three-dimensional orthogonal array of voxnumber images. As another example, the projection els> ^ y, z array where x, y, and z are the coordiimage can be edited for tissue type to "peel away" se- nate ^ of obJect sPace- obJect space is the (x,y,z) lected tissue types, thereby providing a new surface for coordinate system of the patient in the scanner; the cursor to traverse. This can be achieved by duplicat- whereas, image space is the (i,j, k) coordinate system of ing the object memory and accessing the memory hold- 50 the projection image presented in the first port 22. ing data edited with one editing function for the projec- The data processing system C includes transform tion image and accessing the memory edited with the means 60 which translates, rotates, and scales coordiother editing function to display the orthogonal slices. nates, lines, curves, and surfaces from object space to In this manner, the operator can display, for example, a image space and reversely transforms locations, lines, projection view of a section of the patient's skeleton to 55 curves, and surfaces from image space to object space, facilitate accurate placement of the cursor while view- More specifically, the affine transform is a matrix which ing images of all tissue in the orthogonal slices through translates coordinates or vectors x, y, z in object space the cursor position. to corresponding coordinates or vectors i, j, k in image
With reference to FIGS. 4 and 5, in many instances, space, i.e.: the displayed projection image of the volume has one or 60 more oblique surfaces 46. As the cursor 30 moves along an oblique surface, such as along track 48, all three of the transverse, coronal, and sagittal planes are indexed [* y- 4
concurrently. Even after the transverse, coronal, and sagittal views are selected, the operator can rotate the 65
viewing plane or imaged object, such as between the Conversely, the reverse of the affine transform matrix positions of FIGS. 4 and 5, without affecting the orien- converts coordinates or vectors in image space to correlation or other aspects of the display of the transverse sponding coordinates or vectors in object space, i.e.:
|
