USRE36207E - Omniview motionless camera orientation system - Google Patents

Omniview motionless camera orientation system Download PDF

Info

Publication number
USRE36207E
USRE36207E US08/662,410 US66241096A USRE36207E US RE36207 E USRE36207 E US RE36207E US 66241096 A US66241096 A US 66241096A US RE36207 E USRE36207 E US RE36207E
Authority
US
United States
Prior art keywords
image
cos
magnification
input
sin
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
US08/662,410
Inventor
Steven D. Zimmermann
H. Lee Martin
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Interactive Pictures Corp
Original Assignee
Omniview Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=24809001&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=USRE36207(E) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Application filed by Omniview Inc filed Critical Omniview Inc
Priority to US08/662,410 priority Critical patent/USRE36207E/en
Assigned to IPIX reassignment IPIX ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MARTIN, H. LEE
Assigned to INTERACTIVE PICTURES CORPORATION reassignment INTERACTIVE PICTURES CORPORATION CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: OMNIVIEW, INC.
Application granted granted Critical
Publication of USRE36207E publication Critical patent/USRE36207E/en
Assigned to IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC, A DELAWARE LIMITED LIABILITY COMPANY reassignment IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC, A DELAWARE LIMITED LIABILITY COMPANY INTELLECTUAL PROPERTY SECURITY AGREEMENT Assignors: INTERNET PICTURES CORPORATION
Assigned to IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC, A DELAWARE LIMITED LIABILITY COMPANY reassignment IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC, A DELAWARE LIMITED LIABILITY COMPANY INTELLECTUAL PROPERTY SECURITY AGREEMENT Assignors: PW TECHNOLOGY, INC.
Assigned to IMAGE INVESTOR PORFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC, A DELAWARE LIMITED LIABILITY COMPANY reassignment IMAGE INVESTOR PORFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC, A DELAWARE LIMITED LIABILITY COMPANY INTELLECTUAL PROPERTY SECURITY AGREEMENT Assignors: INTERACTIVE PICTURES CORPORATION
Assigned to INTERMET PICTURES CORPORATION reassignment INTERMET PICTURES CORPORATION RELEASE Assignors: IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC
Assigned to INTERACTIVE PICTURES CORPORATION reassignment INTERACTIVE PICTURES CORPORATION RELEASE Assignors: IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC
Assigned to PW TECHNOLOGY, INC. reassignment PW TECHNOLOGY, INC. RELEASE Assignors: IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC
Assigned to SONY CORPORATION reassignment SONY CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: IPIX CORPORATION
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G06T3/047
    • G06T3/12
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19617Surveillance camera constructional details
    • G08B13/19626Surveillance camera constructional details optical details, e.g. lenses, mirrors or multiple lenses
    • G08B13/19628Surveillance camera constructional details optical details, e.g. lenses, mirrors or multiple lenses of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/21Intermediate information storage
    • H04N1/2104Intermediate information storage for one or a few pictures
    • H04N1/2158Intermediate information storage for one or a few pictures using a detachable storage unit
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/21Intermediate information storage
    • H04N1/2166Intermediate information storage for mass storage, e.g. in document filing systems
    • H04N1/217Interfaces allowing access to a single user
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/58Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/698Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment
    • H04N5/262Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2628Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/002Special television systems not provided for by H04N7/007 - H04N7/18
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/183Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/08Accessories or related features not otherwise provided for
    • A61B2090/0813Accessories designed for easy sterilising, i.e. re-usable
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B37/00Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
    • G03B37/06Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe involving anamorphosis

Definitions

  • the invention relates to an apparatus, algorithm, and method for transforming a hemispherical field-of-view image into a non-distorted, normal perspective image at any orientation, rotation, and magnification within the field-of-view.
  • the viewing direction, orientation, and magnification are controlled by either computer or remote control means.
  • this apparatus is the electronic equivalent of a mechanical pan, tilt, zoom, and rotation camera viewing system with no moving mechanisms.
  • Camera viewing systems are utilized in abundance for surveillance, inspection, security, and remote sensing. Remote viewing is critical for robotic manipulation tasks. Close viewing is necessary for detailed manipulation tasks while wide-angle viewing aids positioning of the robotic system to avoid collisions with the work space.
  • the majority of these systems use either a fixed-mount camera with a limited viewing field, or they utilize mechanical pan-and-tilt platforms and mechanized zoom lenses to orient the camera and magnify its image. In the applications where orientation of the camera and magnification of its image are required, the mechanical solution is large and can subtend a significant volume making the viewing system difficult to conceal or use in close quarters.
  • Several cameras are usually necessary to provide wide-angle viewing of the work space.
  • Camera viewing systems that use internal optics to provide wide viewing angles have also been developed in order to minimize the size and volume of the camera and the intrusion into the viewing area. These systems rely on the movement of either a mirror or prism to change the tilt-angle of orientation and provide mechanical rotation of the entire camera to change the pitch angle of orientation. Using this means, the size of the camera orientation system can be minimized, but "blind spots" in the center of the view result. Also, these systems typically have no means of magnifying the image and or producing multiple images from a single camera.
  • a further object of the present invention is to provide the ability to produce multiple images with different orientations and magnifications simultaneously.
  • Another object of the present invention is to be able to provide these images at real-time video rates, that is 30 transformed images per second, and to support various display format standards such as the National Television Standards Committee RS-170 display format.
  • an omnidirectional viewing system that produces the equivalent of pan, tilt, zoom, and rotation within a hemispherical field-of-view with no moving parts.
  • This device includes a means for digitizing an incoming video image signal, transforming a portion of said video image based upon operator commands, and producing one or more output images that are in correct perspective for human viewing.
  • the incoming image is produced by a fisheye lens which has a hemispherical field-of-view.
  • This hemispherical field-of-view image is captured into an electronic memory buffer. A portion of the captured image containing a region-of-interest is transformed into a perspective correct image by image processing computer means.
  • the image processing computer provides direct mapping of the hemispherical image region-of-interest into a corrected image using an orthogonal set of transformation algorithms.
  • the viewing orientation is designated by a command signal generated by either a human operator or computerized input.
  • the transformed image is deposited in a second electronic memory buffer where it is then manipulated to produce the output image as requested by the command signal.
  • FIG. 1 shows a schematic block diagram of the present invention illustrating the major components thereof.
  • FIG. 2 is an example sketch of a typical fisheye image used as input by the present invention.
  • FIG. 3 is an example sketch of the output image after correction for a desired image orientation and magnification within the original image.
  • FIG. 4 is a schematic diagram of the fundamental geometry that the present invention embodies to accomplish the image transformation.
  • FIG. 5 is a schematic diagram demonstrating the projection of the object plane and position vector into image plane coordinates.
  • FIG. 1 Shown schematically at 1 is the fisheye lens that provides an image of the environment with a 180 degree field-of-view.
  • the fisheye lens is attached to a camera 2 which converts the optical image into an electrical signal. These signals are then digitized electronically 3 and stored in an image buffer 4 within the present invention.
  • An image processing system consisting of an X-MAP and a Y-MAP processor shown as 6 and 7, respectively, performs the two-dimensional transform mapping.
  • the image transform processors are controlled by the microcomputer and control interface 5.
  • the microcomputer control interface provides initialization and transform parameter calculation for the system.
  • the control interface also determines the desired transformation coefficients based on orientation angle, magnification, rotation, and light sensitivity input from an input means such as a joystick controller 12 or computer input means 13.
  • the transformed image is filtered by a 2-dimensional convolution filter 8 and the output of the filtered image is stored in an output image buffer 9.
  • the output image buffer 9 is scanned out by display electronics 10 to a video display device 11 for viewing.
  • a range of lens types can be accommodated to support various fields of view.
  • the lens optics 1 correspond directly with the mathematical coefficients used with the X-MAP and Y-MAP processors 6 and 7 to transform the image.
  • the capability to pan and tilt the output image remains even though a different maximum field of view is provided with a different lens element.
  • the invention can be realized by proper combination of a number of optical and electronic devices.
  • the fisheye lens 1 is exemplified by any of a series of wide angle lenses from, for example, Nikon, particularly the 8 mm F2.8.
  • Any video source 2 and image capturing device 3 that converts the optical image into electronic memory can serve as the input for the invention such as a Videk Digital Camera interfaced with Texas Instrument's TMS 34061 integrated circuits.
  • Input and output image buffers 4 and 9 can be constructed using Texas Instrument TMS44C251 video random access memory chips or their equivalents.
  • the control interface can be accomplished with any of a number of microcontrollers including the Intel 80C196.
  • the X-MAP and Y-MAP transform processors 6 and 7 and image filtering 8 can be accomplished with application specific integrated circuits or other means as will be known to persons skilled in the art.
  • the display driver can also be accomplished with integrated circuits such as the Texas Instruments TMS34061.
  • the output video signal can be of the NTSC RS-170, for example, compatible with most commercial television displays in the United States.
  • Remote control 12 and computer control 13 are accomplished via readily available switches and/or computer systems that also will be well known. These components function as a system to select a portion of the input image (fisheye or wide angle) and then mathematically transform the image to provide the proper prospective for output.
  • the keys to the success of the invention include:
  • the transformation that occurs between the input memory buffer 4 and the output memory buffer 9, as controlled by the two coordinated transformation circuits 6 and 7, is better understood by looking at FIG. 2 and FIG. 3.
  • the image shown in FIG. 2 is a pen and ink rendering of the image of a grid pattern produced by a fisheye lens. This image has a field-of-view of 180 degrees and shows the contents of the environment throughout an entire hemisphere. Notice that the resulting image in FIG. 2 is significantly distorted relative to human perception.
  • Vertical grid lines in the environment appear in the image plane as 14a, 14b, and 14c.
  • Horizontal grid lines in the environment appear in the image plane as 15a, 15b, and 15c.
  • the image of an object is exemplified by 16. A portion of the image in FIG.
  • Item 17 shows the corrected representation of the object in the output display.
  • the results shown in the image in FIG. 3 can be produced from any portion of the image of FIG. 2 using the present invention. Note the corrected perspective as demonstrated by the straightening of the grid pattern displayed in FIG. 3.
  • these transformations can be performed at real-time video rates (30 times per second), compatible with commercial video standards.
  • the invention as described has the capability to pan and tilt the output image through the entire field of view of the lens element by changing the input means, e.g. the joystick or computer, to the controller.
  • This allows a large area to be scanned for information as can be useful in security and surveillance applications.
  • the image can also be rotated through 360 degrees on its axis changing the perceived vertical of the displayed image.
  • This capability provides the ability to align the vertical image with the gravity vector to maintain a proper perspective in the image display regardless of the pan or tilt angle of the image.
  • the invention also supports modifications in the magnification used to display the output image. This is commensurate with a zoom function that allows a change in the field of view of the output image. This function is extremely useful for inspection operations.
  • the magnitude of zoom provided is a function of the resolution of the input camera, the resolution of the output display, the clarity of the output display, and the amount of picture element (pixel) averaging that is used in a given display.
  • the invention supports all of these functions to provide capabilities associated with traditional mechanical pan (through 180 degrees), tilt (through 180 degrees), rotation (through 360 degrees), and zoom devices.
  • the digital system also supports image intensity scaling that emulates the functionality of a mechanical iris by shifting the intensity or the displayed image based on commands from the user or an external computer.
  • the postulates and equations that follow are based on the present invention utilizing a fisheye lens as the optical element.
  • the first property of a fisheye lens is that the lens has a 2 ⁇ steradian field-of-view and the image it produces is a circle.
  • the second property is that all objects in the field-of-view are in focus, i.e. the perfect fisheye lens has an infinite depth-of-field.
  • the two important postulates of the fisheye lens system (refer to FIGS. 4 and 5) are stated as follows:
  • Postulate 1 Azimuth angle invariability--For object points that lie in a content plane that is perpendicular to the image plane and passes through the image plane origin, all such points are mapped as image points onto the line of intersection between the image plane and the content plane, i.e. along a radial line.
  • the azimuth angle or the image points is therefore invariant to elevation and object distance changes within the content plane.
  • Postulate 2 Equidistant Projection Rule--The radial distance, r, from the image plane origin along the azimuth angle containing the projection of the object point is linearly proportional to the zenith angle ⁇ , where ⁇ is defined as the angle between a perpendicular line through the image plane origin and the line from the image plane origin to the object point.
  • FIG. 4 shows the coordinate reference frames for the object plane and the image plane.
  • the coordinates u,v describe object points within the object plane.
  • the coordinates x,y,z describe points within the image coordinate frame of reference.
  • the object plane shown in FIG. 4 is a typical region of interest to determine the mapping relationship onto the image plane to properly correct the object.
  • the direction of view vector, DOV x,y,z! determines the zenith and azimuth angles for mapping the object plane, UV, onto the image plane, XY.
  • the object plane is defined to be perpendicular to the vector, DOV x,y,z!.
  • D scalar length from the image plane origin to the object plane origin
  • is the zenith angle
  • is the azimuth angle in image plane spherical coordinates.
  • the origin of object plane is represented as a vector using the components given in equation 1 as:
  • DOV x,y,z! is perpendicular to the object plane and its scalar magnitude D provides the distance to the object plane.
  • the object point relative to the UV plane origin in coordinates relative to the origin of the image plane is given by the following:
  • Projection onto a hemisphere of radius R attached to the image plane is determined by scaling the object vector O x,y,z! to produce a surface vector S x,y,z,!: ##EQU1##
  • Equation 10 represents the length or absolute value of the vector O x,y,z! and can be simplified through algebraic and trigonometric manipulation to give: ##EQU3##
  • image plane center to object plane distance D can be represented in terms of the fisheye image circular radius R by the relation:
  • Equation 14 Equation 14 into Equations 12 and 13 provides a means for obtaining an effective scaling operation or magnification which can be used to provide zoom operation. ##EQU5##
  • the Equations 17 and 18 provide a direct mapping from the UV space to the XY image space and are the fundamental mathematical result that supports the functioning of the present omnidirectional viewing system with no moving parts.
  • the locations of X and y in the imaging array can be determined.
  • This approach provides a means to transform an image from the input video buffer to the output video buffer exactly.
  • the fisheye image system is completely symmetrical about the zenith, therefore, the vector assignments and resulting signs of various components can be chosen differently depending on the desired orientation of the object plane with respect to the image plane.
  • these postulates and mathematical equations can be modified for various lens elements as necessary for the desired field-of-view coverage in a given application.
  • the input means defines the zenith angle, ⁇ , the azimuth angle, ⁇ , the object rotation, .0., and the magnification, m. These values are substituted into Equations 19 to determine values for substitution into Equations 17 and 18.
  • the image circle radius, R is a fixed value that is determined by the camera lens ane element relationship.
  • the variables u and v vary throughout the object plane determining the values for x and y in the image plane coordinates.
  • a fisheye lens provides a hemispherical view that is captured by a camera.
  • the image is then transformed into a corrected image at a desired pan, tilt, magnification, rotation, and focus based on the desired view as described by a control input.
  • the image is then output to a television display with the perspective corrected. Accordingly, no mechanical devices are required to attain this extensive analysis and presentation of the view of an environment through 180 degrees of pan, 180 degrees of tilt, 360 degrees of rotation, and various degrees of zoom magnification.

Abstract

A device for omnidirectional image viewing providing pan-and-tilt orientation, rotation, and magnification within a hemispherical field-of-view that utilizes no moving parts. The imaging device is based on the effect that the image from a fisheye lens, which produces a circular image of at entire hemispherical field-of-view, which can be mathematically corrected using high speed electronic circuitry. More specifically, an incoming fisheye image from any image acquisition source is captured in memory of the device, a transformation is performed for the viewing region of interest and viewing direction, and a corrected image is output as a video image signal for viewing, recording, or analysis. As a result, this device can accomplish the functions of pan, tilt, rotation, and zoom throughout a hemispherical field-of-view without the need for any mechanical mechanisms. The preferred embodiment of the image transformation device can provide corrected images at real-time rates, compatible with standard video equipment. The device can be used for any application where a conventional pan-and-tilt or orientation mechanism might be considered including inspection, monitoring, surveillance, and target acquisition.

Description

.Iadd.This invention was made with Government support under contract NAS1-18855 awarded by NASA. The Government has certain rights in this invention. .Iaddend.
TECHNICAL FIELD
The invention relates to an apparatus, algorithm, and method for transforming a hemispherical field-of-view image into a non-distorted, normal perspective image at any orientation, rotation, and magnification within the field-of-view. The viewing direction, orientation, and magnification are controlled by either computer or remote control means. More particularly, this apparatus is the electronic equivalent of a mechanical pan, tilt, zoom, and rotation camera viewing system with no moving mechanisms.
BACKGROUND ART
Camera viewing systems are utilized in abundance for surveillance, inspection, security, and remote sensing. Remote viewing is critical for robotic manipulation tasks. Close viewing is necessary for detailed manipulation tasks while wide-angle viewing aids positioning of the robotic system to avoid collisions with the work space. The majority of these systems use either a fixed-mount camera with a limited viewing field, or they utilize mechanical pan-and-tilt platforms and mechanized zoom lenses to orient the camera and magnify its image. In the applications where orientation of the camera and magnification of its image are required, the mechanical solution is large and can subtend a significant volume making the viewing system difficult to conceal or use in close quarters. Several cameras are usually necessary to provide wide-angle viewing of the work space.
In order to provide a maximum amount of viewing coverage or subtended angle, mechanical pan/tilt mechanisms usually use motorized drives and gear mechanisms to manipulate the vertical and horizontal orientation. An example of such a device is shown in U.S. Pat. No. 4,728,839 issued to J. B. Coughlan, et al, on Mar. 1, 1988. Collisions with the working environment caused by these mechanical pan/tilt orientation mechanisms can damage both the camera and the work space and impede the remote handling operation. Simultaneously, viewing in said remote environments is extremely important to the performance of inspection and manipulation activities.
Camera viewing systems that use internal optics to provide wide viewing angles have also been developed in order to minimize the size and volume of the camera and the intrusion into the viewing area. These systems rely on the movement of either a mirror or prism to change the tilt-angle of orientation and provide mechanical rotation of the entire camera to change the pitch angle of orientation. Using this means, the size of the camera orientation system can be minimized, but "blind spots" in the center of the view result. Also, these systems typically have no means of magnifying the image and or producing multiple images from a single camera.
Accordingly, it is an object of the present invention to provide an apparatus that can provide an image of any portion of the viewing space within a hemispherical field-of-view without moving the apparatus.
It is another object of the present invention to provide horizontal orientation (pan) of the viewing direction with no moving mechanisms.
It is another object of the present invention to provide vertical orientation (tilt) of the viewing direction with no moving mechanisms.
It is another object of the present invention to provide rotational orientation (rotation) of the viewing direction with no moving mechanisms.
It is another object of the present invention to provide the ability to magnify or scale the image (zoom in and out) electronically.
It is another object of the present invention to provide electronic control of the image intensity (iris level).
It is another object of the present invention to be able to change the image intensity (iris level) without any mechanisms.
It is another object of the present invention to be able to accomplish said pan, tilt, zoom, rotation, and iris with simple inputs made by a lay person from a joystick, keyboard controller, or computer controlled means.
It is also an object of the present invention to provide accurate control of the absolute viewing direction and orientations using said input devices.
A further object of the present invention is to provide the ability to produce multiple images with different orientations and magnifications simultaneously.
Another object of the present invention is to be able to provide these images at real-time video rates, that is 30 transformed images per second, and to support various display format standards such as the National Television Standards Committee RS-170 display format.
These and other objects of the present invention will become apparent upon consideration of the drawings hereinafter in combination with a complete description thereof.
DISCLOSURE OF THE INVENTION
In accordance with the present invention, there is provided an omnidirectional viewing system that produces the equivalent of pan, tilt, zoom, and rotation within a hemispherical field-of-view with no moving parts. This device includes a means for digitizing an incoming video image signal, transforming a portion of said video image based upon operator commands, and producing one or more output images that are in correct perspective for human viewing. In one preferred embodiment, the incoming image is produced by a fisheye lens which has a hemispherical field-of-view. This hemispherical field-of-view image is captured into an electronic memory buffer. A portion of the captured image containing a region-of-interest is transformed into a perspective correct image by image processing computer means. The image processing computer provides direct mapping of the hemispherical image region-of-interest into a corrected image using an orthogonal set of transformation algorithms. The viewing orientation is designated by a command signal generated by either a human operator or computerized input. The transformed image is deposited in a second electronic memory buffer where it is then manipulated to produce the output image as requested by the command signal.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 shows a schematic block diagram of the present invention illustrating the major components thereof.
FIG. 2 is an example sketch of a typical fisheye image used as input by the present invention.
FIG. 3 is an example sketch of the output image after correction for a desired image orientation and magnification within the original image.
FIG. 4 is a schematic diagram of the fundamental geometry that the present invention embodies to accomplish the image transformation.
FIG. 5 is a schematic diagram demonstrating the projection of the object plane and position vector into image plane coordinates.
BEST MODE FOR CARRYING OUT THE INVENTION
In order to minimize the size of the camera orientation system while maintaining the ability to zoom, a camera orientation system that utilizes electronic image transformations rather than mechanisms was developed. While numerous patents on mechanical pan-and-tilt systems have been filed, no approach using strictly electronic transforms and fisheye optics has ever been successfully implemented prior to this effort. In addition, the electrooptical approach utilized in the present invention allows multiple images to be extracted from the output of a single camera. Motivation for this device came from viewing system requirements in remote handling applications where the operating envelop of the equipment is a significant constraint to task accomplishment.
The principles of the present invention can be understood by reference to FIG. 1. Shown schematically at 1 is the fisheye lens that provides an image of the environment with a 180 degree field-of-view. The fisheye lens is attached to a camera 2 which converts the optical image into an electrical signal. These signals are then digitized electronically 3 and stored in an image buffer 4 within the present invention. An image processing system consisting of an X-MAP and a Y-MAP processor shown as 6 and 7, respectively, performs the two-dimensional transform mapping. The image transform processors are controlled by the microcomputer and control interface 5. The microcomputer control interface provides initialization and transform parameter calculation for the system. The control interface also determines the desired transformation coefficients based on orientation angle, magnification, rotation, and light sensitivity input from an input means such as a joystick controller 12 or computer input means 13. The transformed image is filtered by a 2-dimensional convolution filter 8 and the output of the filtered image is stored in an output image buffer 9. The output image buffer 9 is scanned out by display electronics 10 to a video display device 11 for viewing.
A range of lens types can be accommodated to support various fields of view. The lens optics 1 correspond directly with the mathematical coefficients used with the X-MAP and Y-MAP processors 6 and 7 to transform the image. The capability to pan and tilt the output image remains even though a different maximum field of view is provided with a different lens element.
The invention can be realized by proper combination of a number of optical and electronic devices. The fisheye lens 1 is exemplified by any of a series of wide angle lenses from, for example, Nikon, particularly the 8 mm F2.8. Any video source 2 and image capturing device 3 that converts the optical image into electronic memory can serve as the input for the invention such as a Videk Digital Camera interfaced with Texas Instrument's TMS 34061 integrated circuits. Input and output image buffers 4 and 9 can be constructed using Texas Instrument TMS44C251 video random access memory chips or their equivalents. The control interface can be accomplished with any of a number of microcontrollers including the Intel 80C196. The X-MAP and Y-MAP transform processors 6 and 7 and image filtering 8 can be accomplished with application specific integrated circuits or other means as will be known to persons skilled in the art. The display driver can also be accomplished with integrated circuits such as the Texas Instruments TMS34061. The output video signal can be of the NTSC RS-170, for example, compatible with most commercial television displays in the United States. Remote control 12 and computer control 13 are accomplished via readily available switches and/or computer systems that also will be well known. These components function as a system to select a portion of the input image (fisheye or wide angle) and then mathematically transform the image to provide the proper prospective for output. The keys to the success of the invention include:
(1) the entire input image need not be transformed, only the portion of interest
(2) the required mathematical transform is predictable based on the lens characteristics.
The transformation that occurs between the input memory buffer 4 and the output memory buffer 9, as controlled by the two coordinated transformation circuits 6 and 7, is better understood by looking at FIG. 2 and FIG. 3. The image shown in FIG. 2 is a pen and ink rendering of the image of a grid pattern produced by a fisheye lens. This image has a field-of-view of 180 degrees and shows the contents of the environment throughout an entire hemisphere. Notice that the resulting image in FIG. 2 is significantly distorted relative to human perception. Vertical grid lines in the environment appear in the image plane as 14a, 14b, and 14c. Horizontal grid lines in the environment appear in the image plane as 15a, 15b, and 15c. The image of an object is exemplified by 16. A portion of the image in FIG. 2 has been correct, magnified, and rotated to produce the image shown in FIG. 3. Item 17 shows the corrected representation of the object in the output display. The results shown in the image in FIG. 3 can be produced from any portion of the image of FIG. 2 using the present invention. Note the corrected perspective as demonstrated by the straightening of the grid pattern displayed in FIG. 3. In the present invention, these transformations can be performed at real-time video rates (30 times per second), compatible with commercial video standards.
The invention as described has the capability to pan and tilt the output image through the entire field of view of the lens element by changing the input means, e.g. the joystick or computer, to the controller. This allows a large area to be scanned for information as can be useful in security and surveillance applications. The image can also be rotated through 360 degrees on its axis changing the perceived vertical of the displayed image. This capability provides the ability to align the vertical image with the gravity vector to maintain a proper perspective in the image display regardless of the pan or tilt angle of the image. The invention also supports modifications in the magnification used to display the output image. This is commensurate with a zoom function that allows a change in the field of view of the output image. This function is extremely useful for inspection operations. The magnitude of zoom provided is a function of the resolution of the input camera, the resolution of the output display, the clarity of the output display, and the amount of picture element (pixel) averaging that is used in a given display. The invention supports all of these functions to provide capabilities associated with traditional mechanical pan (through 180 degrees), tilt (through 180 degrees), rotation (through 360 degrees), and zoom devices. The digital system also supports image intensity scaling that emulates the functionality of a mechanical iris by shifting the intensity or the displayed image based on commands from the user or an external computer.
The postulates and equations that follow are based on the present invention utilizing a fisheye lens as the optical element. There are two basic properties and two basic postulates that describe the perfect fisheye lens system. The first property of a fisheye lens is that the lens has a 2π steradian field-of-view and the image it produces is a circle. The second property is that all objects in the field-of-view are in focus, i.e. the perfect fisheye lens has an infinite depth-of-field. The two important postulates of the fisheye lens system (refer to FIGS. 4 and 5) are stated as follows:
Postulate 1: Azimuth angle invariability--For object points that lie in a content plane that is perpendicular to the image plane and passes through the image plane origin, all such points are mapped as image points onto the line of intersection between the image plane and the content plane, i.e. along a radial line. The azimuth angle or the image points is therefore invariant to elevation and object distance changes within the content plane.
Postulate 2: Equidistant Projection Rule--The radial distance, r, from the image plane origin along the azimuth angle containing the projection of the object point is linearly proportional to the zenith angle β, where β is defined as the angle between a perpendicular line through the image plane origin and the line from the image plane origin to the object point. Thus the relationship:
r=kβ                                                  (1)
Using these properties and postulates as the foundation of the fisheye lens system, the mathematical transformation for obtaining a perspective corrected image can be determined. FIG. 4 shows the coordinate reference frames for the object plane and the image plane. The coordinates u,v describe object points within the object plane. The coordinates x,y,z describe points within the image coordinate frame of reference.
The object plane shown in FIG. 4 is a typical region of interest to determine the mapping relationship onto the image plane to properly correct the object. The direction of view vector, DOV x,y,z!, determines the zenith and azimuth angles for mapping the object plane, UV, onto the image plane, XY. The object plane is defined to be perpendicular to the vector, DOV x,y,z!.
The location of the origin of the object plane in terms of the image plane x y,z! in spherical coordinates is given by:
x=D sin β cos ∂
y=D sin β sin ∂
x=D cos β                                             (2)
where D=scalar length from the image plane origin to the object plane origin, β, is the zenith angle, and ∂ is the azimuth angle in image plane spherical coordinates. The origin of object plane is represented as a vector using the components given in equation 1 as:
DOV x,y,z!= D sin β cos ∂, D sin β sin ∂, D cos β!                             (3)
DOV x,y,z! is perpendicular to the object plane and its scalar magnitude D provides the distance to the object plane. By aligning the YZ plane with the direction of action of DOV x,y,z!, the azimuth angle α becomes either 90 or 270 degrees and therefore the x component becomes zero resulting in the DOV x,y,z! coordinates:
DOV x,y,z!= 0, -D sin β, D cos β!                (4)
Referring now to FIG. 5, the object point relative to the UV plane origin in coordinates relative to the origin of the image plane is given by the following:
x=u
y=v cos β
z=v sin β                                             (5)
therefore, the coordinates of a point P(u,v) that lies in the object plane can be represented as a vector P x y,z! in image plane coordinates:
P x,y,z!= u, v cos β, v sin β!                   (6)
where P x,y,z! describes the position of the object point in image coordinates relative to the origin of the UV plane. The object vector O x,y,z! that describes the object point in image coordinates is then given by:
O x,y,z!=DOV x,y,z!+P x,y,z!                               (7)
O x,y,z!= u, v cos β-D sin β, v sin β+D cos β!(8)
Projection onto a hemisphere of radius R attached to the image plane is determined by scaling the object vector O x,y,z! to produce a surface vector S x,y,z,!: ##EQU1##
By substituting for the components of O x,y,z! from Equation 8, the vector S x,y,z! describing the image point mapping onto the hemisphere becomes: ##EQU2##
The denominator in Equation 10 represents the length or absolute value of the vector O x,y,z! and can be simplified through algebraic and trigonometric manipulation to give: ##EQU3##
From equation 11, the mapping onto the two-dimensional image plane can be obtained for both x and y as: ##EQU4##
Additionally, the image plane center to object plane distance D can be represented in terms of the fisheye image circular radius R by the relation:
D=mR                                                       (14)
where m represents the scale factor in radial units R from the image plane origin to the object plane origin. Substituting Equation 14 into Equations 12 and 13 provides a means for obtaining an effective scaling operation or magnification which can be used to provide zoom operation. ##EQU5##
Using the equations for two-dimensional rotation of axes for both the UV object plane and the XY image plane the last two equations can be further manipulated to provide a more general set of equations that provides for rotation within the image plane and rotation within the object plane. ##EQU6## where:
A=(cos .0. cos ∂-sin .0. sin ∂ cos β)
B=(sin .0. cos ∂+cos .0. sin ∂ cos β)
C=(cos .0. sin ∂+sin .0. cos ∂ cos β)
D=(sin .0. sin ∂-cos .0. cos ∂ cis β)
and where:
R=radius of the image circle
β=zenith angle
∂=Azimuth angle in image plane
.0.=Object plane rotation angle
m=Magnification
u,v=object plane coordinates
x,y=image plane coordinates
The Equations 17 and 18 provide a direct mapping from the UV space to the XY image space and are the fundamental mathematical result that supports the functioning of the present omnidirectional viewing system with no moving parts. By knowing the desired zenith, azimuth, and object plane rotation angles and the magnification, the locations of X and y in the imaging array can be determined. This approach provides a means to transform an image from the input video buffer to the output video buffer exactly. Also, the fisheye image system is completely symmetrical about the zenith, therefore, the vector assignments and resulting signs of various components can be chosen differently depending on the desired orientation of the object plane with respect to the image plane. In addition, these postulates and mathematical equations can be modified for various lens elements as necessary for the desired field-of-view coverage in a given application.
The input means defines the zenith angle, β, the azimuth angle, ∂, the object rotation, .0., and the magnification, m. These values are substituted into Equations 19 to determine values for substitution into Equations 17 and 18. The image circle radius, R, is a fixed value that is determined by the camera lens ane element relationship. The variables u and v vary throughout the object plane determining the values for x and y in the image plane coordinates.
From the foregoing, it can be seen that a fisheye lens provides a hemispherical view that is captured by a camera. The image is then transformed into a corrected image at a desired pan, tilt, magnification, rotation, and focus based on the desired view as described by a control input. The image is then output to a television display with the perspective corrected. Accordingly, no mechanical devices are required to attain this extensive analysis and presentation of the view of an environment through 180 degrees of pan, 180 degrees of tilt, 360 degrees of rotation, and various degrees of zoom magnification.

Claims (11)

I claim:
1. A device for providing perspective corrected views of a selected portion of a hemispherical view in a desired format that utilizes no moving parts, which comprises:
a camera imaging system for receiving optical images and for producing output signals corresponding to said optical images;
fisheye lens means attached to said camera imaging system for producing said optical images, throughout said hemispherical field-of-view, for optical conveyance to said camera imaging system;
image capture means for receiving said output signals from said camera imaging system and for digitizing said output signals from said camera imaging system;
input image memory means for receiving said digitized signals;
image transform processor means for processing said digitized signals in said input image memory means according to selected viewing angles and magnification, and for producing output transform calculation signals according to a combination of said digitized signals, said selected viewing angles and said selected magnification;
output image memory means for receiving said output signals from said image transform processor means;
input means for selecting said viewing angles and magnification;
microprocessor means for receiving said selected viewing angles and magnification from said input means and for converting said selected viewing angles and magnification for input to said image transform processor means to control said processing of said transform processor means; and
output means connected to said output image memory means for recording said perspective corrected view according to said selected viewing angles and magnification.
2. The device of claim 1 wherein said output means includes image display means for providing a perspective corrected image display according to said selected viewing angle and said magnification.
3. The device of claim 1 wherein said input means further provides for input of a selected portion of said hemispherical view to said transform processor means.
4. The device of claim 1 wherein said input means further provides for input of a selected tilting of said viewing angle through 180 degrees.
5. The device of claim 1 wherein said input means further provides for input of a selected rotation of said viewing angle through 360 degrees to achieve said perspective corrected view.
6. The device of claim 1 wherein said input means further provides for input of a selected pan of said viewing angle through 180 degrees.
7. The device of claim 1 wherein said output transform calculation signals of said image transform processor means are produced in real-time at video rates.
8. The device of claim 1 wherein said input means is a user-operated manipulator switch means.
9. The device of claim 1 wherein said image transform processor means is programmed to implement the following two equations: ##EQU7## where:
A=(cos .O slashed. cos ∂-sin .O slashed. sin ∂cos β)
B=(sin .O slashed. cos ∂+cos .O slashed. sin ∂cos β)
C=(cos .O slashed. sin ∂+sin .O slashed. cos ∂cos β)
D=(sin .O slashed. sin ∂-cos .O slashed. cos ∂cos β)
and where:
R=radius of the image circle
β=zenith angle
∂=Azimuth angle in image plane
.O slashed.=Object plane rotation angle
m=Magnification
u,v=object plane coordinates
x,y=image plane coordinates
10. A device for providing perspective corrected views of a selected portion of a hemispherical view in a desired format that utilizes no moving parts, which comprises:
a camera imaging system for receiving optical images and for producing output signals corresponding to said optical images;
fisheye lens means attached to said camera imaging system for producing said optical images, throughout said hemispherical field-of-view, for optical conveyance to said camera imaging system;
image capture means for receiving said output signals from said camera imaging system and for digitizing said output signals from said camera imaging system;
input image memory means for receiving said digitized signals;
image transform processor means for processing said digitized signals in said input image memory means according to selected viewing angles and magnification, and for producing output signals, said selected viewing angles and said selected magnification, according to the equations; ##EQU8## where:
A=(cos .O slashed. cos ∂-sin .O slashed. sin ∂cos β)
B=(sin .O slashed. cos ∂+cos .O slashed. sin ∂cos β)
C=(cos .O slashed. sin ∂+sin .O slashed. cos ∂cos β)
D=(sin .O slashed. sin ∂-cos .O slashed. cos ∂cos β)
and where:
R=radius of the image circle
β=zenith angle
∂=Azimuth angle in image plane
.O slashed.=Object plane rotation angle
m=Magnification
u,v=object plane coordinates
x,y=image plane coordinates
output image memory means for receiving said output signals from said image transform processor means;
input means for selecting said viewing angles and magnification;
microprocessor means for receiving said selected viewing angles and magnification from said input means and for converting said selected viewing and magnification for input to said image transform processor means to control said processing of said transform processor means; and
output means connected to said output image means for recording said perspective corrected views according to said selected viewing angles and implementation.
11. A device for providing perspective corrected views of a selected portion or a hemispherical view in a desired format that utilizes no moving parts, which comprises:
a camera imaging system for receiving optical images and for producing output signals corresponding to said optical images;
fisheye lens means attached to said camera imaging system for producing said optical images, throughout said hemispherical field-of-view, for optical conveyance to said camera imaging system;
image capture means for receiving said output signals from said camera imaging system and for digitizing said output signals from said camera imaging system;
input image memory means for receiving said digitized signals;
image transform processor means for processing said digitized signals in said input image memory means according to selected viewing angles and magnification, and for producing output transform calculation signals in real-time at video rates according to a combination of said digitized signals, said viewing angles and said selected magnification;
user operated input means for selecting said viewing angles and magnification;
microprocessor means for receiving said selected viewing angles and magnification from said user operated input means and for converting said selected viewing angles and magnification for input to said image transform processor means to control said processing of said transform processor means;
output image memory means for receiving said output transform calculation signals in real-time and at video rates from said image transform processor means; and
output means connected to said output image memory means for recording said perspective corrected views according to said selected viewing angles and magnification.
US08/662,410 1991-05-13 1996-07-12 Omniview motionless camera orientation system Expired - Lifetime USRE36207E (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US08/662,410 USRE36207E (en) 1991-05-13 1996-07-12 Omniview motionless camera orientation system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US07/699,366 US5185667A (en) 1991-05-13 1991-05-13 Omniview motionless camera orientation system
US08/662,410 USRE36207E (en) 1991-05-13 1996-07-12 Omniview motionless camera orientation system

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US07/699,366 Reissue US5185667A (en) 1991-05-13 1991-05-13 Omniview motionless camera orientation system

Publications (1)

Publication Number Publication Date
USRE36207E true USRE36207E (en) 1999-05-04

Family

ID=24809001

Family Applications (2)

Application Number Title Priority Date Filing Date
US07/699,366 Ceased US5185667A (en) 1991-05-13 1991-05-13 Omniview motionless camera orientation system
US08/662,410 Expired - Lifetime USRE36207E (en) 1991-05-13 1996-07-12 Omniview motionless camera orientation system

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US07/699,366 Ceased US5185667A (en) 1991-05-13 1991-05-13 Omniview motionless camera orientation system

Country Status (8)

Country Link
US (2) US5185667A (en)
EP (2) EP0539565A4 (en)
JP (1) JP3051173B2 (en)
AT (1) ATE219875T1 (en)
DE (1) DE69232663T2 (en)
DK (1) DK0971540T3 (en)
ES (1) ES2178328T3 (en)
WO (1) WO1992021208A1 (en)

Cited By (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6181335B1 (en) 1992-12-09 2001-01-30 Discovery Communications, Inc. Card for a set top terminal
US6219089B1 (en) * 1997-05-08 2001-04-17 Be Here Corporation Method and apparatus for electronically distributing images from a panoptic camera system
US6222683B1 (en) 1999-01-13 2001-04-24 Be Here Corporation Panoramic imaging arrangement
US6331869B1 (en) 1998-08-07 2001-12-18 Be Here Corporation Method and apparatus for electronically distributing motion panoramic images
US6341044B1 (en) 1996-06-24 2002-01-22 Be Here Corporation Panoramic imaging arrangement
US6369818B1 (en) 1998-11-25 2002-04-09 Be Here Corporation Method, apparatus and computer program product for generating perspective corrected data from warped information
US6373642B1 (en) 1996-06-24 2002-04-16 Be Here Corporation Panoramic imaging arrangement
US6392687B1 (en) 1997-05-08 2002-05-21 Be Here Corporation Method and apparatus for implementing a panoptic camera system
US6426774B1 (en) 1996-06-24 2002-07-30 Be Here Corporation Panoramic camera
US20020118890A1 (en) * 2001-02-24 2002-08-29 Michael Rondinelli Method and apparatus for processing photographic images
US6466254B1 (en) 1997-05-08 2002-10-15 Be Here Corporation Method and apparatus for electronically distributing motion panoramic images
US6493032B1 (en) 1996-06-24 2002-12-10 Be Here Corporation Imaging arrangement which allows for capturing an image of a view at different resolutions
US6515680B1 (en) 1992-12-09 2003-02-04 Discovery Communications, Inc. Set top terminal for television delivery system
US20030095338A1 (en) * 2001-10-29 2003-05-22 Sanjiv Singh System and method for panoramic imaging
US20030103063A1 (en) * 2001-12-03 2003-06-05 Tempest Microsystems Panoramic imaging and display system with canonical magnifier
US6625812B2 (en) * 1999-10-22 2003-09-23 David Hardin Abrams Method and system for preserving and communicating live views of a remote physical location over a computer network
US6675386B1 (en) 1996-09-04 2004-01-06 Discovery Communications, Inc. Apparatus for video access and control over computer network, including image correction
US6704434B1 (en) * 1999-01-27 2004-03-09 Suzuki Motor Corporation Vehicle driving information storage apparatus and vehicle driving information storage method
US20040202380A1 (en) * 2001-03-05 2004-10-14 Thorsten Kohler Method and device for correcting an image, particularly for occupant protection
US20050058360A1 (en) * 2003-09-12 2005-03-17 Thomas Berkey Imaging system and method for displaying and/or recording undistorted wide-angle image data
US20050062845A1 (en) * 2003-09-12 2005-03-24 Mills Lawrence R. Video user interface system and method
EP1600890A2 (en) * 2004-05-28 2005-11-30 Kabushiki Kaisha Toshiba Distortion correction of fish-eye image
US20060028548A1 (en) * 2004-08-06 2006-02-09 Salivar William M System and method for correlating camera views
US20060028550A1 (en) * 2004-08-06 2006-02-09 Palmer Robert G Jr Surveillance system and method
US20060033813A1 (en) * 2004-08-06 2006-02-16 Provinsal Mark S Immersive surveillance system interface
US7123777B2 (en) 2001-09-27 2006-10-17 Eyesee360, Inc. System and method for panoramic imaging
US20070074252A1 (en) * 2005-09-29 2007-03-29 Nazarian David S Method and apparatus for browsing media content based on user affinity
US20070124783A1 (en) * 2005-11-23 2007-05-31 Grandeye Ltd, Uk, Interactive wide-angle video server
US7382399B1 (en) 1991-05-13 2008-06-03 Sony Coporation Omniview motionless camera orientation system
US20080129723A1 (en) * 2006-11-30 2008-06-05 Comer Robert P System and method for converting a fish-eye image into a rectilinear image
US20090128686A1 (en) * 2007-11-19 2009-05-21 Tatsumaro Yamashita Imaging apparatus
US7714936B1 (en) * 1991-05-13 2010-05-11 Sony Corporation Omniview motionless camera orientation system
US7834907B2 (en) 2004-03-03 2010-11-16 Canon Kabushiki Kaisha Image-taking apparatus and image processing method
US7865405B2 (en) 1992-12-09 2011-01-04 Discovery Patent Holdings, Llc Electronic book having electronic commerce features
US20120114262A1 (en) * 2010-11-09 2012-05-10 Chi-Chang Yu Image correction method and related image correction system thereof
US8284258B1 (en) 2008-09-18 2012-10-09 Grandeye, Ltd. Unusual event detection in wide-angle video (based on moving object trajectories)
US20130044258A1 (en) * 2011-08-15 2013-02-21 Danfung Dennis Method for presenting video content on a hand-held electronic device
US8547423B2 (en) 2009-09-24 2013-10-01 Alex Ning Imaging system and device
US8578410B2 (en) 2001-08-03 2013-11-05 Comcast Ip Holdings, I, Llc Video and digital multimedia aggregator content coding and formatting
US8621521B2 (en) 2001-08-03 2013-12-31 Comcast Ip Holdings I, Llc Video and digital multimedia aggregator
US9286294B2 (en) 1992-12-09 2016-03-15 Comcast Ip Holdings I, Llc Video and digital multimedia aggregator content suggestion engine
US9529824B2 (en) * 2013-06-05 2016-12-27 Digitalglobe, Inc. System and method for multi resolution and multi temporal image search
US9813641B2 (en) 2000-06-19 2017-11-07 Comcast Ip Holdings I, Llc Method and apparatus for targeting of interactive virtual objects
US9930225B2 (en) 2011-02-10 2018-03-27 Villmer Llc Omni-directional camera and related viewing software
US10225511B1 (en) 2015-12-30 2019-03-05 Google Llc Low power framework for controlling image sensor mode in a mobile image capture device
US10681268B2 (en) 2014-05-15 2020-06-09 Ricoh Company, Ltd. Imaging system, imaging apparatus, and system
US10732809B2 (en) 2015-12-30 2020-08-04 Google Llc Systems and methods for selective retention and editing of images captured by mobile image capture device

Families Citing this family (280)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6442465B2 (en) 1992-05-05 2002-08-27 Automotive Technologies International, Inc. Vehicular component control systems and methods
US6772057B2 (en) 1995-06-07 2004-08-03 Automotive Technologies International, Inc. Vehicular monitoring systems using image processing
US5845000A (en) * 1992-05-05 1998-12-01 Automotive Technologies International, Inc. Optical identification and monitoring system using pattern recognition for use with vehicles
US6856873B2 (en) 1995-06-07 2005-02-15 Automotive Technologies International, Inc. Vehicular monitoring systems using image processing
US6507779B2 (en) 1995-06-07 2003-01-14 Automotive Technologies International, Inc. Vehicle rear seat monitor
US6553296B2 (en) 1995-06-07 2003-04-22 Automotive Technologies International, Inc. Vehicular occupant detection arrangements
US6393133B1 (en) 1992-05-05 2002-05-21 Automotive Technologies International, Inc. Method and system for controlling a vehicular system based on occupancy of the vehicle
US5815199A (en) * 1991-01-31 1998-09-29 Matsushita Electric Works, Ltd. Interphone with television
US5903319A (en) * 1991-05-13 1999-05-11 Interactive Pictures Corporation Method for eliminating temporal and spacial distortion from interlaced video signals
US6201574B1 (en) 1991-05-13 2001-03-13 Interactive Pictures Corporation Motionless camera orientation system distortion correcting sensing element
US5384588A (en) * 1991-05-13 1995-01-24 Telerobotics International, Inc. System for omindirectional image viewing at a remote location without the transmission of control signals to select viewing parameters
US5528194A (en) * 1991-05-13 1996-06-18 Sony Corporation Apparatus and method for performing geometric transformations on an input image
US5990941A (en) * 1991-05-13 1999-11-23 Interactive Pictures Corporation Method and apparatus for the interactive display of any portion of a spherical image
US6002430A (en) * 1994-01-31 1999-12-14 Interactive Pictures Corporation Method and apparatus for simultaneous capture of a spherical image
US6243131B1 (en) 1991-05-13 2001-06-05 Interactive Pictures Corporation Method for directly scanning a rectilinear imaging element using a non-linear scan
US5313306A (en) * 1991-05-13 1994-05-17 Telerobotics International, Inc. Omniview motionless camera endoscopy system
US5764276A (en) * 1991-05-13 1998-06-09 Interactive Pictures Corporation Method and apparatus for providing perceived video viewing experiences using still images
US5359363A (en) * 1991-05-13 1994-10-25 Telerobotics International, Inc. Omniview motionless camera surveillance system
KR940010592B1 (en) * 1991-10-01 1994-10-24 삼성전자 주식회사 Method of and apparatus for pursueing object of camera
US6005984A (en) * 1991-12-11 1999-12-21 Fujitsu Limited Process and apparatus for extracting and recognizing figure elements using division into receptive fields, polar transformation, application of one-dimensional filter, and correlation between plurality of images
US5631973A (en) 1994-05-05 1997-05-20 Sri International Method for telemanipulation with telepresence
US7887089B2 (en) * 1992-05-05 2011-02-15 Automotive Technologies International, Inc. Vehicular occupant protection system control arrangement and method using multiple sensor systems
US5835613A (en) * 1992-05-05 1998-11-10 Automotive Technologies International, Inc. Optical identification and monitoring system using pattern recognition for use with vehicles
US8152198B2 (en) * 1992-05-05 2012-04-10 Automotive Technologies International, Inc. Vehicular occupant sensing techniques
JP3391405B2 (en) * 1992-05-29 2003-03-31 株式会社エフ・エフ・シー Object identification method in camera image
EP0576843A2 (en) * 1992-06-24 1994-01-05 Siemens Corporate Research, Inc. A method and apparatus for orienting a camera
US5329310A (en) * 1992-06-30 1994-07-12 The Walt Disney Company Method and apparatus for controlling distortion of a projected image
US5581297A (en) * 1992-07-24 1996-12-03 Intelligent Instruments Corporation Low power video security monitoring system
US6121966A (en) * 1992-11-02 2000-09-19 Apple Computer, Inc. Navigable viewing system
US5563650A (en) * 1992-11-24 1996-10-08 Geeris Holding Nederland B.V. Method and device for producing panoramic images, and a method and device for consulting panoramic images
US7401286B1 (en) * 1993-12-02 2008-07-15 Discovery Communications, Inc. Electronic book electronic links
US7336788B1 (en) * 1992-12-09 2008-02-26 Discovery Communicatoins Inc. Electronic book secure communication with home subsystem
US7269841B1 (en) 1992-12-09 2007-09-11 Sedna Patent Services, Llc Digital cable headend for cable television delivery system
US7835989B1 (en) 1992-12-09 2010-11-16 Discovery Communications, Inc. Electronic book alternative delivery systems
US7298851B1 (en) * 1992-12-09 2007-11-20 Discovery Communications, Inc. Electronic book security and copyright protection system
US5659350A (en) 1992-12-09 1997-08-19 Discovery Communications, Inc. Operations center for a television program packaging and delivery system
US7849393B1 (en) 1992-12-09 2010-12-07 Discovery Communications, Inc. Electronic book connection to world watch live
US8073695B1 (en) 1992-12-09 2011-12-06 Adrea, LLC Electronic book with voice emulation features
US6731284B1 (en) 1992-12-14 2004-05-04 Ford Oxaal Method of and apparatus for performing perspective transformation of visible stimuli
US5903782A (en) * 1995-11-15 1999-05-11 Oxaal; Ford Method and apparatus for producing a three-hundred and sixty degree spherical visual data set
US5684937A (en) * 1992-12-14 1997-11-04 Oxaal; Ford Method and apparatus for performing perspective transformation on visible stimuli
US5410232A (en) * 1992-12-18 1995-04-25 Georgia Tech Research Corporation Spherical motor and method
US5402049A (en) * 1992-12-18 1995-03-28 Georgia Tech Research Corporation System and method for controlling a variable reluctance spherical motor
EP0605045B1 (en) * 1992-12-29 1999-03-31 Laboratoires D'electronique Philips S.A.S. Image processing method and apparatus for generating one image from adjacent images
JPH08506225A (en) * 1993-01-20 1996-07-02 ジョン アリグザンドロビクス,ピーター Line monitor system
US9053640B1 (en) 1993-12-02 2015-06-09 Adrea, LLC Interactive electronic book
US8095949B1 (en) 1993-12-02 2012-01-10 Adrea, LLC Electronic book with restricted access features
US7861166B1 (en) 1993-12-02 2010-12-28 Discovery Patent Holding, Llc Resizing document pages to fit available hardware screens
US7865567B1 (en) 1993-12-02 2011-01-04 Discovery Patent Holdings, Llc Virtual on-demand electronic book
FR2714503A1 (en) * 1993-12-29 1995-06-30 Philips Laboratoire Electroniq Image processing method and device for constructing from a source image a target image with change of perspective.
US7768380B2 (en) * 1994-05-09 2010-08-03 Automotive Technologies International, Inc. Security system control for monitoring vehicular compartments
CA2190596C (en) * 1994-05-19 2002-03-26 Theodore M. Lachinski Method for collecting and processing visual and spatial position information
US5796426A (en) * 1994-05-27 1998-08-18 Warp, Ltd. Wide-angle image dewarping method and apparatus
USRE43490E1 (en) 1994-05-27 2012-06-26 B.H. Image Co. Llc Wide-angle dewarping method and apparatus
US5508734A (en) 1994-07-27 1996-04-16 International Business Machines Corporation Method and apparatus for hemispheric imaging which emphasizes peripheral content
WO1996008105A1 (en) * 1994-09-09 1996-03-14 Motorola Inc. Method for creating image data
TW250616B (en) * 1994-11-07 1995-07-01 Discovery Communicat Inc Electronic book selection and delivery system
EP0713331B1 (en) * 1994-11-17 2001-03-14 Canon Kabushiki Kaisha Camera control device and method
CA2155719C (en) * 1994-11-22 2005-11-01 Terry Laurence Glatt Video surveillance system with pilot and slave cameras
JPH08214201A (en) * 1994-11-28 1996-08-20 Canon Inc Image pickup device
US5489940A (en) * 1994-12-08 1996-02-06 Motorola, Inc. Electronic imaging system and sensor for correcting the distortion in a wide-angle lens
US5594935A (en) * 1995-02-23 1997-01-14 Motorola, Inc. Interactive image display system of wide angle images comprising an accounting system
US5646677A (en) * 1995-02-23 1997-07-08 Motorola, Inc. Method and apparatus for interactively viewing wide-angle images from terrestrial, space, and underwater viewpoints
US5706421A (en) * 1995-04-28 1998-01-06 Motorola, Inc. Method and system for reproducing an animated image sequence using wide-angle images
AU5027796A (en) * 1995-03-07 1996-09-23 Interval Research Corporation System and method for selective recording of information
US6493031B1 (en) 1995-03-31 2002-12-10 Canon Kabushiki Kaisha Visual information processing method and apparatus for extracting feature quantities from a two-dimensional image signal
US5703604A (en) * 1995-05-22 1997-12-30 Dodeca Llc Immersive dodecaherdral video viewing system
US5657073A (en) * 1995-06-01 1997-08-12 Panoramic Viewing Systems, Inc. Seamless multi-camera panoramic imaging with distortion correction and selectable field of view
US5920327A (en) * 1995-06-06 1999-07-06 Microsoft Corporation Multiple resolution data display
WO1997005744A1 (en) * 1995-07-27 1997-02-13 Sensormatic Electronics Corporation Image splitting, forming and processing device and method for use with no moving parts camera
US5691765A (en) * 1995-07-27 1997-11-25 Sensormatic Electronics Corporation Image forming and processing device and method for use with no moving parts camera
JPH0955925A (en) * 1995-08-11 1997-02-25 Nippon Telegr & Teleph Corp <Ntt> Picture system
US6031540A (en) 1995-11-02 2000-02-29 Imove Inc. Method and apparatus for simulating movement in multidimensional space with polygonal projections from subhemispherical imagery
US5694531A (en) * 1995-11-02 1997-12-02 Infinite Pictures Method and apparatus for simulating movement in multidimensional space with polygonal projections
US7542035B2 (en) * 1995-11-15 2009-06-02 Ford Oxaal Method for interactively viewing full-surround image data and apparatus therefor
JPH09187038A (en) * 1995-12-27 1997-07-15 Canon Inc Three-dimensional shape extract device
JPH09214932A (en) * 1996-01-30 1997-08-15 Nippon Telegr & Teleph Corp <Ntt> Image device and image communication system
SG70586A1 (en) * 1996-02-21 2000-02-22 Samsung Aerospace Ind Video overhead display system
AU1734197A (en) * 1996-02-23 1997-09-10 Kabushiki Kaisha Yokota Seisakusho Wide visual-field recognizing system
US6118474A (en) * 1996-05-10 2000-09-12 The Trustees Of Columbia University In The City Of New York Omnidirectional imaging apparatus
US5760826A (en) * 1996-05-10 1998-06-02 The Trustees Of Columbia University Omnidirectional imaging apparatus
JPH10164326A (en) * 1996-11-28 1998-06-19 Minolta Co Ltd Image fetching device
US5893062A (en) 1996-12-05 1999-04-06 Interval Research Corporation Variable rate video playback with synchronized audio
US6263507B1 (en) * 1996-12-05 2001-07-17 Interval Research Corporation Browser for use in navigating a body of information, with particular application to browsing information represented by audiovisual data
US6147709A (en) * 1997-04-07 2000-11-14 Interactive Pictures Corporation Method and apparatus for inserting a high resolution image into a low resolution interactive image to produce a realistic immersive experience
US6333826B1 (en) 1997-04-16 2001-12-25 Jeffrey R. Charles Omniramic optical system having central coverage means which is associated with a camera, projector, or similar article
US6449103B1 (en) 1997-04-16 2002-09-10 Jeffrey R. Charles Solid catadioptric omnidirectional optical system having central coverage means which is associated with a camera, projector, medical instrument, or similar article
US5973734A (en) 1997-07-09 1999-10-26 Flashpoint Technology, Inc. Method and apparatus for correcting aspect ratio in a camera graphical user interface
US6188830B1 (en) 1997-07-14 2001-02-13 Sony Corporation Audiovisual effects processing method and apparatus for instantaneous storage-based playback of audio data in synchronization with video data
US6624846B1 (en) 1997-07-18 2003-09-23 Interval Research Corporation Visual user interface for use in controlling the interaction of a device with a spatial region
US6783285B1 (en) * 1997-08-18 2004-08-31 Igor Alexeff Image viewing device
FR2770072B1 (en) * 1997-08-19 2000-07-28 Serge Dulin VIRTUAL CAMERA
US6011558A (en) * 1997-09-23 2000-01-04 Industrial Technology Research Institute Intelligent stitcher for panoramic image-based virtual worlds
US6930709B1 (en) * 1997-12-04 2005-08-16 Pentax Of America, Inc. Integrated internet/intranet camera
US7280134B1 (en) 1998-01-26 2007-10-09 Thales Avionics, Inc. Landscape camera system with electronic field of view switching
EP0933666A1 (en) * 1998-01-30 1999-08-04 SONY DEUTSCHLAND GmbH Image detecting apparatus controlled by detecting the viewing direction
US6226035B1 (en) 1998-03-04 2001-05-01 Cyclo Vision Technologies, Inc. Adjustable imaging system with wide angle capability
US6215519B1 (en) 1998-03-04 2001-04-10 The Trustees Of Columbia University In The City Of New York Combined wide angle and narrow angle imaging system and method for surveillance and monitoring
US6097434A (en) * 1998-03-25 2000-08-01 Intel Corporation System and method for correcting pixel data in an electronic device
US6486908B1 (en) * 1998-05-27 2002-11-26 Industrial Technology Research Institute Image-based method and system for building spherical panoramas
WO1999062252A1 (en) * 1998-05-28 1999-12-02 Bamboo.Com Method and apparatus for creating seamless digital panoramic images
US20010005217A1 (en) * 1998-06-01 2001-06-28 Hamilton Jeffrey Allen Incident recording information transfer device
US6950013B2 (en) 1998-06-01 2005-09-27 Robert Jeffery Scaman Incident recording secure database
US7245319B1 (en) * 1998-06-11 2007-07-17 Fujifilm Corporation Digital image shooting device with lens characteristic correction unit
US6924832B1 (en) 1998-08-07 2005-08-02 Be Here Corporation Method, apparatus & computer program product for tracking objects in a warped video image
US6130704A (en) * 1998-10-22 2000-10-10 Sensormatics Electronics Corporation Controlling movement of video surveillance cameras
US6330022B1 (en) * 1998-11-05 2001-12-11 Lucent Technologies Inc. Digital processing apparatus and method to support video conferencing in variable contexts
JP2002532795A (en) 1998-12-07 2002-10-02 ユニバーサル シティ スタジオズ インコーポレイテッド Image correction method for compensating viewpoint image distortion
US6290645B1 (en) 1998-12-16 2001-09-18 Dynamic Surgical Inventions Llc Endoscopic video camera lamp and coupling assembly
US6747702B1 (en) * 1998-12-23 2004-06-08 Eastman Kodak Company Apparatus and method for producing images without distortion and lateral color aberration
US6317141B1 (en) 1998-12-31 2001-11-13 Flashpoint Technology, Inc. Method and apparatus for editing heterogeneous media objects in a digital imaging device
ATE278202T1 (en) 1999-01-04 2004-10-15 Cyclovision Technologies Inc DEVICE FOR TAKING PANORAMIC IMAGES
US6563532B1 (en) 1999-01-05 2003-05-13 Internal Research Corporation Low attention recording unit for use by vigorously active recorder
US6934461B1 (en) * 1999-01-05 2005-08-23 Interval Research Corporation Low attention recording, with particular application to social recording
JP3126955B2 (en) * 1999-02-12 2001-01-22 株式会社アドバネット Arithmetic unit for image conversion
AU4221000A (en) * 1999-04-08 2000-10-23 Internet Pictures Corporation Remote controlled platform for camera
US6778211B1 (en) 1999-04-08 2004-08-17 Ipix Corp. Method and apparatus for providing virtual processing effects for wide-angle video images
AU3638499A (en) * 1999-04-08 2000-11-14 Interactive Pictures Corporation Apparatus, media and method for capturing and processing spherical images
CA2309459A1 (en) * 1999-06-10 2000-12-10 International Business Machines Corporation System for personalized field of view in a broadcast environment
KR100711950B1 (en) * 1999-06-29 2007-05-02 코닌클리케 필립스 일렉트로닉스 엔.브이. Real-time tracking of an object of interest using a hybrid optical and virtual zooming mechanism
US7015954B1 (en) * 1999-08-09 2006-03-21 Fuji Xerox Co., Ltd. Automatic video system using multiple cameras
AU1569201A (en) * 1999-09-20 2001-04-24 Trustees Of Columbia University In The City Of New York, The Systems and methods for generating spherical mosaic images
US7155735B1 (en) 1999-10-08 2006-12-26 Vulcan Patents Llc System and method for the broadcast dissemination of time-ordered data
FR2801757B1 (en) * 1999-11-30 2002-01-18 Raoul Parienti VIDEO IMAGE VIEWING SYSTEM FOR INTERACTIVE TELEVISION, VIDEO VISOR AND REMOTE MONITORING
US6687387B1 (en) 1999-12-27 2004-02-03 Internet Pictures Corporation Velocity-dependent dewarping of images
US6757682B1 (en) 2000-01-28 2004-06-29 Interval Research Corporation Alerting users to items of current interest
JP4286420B2 (en) * 2000-02-18 2009-07-01 Hoya株式会社 Internet camera
JP2001238199A (en) * 2000-02-25 2001-08-31 Asahi Optical Co Ltd Internet camera system
JP4262384B2 (en) * 2000-02-28 2009-05-13 Hoya株式会社 Internet camera
DE10020260C2 (en) * 2000-04-25 2003-03-27 Rainer Mockler Digital camera or digital video camera and image capturing method
DE10020261C2 (en) * 2000-04-25 2003-04-17 Rainer Mockler Digital camera or digital video camera and image capturing method
JP2001333303A (en) 2000-05-23 2001-11-30 Sharp Corp Omnidirectional vision system
JP2002203254A (en) * 2000-08-30 2002-07-19 Usc Corp Curved surface image transforming method and recording medium with the curved surface image transforming method recorded thereon
JP3650578B2 (en) 2000-09-28 2005-05-18 株式会社立山アールアンドディ Panoramic image navigation system using neural network to correct image distortion
US20020111969A1 (en) * 2000-09-28 2002-08-15 Halstead Robert H. System and method for processing graphical objects for layout using an elastic difference operation
US7562380B2 (en) * 2000-10-27 2009-07-14 Hoya Corporation Internet camera system
CA2328795A1 (en) 2000-12-19 2002-06-19 Advanced Numerical Methods Ltd. Applications and performance enhancements for detail-in-context viewing technology
CA2341965A1 (en) 2000-12-19 2002-06-19 Advanced Numerical Methods Ltd. A method and system for inversion of detail-in-context presentations
KR100409194B1 (en) * 2001-01-05 2003-12-06 주식회사 썸엔터미디어 System and method for making circle vision using virtuality circle vision camera system, and computer-readable media for storing program thereof
US6853809B2 (en) 2001-01-30 2005-02-08 Koninklijke Philips Electronics N.V. Camera system for providing instant switching between wide angle and full resolution views of a subject
US6754400B2 (en) 2001-02-06 2004-06-22 Richard Wilson, Jr. System and method for creation, processing and visualization of omni-directional images
JP3804916B2 (en) * 2001-02-09 2006-08-02 シャープ株式会社 Imaging system, program used for controlling image data thereof, method for correcting distortion of captured image in imaging system, and storage medium storing procedure thereof
EP1370830A1 (en) * 2001-03-13 2003-12-17 Tacshot, Inc. Panoramic aerial imaging device
US20020141657A1 (en) * 2001-03-30 2002-10-03 Robert Novak System and method for a software steerable web Camera
WO2002080521A2 (en) * 2001-03-30 2002-10-10 Digeo, Inc. System and method for a software steerable web camera with multiple image subset capture
WO2002082796A1 (en) * 2001-04-03 2002-10-17 Raoul Parienti Video image display system that is used for interactive television, videoconferencing and telemonitoring
US20020147991A1 (en) * 2001-04-10 2002-10-10 Furlan John L. W. Transmission of panoramic video via existing video infrastructure
US6621518B2 (en) 2001-04-25 2003-09-16 Denis R. Lietgeb Video surveillance system
US8416266B2 (en) 2001-05-03 2013-04-09 Noregin Assetts N.V., L.L.C. Interacting with detail-in-context presentations
CA2345803A1 (en) * 2001-05-03 2002-11-03 Idelix Software Inc. User interface elements for pliable display technology implementations
JP2002334322A (en) * 2001-05-10 2002-11-22 Sharp Corp System, method and program for perspective projection image generation, and storage medium stored with perspective projection image generating program
US7213214B2 (en) * 2001-06-12 2007-05-01 Idelix Software Inc. Graphical user interface with zoom for detail-in-context presentations
JP3698420B2 (en) * 2001-06-12 2005-09-21 シャープ株式会社 Image monitoring apparatus, image monitoring method, and image monitoring processing program
US9760235B2 (en) * 2001-06-12 2017-09-12 Callahan Cellular L.L.C. Lens-defined adjustment of displays
US7084886B2 (en) 2002-07-16 2006-08-01 Idelix Software Inc. Using detail-in-context lenses for accurate digital image cropping and measurement
JP2003069990A (en) * 2001-06-13 2003-03-07 Usc Corp Remote video recognition system
US7079707B2 (en) * 2001-07-20 2006-07-18 Hewlett-Packard Development Company, L.P. System and method for horizon correction within images
JP3624288B2 (en) * 2001-09-17 2005-03-02 株式会社日立製作所 Store management system
CA2361341A1 (en) * 2001-11-07 2003-05-07 Idelix Software Inc. Use of detail-in-context presentation on stereoscopically paired images
DE60213746T2 (en) * 2001-11-28 2007-08-16 Matsushita Electric Industrial Co., Ltd., Kadoma Security system for a house
WO2003067517A2 (en) * 2002-02-04 2003-08-14 Polycom, Inc. Apparatus and method for providing electronic image manipulation in video conferencing applications
CA2370752A1 (en) * 2002-02-05 2003-08-05 Idelix Software Inc. Fast rendering of pyramid lens distorted raster images
DE10213931C1 (en) * 2002-03-28 2003-03-27 Hunger Ibak H Gmbh & Co Kg Method of inspecting drainpipes has discrete two dimensional images used to produce continuous three dimensional image of pipe interior
CA2386702A1 (en) * 2002-05-17 2003-11-17 Idelix Software Inc. Computing the inverse of a pdt distortion
JP4211292B2 (en) * 2002-06-03 2009-01-21 ソニー株式会社 Image processing apparatus, image processing method, program, and program recording medium
US7714943B2 (en) * 2002-06-12 2010-05-11 Geo Semiconductor Inc. Ultra-thin image projection system
US20070165192A1 (en) * 2006-01-13 2007-07-19 Silicon Optix Inc. Reduced field angle projection display system
AU2003243860A1 (en) * 2002-06-12 2003-12-31 Silicon Optix Inc. Short throw projection system and method
AU2003280516A1 (en) * 2002-07-01 2004-01-19 The Regents Of The University Of California Digital processing of video images
US8120624B2 (en) * 2002-07-16 2012-02-21 Noregin Assets N.V. L.L.C. Detail-in-context lenses for digital image cropping, measurement and online maps
CA2393887A1 (en) 2002-07-17 2004-01-17 Idelix Software Inc. Enhancements to user interface for detail-in-context data presentation
US7042508B2 (en) * 2002-07-26 2006-05-09 Appro Technology Inc. Method for presenting fisheye-camera images
WO2004020926A1 (en) * 2002-08-27 2004-03-11 Ircon, Inc. Apparatus and method of sensing the temperature of a molten metal vehicle
DE10243620A1 (en) * 2002-09-19 2004-04-01 Valeo Schalter Und Sensoren Gmbh Process for image processing of the optical signals detected by a system for the environment detection of a vehicle and system for environment detection of a vehicle
CA2406131A1 (en) * 2002-09-30 2004-03-30 Idelix Software Inc. A graphical user interface using detail-in-context folding
CA2449888A1 (en) 2003-11-17 2005-05-17 Idelix Software Inc. Navigating large images using detail-in-context fisheye rendering techniques
CA2407383A1 (en) * 2002-10-10 2004-04-10 Idelix Software Inc. Editing multiple layers of a presentation using detail-in-context lenses
US20070097109A1 (en) * 2005-10-18 2007-05-03 Idelix Software Inc. Method and system for generating detail-in-context presentations in client/server systems
CA2411898A1 (en) 2002-11-15 2004-05-15 Idelix Software Inc. A method and system for controlling access to detail-in-context presentations
US7202888B2 (en) * 2002-11-19 2007-04-10 Hewlett-Packard Development Company, L.P. Electronic imaging device resolution enhancement
US7567274B2 (en) * 2002-12-09 2009-07-28 Frank Edughom Ekpar Method and apparatus for creating interactive virtual tours
TWI231886B (en) * 2003-01-08 2005-05-01 Silicon Optix Inc Image projection system and method
US7391451B2 (en) * 2003-01-09 2008-06-24 Lockheed Martin Corporation Reconfigurable, multi-output frame grabber for machine vision applications
US20040254424A1 (en) * 2003-04-15 2004-12-16 Interscience, Inc. Integrated panoramic and forward view endoscope
TW565735B (en) * 2003-04-18 2003-12-11 Guo-Jen Jan Method for determining the optical parameters of a camera
TW565736B (en) * 2003-04-18 2003-12-11 Guo-Jen Jan Method for determining the optical parameters of a camera
US7528881B2 (en) * 2003-05-02 2009-05-05 Grandeye, Ltd. Multiple object processing in wide-angle video camera
US7450165B2 (en) * 2003-05-02 2008-11-11 Grandeye, Ltd. Multiple-view processing in wide-angle video camera
US20050007453A1 (en) * 2003-05-02 2005-01-13 Yavuz Ahiska Method and system of simultaneously displaying multiple views for video surveillance
US20100002070A1 (en) 2004-04-30 2010-01-07 Grandeye Ltd. Method and System of Simultaneously Displaying Multiple Views for Video Surveillance
US7529424B2 (en) * 2003-05-02 2009-05-05 Grandeye, Ltd. Correction of optical distortion by image processing
US20050028215A1 (en) * 2003-06-03 2005-02-03 Yavuz Ahiska Network camera supporting multiple IP addresses
FR2861525B1 (en) * 2003-10-24 2006-04-28 Winlight System Finance METHOD AND DEVICE FOR CAPTURING A LARGE FIELD IMAGE AND A REGION OF INTEREST THEREOF
JP2005252625A (en) * 2004-03-03 2005-09-15 Canon Inc Image pickup device and image processing method
US7893985B1 (en) 2004-03-15 2011-02-22 Grandeye Ltd. Wide angle electronic camera with improved peripheral vision
US7486302B2 (en) * 2004-04-14 2009-02-03 Noregin Assets N.V., L.L.C. Fisheye lens graphical user interfaces
US20060031879A1 (en) * 2004-04-30 2006-02-09 Vulcan Inc. Management and non-linear presentation of news-related broadcasted or streamed multimedia content
US8427538B2 (en) * 2004-04-30 2013-04-23 Oncam Grandeye Multiple view and multiple object processing in wide-angle video camera
US20060031916A1 (en) * 2004-04-30 2006-02-09 Vulcan Inc. Management and non-linear presentation of broadcasted or streamed multimedia content
US20060031885A1 (en) * 2004-04-30 2006-02-09 Vulcan Inc. Management and non-linear presentation of music-related broadcasted or streamed multimedia content
US8106927B2 (en) 2004-05-28 2012-01-31 Noregin Assets N.V., L.L.C. Graphical user interfaces and occlusion prevention for fisheye lenses with line segment foci
US9317945B2 (en) * 2004-06-23 2016-04-19 Callahan Cellular L.L.C. Detail-in-context lenses for navigation
US7366359B1 (en) * 2004-07-08 2008-04-29 Grandeye, Ltd. Image processing of regions in a wide angle video camera
US7990422B2 (en) 2004-07-19 2011-08-02 Grandeye, Ltd. Automatically expanding the zoom capability of a wide-angle video camera
US7576767B2 (en) * 2004-07-26 2009-08-18 Geo Semiconductors Inc. Panoramic vision system and method
US20060062478A1 (en) * 2004-08-16 2006-03-23 Grandeye, Ltd., Region-sensitive compression of digital video
US7714859B2 (en) 2004-09-03 2010-05-11 Shoemaker Garth B D Occlusion reduction and magnification for multidimensional data presentations
US8860780B1 (en) 2004-09-27 2014-10-14 Grandeye, Ltd. Automatic pivoting in a wide-angle video camera
US7995078B2 (en) 2004-09-29 2011-08-09 Noregin Assets, N.V., L.L.C. Compound lenses for multi-source data presentation
WO2006044476A2 (en) 2004-10-12 2006-04-27 Robert Vernon Vanman Method of and system for mobile surveillance and event recording
US9141615B1 (en) 2004-11-12 2015-09-22 Grandeye, Ltd. Interactive media server
US7894531B1 (en) 2005-02-15 2011-02-22 Grandeye Ltd. Method of compression for wide angle digital video
US7580036B2 (en) 2005-04-13 2009-08-25 Catherine Montagnese Detail-in-context terrain displacement algorithm with optimizations
US20060244831A1 (en) * 2005-04-28 2006-11-02 Kraft Clifford H System and method for supplying and receiving a custom image
US8031206B2 (en) 2005-10-12 2011-10-04 Noregin Assets N.V., L.L.C. Method and system for generating pyramid fisheye lens detail-in-context presentations
US7834910B2 (en) * 2006-03-01 2010-11-16 David M. DeLorme Method and apparatus for panoramic imaging
US7983473B2 (en) * 2006-04-11 2011-07-19 Noregin Assets, N.V., L.L.C. Transparency adjustment of a presentation
US7872593B1 (en) 2006-04-28 2011-01-18 At&T Intellectual Property Ii, L.P. System and method for collecting image data
US9224145B1 (en) 2006-08-30 2015-12-29 Qurio Holdings, Inc. Venue based digital rights using capture device with digital watermarking capability
JP2008225522A (en) * 2007-03-08 2008-09-25 Sony Corp Image processor, camera device, image processing method, and program
DE102007013239A1 (en) * 2007-03-15 2008-09-18 Mobotix Ag supervision order
WO2008123675A1 (en) * 2007-04-06 2008-10-16 Korea Expressway Corporation Multi-area monitoring system from single cctv having a camera quadratic curved surface mirror structure and it, and unwrapping method for the same
US20080298674A1 (en) * 2007-05-29 2008-12-04 Image Masters Inc. Stereoscopic Panoramic imaging system
US8154578B2 (en) * 2007-05-31 2012-04-10 Eastman Kodak Company Multi-camera residential communication system
JP5109803B2 (en) * 2007-06-06 2012-12-26 ソニー株式会社 Image processing apparatus, image processing method, and image processing program
US8543788B2 (en) * 2007-06-06 2013-09-24 Aptina Imaging Corporation Conformal rolling buffer apparatus, systems, and methods
WO2009013845A1 (en) * 2007-07-20 2009-01-29 Techwell Japan K.K. Image processing device and camera system
US9026938B2 (en) * 2007-07-26 2015-05-05 Noregin Assets N.V., L.L.C. Dynamic detail-in-context user interface for application access and content access on electronic displays
KR100882011B1 (en) * 2007-07-29 2009-02-04 주식회사 나노포토닉스 Methods of obtaining panoramic images using rotationally symmetric wide-angle lenses and devices thereof
US7961980B2 (en) * 2007-08-06 2011-06-14 Imay Software Co., Ltd. Method for providing output image in either cylindrical mode or perspective mode
KR101404527B1 (en) * 2007-12-26 2014-06-09 다이니폰 인사츠 가부시키가이샤 Image converter and image converting method
US9860536B2 (en) 2008-02-15 2018-01-02 Enforcement Video, Llc System and method for high-resolution storage of images
US8771177B2 (en) 2008-07-08 2014-07-08 Karl Storz Imaging, Inc. Wide angle flexible endoscope
US10092169B2 (en) 2008-07-08 2018-10-09 Karl Storz Imaging, Inc. Solid state variable direction of view endoscope
US8758234B2 (en) 2008-07-08 2014-06-24 Karl Storz Imaging, Inc. Solid state variable direction of view endoscope
US8814782B2 (en) * 2008-07-08 2014-08-26 Karl Storz Imaging, Inc. Solid state variable direction of view endoscope
US8712362B2 (en) 2008-07-26 2014-04-29 Enforcement Video, Llc Method and system of extending battery life of a wireless microphone unit
JP4629131B2 (en) * 2008-09-03 2011-02-09 大日本印刷株式会社 Image converter
US8326077B2 (en) * 2008-10-31 2012-12-04 General Instrument Corporation Method and apparatus for transforming a non-linear lens-distorted image
JP5235127B2 (en) 2008-11-28 2013-07-10 ヤマハ発動機株式会社 Remote control system and remote control device
CN101750417B (en) * 2008-12-12 2012-03-14 鸿富锦精密工业(深圳)有限公司 Detecting device
JP5169787B2 (en) * 2008-12-12 2013-03-27 大日本印刷株式会社 Image conversion apparatus and image conversion method
US20110032368A1 (en) * 2009-08-07 2011-02-10 Nicholas John Pelling System for Emulating Continuous Pan/Tilt Cameras
JP5536060B2 (en) * 2010-03-18 2014-07-02 パナソニック株式会社 Omnidirectional image processing apparatus and omnidirectional image processing method
KR20120046802A (en) * 2010-10-27 2012-05-11 삼성전자주식회사 Apparatus and method of creating 3 dimension panorama image by using a camera
JP5914813B2 (en) * 2011-01-06 2016-05-11 パナソニックIpマネジメント株式会社 Camera, distortion correction apparatus, and distortion correction method
JP5678324B2 (en) 2011-02-10 2015-03-04 パナソニックIpマネジメント株式会社 Display device, computer program, and display method
JP5966341B2 (en) * 2011-12-19 2016-08-10 大日本印刷株式会社 Image processing apparatus, image processing method, program for image processing apparatus, and image display apparatus
KR20140121431A (en) 2012-01-09 2014-10-15 아이시360, 인코포레이티드 Panoramic optical systems
EP2617349B1 (en) 2012-01-20 2017-09-27 Karl Storz Imaging Inc. Wide angle flexible endoscope
US9763563B2 (en) 2012-07-11 2017-09-19 Karl Storz Imaging, Inc. Endoscopic camera single-button mode activation
PL400346A1 (en) 2012-08-13 2014-02-17 Politechnika Poznanska Method for obtaining a single image from many images of different viewing angle with the use barrel distortion
US9408527B2 (en) 2012-11-01 2016-08-09 Karl Storz Imaging, Inc. Solid state variable direction of view endoscope with rotatable wide-angle field for maximal image performance
EP2745763B1 (en) 2012-12-20 2017-09-27 Karl Storz Imaging Inc. Solid state variable direction of view endoscope
DE102013226196A1 (en) 2013-12-17 2015-06-18 Volkswagen Aktiengesellschaft Optical sensor system
JP2017508351A (en) * 2014-01-10 2017-03-23 リボルブ ロボティクス インク System and method for controlling a robot stand during a video conference
US10932657B2 (en) * 2014-04-02 2021-03-02 Transenterix Europe S.A.R.L. Endoscope with wide angle lens and adjustable view
US9883101B1 (en) * 2014-07-23 2018-01-30 Hoyos Integrity Corporation Providing a real-time via a wireless communication channel associated with a panoramic video capture device
US9660744B1 (en) 2015-01-13 2017-05-23 Enforcement Video, Llc Systems and methods for adaptive frequency synchronization
US9602761B1 (en) 2015-01-22 2017-03-21 Enforcement Video, Llc Systems and methods for intelligently recording a live media stream
JP6518115B2 (en) * 2015-04-13 2019-05-22 キヤノン株式会社 IMAGE PROCESSING APPARATUS, IMAGING APPARATUS, CONTROL METHOD OF IMAGE PROCESSING APPARATUS, AND PROGRAM
JPWO2016189644A1 (en) * 2015-05-26 2018-03-15 オリンパス株式会社 Optical system, imaging device, endoscope system, and distance measuring system
EP3130276B8 (en) * 2015-08-12 2020-02-26 TransEnterix Europe Sàrl Endoscope with wide angle lens and adjustable view
US10440307B2 (en) * 2015-12-22 2019-10-08 Casio Computer Co., Ltd. Image processing device, image processing method and medium
US10334224B2 (en) * 2016-02-19 2019-06-25 Alcacruz Inc. Systems and method for GPU based virtual reality video streaming server
US10250433B1 (en) 2016-03-25 2019-04-02 WatchGuard, Inc. Method and system for peer-to-peer operation of multiple recording devices
US10341605B1 (en) 2016-04-07 2019-07-02 WatchGuard, Inc. Systems and methods for multiple-resolution storage of media streams
EP3249651B1 (en) 2016-05-23 2018-08-29 Axis AB Generating a summary video sequence from a source video sequence
US10699389B2 (en) * 2016-05-24 2020-06-30 Qualcomm Incorporated Fisheye rendering with lens distortion correction for 360-degree video
US20180018807A1 (en) * 2016-07-15 2018-01-18 Aspeed Technology Inc. Method and apparatus for generating panoramic image with texture mapping
KR101889225B1 (en) 2016-09-06 2018-08-16 주식회사 에스360브이알 Method of obtaining stereoscopic panoramic images, playing the same and stereoscopic panoramic camera
EP3315907A1 (en) * 2016-10-27 2018-05-02 Leica Geosystems AG Verfahren zur visuellen darstellung von scandaten
US10999602B2 (en) 2016-12-23 2021-05-04 Apple Inc. Sphere projected motion estimation/compensation and mode decision
US11259046B2 (en) 2017-02-15 2022-02-22 Apple Inc. Processing of equirectangular object data to compensate for distortion by spherical projections
US10924747B2 (en) 2017-02-27 2021-02-16 Apple Inc. Video coding techniques for multi-view video
US11093752B2 (en) 2017-06-02 2021-08-17 Apple Inc. Object tracking in multi-view video
US10754242B2 (en) 2017-06-30 2020-08-25 Apple Inc. Adaptive resolution and projection format in multi-direction video
US20190005709A1 (en) * 2017-06-30 2019-01-03 Apple Inc. Techniques for Correction of Visual Artifacts in Multi-View Images
US11050932B2 (en) * 2019-03-01 2021-06-29 Texas Instruments Incorporated Using real time ray tracing for lens remapping
CN112150554B (en) * 2019-06-28 2023-08-04 杭州海康威视数字技术股份有限公司 Picture display method, device, terminal and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0011909A1 (en) * 1978-08-11 1980-06-11 E.I. Du Pont De Nemours And Company X-ray intensifying screen based on a tantalate phosphor and process for producing the phosphor
WO1982003712A1 (en) * 1981-04-10 1982-10-28 Gabriel Steven Allen Controller for system for spatially transforming images
US4772942A (en) * 1986-01-11 1988-09-20 Pilkington P.E. Limited Display system having wide field of view
JPH02127877A (en) * 1988-11-08 1990-05-16 Casio Comput Co Ltd Electronic still camera provided with fisheye lens
US5023725A (en) * 1989-10-23 1991-06-11 Mccutchen David Method and apparatus for dodecahedral imaging system
US5067019A (en) * 1989-03-31 1991-11-19 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Programmable remapper for image processing
US5068735A (en) * 1989-08-22 1991-11-26 Fuji Photo Optical Co., Ltd. System for controlling the aiming direction, focus, zooming, and/or position of a television camera

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS59115677A (en) * 1982-12-22 1984-07-04 Hitachi Ltd Picture processor
WO1986004204A1 (en) * 1985-01-07 1986-07-17 Tech-21 Pty Limited Imaging method and apparatus

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0011909A1 (en) * 1978-08-11 1980-06-11 E.I. Du Pont De Nemours And Company X-ray intensifying screen based on a tantalate phosphor and process for producing the phosphor
WO1982003712A1 (en) * 1981-04-10 1982-10-28 Gabriel Steven Allen Controller for system for spatially transforming images
US4772942A (en) * 1986-01-11 1988-09-20 Pilkington P.E. Limited Display system having wide field of view
JPH02127877A (en) * 1988-11-08 1990-05-16 Casio Comput Co Ltd Electronic still camera provided with fisheye lens
US5067019A (en) * 1989-03-31 1991-11-19 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Programmable remapper for image processing
US5068735A (en) * 1989-08-22 1991-11-26 Fuji Photo Optical Co., Ltd. System for controlling the aiming direction, focus, zooming, and/or position of a television camera
US5023725A (en) * 1989-10-23 1991-06-11 Mccutchen David Method and apparatus for dodecahedral imaging system

Non-Patent Citations (76)

* Cited by examiner, † Cited by third party
Title
"Declaration of Scott Gilbert in Support of Defendant Infinite Pictures' Memorandum in Opposition to Plaintiff's Motion for Preliminary Injunction", Omniview, Inc. v. Infinite Pictures, Inc., Civ. Action No. 3-96-849
A. Paeth, "Digital Cartography for Computer Graphics", Graphics Gems, 1990, pp. 307-320.
A. Paeth, Digital Cartography for Computer Graphics , Graphics Gems, 1990, pp. 307 320. *
Block diagram of Greene/NYIT system (1986), Greene testimony from Transcript, Feb. 2, 1998, pp. 33 37. *
Block diagram of Greene/NYIT system (1986), Greene testimony from Transcript, Feb. 2, 1998, pp. 33-37.
Color copies of 12 prior art slides (1984 86) shown and described in the Greene trial testimony in Transcript pages identified in captions, Feb. 2, 1998. *
Color copies of 12 prior art slides (1984-86) shown and described in the Greene trial testimony in Transcript pages identified in captions, Feb. 2, 1998.
Color image of the Hawthorne Bridge showing distortion at different magnification (Birdwell Transcript of Feb. 5, 1998, pp. 149 152). *
Color image of the Hawthorne Bridge showing distortion at different magnification (Birdwell Transcript of Feb. 5, 1998, pp. 149-152).
Declaration of Scott Gilbert in Support of Defendant Infinite Pictures Memorandum in Opposition to Plaintiff s Motion for Preliminary Injunction , Omniview, Inc. v. Infinite Pictures, Inc. , Civ. Action No. 3 96 849 *
Diagrams of the geometry employed in the "Fisheye to Box" and "Poly" software used in the DX 280 system, Greene trial testimony from Transcript, Feb. 2, 1998, pp. 43-50.
Diagrams of the geometry employed in the Fisheye to Box and Poly software used in the DX 280 system, Greene trial testimony from Transcript, Feb. 2, 1998, pp. 43 50. *
Drawing from Zimmermann testimony, Jan. 6, 1998. *
DX 402 showing similarity of transforms performed by TRW TMC2302 and TMC 2301 ASICs TRW LSI Products, Inc., LaJolla, CA, 1988. (Birdwell Transcript, Feb. 5, 1998. pp. 157 160). *
DX 402- showing similarity of transforms performed by TRW TMC2302 and TMC 2301 ASICs TRW LSI Products, Inc., LaJolla, CA, 1988. (Birdwell Transcript, Feb. 5, 1998. pp. 157-160).
F. Kenton Musgrave, "A Panoramic Virtual Screen for Ray Tracing", Graphics Gems, 1992, pp. 288-294.
F. Kenton Musgrave, A Panoramic Virtual Screen for Ray Tracing , Graphics Gems, 1992, pp. 288 294. *
F. Pearson II, "Map Projections Theory and Applications", CRC Press, Inc., 1990, pp. 215-345.
F. Pearson II, Map Projections Theory and Applications , CRC Press, Inc., 1990, pp. 215 345. *
Function, Statistics and Trigonometry , Scott, Foresman & Company, 1992, pp. i x, 143, 709 720. *
Function, Statistics and Trigonometry, Scott, Foresman & Company, 1992, pp. i-x, 143, 709-720.
G. David Ripley, "DVI--A Digital Multimedia Technology", Communications of the ACM Jul. 1989, vol. 32, No. 7, pp. 811-822.
G. David Ripley, DVI A Digital Multimedia Technology , Communications of the ACM Jul. 1989, vol. 32, No. 7, pp. 811 822. *
G. Wolberg, "Digital Image Warping", IEEE Computer Society Press, 1988.
G. Wolberg, Digital Image Warping , IEEE Computer Society Press, 1988. *
Greene, Ned and Heckbert, Mark, Creating Raster Omnimax Images from Multiple Perspective Views Using the Elliptical Weighted Average Filter, IEEE Computer Graphics and Applications , Jun. 1986, pp. 21 27. *
Greene, Ned and Heckbert, Mark, Creating Raster Omnimax Images from Multiple Perspective Views Using the Elliptical Weighted Average Filter, IEEE Computer Graphics and Applications, Jun. 1986, pp. 21-27.
Greene, Ned, A Method of Modeling Sky for Computer Animation , Proceedings for a computer animation conference, 1984. *
Greene, Ned, A Method of Modeling Sky for Computer Animation, Proceedings for a computer animation conference, 1984.
Greene, Ned, Environment Mapping and Other Applications of World Projections, IEEE Computer Graphics and Applications , Nov. 1986, pp. 21 29. *
Greene, Ned, Environment Mapping and Other Applications of World Projections, IEEE Computer Graphics and Applications, Nov. 1986, pp. 21-29.
Heckbert, "Fundamentals of Texture Mapping and Image Warping", Report No. UCB/CSD 89/516, Jun. 1989.
Heckbert, "The PMAT and Poly User's Manual", NYIT Document, 1983.
Heckbert, Fundamentals of Texture Mapping and Image Warping , Report No. UCB/CSD 89/516, Jun. 1989. *
Heckbert, Paul, Fundamentals of Texture Mapping and Image Warping , Computer Science Division, University of California, Berkeley, Masters Thesis, Jun. 1989. *
Heckbert, Paul, Fundamentals of Texture Mapping and Image Warping, Computer Science Division, University of California, Berkeley, Masters Thesis, Jun. 1989.
Heckbert, Paul, NYIT PMAT and Poly Users Manual, 1983. *
Heckbert, The PMAT and Poly User s Manual , NYIT Document, 1983. *
Intel Corporation, "Action Media 750 Production Tool Reference", 1988, 1991.
Intel Corporation, Action Media 750 Production Tool Reference , 1988, 1991. *
J. Blinn et al., "Texture and Reflection in Computer Generated Images," Comm. ACM, vol. 19, No. 10, 1976, pp. 542-547.
J. Blinn et al., Texture and Reflection in Computer Generated Images, Comm. ACM, vol. 19, No. 10, 1976, pp. 542 547. *
J. D. Foley et al., "Computer Graphics: Principles and Practice", 1990, 1996, pp. 229-381.
J. D. Foley et al., Computer Graphics: Principles and Practice , 1990, 1996, pp. 229 381. *
M. Onoe et al., "Digital Processing of Images Taken by Fish-Eye Lens", IEEE: Proceedings, New York, 1982, vol. 1, pp. 105-108.
M. Onoe et al., Digital Processing of Images Taken by Fish Eye Lens , IEEE: Proceedings, New York, 1982, vol. 1, pp. 105 108. *
N. Greene et al., "Creating Raster Omnimax Images from Multiple Perspective Views Using the Elliptical Weighted Average Filter", IEEE Computer Graphics and Applications, Jun. 1986, pp. 21-27.
N. Greene et al., Creating Raster Omnimax Images from Multiple Perspective Views Using the Elliptical Weighted Average Filter , IEEE Computer Graphics and Applications, Jun. 1986, pp. 21 27. *
N. Greene, "A Method of Modeling Sky for Computer Animations", Proc. First Int'l. Conf. Engineering and Computer Graphics, Aug. 1984, pp. 297-300.
N. Greene, "Environment Mapping and Other Applications of World Projections", IEEE Computer Graphics and Applications, Nov. 1986, pp. 21-29.
N. Greene, A Method of Modeling Sky for Computer Animations , Proc. First Int l. Conf. Engineering and Computer Graphics, Aug. 1984, pp. 297 300. *
N. Greene, Environment Mapping and Other Applications of World Projections , IEEE Computer Graphics and Applications, Nov. 1986, pp. 21 29. *
Plaintiff s exhibits PX 559 and PX560 showing the failure of perspective correction of a fisheye image of Hawthorne Bridge side rail using 667 patent test program (previously submitted) when actual image radius R 256 pixels is used and with a reduced radius, R 230 pixels, to get a somewhat less distorted output image. (Birdwell Feb. 5, 1998 Transcript at pp. 149 150, line 1). *
Plaintiff's exhibits PX 559 and PX560 showing the failure of perspective correction of a fisheye image of Hawthorne Bridge side rail using '667 patent test program (previously submitted) when actual image radius R˜256 pixels is used and with a reduced radius, R˜230 pixels, to get a somewhat less distorted output image. (Birdwell Feb. 5, 1998 Transcript at pp. 149-150, line 1).
R. Kingslake, "Optical System Design", Academic Press, 1983, pp. 86-87.
R. Kingslake, Optical System Design , Academic Press, 1983, pp. 86 87. *
S. Morris, "Digital Video Interactive--A New Integrated Format for Multi-Media Information", Microcomputer for Information Management, Dec. 1987, 4(4):249-261.
S. Morris, Digital Video Interactive A New Integrated Format for Multi Media Information , Microcomputer for Information Management, Dec. 1987, 4(4):249 261. *
S. Ray, "The Lens in Action", Hastings House, 1976, pp. 114-117.
S. Ray, The Lens in Action , Hastings House, 1976, pp. 114 117. *
TMC2301 ASICS Data Sheet, TRW LSI Products, Inc., LaJolla, CA, 1988. *
TMC2302 ASICS Data Sheets, TRW LSI Products, Inc., LaJolla, CA, 1990. *
Transcript of trial testimony of Dr. Ned Greene, Interactive Pictures Corporation, f/k/a Omniview, Inc., v. Infinite Pictures, Inc. and Bill Tillman , Civil Action No. 3 96 849, U.S. District Court, Eastern District of Tennessee, Feb. 2, 1998. *
Transcript of trial testimony of Dr. Ned Greene, Interactive Pictures Corporation, f/k/a Omniview, Inc., v. Infinite Pictures, Inc. and Bill Tillman, Civil Action No. 3-96-849, U.S. District Court, Eastern District of Tennessee, Feb. 2, 1998.
Transcript of trial testimony of Steven D. Zimmermann, Interactive Pictures Corporation, f/k/a Omniview, Inc. v. Infinite Pictures, Inc. and Bill Tillman , Civil Action No. 3 96 849, U.S. District Court, Eastern District of Tennessee, Jan. 6, 1998, pp. 77 142, 152 156. *
Transcript of trial testimony of Steven D. Zimmermann, Interactive Pictures Corporation, f/k/a Omniview, Inc. v. Infinite Pictures, Inc. and Bill Tillman, Civil Action No. 3-96-849, U.S. District Court, Eastern District of Tennessee, Jan. 6, 1998, pp. 77-142, 152-156.
Transcripts of relevant trial testimony of Dr. Douglas Birdwell: Transcript of Jan. 7, 1998, pp. 61 72, Transcript of Jan. 8, 1998, pp. 27 42, Transcript of Feb. 5, 1998, pp. 65 165. *
Transcripts of relevant trial testimony of Dr. Douglas Birdwell: Transcript of Jan. 7, 1998, pp. 61-72, Transcript of Jan. 8, 1998, pp. 27-42, Transcript of Feb. 5, 1998, pp. 65-165.
Two (2) Japanese prior art articles authored by Dr. Murio Kuno (1980). *
Two sketches drawn by Dr. Douglas Birdwell, Transcript, Jan. 8, 1998, pp. 27 41. *
Two sketches drawn by Dr. Douglas Birdwell, Transcript, Jan. 8, 1998, pp. 27-41.
U.S. Geological Survey Professional Paper 1395, Map Projections A Working Manual , 1987, pp. viii ix, 3 10, 33 35, 90 91, 164 168. *
U.S. Geological Survey Professional Paper 1395, Map Projections--A Working Manual, 1987, pp. viii-ix, 3-10, 33-35, 90-91, 164-168.
Upstill, Steve, Building Strong Images , UNIX Review, Oct. 1988, pp. 63 73., *
Upstill, Steve, Building Strong Images, UNIX Review, Oct. 1988, pp. 63-73.,
Zimmermann et al, excerpts from Phase I NASA Test Report, Aug. 1988. *

Cited By (79)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7382399B1 (en) 1991-05-13 2008-06-03 Sony Coporation Omniview motionless camera orientation system
US7714936B1 (en) * 1991-05-13 2010-05-11 Sony Corporation Omniview motionless camera orientation system
US6181335B1 (en) 1992-12-09 2001-01-30 Discovery Communications, Inc. Card for a set top terminal
US7865405B2 (en) 1992-12-09 2011-01-04 Discovery Patent Holdings, Llc Electronic book having electronic commerce features
US9286294B2 (en) 1992-12-09 2016-03-15 Comcast Ip Holdings I, Llc Video and digital multimedia aggregator content suggestion engine
US6515680B1 (en) 1992-12-09 2003-02-04 Discovery Communications, Inc. Set top terminal for television delivery system
US8060905B1 (en) 1992-12-09 2011-11-15 Comcast Ip Holdings I, Llc Television delivery system having interactive electronic program guide
US20030193607A1 (en) * 1996-06-24 2003-10-16 Be Here Corporation Panoramic camera
US6373642B1 (en) 1996-06-24 2002-04-16 Be Here Corporation Panoramic imaging arrangement
US6426774B1 (en) 1996-06-24 2002-07-30 Be Here Corporation Panoramic camera
US6593969B1 (en) 1996-06-24 2003-07-15 Be Here Corporation Preparing a panoramic image for presentation
US20030193606A1 (en) * 1996-06-24 2003-10-16 Be Here Corporation Panoramic camera
US6480229B1 (en) 1996-06-24 2002-11-12 Be Here Corporation Panoramic camera
US6493032B1 (en) 1996-06-24 2002-12-10 Be Here Corporation Imaging arrangement which allows for capturing an image of a view at different resolutions
US6341044B1 (en) 1996-06-24 2002-01-22 Be Here Corporation Panoramic imaging arrangement
US6515696B1 (en) 1996-06-24 2003-02-04 Be Here Corporation Method and apparatus for presenting images from a remote location
US6542184B1 (en) 1996-06-24 2003-04-01 Edward Driscoll, Jr. Methods, apparatus, and program products for presenting panoramic images of a remote location
USRE44087E1 (en) 1996-06-24 2013-03-19 B.H. Image Co. Llc Presenting panoramic images with geometric transformation
US6337708B1 (en) 1996-06-24 2002-01-08 Be Here Corporation Method and apparatus for electronically distributing motion panoramic images
US6583815B1 (en) 1996-06-24 2003-06-24 Be Here Corporation Method and apparatus for presenting images from a remote location
US6675386B1 (en) 1996-09-04 2004-01-06 Discovery Communications, Inc. Apparatus for video access and control over computer network, including image correction
US20040010804A1 (en) * 1996-09-04 2004-01-15 Hendricks John S. Apparatus for video access and control over computer network, including image correction
US6466254B1 (en) 1997-05-08 2002-10-15 Be Here Corporation Method and apparatus for electronically distributing motion panoramic images
US6392687B1 (en) 1997-05-08 2002-05-21 Be Here Corporation Method and apparatus for implementing a panoptic camera system
US6219089B1 (en) * 1997-05-08 2001-04-17 Be Here Corporation Method and apparatus for electronically distributing images from a panoptic camera system
US6331869B1 (en) 1998-08-07 2001-12-18 Be Here Corporation Method and apparatus for electronically distributing motion panoramic images
US6369818B1 (en) 1998-11-25 2002-04-09 Be Here Corporation Method, apparatus and computer program product for generating perspective corrected data from warped information
US6222683B1 (en) 1999-01-13 2001-04-24 Be Here Corporation Panoramic imaging arrangement
US6704434B1 (en) * 1999-01-27 2004-03-09 Suzuki Motor Corporation Vehicle driving information storage apparatus and vehicle driving information storage method
US6625812B2 (en) * 1999-10-22 2003-09-23 David Hardin Abrams Method and system for preserving and communicating live views of a remote physical location over a computer network
US9813641B2 (en) 2000-06-19 2017-11-07 Comcast Ip Holdings I, Llc Method and apparatus for targeting of interactive virtual objects
US20020118890A1 (en) * 2001-02-24 2002-08-29 Michael Rondinelli Method and apparatus for processing photographic images
US7304680B2 (en) * 2001-03-05 2007-12-04 Siemens Aktiengesellschaft Method and device for correcting an image, particularly for occupant protection
US20040202380A1 (en) * 2001-03-05 2004-10-14 Thorsten Kohler Method and device for correcting an image, particularly for occupant protection
US8578410B2 (en) 2001-08-03 2013-11-05 Comcast Ip Holdings, I, Llc Video and digital multimedia aggregator content coding and formatting
US8621521B2 (en) 2001-08-03 2013-12-31 Comcast Ip Holdings I, Llc Video and digital multimedia aggregator
US10349096B2 (en) 2001-08-03 2019-07-09 Comcast Ip Holdings I, Llc Video and digital multimedia aggregator content coding and formatting
US10140433B2 (en) 2001-08-03 2018-11-27 Comcast Ip Holdings I, Llc Video and digital multimedia aggregator
US7123777B2 (en) 2001-09-27 2006-10-17 Eyesee360, Inc. System and method for panoramic imaging
US20030095338A1 (en) * 2001-10-29 2003-05-22 Sanjiv Singh System and method for panoramic imaging
US7058239B2 (en) 2001-10-29 2006-06-06 Eyesee360, Inc. System and method for panoramic imaging
US6833843B2 (en) * 2001-12-03 2004-12-21 Tempest Microsystems Panoramic imaging and display system with canonical magnifier
US7274381B2 (en) * 2001-12-03 2007-09-25 Tempest Microsystems, Inc. Panoramic imaging and display system with canonical magnifier
US20030103063A1 (en) * 2001-12-03 2003-06-05 Tempest Microsystems Panoramic imaging and display system with canonical magnifier
US20050259118A1 (en) * 2001-12-03 2005-11-24 Michael Mojaver Panoramic imaging and display system with canonical magnifier
US9955073B2 (en) 2003-09-12 2018-04-24 Sensormatic Electronics, LLC Video user interface system and method
US20050062845A1 (en) * 2003-09-12 2005-03-24 Mills Lawrence R. Video user interface system and method
US20050058360A1 (en) * 2003-09-12 2005-03-17 Thomas Berkey Imaging system and method for displaying and/or recording undistorted wide-angle image data
US7834907B2 (en) 2004-03-03 2010-11-16 Canon Kabushiki Kaisha Image-taking apparatus and image processing method
EP1600890A2 (en) * 2004-05-28 2005-11-30 Kabushiki Kaisha Toshiba Distortion correction of fish-eye image
EP1600890A3 (en) * 2004-05-28 2011-12-07 Kabushiki Kaisha Toshiba Distortion correction of fish-eye image
US7629995B2 (en) 2004-08-06 2009-12-08 Sony Corporation System and method for correlating camera views
US20060028548A1 (en) * 2004-08-06 2006-02-09 Salivar William M System and method for correlating camera views
US7750936B2 (en) 2004-08-06 2010-07-06 Sony Corporation Immersive surveillance system interface
US9749525B2 (en) 2004-08-06 2017-08-29 Sony Semiconductor Solutions Corporation System and method for correlating camera views
US20090262196A1 (en) * 2004-08-06 2009-10-22 Sony Corporation System and method for correlating camera views
US8692881B2 (en) 2004-08-06 2014-04-08 Sony Corporation System and method for correlating camera views
US20060033813A1 (en) * 2004-08-06 2006-02-16 Provinsal Mark S Immersive surveillance system interface
US20060028550A1 (en) * 2004-08-06 2006-02-09 Palmer Robert G Jr Surveillance system and method
US20070074252A1 (en) * 2005-09-29 2007-03-29 Nazarian David S Method and apparatus for browsing media content based on user affinity
US7707137B2 (en) * 2005-09-29 2010-04-27 Sun Microsystems, Inc. Method and apparatus for browsing media content based on user affinity
US8723951B2 (en) 2005-11-23 2014-05-13 Grandeye, Ltd. Interactive wide-angle video server
US20070124783A1 (en) * 2005-11-23 2007-05-31 Grandeye Ltd, Uk, Interactive wide-angle video server
US20080129723A1 (en) * 2006-11-30 2008-06-05 Comer Robert P System and method for converting a fish-eye image into a rectilinear image
US8670001B2 (en) 2006-11-30 2014-03-11 The Mathworks, Inc. System and method for converting a fish-eye image into a rectilinear image
US8134608B2 (en) * 2007-11-19 2012-03-13 Alps Electric Co., Ltd. Imaging apparatus
US20090128686A1 (en) * 2007-11-19 2009-05-21 Tatsumaro Yamashita Imaging apparatus
US8284258B1 (en) 2008-09-18 2012-10-09 Grandeye, Ltd. Unusual event detection in wide-angle video (based on moving object trajectories)
US8547423B2 (en) 2009-09-24 2013-10-01 Alex Ning Imaging system and device
US9153014B2 (en) * 2010-11-09 2015-10-06 Avisonic Technology Corporation Image correction method and related image correction system thereof
US20120114262A1 (en) * 2010-11-09 2012-05-10 Chi-Chang Yu Image correction method and related image correction system thereof
US9930225B2 (en) 2011-02-10 2018-03-27 Villmer Llc Omni-directional camera and related viewing software
US20130044258A1 (en) * 2011-08-15 2013-02-21 Danfung Dennis Method for presenting video content on a hand-held electronic device
US9529824B2 (en) * 2013-06-05 2016-12-27 Digitalglobe, Inc. System and method for multi resolution and multi temporal image search
US10681268B2 (en) 2014-05-15 2020-06-09 Ricoh Company, Ltd. Imaging system, imaging apparatus, and system
US10225511B1 (en) 2015-12-30 2019-03-05 Google Llc Low power framework for controlling image sensor mode in a mobile image capture device
US10728489B2 (en) 2015-12-30 2020-07-28 Google Llc Low power framework for controlling image sensor mode in a mobile image capture device
US10732809B2 (en) 2015-12-30 2020-08-04 Google Llc Systems and methods for selective retention and editing of images captured by mobile image capture device
US11159763B2 (en) 2015-12-30 2021-10-26 Google Llc Low power framework for controlling image sensor mode in a mobile image capture device

Also Published As

Publication number Publication date
ATE219875T1 (en) 2002-07-15
JP3051173B2 (en) 2000-06-12
EP0539565A1 (en) 1993-05-05
JPH06501585A (en) 1994-02-17
ES2178328T3 (en) 2002-12-16
US5185667A (en) 1993-02-09
DE69232663T2 (en) 2003-03-27
DK0971540T3 (en) 2002-10-21
EP0971540A1 (en) 2000-01-12
WO1992021208A1 (en) 1992-11-26
EP0971540B1 (en) 2002-06-26
DE69232663D1 (en) 2002-08-01
EP0539565A4 (en) 1993-09-29

Similar Documents

Publication Publication Date Title
USRE36207E (en) Omniview motionless camera orientation system
US6201574B1 (en) Motionless camera orientation system distortion correcting sensing element
US5359363A (en) Omniview motionless camera surveillance system
US7714936B1 (en) Omniview motionless camera orientation system
US5313306A (en) Omniview motionless camera endoscopy system
US7382399B1 (en) Omniview motionless camera orientation system
US6002430A (en) Method and apparatus for simultaneous capture of a spherical image
US7161615B2 (en) System and method for tracking objects and obscuring fields of view under video surveillance
US7274381B2 (en) Panoramic imaging and display system with canonical magnifier
US6603502B2 (en) System for omnidirectional image viewing at a remote location without the transmission of control signals to select viewing parameters
Peri et al. Generation of perspective and panoramic video from omnidirectional video
TWI535285B (en) Conference system, surveillance system, image processing device, image processing method and image processing program, etc.
JP4268206B2 (en) Fisheye lens camera device and image distortion correction method thereof
EP1580986A2 (en) Method and apparatus for creating spherical images
JP2008048443A (en) Fisheye lens camera apparatus and image extraction method thereof
Zimmermann et al. A video pan/tilt/magnify/rotate system with no moving parts
EP0757273A1 (en) Combination of several viewing devices with common point of vision
Martin et al. Omniview motionless camera orientation system
Zimmermann et al. Electronic pan/tilt/zoom camera system
WO1999065245A1 (en) A method and system for providing a seamless omniview image from fisheye images
JP2021131501A (en) Imaging apparatus
JPS60191373A (en) Recognizer for three-dimensional object

Legal Events

Date Code Title Description
AS Assignment

Owner name: IPIX, TENNESSEE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MARTIN, H. LEE;REEL/FRAME:009087/0230

Effective date: 19980403

AS Assignment

Owner name: INTERACTIVE PICTURES CORPORATION, TENNESSEE

Free format text: CHANGE OF NAME;ASSIGNOR:OMNIVIEW, INC.;REEL/FRAME:009401/0428

Effective date: 19971208

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Free format text: PAT HLDR NO LONGER CLAIMS SMALL ENT STAT AS SMALL BUSINESS (ORIGINAL EVENT CODE: LSM2); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FPAY Fee payment

Year of fee payment: 8

SULP Surcharge for late payment
AS Assignment

Owner name: IMAGE INVESTOR PORFOLIO, A SEPARATE SERIES OF MEMP

Free format text: INTELLECTUAL PROPERTY SECURITY AGREEMENT;ASSIGNOR:INTERACTIVE PICTURES CORPORATION;REEL/FRAME:011837/0431

Effective date: 20010514

Owner name: IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEM

Free format text: INTELLECTUAL PROPERTY SECURITY AGREEMENT;ASSIGNOR:PW TECHNOLOGY, INC.;REEL/FRAME:011828/0088

Effective date: 20010514

Owner name: IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEM

Free format text: INTELLECTUAL PROPERTY SECURITY AGREEMENT;ASSIGNOR:INTERNET PICTURES CORPORATION;REEL/FRAME:011828/0054

Effective date: 20010514

AS Assignment

Owner name: PW TECHNOLOGY, INC., CALIFORNIA

Free format text: RELEASE;ASSIGNOR:IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC;REEL/FRAME:012295/0978

Effective date: 20010926

Owner name: INTERACTIVE PICTURES CORPORATION, TENNESSEE

Free format text: RELEASE;ASSIGNOR:IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC;REEL/FRAME:012295/0982

Effective date: 20010926

Owner name: INTERMET PICTURES CORPORATION, TENNESSEE

Free format text: RELEASE;ASSIGNOR:IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC;REEL/FRAME:012295/0986

Effective date: 20010926

Owner name: INTERMET PICTURES CORPORATION,TENNESSEE

Free format text: RELEASE;ASSIGNOR:IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC;REEL/FRAME:012295/0986

Effective date: 20010926

Owner name: INTERACTIVE PICTURES CORPORATION,TENNESSEE

Free format text: RELEASE;ASSIGNOR:IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC;REEL/FRAME:012295/0982

Effective date: 20010926

Owner name: PW TECHNOLOGY, INC.,CALIFORNIA

Free format text: RELEASE;ASSIGNOR:IMAGE INVESTOR PORTFOLIO, A SEPARATE SERIES OF MEMPHIS ANGELS, LLC;REEL/FRAME:012295/0978

Effective date: 20010926

FPAY Fee payment

Year of fee payment: 12

AS Assignment

Owner name: SONY CORPORATION,JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:IPIX CORPORATION;REEL/FRAME:019084/0034

Effective date: 20070222

Owner name: SONY CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:IPIX CORPORATION;REEL/FRAME:019084/0034

Effective date: 20070222