WO2009089369A1 - Point and go navigation system and method - Google Patents

Point and go navigation system and method Download PDF

Info

Publication number
WO2009089369A1
WO2009089369A1 PCT/US2009/030464 US2009030464W WO2009089369A1 WO 2009089369 A1 WO2009089369 A1 WO 2009089369A1 US 2009030464 W US2009030464 W US 2009030464W WO 2009089369 A1 WO2009089369 A1 WO 2009089369A1
Authority
WO
WIPO (PCT)
Prior art keywords
robotic vehicle
environment
sensor
target point
visual representation
Prior art date
Application number
PCT/US2009/030464
Other languages
French (fr)
Inventor
Stephen C. Jacobsen
Marc Olivier
Original Assignee
Raytheon Sarcos, Llc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Raytheon Sarcos, Llc filed Critical Raytheon Sarcos, Llc
Publication of WO2009089369A1 publication Critical patent/WO2009089369A1/en

Links

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/0011Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot associated with a remote control arrangement
    • G05D1/0044Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot associated with a remote control arrangement by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0227Control of position or course in two dimensions specially adapted to land vehicles using mechanical sensing means, e.g. for sensing treated area
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0238Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors
    • G05D1/024Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors in combination with a laser
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0251Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0255Control of position or course in two dimensions specially adapted to land vehicles using acoustic signals, e.g. ultra-sonic singals
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0257Control of position or course in two dimensions specially adapted to land vehicles using a radar
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0259Control of position or course in two dimensions specially adapted to land vehicles using magnetic or electromagnetic means
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0268Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means
    • G05D1/027Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means comprising intertial navigation means, e.g. azimuth detector
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
    • G05D1/0278Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using satellite positioning signals, e.g. GPS

Definitions

  • the present invention relates to navigation control for robotic vehicles.
  • BACKGROUND OF THE INVENTION AND RELATED ART Navigating a robotic vehicle remotely through an environment can be quite challenging due to a number of factors.
  • a complex robotic vehicle typically has a large number of operator controls available.
  • One control approach is the so-called replica master mode of operation.
  • the robot is slaved to a replica master, which provides the same joints as the robot.
  • the master communicates positions of joints to the robot, which attempts to move to the same position.
  • replica master mode is handy for some types of robot manipulation (e.g., telemanipulation), this type of control can be a tedious way to remotely navigate a semi- autonomous robot through a varied terrain.
  • An alternate approach to robotic navigation is to preprogram the robot movements beforehand. While preprogramming is efficient for some types of robot manipulation (e.g., manufacturing), this type of control is not particularly suitable when preplanning is not possible, such as in exploratory operations.
  • the present invention includes a method and system for providing point and go navigation for remote control of a robotic vehicle.
  • a method of providing point and go navigation of a robotic vehicle within an environment is provided.
  • the method can include obtaining sensor information from the robotic vehicle and displaying a visual representation of the environment on an operator console.
  • the method can also include designating a target point within the visual representation of the environment based on operator input, and advancing the robotic vehicle toward a target location in the environment that corresponds to the target point. Movement of the robotic vehicle can be based on an automatic navigation response to the designated target point.
  • FIG. 1 illustrates a schematic illustration of a system for providing point and go navigation of a robotic vehicle according to an exemplary embodiment of the present invention
  • FIG. 2 illustrates a block diagram of a remote operator console for point and go navigation in accordance with an embodiment of the present invention
  • FIG. 3 illustrates a flow chart of a method of providing point and go navigation for remote control of a robotic vehicle in accordance with another embodiment of the present invention.
  • FIG. 1 illustrates the system 10 as including a robotic vehicle 12 and a remote operator console 14.
  • the remote operator console is in bidirectional communication with the robotic vehicle via a communications link 16.
  • the communication link can be, for example, a wireless radio frequency link, a free space optical link, a free space ultrasonic link, a wired link, a fiber optic link, or a combination thereof.
  • the communication link may also include one or more relay nodes (not shown), for example, to extend the range for communication between the remote operator console and the robotic vehicle.
  • the relay node may be mounted on a second robotic vehicle.
  • the robotic vehicle 12 includes at least one environmental sensor 18, such as a camera, a stereo camera, a sound sensor, an electromagnetic sensor, a chemical sensor, a radar, a lidar, a range finder, a scanning range finder, a sonar, a contact sensor, a sniff sensor, a Global Positioning System (GPS) receiver, an inertial measurement unit, an orientation sensor, or a combination thereof.
  • the sensor allows the robotic vehicle to obtain information regarding the environment in which it is operating.
  • the robotic vehicle transmits sensor information to the remote operator console 14 via the communication link 16.
  • the remote operator console 14 includes a display 20 to display a visual representation of the environment based on sensor information received from the robotic vehicle 12.
  • the remote operator console includes an operator input function for defining a target point on the visual representation.
  • the operator input function can be, for example, a touch screen integrated with the display.
  • the operator input function can be buttons 23 or similar controls on the remote operator console. Navigational commands are transmitted from the remote operator console to the robotic vehicle via the communication link 16 to control movement of the robotic vehicle.
  • An exemplary embodiment of the operator console 14 is shown in further detail in
  • the operator console includes a receiver 22 for receiving sensor data from the robotic vehicle 12 (FIG. 1).
  • the sensor data is provided to the display 20.
  • the operator console also includes an operator input function 21 to allow designation of a target point.
  • the display can include a touch sensitive screen that allows the operator to designate a target point by touching the display.
  • the remote operator console can include a keyboard, cursor controls, or other operator input device.
  • an electronic cursor may be moved on the display to designate a target point.
  • an operator may enter coordinates using a keyboard.
  • the designated target point on the display 20 corresponds to a target location within the environment in which the robotic vehicle 12 is operating.
  • the environment includes a yellow fire hydrant.
  • the yellow fire hydrant may be designated as the target location by touching the portion of the image corresponding to the yellow fire hydrant on the display.
  • the remote operator console 14 may transmit navigation commands to the robotic vehicle 12 using the transmitter 24 to cause the robotic vehicle to advance toward the designated target location.
  • Navigation of the robotic vehicle 12 toward the designated target location can be performed in various ways.
  • coordinates corresponding to the designated target location can be transmitted to the robotic vehicle.
  • the coordinates may be defined relative to the position on the display, and translated into an actual location within the environment by the robotic vehicle.
  • the coordinates may be defined relative to references points within the environment or other reference datum and coordinate translation provided by the remote operator console.
  • Absolute coordinates may be defined, for example, by combining the displayed sensor data with external information such as maps, GPS position fixes, or the like.
  • the display may also be augmented with additional environmental information obtained independently of the robotic vehicle, such as, for example, an electronic map, environmental sensors, or sensor information from a second robotic vehicle operating within the environment.
  • a second environmental sensor can be mounted on a second robotic vehicle.
  • the second environmental sensor may include, for example, a camera, a stereo camera, a sound sensor, an electromagnetic sensor, a chemical sensor, a radar, a lidar, a range finder, a scanning range finder, a sonar, a contact sensor, a sniff sensor, a Global Positioning System (GPS) receiver, or a combination thereof.
  • the target location may be designated using an operator-defined displacement from the target point.
  • the fire hydrant may be designated using the touch screen and additional operator input provided to specify that the target location is 10 feet to the left of the fire hydrant.
  • Navigation commands to the robotic vehicle may alternately include a series of movement commands.
  • the remote operator console may include processing to determine a route toward the target point, and then transmit incremental movement commands to the robotic vehicle to navigate toward the target point.
  • the automatic navigation For example, one relatively straightforward automatic navigation approach is to turn the vehicle in a direction corresponding to the designated target and move forward in that direction.
  • the display may include processing to automatically determine a planned route from a current position of the robotic vehicle toward the target location. Processing may include object identification using image-processing techniques to allow obstacles within the environment to be identified and avoided.
  • the environmental sensor may include range finding, allowing for improved obstacle identification and navigation.
  • operator input can identify a target path, for example, by tracing a curve on the touch sensitive screen to define the target path.
  • the robotic vehicle can then be automatically navigated to follow a travel path in the environment correlated with the target path.
  • the display can be updated as new sensor data is received.
  • the target point may be re-designated by the operator. For example, navigation may be controlled incrementally, designating a target point, navigating the robotic vehicle toward the target point, and re-designating the target point as the robotic vehicle moves.
  • robotic vehicles are capable of multiple movement modes.
  • commonly owned co-pending U.S. Provisional Patent Application Serial No. 60/858,917, filed November 13, 2006, and entitled "Serpentine Robotic Crawler” describes one exemplary unmanned ground robotic vehicle having multiple movement modes that may be used in embodiments of the present invention and is incorporated herein by reference.
  • the operator may also identify transition points on the target path where a movement move or other behavior of the robotic vehicle is to change. When the robotic vehicle reaches the point in the environment corresponding to the transition point, robotic vehicle may change from a first mode of operation to a second mode of operation.
  • the robotic vehicle describe above includes a stair climbing mode, a tank configuration for movement on a relatively flat surface, and an inside surface climbing mode.
  • a desired path may include movement across a floor area, up a set of stairs, and into a vent duct. Transition points may be designated at the beginning of the stairs, the top of the stairs, and the entry to vent duct.
  • the robotic vehicle may then switch between appropriate movement modes as each different area is entered.
  • the robotic vehicle mode of operation may be adjusted automatically based on characteristics of the environment obtained from the sensor information. For example, image recognition software may recognize certain types of obstacles and movement mode changed automatically.
  • FIG. 3 illustrates a method of providing point and go navigation.
  • the method 30 includes obtaining 32-sensor information describing an environment in which the robotic vehicle is operating. For example, sensor information may be obtained using sensors mounted on the robotic vehicle as described above.
  • the method also includes communicating 34 the sensor information to a remotely located operator console. For example, a communication link as described above can be used.
  • the method also includes displaying 36 a visual representation of the environment on the operator console, the visual representation being based on the sensor information.
  • the method also includes 38 designating a target point within the visual representation based on operator input. For example, operator input may be via a touch screen or other input device as described above.
  • the method also includes advancing 40 the robotic vehicle toward a target location in the environment corresponding to the target point. Advancing the robotic vehicle can be based on an automatic navigation response to the designated target point, for example as described above.
  • point and go navigation in accordance with embodiments of the present invention can ease the difficulty of controlling a robotic vehicle.
  • An operator can point to a desired target point on a display screen and the robotic vehicle will automatically go to the designated location.
  • Processing within either the remote operator console or the robotic vehicle can translate a designated target point or path into navigational commands for the robotic vehicle.
  • transition points can be designated, and robotic vehicle operation mode changed at the transition points.
  • Applications of such a navigation technique may include search and rescue, military operations, and industrial operations.
  • the term "preferably” is non-exclusive where it is intended to mean “preferably, but not limited to.” Any steps recited in any method or process claims may be executed in any order and are not limited to the order presented in the claims. Means-plus-function or step-plus- function limitations will only be employed where for a specific claim limitation all of the following conditions are present: a) "means for” or “step for” is expressly recited in that limitation; b) a corresponding function is expressly recited in that limitation; and c) structure, material or acts that support that function are described within the specification. Accordingly, the scope of the invention should be determined solely by the appended claims and their legal equivalents, rather than by the descriptions and examples given above.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • General Engineering & Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Traffic Control Systems (AREA)

Abstract

A remote operator console provides point and go navigation of a robotic vehicle. The remote operator console provides a display for visual representation of the environment in which the robotic vehicle is operating based on sensor information received from the robotic vehicle. An operator may designate a target point on the display. The robotic vehicle is automatically navigated toward a location in the environment corresponding to the designated target point.

Description

POINT AND GO NAVIGATION SYSTEM AND METHOD
FIELD OF THE INVENTION
The present invention relates to navigation control for robotic vehicles.
BACKGROUND OF THE INVENTION AND RELATED ART Navigating a robotic vehicle remotely through an environment can be quite challenging due to a number of factors. For example, a complex robotic vehicle typically has a large number of operator controls available. One control approach is the so-called replica master mode of operation. The robot is slaved to a replica master, which provides the same joints as the robot. As the operator manipulates the master, the master communicates positions of joints to the robot, which attempts to move to the same position. While replica master mode is handy for some types of robot manipulation (e.g., telemanipulation), this type of control can be a tedious way to remotely navigate a semi- autonomous robot through a varied terrain.
An alternate approach to robotic navigation is to preprogram the robot movements beforehand. While preprogramming is efficient for some types of robot manipulation (e.g., manufacturing), this type of control is not particularly suitable when preplanning is not possible, such as in exploratory operations.
SUMMARY OF THE INVENTION The present invention includes a method and system for providing point and go navigation for remote control of a robotic vehicle. In one embodiment, a method of providing point and go navigation of a robotic vehicle within an environment is provided. The method can include obtaining sensor information from the robotic vehicle and displaying a visual representation of the environment on an operator console. The method can also include designating a target point within the visual representation of the environment based on operator input, and advancing the robotic vehicle toward a target location in the environment that corresponds to the target point. Movement of the robotic vehicle can be based on an automatic navigation response to the designated target point. BRIEF DESCRIPTION OF THE DRAWINGS
The present invention will become more fully apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. Understanding that these drawings merely depict exemplary embodiments of the present invention, they are, therefore, not to be considered limiting of its scope. It will be readily appreciated that the components of the present invention, as generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Nonetheless, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:
FIG. 1 illustrates a schematic illustration of a system for providing point and go navigation of a robotic vehicle according to an exemplary embodiment of the present invention;
FIG. 2 illustrates a block diagram of a remote operator console for point and go navigation in accordance with an embodiment of the present invention; and
FIG. 3 illustrates a flow chart of a method of providing point and go navigation for remote control of a robotic vehicle in accordance with another embodiment of the present invention.
DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS
The following detailed description of exemplary embodiments of the invention makes reference to the accompanying drawings, which form a part hereof and in which are shown, by way of illustration, exemplary embodiments in which the invention may be practiced. While these exemplary embodiments are described in sufficient detail to enable those skilled in the art practice the invention, it should be understood that other embodiments may be realized and that various changes to the invention may be made without departing from the spirit and scope of the present invention. Thus, the following more detailed description of the embodiments of the present invention is not intended to limit the scope of the invention, as claimed, but is presented for purposes of illustration only and not limitation to describe the features and characteristics of the present invention, and to sufficiently enable one skilled in the art to practice the invention. Accordingly, the scope of the present invention is to be defined solely by the appended claims. The following detailed description and exemplary embodiments of the invention will be best understood by reference to the accompanying drawings, wherein the elements and features of the invention are designated by numerals throughout.
With reference to FIG. 1, shown is an illustration of a system for providing point and go navigation of a robotic vehicle according to a first exemplary embodiment of the present invention. Specifically, FIG. 1 illustrates the system 10 as including a robotic vehicle 12 and a remote operator console 14. The remote operator console is in bidirectional communication with the robotic vehicle via a communications link 16. The communication link can be, for example, a wireless radio frequency link, a free space optical link, a free space ultrasonic link, a wired link, a fiber optic link, or a combination thereof. The communication link may also include one or more relay nodes (not shown), for example, to extend the range for communication between the remote operator console and the robotic vehicle. The relay node may be mounted on a second robotic vehicle. The robotic vehicle 12 includes at least one environmental sensor 18, such as a camera, a stereo camera, a sound sensor, an electromagnetic sensor, a chemical sensor, a radar, a lidar, a range finder, a scanning range finder, a sonar, a contact sensor, a sniff sensor, a Global Positioning System (GPS) receiver, an inertial measurement unit, an orientation sensor, or a combination thereof. The sensor allows the robotic vehicle to obtain information regarding the environment in which it is operating. The robotic vehicle transmits sensor information to the remote operator console 14 via the communication link 16.
The remote operator console 14 includes a display 20 to display a visual representation of the environment based on sensor information received from the robotic vehicle 12. The remote operator console includes an operator input function for defining a target point on the visual representation. The operator input function can be, for example, a touch screen integrated with the display. As another example, the operator input function can be buttons 23 or similar controls on the remote operator console. Navigational commands are transmitted from the remote operator console to the robotic vehicle via the communication link 16 to control movement of the robotic vehicle. An exemplary embodiment of the operator console 14 is shown in further detail in
FIG. 2. The operator console includes a receiver 22 for receiving sensor data from the robotic vehicle 12 (FIG. 1). The sensor data is provided to the display 20. The operator console also includes an operator input function 21 to allow designation of a target point. For example, the display can include a touch sensitive screen that allows the operator to designate a target point by touching the display. As another example, the remote operator console can include a keyboard, cursor controls, or other operator input device. For example, an electronic cursor may be moved on the display to designate a target point. As another example, an operator may enter coordinates using a keyboard.
The designated target point on the display 20 corresponds to a target location within the environment in which the robotic vehicle 12 is operating. For example, consider an example where the environment includes a yellow fire hydrant. The yellow fire hydrant may be designated as the target location by touching the portion of the image corresponding to the yellow fire hydrant on the display.
The remote operator console 14 may transmit navigation commands to the robotic vehicle 12 using the transmitter 24 to cause the robotic vehicle to advance toward the designated target location. Navigation of the robotic vehicle 12 toward the designated target location can be performed in various ways. In one example, coordinates corresponding to the designated target location can be transmitted to the robotic vehicle. The coordinates may be defined relative to the position on the display, and translated into an actual location within the environment by the robotic vehicle. Alternately, the coordinates may be defined relative to references points within the environment or other reference datum and coordinate translation provided by the remote operator console. Absolute coordinates may be defined, for example, by combining the displayed sensor data with external information such as maps, GPS position fixes, or the like. The display may also be augmented with additional environmental information obtained independently of the robotic vehicle, such as, for example, an electronic map, environmental sensors, or sensor information from a second robotic vehicle operating within the environment. For example, a second environmental sensor can be mounted on a second robotic vehicle. The second environmental sensor may include, for example, a camera, a stereo camera, a sound sensor, an electromagnetic sensor, a chemical sensor, a radar, a lidar, a range finder, a scanning range finder, a sonar, a contact sensor, a sniff sensor, a Global Positioning System (GPS) receiver, or a combination thereof. The target location may be designated using an operator-defined displacement from the target point. For example, the fire hydrant may be designated using the touch screen and additional operator input provided to specify that the target location is 10 feet to the left of the fire hydrant. Navigation commands to the robotic vehicle may alternately include a series of movement commands. For example, the remote operator console may include processing to determine a route toward the target point, and then transmit incremental movement commands to the robotic vehicle to navigate toward the target point. Various degrees of sophistication are possible for the automatic navigation. For example, one relatively straightforward automatic navigation approach is to turn the vehicle in a direction corresponding to the designated target and move forward in that direction. As another example, the display may include processing to automatically determine a planned route from a current position of the robotic vehicle toward the target location. Processing may include object identification using image-processing techniques to allow obstacles within the environment to be identified and avoided. The environmental sensor may include range finding, allowing for improved obstacle identification and navigation.
Alternately, operator input can identify a target path, for example, by tracing a curve on the touch sensitive screen to define the target path. The robotic vehicle can then be automatically navigated to follow a travel path in the environment correlated with the target path.
As the robotic vehicle moves through the environment, the display can be updated as new sensor data is received. If desired, the target point may be re-designated by the operator. For example, navigation may be controlled incrementally, designating a target point, navigating the robotic vehicle toward the target point, and re-designating the target point as the robotic vehicle moves.
As mentioned above, some robotic vehicles are capable of multiple movement modes. For example, commonly owned co-pending U.S. Provisional Patent Application Serial No. 60/858,917, filed November 13, 2006, and entitled "Serpentine Robotic Crawler," describes one exemplary unmanned ground robotic vehicle having multiple movement modes that may be used in embodiments of the present invention and is incorporated herein by reference. In addition to identifying a target path on the operator console, the operator may also identify transition points on the target path where a movement move or other behavior of the robotic vehicle is to change. When the robotic vehicle reaches the point in the environment corresponding to the transition point, robotic vehicle may change from a first mode of operation to a second mode of operation.
For example, the robotic vehicle describe above includes a stair climbing mode, a tank configuration for movement on a relatively flat surface, and an inside surface climbing mode. A desired path may include movement across a floor area, up a set of stairs, and into a vent duct. Transition points may be designated at the beginning of the stairs, the top of the stairs, and the entry to vent duct. The robotic vehicle may then switch between appropriate movement modes as each different area is entered. As another example, the robotic vehicle mode of operation may be adjusted automatically based on characteristics of the environment obtained from the sensor information. For example, image recognition software may recognize certain types of obstacles and movement mode changed automatically.
FIG. 3 illustrates a method of providing point and go navigation. The method 30 includes obtaining 32-sensor information describing an environment in which the robotic vehicle is operating. For example, sensor information may be obtained using sensors mounted on the robotic vehicle as described above. The method also includes communicating 34 the sensor information to a remotely located operator console. For example, a communication link as described above can be used. The method also includes displaying 36 a visual representation of the environment on the operator console, the visual representation being based on the sensor information. The method also includes 38 designating a target point within the visual representation based on operator input. For example, operator input may be via a touch screen or other input device as described above. The method also includes advancing 40 the robotic vehicle toward a target location in the environment corresponding to the target point. Advancing the robotic vehicle can be based on an automatic navigation response to the designated target point, for example as described above.
Summarizing and reiterating to some extent, point and go navigation in accordance with embodiments of the present invention can ease the difficulty of controlling a robotic vehicle. An operator can point to a desired target point on a display screen and the robotic vehicle will automatically go to the designated location. Processing within either the remote operator console or the robotic vehicle can translate a designated target point or path into navigational commands for the robotic vehicle. Thus, the operator is freed from having to specify every detailed movement of the robotic vehicle. If desired, transition points can be designated, and robotic vehicle operation mode changed at the transition points. Applications of such a navigation technique may include search and rescue, military operations, and industrial operations. The foregoing detailed description describes the invention with reference to specific exemplary embodiments. However, it will be appreciated that various modifications and changes can be made without departing from the scope of the present invention as set forth in the appended claims. The detailed description and accompanying drawings are to be regarded as merely illustrative, rather than as restrictive, and all such modifications or changes, if any, are intended to fall within the scope of the present invention as described and set forth herein.
More specifically, while illustrative exemplary embodiments of the invention have been described herein, the present invention is not limited to these embodiments, but includes any and all embodiments having modifications, omissions, combinations (e.g., of aspects across various embodiments), adaptations and/or alterations as would be appreciated by those in the art based on the foregoing detailed description. The limitations in the claims are to be interpreted broadly based the language employed in the claims and not limited to examples described in the foregoing detailed description or during the prosecution of the application, which examples are to be construed as nonexclusive. For example, in the present disclosure, the term "preferably" is non-exclusive where it is intended to mean "preferably, but not limited to." Any steps recited in any method or process claims may be executed in any order and are not limited to the order presented in the claims. Means-plus-function or step-plus- function limitations will only be employed where for a specific claim limitation all of the following conditions are present: a) "means for" or "step for" is expressly recited in that limitation; b) a corresponding function is expressly recited in that limitation; and c) structure, material or acts that support that function are described within the specification. Accordingly, the scope of the invention should be determined solely by the appended claims and their legal equivalents, rather than by the descriptions and examples given above.

Claims

CLAIMSWhat is claimed is:
1. A method of providing point and go navigation for remote control of an unmanned robotic vehicle, comprising:
Obtaining sensor information describing an environment in which the unmanned robotic vehicle is operating;
Communicating the sensor information to a remotely located operator console; Displaying a visual representation of the environment on the operator console based on the sensor information;
Designating a target point within the visual representation based on operator input; and Advancing the unmanned robotic vehicle toward a target location in the environment corresponding to the target point based on an automatic navigation response to the designated target point.
2. The method of claim 1, wherein the target location is defined as an operator-defined displacement from a point in the environment corresponding to the target point.
3. The method of claim 1 , further comprising:
Updating the visual representation of the environment on the operator console as the unmanned robotic vehicle moves through the environment; and Re-designating the target point within the updated visual representation based on additional operator input.
4. The method of claim 1 , wherein advancing the unmanned robotic vehicle comprises automatically determining a planned route from a current unmanned robotic vehicle position to the target location.
5. The method of claim 4, further comprising displaying a visual representation of the planned route on the operator console.
6. The method of claim 5, further comprising modifying the planned route based on operator input.
7. The method of claim 1, further comprising: Identifying a target path within the visual representation based on the operator input; and
Automatically navigating the unmanned robotic vehicle to follow a travel path in the environment correlated with the target path.
8. The method of claim 7, further comprising:
Designating a transition point on the target path based on the operator input; and Adjusting unmanned robotic vehicle behavior from a first mode of operation to a second mode of operation at a point in the environment corresponding to the transition point.
9. The method of claim 1 , wherein designating the target point comprises touching the operator console at the target point on the visual representation of the environment.
10. The method of claim 1 , wherein designating the target point comprises placing an electronic cursor at the target point on the visual representation of the environment.
11. The method of claim 1 , wherein the target point is defined by entering coordinates into the operator console.
12. The method of claim 1 , further comprising augmenting the visual representation of the environment using additional environmental information obtained independently of the unmanned robotic vehicle.
13. The method claim 12, wherein the additional environmental information is obtained from an electronic map.
14. The method claim 12, wherein the additional environmental information is obtained from a second environmental sensor.
15. The method claim 12, wherein the additional environmental information is obtained from a GPS receiver.
16. The method of claim 14, wherein the second environmental sensor is mounted on a second unmanned robotic vehicle.
17. The method of claim 1 , further comprising:
Identifying a characteristic of the environment from the sensor information; and Modifying unmanned robotic vehicle behavior based on the characteristic.
18. A system for providing point and go navigation of an unmanned robotic vehicle, the system comprising: a) An unmanned robotic vehicle having an environmental sensor and a transmitter and receiver unit, and being capable of controlled movement within an environment; and b) A remote operator console in bi-directional communication with the unmanned robotic vehicle, the remote operator console comprising: i) A receiver for receiving sensor information from the unmanned robotic vehicle; ii) A display for displaying a visual representation of the environment based on the sensor information received from the environmental sensor; iii) An operator input function for defining a target point on the visual representation, and iv) A transmitter for transmitting navigational commands to the unmanned robotic vehicle based on the target point.
19. The system of claim 18, wherein the navigational commands comprise target point coordinates.
20. The system of claim 19, wherein the target point coordinates are defined relative to the visual representation.
21. The system of claim 18, wherein the navigational commands comprise a series of movement commands.
22. The system of claim 18, wherein the environmental sensor is chosen from the group consisting of a camera, a stereo camera, a sound sensor, an electromagnetic sensor, a chemical sensor, a radar, a lidar, a range finder, a scanning range finder, a sonar, a contact sensor, a sniff sensor, a GPS receiver, an inertial measurement unit, an orientation sensor, and combinations thereof.
23. The system of claim 18, wherein the display and operator input are provided by a touch screen.
24. The system of claim 18, further comprising a communication link between the unmanned robotic vehicle and the remote operator console.
25. The system of claim 24, wherein the communication link is selected from the group consisting of a wireless radio frequency link, a free space optical link, a free space ultrasonic link, a wired link, a fiber optic link, and combinations thereof.
26. The system of claim 25, wherein the communication link further comprises at least one relay node.
PCT/US2009/030464 2008-01-08 2009-01-08 Point and go navigation system and method WO2009089369A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US1050108P 2008-01-08 2008-01-08
US61/010,501 2008-01-08

Publications (1)

Publication Number Publication Date
WO2009089369A1 true WO2009089369A1 (en) 2009-07-16

Family

ID=40548976

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2009/030464 WO2009089369A1 (en) 2008-01-08 2009-01-08 Point and go navigation system and method

Country Status (1)

Country Link
WO (1) WO2009089369A1 (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012021192A3 (en) * 2010-05-12 2012-08-23 Irobot Corporation System and method to remotely control a vehicle
US8774981B2 (en) 2009-09-14 2014-07-08 Israel Aerospace Industries Ltd. Infantry robotic porter system and methods useful in conjunction therewith
CN104049634A (en) * 2014-07-02 2014-09-17 燕山大学 Intelligent body fuzzy dynamic obstacle avoidance method based on Camshift algorithm
CN105116785A (en) * 2015-06-26 2015-12-02 北京航空航天大学 Multi-platform remote robot general control system
CN106444756A (en) * 2016-09-22 2017-02-22 纳恩博(北京)科技有限公司 Information processing method and electronic equipment
JP2017054475A (en) * 2015-09-11 2017-03-16 富士ゼロックス株式会社 Teleoperation device, method and program
WO2017063652A1 (en) * 2015-10-12 2017-04-20 Intelligent Marking Aps Self-propelled robot unit including a navigation system and movement items for positioning of the robot unit on an even base
CN109263641A (en) * 2017-07-12 2019-01-25 罗伯特·博世有限公司 For positioning and the method and apparatus of automatic operating vehicle
WO2019047643A1 (en) * 2017-09-05 2019-03-14 百度在线网络技术(北京)有限公司 Control method and device for unmanned vehicle
CN110570687A (en) * 2018-06-06 2019-12-13 杭州海康机器人技术有限公司 AGV control method and device and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010037163A1 (en) * 2000-05-01 2001-11-01 Irobot Corporation Method and system for remote control of mobile robot
EP1510896A1 (en) * 2002-05-31 2005-03-02 Fujitsu Limited Remotely-operated robot, and robot self position identifying method
US20070156286A1 (en) * 2005-12-30 2007-07-05 Irobot Corporation Autonomous Mobile Robot

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010037163A1 (en) * 2000-05-01 2001-11-01 Irobot Corporation Method and system for remote control of mobile robot
EP1510896A1 (en) * 2002-05-31 2005-03-02 Fujitsu Limited Remotely-operated robot, and robot self position identifying method
US20070156286A1 (en) * 2005-12-30 2007-07-05 Irobot Corporation Autonomous Mobile Robot

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8774981B2 (en) 2009-09-14 2014-07-08 Israel Aerospace Industries Ltd. Infantry robotic porter system and methods useful in conjunction therewith
US9658615B2 (en) 2010-05-12 2017-05-23 Irobot Defense Holdings, Inc. Remote vehicle control system and method
GB2494807A (en) * 2010-05-12 2013-03-20 Irobot Corp System and method to remotely control a vehicle
US8954194B2 (en) 2010-05-12 2015-02-10 Irobot Corporation Remote vehicle control system and method
GB2523484A (en) * 2010-05-12 2015-08-26 Irobot Corp Remote vehicle control system and method
GB2494807B (en) * 2010-05-12 2015-11-18 Irobot Corp System and method to remotely control a vehicle
WO2012021192A3 (en) * 2010-05-12 2012-08-23 Irobot Corporation System and method to remotely control a vehicle
GB2523484B (en) * 2010-05-12 2016-04-13 Irobot Corp Remote vehicle control system and method
CN104049634A (en) * 2014-07-02 2014-09-17 燕山大学 Intelligent body fuzzy dynamic obstacle avoidance method based on Camshift algorithm
CN104049634B (en) * 2014-07-02 2017-02-01 燕山大学 Intelligent body fuzzy dynamic obstacle avoidance method based on Camshift algorithm
CN105116785B (en) * 2015-06-26 2018-08-24 北京航空航天大学 A kind of multi-platform tele-robotic general-purpose control system
CN105116785A (en) * 2015-06-26 2015-12-02 北京航空航天大学 Multi-platform remote robot general control system
JP2017054475A (en) * 2015-09-11 2017-03-16 富士ゼロックス株式会社 Teleoperation device, method and program
WO2017063652A1 (en) * 2015-10-12 2017-04-20 Intelligent Marking Aps Self-propelled robot unit including a navigation system and movement items for positioning of the robot unit on an even base
DK178897B1 (en) * 2015-10-12 2017-05-08 Intelligent Marking Aps Self-propelled robot unit comprising a navigation system and moving means for positioning the robot unit on a level surface.
US11467601B2 (en) 2015-10-12 2022-10-11 Intelligent Marking Aps Self-propelled robot unit including a navigation system and movement items for positioning of the robot unit on an even base
CN106444756A (en) * 2016-09-22 2017-02-22 纳恩博(北京)科技有限公司 Information processing method and electronic equipment
CN109263641A (en) * 2017-07-12 2019-01-25 罗伯特·博世有限公司 For positioning and the method and apparatus of automatic operating vehicle
CN109263641B (en) * 2017-07-12 2023-04-07 罗伯特·博世有限公司 Method and device for locating and automatically operating a vehicle
WO2019047643A1 (en) * 2017-09-05 2019-03-14 百度在线网络技术(北京)有限公司 Control method and device for unmanned vehicle
CN110570687A (en) * 2018-06-06 2019-12-13 杭州海康机器人技术有限公司 AGV control method and device and storage medium

Similar Documents

Publication Publication Date Title
US8392036B2 (en) Point and go navigation system and method
WO2009089369A1 (en) Point and go navigation system and method
US11294060B2 (en) System and method for lidar-based vehicular localization relating to autonomous navigation
AU2019210565B2 (en) Moving robot, method for controlling moving robot, and moving robot system
US10598505B2 (en) Travel route generation apparatus and method for generating travel route
US7539557B2 (en) Autonomous mobile robot
KR101202695B1 (en) Autonomous movement device
KR101703177B1 (en) Apparatus and method for recognizing position of vehicle
US20120072052A1 (en) Navigation Portals for a Remote Vehicle Control User Interface
US11351986B2 (en) In-vehicle processing apparatus
JP6982999B2 (en) Route determination device and route determination method
JP2011170844A (en) Control method for unmanned vehicle
CN103294054A (en) Robot navigation method and system
US20190258265A1 (en) Method for Operating A Partially Autonomous or Autonomous Motor Vehicle, and Motor Vehicle
JP6910023B2 (en) How to control unmanned moving objects
KR101146942B1 (en) Device for generating path of moving robot, moving robot having the same and method for generating path of moving robot
WO2015167411A1 (en) Smart navigation system for brainwave controlled wheelchairs
JP5187757B2 (en) Unmanned mobile system
WO2020121900A1 (en) Mobile body controller, mobile body and method of controlling mobile body
JP5969903B2 (en) Control method of unmanned moving object
JP6947563B2 (en) Mobile robot control device and control method
JP6934760B2 (en) Travel device, travel control system, travel control method and program
JP6994427B2 (en) Posture recognition system for construction machinery
JP6924160B2 (en) Remote control system
AU2011293447B2 (en) Remote vehicle missions and systems for supporting remote vehicle missions

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09700561

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09700561

Country of ref document: EP

Kind code of ref document: A1