WO2008120971A1

WO2008120971A1 - Method of and apparatus for providing tracking information together with environmental information using a personal mobile device

Info

Publication number: WO2008120971A1
Application number: PCT/NL2007/050134
Authority: WO
Inventors: Linde Van De Velde
Original assignee: Tele Atlas B.V.
Priority date: 2007-04-02
Filing date: 2007-04-02
Publication date: 2008-10-09

Abstract

The invention relates to a method of and a personal mobile device for collecting tracking data and environmental data associated with a person using the personal mobile device. In the method the following actions are performed: upon sensing a first event, collecting with the personal mobile device the track data representative of the track of the person including at least position and time information of the personal mobile device along the track, as well as the environmental data of an environment along the track; storing the track data and environmental data in a memory, the memory being either an internal memory of the personal mobile device or a memory external to the personal mobile device.

Description

Method of and apparatus for providing tracking information together with environmental information using a personal mobile device.

Field of the invention The present invention relates to a method of providing tracking information together with environmental information associated with a moving object to an observation station. The invention further relates to an apparatus for providing the tracking information together with environmental information, a computer program product and a processor readable medium carrying the computer program product.

Prior art

Currently, there is a significant amount of crime occurring. For instance, pedestrians are attacked or robbed in the middle of a station or people are killed on a bus. In such events there are often a number of people who are witnesses, but few dare to act as they fear for their own safety Additionally, accidents frequently happen and no information is available regarding the pre-accident conditions.

Nowadays, millions of people have a mobile device equipped with a mobile phone, camera and voice recorder and/or transmitter. It is expected that in the near future, such devices will be equipped with an enabled positioning device, e.g. a GPS or the like, or are capable of communicating wirelessly with a positioning device. An example of such a navigation support software for mobile devices is PathAway.

With current mobile devices, if present, the camera and the voice recorder/transmitter have to be activated individually by a user to generate audiovisual data. A police centre which should investigate a crime, would be greatly aided if users of such mobile devices have registered information with them and have provided this to the police centre. Additionally, for determining the cause of an accident it is important to know the circumstances before the accident happened. As humans can only react to an accident as or after it has happened, little or no information is generally captured by a mobile device of current design before an accident occurs. If the police had more information about a crime provided by the victim himself or a witness, more crimes could be solved. The chance that a criminal will be punished would increase. This may discourage criminals from committing crimes thereby decreasing the number of crimes, which will make people safer. Moreover, after an accident has happened both police investigators and insurance company investigators wish to know what or whom caused the accident. Therefore, there is a need for mobile equipment which can easily be used by a user to provide useful information about a crime or accident or any other critical situation that afterwards needs to be investigated.

Summary of the invention

The present invention seeks to provide a method of providing information about the circumstances of an event to an appropriate observation/investigation station. More specifically the invention seeks to provide a method of collecting tracking information associated with a moving person that can be sent to an observation station, which uses the functionality of mobile devices to easily capture evidence about the circumstances of an accident, a crime or a special moment in time. Tracking information is important for crime and accident investigators since they may want to know locations and location history of certain events. The observation station might be any public or private individual or organization that can perform one of the following functions: store the mobile data for future reference; respond to the data in real time or near real time; use the data in analysis, investigations, legal or business proceedings or for private purposes.

According to the present invention, the method comprises the features of claim 1.

The invention also relates to a personal mobile device as claimed in claim 15. Moreover, the invention relates to a telecommunication system comprising such a personal mobile device, to a computer program product comprising data and instructions allowing a processor to perform the method of the invention and a data carrier comprising such a computer program.

The invention is based on the recognition that a personal mobile device with a GPS, a camera and/or voice recorder or other voice sensor as carried by a person, could easily be used to make a picture or voice recording of a crime or accident in a discrete way. If the personal mobile device records the position of the personal mobile device and a picture or sound sample at predefined intervals, a track of the person is generated together with audio and/or video information representative of the environment the person is in and/or has been in during a past period of time. In the context of this invention, the term "environmental data" is used which refers to such audio or video data or both. The pictures provide visual information of the track and the sound samples provide information about voices of persons in the neighborhood of the person, background sounds, etc. The data is stored for possible future use, e.g. on the personal mobile device. However, storage may, alternatively, be done in an external memory. It is observed that the term "video information" may refer to any sequence of images with any time interval between consecutive images. The images may, for instance, be part of a video film made by a cell phone or the like, or may be separate images consecutively made by a user of the personal mobile device. Image sequences or video could be color, black and white, or infrared nature.

Starting the collecting and recording of information from the camera, the microphone and/or the GPS receiver may be done manually by the user of the personal mobile device in any technically feasible way. He may, for instance, turn on these devices in the way currently implemented on personal mobile devices available on the market. However, this may mean that he has to turn on three separate devices or features. So, in an embodiment, the application running on the personal mobile device may provide the user with the option to press only one special button on the personal mobile device or a unique combination of two or more buttons on the personal mobile device.

Moreover, by pressing a single button, button combination or special number, such as 911, 112 or the like on his personal mobile device, a user may activate a command equivalent to an emergency call. After receiving the command, the personal mobile device provides the stored and/or current data to an observation station, which could be an emergency centre, a police centre, a security centre, a personal computer at the premises of a parent or the like. "Provides" means that the stored and/or current data is sent to the observation centre or that the observation station retrieves information where to find the stored data on, for example, a server. After the emergency call, an operator of the observation station knows that an event, for example a crime or accident, has happened or is in progress and can evaluate what had happened before the emergency call by reviewing the information received from the personal mobile device on a screen and reproducing the associated audio and/or visual information. Furthermore, the operator of the observation station immediately knows the location of the event because the position information is also sent. After making the emergency call, the personal mobile device may provide further information about the event on a regular basis which may be nearly continuous, which could be used by the operator of the observation station to follow the event and to determine which sources are needed to resolve any ongoing problem. The pictures could be used to recognize faces of people that are part of the crime. Having such a simple-to-use safety device and service could make people feel safer. This will help people reduce their fear and anxiety in using for example public transport, walking in a metro station, waiting at a bus station and the like. In a further embodiment of the invention the length of the intervals for collecting data has a relationship with the position information. This feature allows for a reduction of the amount of data to be stored. It has been found that places of crimes, attacks and accidents, are not evenly distributed over a city, but are concentrated at particular places. By knowing the places with a high rate of occurrence of an event like a crime or accident, this position information could be used to determine the length of the interval between two instants of collecting data representative of the position, time, and circumstances of a person at a particular moment in time. At places with a high rate of occurrence of a crime, the length of the sampling interval should be shorter then for places with a relative low rate of occurrence of a crime. In this way the personal mobile device will monitor the environment of the person better at locations or areas with a high risk of an event. This is in line with the safety feelings of the person, as a person has a safer feeling when he is monitored more frequently. This feature enables the user to have a feeling of safety which is evenly distributed over a city, whereas the amount of data to be stored can be reduced when a person is in an area with a low risk of becoming part of an event.

In a further embodiment of the invention the length of the intervals for collecting data has a relationship with characteristics obtained from a map database, the characteristics being associated in the map database with a position corresponding to the position information. This feature enables the application according to the invention to determine a risk factor from data obtained from a map database. A map database in a navigation system comprises road information, such as type of road, crossings, number of lanes, etc. The map databases used in navigation systems more and more comprise additional information that could be of interest to a user, for example the location of bus stations, metro stations, train stations, shopping centers, bars, police stations, hotels, museums, gas stations, etc. The characteristics in the map database can be used to determine a risk factor for a particular position, wherein the risk factor depends on the characteristics in the map database in the vicinity of said particular position. In this way, already available and relatively accurate and up to date information is used to derive the interval length and it is not needed to generate a dedicated risk map, in which for each position a risk factor has to be determined, although such a risk map if available could also be used.

In further embodiment of the invention, the length of the interval depends on the moment of time. It is known that there is more criminal activity during the night than between dawn and dusk. Furthermore, it is known that there is more criminal activity during rush hour at stations. This knowledge can be taken into account for deriving the frequency to collect data.

A map database could also comprise information related to opening and closing times of shopping centers. There is a correlation between these times and the amount of criminal activity in shopping centers. This information can be taken into account in deriving the length of the interval between the current and next collection of data to obtain the tracking and environmental information associated with the moving object. In a further embodiment of the invention, the length of the interval depends on the movement of the object. The duration that a person is located at a specific position close to the environment of specific risk features in combination with, for example, the dusk-till-dawn time also gives an indication of the safety of a person. For example, if a person is located in the near vicinity of a bus station or a metro station for a few minutes, say less than 30 minutes, during dusk till dawn, this is an indication that the person is waiting for the next bus or metro. A safety risk may be attributed to such a situation. Furthermore, the fact that the speed of travel along the path is suddenly increased to a normal driving speed after the person was recorded as being in the environment of a bus station means that the chance that this person is now located in a public transport vehicle is very high. Again, a lower probability of remaining safe may be attributed to such a situation. The different attributed lower probability of remaining safe may be translated into a different length of the collection interval.

In a further embodiment of the invention the collecting action comprises - collecting position and time data representative of the track of the personal mobile device;

- collecting audiovisual data representative of the tracking circumstances of the personal mobile device; - generating association data to associate the position and time information with the audio visual data. These features enable someone practicing the invention to retrieve data from different sources and to generate the connection or relationship between the data. For example, pictures from a camera do not necessarily comprise time or position information. If the images are stored without linking information, it will be very hard to determine the time and/or position of the camera when the picture is taken.

Furthermore, the clock time of the camera could differ from the clock time of the personal mobile device. This could lead to mistakes in interpreting the collected data from the camera and personal mobile device. Therefore, according to this embodiment of the invention association data is generated to define unambiguously the relationship between the data supplied by different devices with there own clock time and/or position information system. This will increase the ease of interpreting the different types of data at the observation station.

In an embodiment of the invention, the method further comprises the action:

- if the data stored in the memory satisfies a predefined condition, deleting a part of the stored data from the memory. This feature allows the personal mobile device to limit the required storage capacity and the amount of data to be transmitted to the observation station.

The present invention can be implemented using software, hardware, or a combination of software and hardware. When all or portions of the present invention are implemented in software, that software can reside on a processor readable storage medium. Examples of appropriate processor readable storage medium include a floppy disk, hard disk, CD ROM, DVD, memory IC, etc. When the implementation includes hardware, the hardware may include an output device (e. g. a monitor, speaker or printer), an input device (e.g. a keyboard, pointing device and/or a microphone), and a processor in communication with the output device and processor readable storage medium in communication with the processor. The processor readable storage medium stores code capable of programming the processor to perform the actions to implement the present invention. The process of the present invention can also be implemented on a server that can be accessed over telephone lines or other network or internet connections.

In an embodiment, the tracking information and environmental information are pushed in a manual or semi automatic way to an emergency center in case of an emergency.

Short description of drawings

The present invention will be discussed in more detail below, using a number of exemplary embodiments, with reference to the attached drawings that are intended to illustrate the invention but not to limit its scope which is defined by the annexed claims and its equivalent embodiment, in which

Figure 1 shows an exemplar embodiment of a system for implementing the invention;

Figure 2 shows a flow diagram of an exemplar implementation of the process for providing track information according to the invention;

Figure 3 illustrates an exemplar embodiment of the usage of the invention and corresponding information generation;

Figure 4 illustrates an exemplar embodiment of the usage of the invention and corresponding representation at an observation station; Figure 5 illustrates another exemplar embodiment of representation of track information at an observation station; and,

Figure 6 shows a block diagram of a computer arrangement with which the invention can be performed.

Detailed description of exemplary embodiments

Figure 1 shows an exemplar embodiment of a system for implementing the invention. The system is arranged to perform a Location Based Service (LBS) Safety Application. The objective of the system is to provide tracking information of a person with a personal mobile device, as well as audio and/or video information of the environment of that personal mobile device along a traveled track. The system comprises a personal mobile device 102, such as a mobile phone, smart phone, PDA or the like. A mobile device is defined to be "personal" in the context of this document when it can be carried by a person and it has, substantially, a pocket size. The personal mobile device 102 could include a built-in camera 104, for capturing images or videos, a built-in microphone 106 and a GPS receiver 108. However, the camera 104, microphone 106 and GPS receiver 108 could be wirelessly connected to the personal mobile device 102, for example by using Bluetooth. In that case the camera 104 and/or microphone 106 could be worn by a user and for example be integrated in glasses, a hair slide or clothing. If the camera 104 is integrated in glasses, the pictures taken could always contain the information in the field of vision of the user wearing the glasses. The personal mobile device 102 could for example be integrated in a watch, which could communicate with the mobile phone 102 which is carried by the user. In that case the mobile phone 102 is used to communicate over long distances.

Instead of a mobile phone 102, the personal mobile device may be a transceiver in a movable object, the transceiver being able to communicate with the camera 104 and/or microphone 106 and/or GPS receiver 108. Such an external device can be built in a bus or any other vehicle, enabling the user carrying the personal mobile device to record the sound and images in the proximity of the user in a life environment.

By means of the camera 104, microphone 106 and GPS receiver 108, data representative of the track and environment of the user is collected. Starting collecting and recording information from the camera 104, the microphone 106 and/or the GPS receiver 108 may be done manually by the user of the personal mobile device 102 in any technically feasible way. He may, for instance, turn on these devices in the way as nowadays implemented on personal mobile devices 102 available on the market. However, this may mean that he has to turn on three separate devices. So, in an embodiment, the application running on the personal mobile device 102 may provide the user with the option to press only one special button on the personal mobile device 102 or a unique combination of two or more buttons on the personal mobile device.

The collected data comprises at least position information obtained by the GPS receiver 108 and corresponding time information. The position information could also be based on cell-id information. In this case, a mobile network station , like a GSM base station sends a Cell ID. The mobile phone 102 reads this Cell ID. By gathering the Cell IDs with associated position information of the GSM mast, it is possible to determine the current location of the personal mobile device 102. With this information it is possible to develop location based applications using a mobile phone 102, such as the Location Based Service (LBS) Safety Application according to the invention. Furthermore, by using the location information, geo-coded images, videos and sound tracks can be generated. This could be done by adding position information such as the cell id or geographical coordinates from GPS, or other satellites as meta data to the respective files carrying the collected data. It is observed that, the camera 104 and the microphone 108 are just examples of sensor devices arranged to sense some feature of the environment, like a visible object, a sound or a location . For instance, future cars may collect data from radar and laser scanners as well as from cameras fixed to the car. An orientation device may exist on the camera or position device as well. Such an orientation device provides orientation data that may be collected to help define the direction of the camera when pictures were taken. Such radar and laser devices are other examples of sensor devices.

By means of the camera 104, images representative of the circumstances around the user carrying the personal mobile device 102 are collected and by means of the microphone 106 sounds around the user are collected. By means of the tracking information as collected by the GPS receiver 108, the circumstances leading to a particular event are recorded. The tracking and environmental information when evaluated by a police officer, could help him to reconstruct the circumstances before, for example, an attempted robbery and what happened thereafter.

The collected tracking information could be stored locally in a memory of the personal mobile device 102 or externally on storage space on a server 110. In the latter case, the personal mobile device 102 contacts the server 110 on a regular basis to upload the collected data and to store the data on a storage medium, for example a hard disk or the like. In this way the collected data is safely stored in a remote place in the case that the data is needed for further processing because some event occurred. An LBS safety application system according to the invention, as shown in figure 1, further comprises an observation station 112. The observation station 112 could be a computer system located in an emergency centre, a police centre, a security centre or the like, with a mouse 114, a keyboard 116 and a loudspeaker 118. Moreover, the observation station is preferably implemented as a computer system shown in more detail in figure 6. The observation station 112 is arranged to receive the collected and stored data representative of the track and environment of the user carrying the personal mobile device 102 generally after the user has activated the personal mobile device to do so. In an embodiment of the invention the personal mobile device 102 comprises a dedicated emergency key which when pressed generates a command which sets up an emergency call and starts to send the collected and stored data in its memory to the observation station 112 for further processing. After using the emergency key the personal mobile device 102 may still collect data and send the data immediately on to the observation station 112. The data comprises at least position and time information and some audio and/or visual information. By means of the data, the observation station may superimpose the track of the user over a road map or other spatial representation such as a building or stadium plan and display the resulting data on a screen. Further, the visual information can be reproduced on the screen of the observation station 112 and the audio signals can be reproduced via loudspeaker 118 or a headphone. By means of the stored data the circumstances leading to the event, can be shown and by means of the data collected during the emergency call circumstances immediately following the event can be shown, and thus an impression and evidence of the accident/crime circumstances can be generated. As the data before the emergency call is generated automatically, there is a fair chance that the data comprise some facts, that could help an official recognize the suspected offender or the causes of the accident.

Instead of a dedicated emergency key, a defined key-combination or key sequence could be used to generate an emergency command. A key sequence could be an emergency number such as 911 or 112. Furthermore, the emergency command could be generated by voice control. This activation could put the user in direct contact with an operator at the emergency centre and could even send a configuration flag instructing the operator to either talk to the user or remain silent and listen only. The observation station 112 could also be a home computer of an observer of a certain person, for example a parent of a child. In this case, the child could use the personal mobile device 102 to indicate to the observer that he has a feeling of insecurity. The observer can see where the child is, see the circumstances that lead to the feeling of insecurity and the current situation. Based on the observations the observer can take the appropriate actions. The observer can for example contact the child by means of the personal mobile device, reassure the child, provide some instructions and if necessary contact an emergency center. For minors, the parent might want the ability to see where the child is at anytime. Moreover, knowing where ones child is and being able to "see" his environment and hear sounds from his environment adds to safety.

It should be noted that the server 110 could be part of the observation station 112. In that case the collected data is transmitted continuously to the observation station 112. In case of selecting the emergency key the observer using the observation station 112 may get an alert to check the received data.

The functioning of the LSB safety application according to the invention will be disclosed in more detail below.

Figure 2 shows a flow diagram of an exemplar implementation of the process for providing track information according to the invention performed on the personal mobile device 102. The process starts with action 202 by turning on the personal mobile device 102. Moreover, at least the GPS device 108 or any other position collecting device is turned on and at least one of the camera 104 and microphone 106. This may be done in any way explained earlier. After turning on the personal mobile device 102, in action 204, data is collected about the current environment of the user carrying the personal mobile device 102. The data includes at a minimum, position information and time information. The position information could be derived from data generated by a GPS receiver or be based on cell-id information or other location determination technology well understood by those versed in the art. Based on the functionality coupled or connected to the personal mobile device 102, an image or a video sequence about the environment of the user and/or sounds from the environment of the user will be collected. Possibly other source data may be collected such as radar or laser scan information. The data obtained by the different sources (GPS, Cell-ID, microphone, camera, radar etc.) is linked together. All data obtained by each of these sub devices receives a time stamp from the internal processor of the personal mobile device. This time stamp is the unique link between all content gathered by the different sub devices. The data collected during a time period could be combined into one data package, with one package for each time period. Such a period of time may extend over a few seconds or longer. In another embodiment the audiovisual information is stored in separate files and a text file is generated with reference to the respective files for each time period. An example of the content of such a comma-separate-values (CSV) text file is given below: T100,GPS xlOOylOOzlOO, jpeglOO, cell id xlOOylOOzlOO, wavlOO

TlOl₅GPS xlOlylOlzlOl, jpeglOl, cell id xlOlylOlzlOl, wavlOl

T102,GPS xlO2ylO2zlO2, jpeglO2, cell id xlO2ylO2zlO2, wavlO2

TlO3,GPS xlO3ylO3zlO3, jpeglO3, cell id xlO3ylO3zlO3, wavlO3

The first column indicates the time instant. The second column represents the position obtained from the GPS receiver at the time instant indicated in the first column. The positions in the second column form a GPS track. The third column indicates the name of an image file in jpeg compression format associated with the time in the first column. The fourth column represents the position derived from the cell-id information for that same time and the fifth column indicates the name of an audio file in wav- format associated with the same time. The wav-file could for example correspond to 30 seconds of sound before the time instant. It should be noted that the GPS position may not be derived all the time, for example when the user is in a tunnel or in an underground metro station, a large building, etc. In that case storing both the GPS position and cell-id information has the advantage that the track of the user can still be recorded but with less accurate cell- id position information. Units attached to vehicles may have more sophisticated position determination equipment that might still provide an accurate location even where GPS reception and/or cell id reception fail.

The data collected in action 204 is subsequently stored in a memory, action 206. The memory could be local storage space in the personal mobile device 102 or storage space on server 110. An advantage of using storage space on server 110 is that the stored data is not destroyed if the personal mobile device 102 is destroyed and/or missing, for example due to a fire or due to an auto crash, theft or intentional destruction. In that case, when the user and the corresponding personal mobile device 102 are identified, the data on the server 110 could be recovered and sent to the observation station 112 and the environmental data up to the destruction or loss of the personal mobile device 102 can be reproduced. (In the case of loss, for example by theft, the data may still continue making recovery much easier.)

After storing the collected data, in action 208 it is verified whether the emergency command is given. If no emergency command is given after a predefined interval the actions 204 and 206 are repeated. By collecting and storing data, at intervals, a representative sample of the track and of the surrounding circumstances of the user is generated. The storing process of the collected data is such that any data apart from data collected during a last predetermined period of time may be overwritten. Such a last predetermined period of time may, for instance have a length of between 5 and 60 minutes. Thus, environmental data as to an event that triggers the user to start sending the stored data includes at least data from that last period directly prior to the event.

If an emergency command is given the data stored in the memory is sent to the observation station 112. This action in indicated by block 210. Preferably, a user identification or personal mobile device identification is submitted to the observation station 112 to identify the data transmitted to the observation system 112. The identification could also be used to identify the user of the personal mobile device 102. If the data is stored in the memory of the personal mobile device 102, the data is transmitted directly to the observation station 112. The transmission could be performed by means of GSM / GPRS / UMTS / CDMA / IP Multimedia Subsystem (IMS) or any other suitable communication protocol.

The transmission of data will take some time and should preferably be error free. However, the data transmitted to the observation station 112 should be reproduced as soon as possible, to enable the observer to take appropriate actions. Therefore, the data is preferably not sent as one big data file but sent in portions. The most important data should be transmitted first. If a CSV text file is used to link the respective data sources, the text file is transmitted first. In an embodiment, the remaining data files, comprising the audio, image and video information, are transmitted in reverse time order, starting with the latest created files first. The remaining data files could also be transmitted in packages corresponding to a period of, say, 5 minutes. In another embodiment, after sending the text file, the sound files are sent first, and the bigger image and video files are sent after that. The application defines which method is most suitable. If the data is stored on a server, the personal mobile device 102 sends an instruction to the server 110 to forward the information to the observation station 112. The transmission could be performed by means of GSM / GPRS / UMTS / IP Multimedia Subsystem (IMS) or any other suitable communication protocol. If the data is stored on the server 110, the data could also be obtained with a TCP/IP interface. In that event, the observation station 112 needs to retrieve an identification signal from the personal mobile device 102 to be able to access the data stored on the server 110. The application running on the observation station 112 controls the retrieval of data from the server 110. The emergency command could also be submitted to the observation station 112 in the form of an SMS-message to one standard emergency number of the number associated with the observation station 112. The observation station 112 will then establish the connection between the personal mobile device 102 and if necessary the server 110 to retrieve the collected data from the memory. After submitting the stored data or the instruction to either send the data or to retrieve the data, the personal mobile device 102 preferably continues to collect data in action 212. Action 212 could be arranged to collect the same type of information as in action 204. However, as action 212 is performed during an emergency situation, in stead of a single image collected in action 204 short image sequences, for example an AVI file of 5 seconds, could be collected and retrieved so as to allow the personal mobile device 102 to provide more detailed information about the circumstances of the emergency situation. In another embodiment, the microphone 106 could be switched on continuously and the pictures could be taken as fast as processing and bandwidth allow. In action 214, the collected data is sent directly to the observation station 112 and depending on the application, stored in the local memory or on the server 110. A user identification or a personal mobile device identification is sent together with the collected data.

In action 216 it is verified whether the emergency command is released. If the emergency command is still active, actions 212 and 214 are is repeated at a predefined interval. The predefined data collecting interval during the emergency command active period after which the data is sent is preferably shorter than the interval if the emergency command is not active. In this way, more frequent data about the circumstances of an accident or crime itself is provided, enabling the observer to obtain a more detailed view of the circumstances after the event has begun. If the emergency command is released, the process will return to action 204. The emergency command could be released by pressing again the dedicated emergency key, another dedicated key, for example the "#" key on a mobile phone, a predefined key sequence or combination or voice message.

In action 206 the collected data is stored and in action 210 the stored data is transmitted to observation station 112. The amount of data collected and stored may become enormous. Generally, only a limited period of time before an incident will be needed to provide evidence. Therefore, in action 206 the data in the memory, older than a predefined period, will be deleted from the memory. This restricts the amount of data to be transmitted to the observation station 112 after the emergency command is activated. The time period of data stored in the memory is preferably limited to the last 30-60 minutes. In another embodiment of the invention, the memory contains the information of for example a maximum of 6 hours. As soon as the maximum is reached, three hours representing the oldest tracking information are deleted from the memory. In this way only occasionally information has to be deleted.

A common implementation would be a circular memory. The memory size might be fixed but the time would depend on the frequency of collection. In such a system the oldest data is always overwritten by the newest data. When the emergency key is pressed the whole memory is dumped out to the observation station 112.

If the data is stored at server 110, data is sent periodically to the server 110. Then, the server 110 may be running an algorithm that keeps a predetermined amount of received data. That amount of data may be much larger than the amount stored on the personal mobile device 102 itself. If the server 110 stops receiving data from the personal mobile device 102, the data received during a predetermined period of time is kept in its memory even though time passes. This way even if a key was not pressed by a user of the personal mobile device 102 the last events might still be captured. User preferences allow the user to set the maximum duration of the tracking information and indicate when and how much information is deleted.

There might also be an option that the personal mobile device 102 stores first the collected data locally in a memory and sends the stored information to server 110 each quarter of an hour. The collected data could also be sent to an email address. In this way, the collected data could easily be sent to different destinations. A destination could be the mail directory of the user himself or the mail directory of the server 110. In the latter case, the email application running on the server 110 is capable of detecting the email and to process the email accordingly, so as to store the collected data in the memory.

As disclosed above, actions 204 and 206 are performed at intervals. The interval could be fixed. However, the risk of an accident / crime has a relationship with the location where the user is situated and perhaps the time of day. Furthermore, the feeling of safety of the user may depend on the degree he is observed. The more the user is observed the more safe he is likely to feel. To provide a uniform feeling of safety, according to an embodiment of the invention, the interval decreases when the estimated risk of an accident / crime increases. This increases the frequency of logging the collected data. This feature is made available by using a digital map database to estimate how safe or less safe the current position of the user carrying the personal mobile device 102 is. This can be adjusted by time and / or the user himself. To enable this feature, the personal mobile device 102 is capable to access a digital map database that stores a safety indication as a function of location. The safety indication is based on historical data as to number of accidents and/or number of crimes at that location, making use of the crime spot features. Each location may be defined as an area of a predetermined size. The personal mobile device 102 could be a combined mobile phone and navigation system. In this case the digital map database is available in the memory of the personal mobile device itself. In another embodiment, the personal mobile device 102 could access an external navigation system carrying the digital map database to retrieve the necessary features to estimate an indication on how safe or unsafe the current location of the user is. The map database could also be stored on server 110 or another server. In that case the personal mobile device 102 regularly contacts the server 110 to retrieve the features from the map database. It should be noted that the server carrying the digital map database could be different from the server storing the collected data.

The data logging frequency or interval between two subsequent instants of data collection can be related to the position of the user. Map features and attributes could help to estimate how safe or unsafe an area is and, thus, the value of the safety indication. The presence of certain features and attributes in the map could be linked to the time of the day and/or related to the intensity of the daylight. So the safety level of a particular area at a particular time of the day may differ during winter and summer. It may also vary by day of the week. The safety level of a position or area may be derived from static features obtained from the map and dynamic features which might depend on the light intensity or the time of day.

Safety with respect to automobiles could also be a function of dynamic data that is provided by a map supplier or others including traffic information and weather, for instance, via a live telecommunication link between a car mobile phone / navigation system and a server of that map supplier (or someone else).

Any of the following features and attributes and combinations of them give an indication of the safety level of an area:

• the amount of roads of a low functional road class in the area. The road class feature indicates the importance of a street. The higher the road class the higher the safety level - (there may be exceptions, like a situation where roads are running for hundreds of km through desolated areas).

• the number of cafes and restaurants in an area;

• the number of points of interest in an area;

• the number of shops in an area; • the width of the road. The smaller the road, the less safe;

• the presence of a park;

• the presence of a metro station, railway station, bus stop or tram stop;

• the presence of street lighting;

• a crime spot identification which identifies the safety level of an area; • an accident risk identification which identifies the risk of an accident in a certain area;

• time of day, day of week, week of year

• weather conditions

• traffic conditions.

The crime spot identification and accident risk identification could be based on statistical information about incidents reported by the police or other governmental offices. The list of features and attributes should not be interpreted to limit the invention to the mentioned features. Other features or combination of features might be found to be suitable as well for the LBS safety application. In one embodiment, the user could also set locations that he believes to be at higher or lower in risk and the device would respond appropriately when the positioning unit determines that the user is at or near the set locations.

The more of the before-mentioned features that are found to occur in a particular area, the stronger the evidence that the area should be considered as unsafe. In an embodiment, the LSB safety application running on the personal mobile device 102 uses the estimated safety level to determine the appropriate frequency of logging the position information and audio and/or visual information.

Conversely, a location, area or path, in combination with a time or time interval might be considered safe. So in the application of a child coming home from school, the time window of coming home and some spatial window around the path of travel or other spatially defined area, are considered safe. If either window is exceeded it could trigger a change in the recording and/or communicating of safety information.

Furthermore, the interval could depend on the duration a person carrying the personal mobile device 102 is positioned at a specific location close to or in the environment of specific unsafe features. The combination also gives an indication on the safety rate of a person. For example, when a person is located in the near environment of a bus station or a metro station and not or hardly moving for a few minutes during dusk till dawn, there is a reasonable chance that person is less safe than when he is walking. Waiting at a station is regarded to be less safe than walking along the station. Furthermore, the fact that the path of a user and the movement of the user is suddenly increased to driving speed after it was derived from the map database that he could have been waiting for a bus, could be an indication that the user entered the bus. When sitting in a bus or other public transport vehicle is regarded to be a less safe environment, this sudden increase in speed could be used to estimate a new safety level and to select the corresponding interval for collecting data.

Figure 3 illustrates an exemplar embodiment of the usage of the invention and corresponding information generation. At time a, a user walks along the street. The personal mobile device 102 is switched on and logs periodically (that may be near continuously) and automatically the track information and related audio and/or visual information. The information is captured automatically at intervals for example of each 1, 5, 20 minutes. The intervals could be user controlled, wherein the user selects the interval corresponding to his feelings of safety. The interval could also be automatically adapted based on the estimated safety of the present position as described earlier. The estimated safety could be a parameter in a digital map associated with positions or could be derived from other map features. At the actual interval, data representative of the current circumstances is collected and stored in a memory. The position data and linking data is stored in a text file and the pictures are stored in a predefined directory. The linking data indicates which files are associated with a time instant. The camera could have sensors to switch automatically to the most optimal mode: black and white, color, or infrared.

At time b, the user takes the bus. The personal mobile device 102 continues collecting and storing the data so as to obtain the tracking data and environmental data. The interval between two subsequent collections of data could be selected in a similar way as in the time between time a and time b. If the feeling of safety is generally less than when outside the bus, the user could select the interval corresponding to a less safe level or the personal mobile device could detect or determine that the user is in the bus and that the interval corresponding to a less safe level could be automatically selected or the personal mobile device 102 could continue at its original settings if so configured.

At time c, a criminal enters the bus and hijacks the bus by putting a gun on the head of the driver. The user presses the emergency key on his personal mobile device 102. The personal mobile device 102 immediately sends the log file, for example the CSV text file, of the last 30 - 60 minutes to the observation station 112 and the corresponding audio and/or visual data files. In addition, from the moment the user pushes the emergency key, fresh collected data including the GPS-positions, GSM cell- id's, and audio and/or visual data is sent immediately and continuously to the observation station 112.

Image sequences or video could be color, black and white, or infrared nature. Based on light conditions, the system may automatically detect which type of imagery needs to be captured. In case GPS track data is stored each 30" and picture imagery is stored with a low resolution at a picture size of 50 kb per picture and capture each 30", a total data package over a period of 30 minutes has a data volume of about 3 MB. However, consecutive images need not be part of a film. They may alternatively be separate images taken by a user of the personal mobile device 102 at moments he/she decided to do so.

The observation station 112 stores the collected data, displays received video data via its monitor and outputs received audio data via its loudspeaker 118. The received data can be forwarded to the police (or other authority) for further evaluation. If real time resources with the police are available, based on the audiovisual data, the police can make an assessment as to the nature of the event to determine if a crime is being committed or some emergency response is necessary. The GPS tracking data can tell the police where to go, whereas audio/video data can tell the police how to respond. E.g., the police may be sent to the location of the bus, to be able to stop the bus and overwhelm the hijacker.

If no real time resources are available or if the assessment made results in the conclusion not to send an immediate response, the data as received can be used in a follow-up investigation/analysis.

In another embodiment of the invention, up to time c, the personal mobile device 102 collects single images and as soon as the emergency key is pressed the personal mobile device 102 collects video material and sends the captured frames each minute as a new file to the observation station.

At time d, the bus is stopped by the police and the hijacker is overwhelmed. The user stops sending the information to the observation station by pushing again the emergency key. To encourage people to use the LBS safety application, the user may be awarded for the fact that the criminal could be arrested

Figure 4 illustrates an exemplar embodiment of the usage of the invention and corresponding representation at observation station 112 on a display unit. The position information in the CSV file is used to retrieve a map corresponding to the track traveled by the user of personal mobile device 102. Subsequently, the track is superimposed over the map. In this way, the observer at observation station 112 can see the track of the user in the past and follow the present path of the user. Figure 5 illustrates another exemplar embodiment of representation of track information at observation station 112. Next to the track information and superimposed over a map, the collected visual information could be displayed on a display unit. The respective images or frames of a video could be displayed next to the map. In this way the observer can see in a glance when something suspicious had happened. In the example of figure 5, the upper right portion shows nine images consecutively taken along a small portion of the route a-b-c-d indicated with a dash-dot line in the left portion of figure 5. Of course, more images may be taken along that route. The image at the lower right portion has been taken on a moment that a crime was observed by the user of the personal mobile device 102.

The collected audio and/or visual data and position information could be used at the observation station to automatically determine accurately the position and orientation of camera 104 connected to the personal mobile device 102. The visual information may be used by observation station 112 to find similarities with image information in the map database. By means of image processing tools characteristics of for example buildings can be derived from the visual information. These characteristics may be used to find similar characteristics in the map database. A geo- position is associated with the characteristics in the map database. The geo-position of a characteristic in combination with the position associated with the visual data determines the orientation of the viewing direction of the camera.

The safety application software on the observation station 112 could also be arranged to reproduce the audio and/or visual information corresponding to a particular position by moving a cursor over the superposed track of the user. The system described above could easily be adapted to provide more safety services to a user. The safety services could be bought by a user from a security firm and depend on the level or degree of feeling of safety the user may need. Some examples of safety services are: Service 0: The data is collected and sent to the observation station 112 continuously or as near to continuously as bandwidth limitations will allow. The observation station 112 continuously monitors the circumstances until the user feels he is in a more secure area. Service 1 : This service is for high risk areas, for example during night, dusk, dawn, in metro's and stations. After selecting this service, each 5 minutes data is collected by the personal mobile device 102 and transmitted to the observation station 112; Service 2: This service is for less risky areas. After selecting this service, each 30 minutes data is collected by the personal mobile device 102 and transmitted to the observation station 112;

Service 3: This service provides automatic adaptation of the frequency of logging, wherein the frequency is based on a combination of map features giving an indication on the degree of safety of an area.

The services provide a manner wherein a criminal has no way to avoid some level of evidence capture. Even if the person is surprised and never hits the emergency key, once a crime is reported, even days after finding the body of the victim, the police may have a record of what had happened. Figure 6 illustrates a high level block diagram of a computer system which can be used to implement a personal mobile device 102 and observation station 112 performing operations described above.

The computer system of Figure 6 includes a processor unit 612 and main memory 614. Processor unit 612 may contain a single microprocessor, or may contain a plurality of microprocessors for configuring the computer system as a multi-processor system. Main memory 614 stores, in part, instructions and data for execution by processor unit 612. If the method of the present invention is wholly or partially implemented in software, main memory 614 stores the executable code when in operation. Main memory 614 may include banks of dynamic random access memory (DRAM) as well as high speed cache memory.

The system of Figure 6 further includes a mass storage device 616, peripheral device(s) 618, input device(s) 620, portable storage medium drive(s) 622, a graphics subsystem 624 and an output display 626. For purposes of simplicity, the components shown in Figure 6 are depicted as being connected via a single bus 628. However, the components may be connected through one or more data transport means. For example, processor unit 612 and main memory 614 may be connected via a local microprocessor bus, and the mass storage device 616, peripheral device(s) 618, portable storage medium drive(s) 622, and graphics subsystem 624 may be connected via one or more input/output (I/O) buses. Input devices 620 such as a camera or microphone, may be connected via a wireless connection such as Bluetooth or the like. Mass storage device 616, which may be implemented with a magnetic disk drive or an optical disk drive, is a non-volatile storage device for storing data, such as CSV text files, image files, video files, audio files, and instructions for use by processor unit 612. In one embodiment, mass storage device 616 stores the system software or computer program for implementing the present invention for purposes of loading to main memory 614.

Portable storage medium drive 622 operates in conjunction with a portable non- volatile storage medium, such as a floppy disk, micro drive and flash memory, to input and output data and code to and from the computer system of Figure 6. In one embodiment, the system software for implementing the present invention is stored on a processor readable medium in the form of such a portable medium, and is input to the computer system via the portable storage medium drive 622. Peripheral device(s) 618 may include any type of computer support device, such as an input/output (I/O) interface, to add additional functionality to the computer system. For example, peripheral device(s) 618 may include a network interface card for interfacing computer system to a network, a modem, etc.

Input device(s) 620 provide a portion of a user interface. Input device(s) 620 may include an alpha-numeric keypad for inputting alpha-numeric and other key information, or a pointing device, such as a mouse, a trackball, stylus, touch screen or cursor direction keys. In order to display textual and graphical information, the computer system of Figure 6 includes graphics subsystem 624 and output display 626. Output display 626 may include a cathode ray tube (CRT) display, liquid crystal display (LCD) or other suitable display device. Graphics subsystem 624 receives textual and graphical information, and processes the information for output to display 626. Output display 626 can be used to show the actual status of the personal mobile device 102, a digital map with an overlay of the track, the collected image and/or display other information that is part of a user interface. The system of Figure 6 also includes an audio system 628. In one embodiment, audio system 628 includes a sound card that receives audio signals from a microphone. Additionally, the system of Figure 6 includes output devices 632. Examples of suitable output devices include speakers, printers, etc. The components contained in the computer system of Figure 6 are those typically found in general purpose computer systems, and are intended to represent a broad category of such computer components that are well known in the art.

Thus, the computer system of Figure 6 can be a personal computer, workstation, minicomputer, mainframe computer, personal digital assistant PDA, laptop, etc with peripherals. The computer can also include different bus configurations, networked platforms, multi-processor platforms, etc. Various operating systems can be used including UNIX, Solaris, Linux, Windows, Macintosh OS, and other suitable operating systems. In the embodiments given above, a user is carrying the personal mobile device

102. However, the personal mobile device 102 could also be built in for example a car or a bus. For example if the personal mobile device is built in a bus, and the bus is equipped with camera's and microphones collecting data about the circumstances in and around the bus, a passenger can push an emergency button in the bus and according to the invention the collected and stored data is subsequently transmitted to the observation station. Furthermore, if the personal mobile device is built in a vehicle, the personal mobile device could generate automatically the emergency command by detecting for example an abnormal orientation of the bus or abnormal movement of the bus. Data as to such a sudden movement or abnormal orientation can be derived from for instance the GPS receiver 108, as will be evident to persons skilled in the art. The foregoing detailed description of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. For example, the invention is also suitable to implement a children LBS safety application. Parents (or other attendants) want to know where their child is located, when playing outside. What is he or she doing? The invention could be configured such that the position of the child and the related pictures and sounds are sent with a certain frequency to an application running on the parents personal computer or laptop. So instead of sending the collected data to an emergency centre the data is sent to a PC. The frequency of data collection can be based on features retrieved from a digital map database. Furthermore, in this application the time is an indication of the desired safety level. For example, the child may leave school at 15:00 and will normally arrive at home at 15:15. During this period the risk for an incident is high and consequently the logging frequency is increased to the highest level. For instance, when a kid is kidnapped and the personal mobile device 102 is later discovered valuable data may be retrieved from the memory of the personal mobile device 102 showing for instance a color and model of a car of the potential kidnapper.

It is observed that the data could alternatively be sent to a personal, mobile telecommunication device of the parent, like a mobile telephone, smart phone or pda. The PC or mobile telecommunication device could be programmed to have an easy forwarding function to forward received data to the observation station 112. In a further embodiment the personal mobile device could be programmed to respond as though an emergency button has been pressed if certain spatial or temporal situations arise. For example, the parents of the device of the child coming home from school may program the path home as taken normally by the child. If the personal mobile device provided with location sensors senses that it has substantially left the path it would automatically set the emergency mode. Alternately, if the child does not reach a programmed location such as home, by a certain time, again the device can be programmed to set a certain level of emergency function.

The user may know upfront what are less safe tracks or areas for himself or for users of the system. The predefined tracks may be entered by guides, parents, guards, into the memory of the personal mobile device. For example, in the case of a parent - child relation, the personal mobile device may automatically start recording track, sound and picture data with the highest capturing rate, the moment the personal mobile device senses that the child is moving more than a predefined amount of meters or kilometers away from the predefined path (corresponding to a minimum deviation from that path) as stored in the personal mobile device memory. The personal mobile device may then be programmed to automatically send all captured information (starting from the predefined minimum deviation to the current position) to the observation station, the moment the child reaches a maximum predefined accepted deviation from the predefined path. In such a case the observation station may be a PC at the premises of the parents. So, an observation station in the sense of the invention may be any centrally or locally located computer arrangement operated by an operator. Apart from the current position of their child, the PC of the parents also receives data as to the track followed by the child and related pictures and sound. Moreover, from that time

Claims

28 CLAIMS

1. Method of collecting tracking data and environmental data associated with a person with a personal mobile device, the method comprising: • upon sensing a first event, collecting with said personal mobile device said track data representative of the track of the person including at least position and time information of the personal mobile device along the track, as well as said environmental data of an environment along said track;

• storing the track data and environmental data in a memory, the memory being either an internal memory of said personal mobile device or a memory external to said personal mobile device.

2. Method according to claim 1, including, upon sensing a second event, providing the stored data from said memory to an observation station

3. Method according to claim 2, including, after said second event, collecting and sending to the observation station continuing track data representative of the actual track of the personal mobile device including at least position information, as well as continuing environmental data.

4. Method according to claim 2 or 3, wherein said second event is a command generated automatically by said personal mobile device if said personal mobile device has moved away more than a predetermined distance from one of a predetermined location, area and path.

5. Method according to claim 2 or 3, wherein said second event is a command generated automatically by said personal mobile device if said personal mobile device is not within a predetermined distance from one of a predetermined location, area and path in a predetermined time interval.

6. Method according to claim 2 or 3, wherein said second event is a command generated automatically by said personal mobile device if said personal mobile device is within a predetermined distance from one of a predetermined location, area and path. 29

7. Method according to any of the preceding claims, wherein said first event is a command received from a user of said personal mobile device.

8. Method according to any of the preceding claims, wherein a length of intervals for collecting said track data and environmental data can be varied.

9. Method according to claim 8, wherein the length of the intervals for collecting said track data and environmental data have a relationship with a safety level indication of an area along said track.

10. Method according to claim 9, wherein the safety level indication is related to at least one of:

• amount of roads of a low functional road class in the area, which road class feature indicates importance of a road;

• number of cafes and restaurants in said area;

• number of points of interest in said area;

• number of shops in said area;

• width of a road; • presence of a park in said area;

• presence of a metro station, railway station, bus stop or tram stop in said area;

• presence of street lighting in said area;

• number of crimes in said area per unit of time;

• number of accidents in said area per unit of time; • time of day, day of week, week of year;

• weather conditions in said area;

• traffic conditions in said area;

• movement of the personal mobile device;

• specific location, area or path; • specific location in combination with a time window;

• specific area in combination with a time window;

• specific paths in combination with a time window. 30

11. Method according to any one of the preceding claims, wherein said environmental data relates to at least one of audio and video data.

12. Method according to claim 11, wherein said method includes generating association data to associate the position and time information with said at least one of said audio and video data.

13. Method according to any one of the preceding claims, further comprising, if the data stored in the memory satisfies a predefined condition, deleting a part of the stored track and environmental data from the memory.

14. Method according to claim 13, wherein at least environmental data relating to a predetermined time period is kept in said memory.

15. A personal mobile device comprising a processor readable storage medium, a processor in communication with said processor readable storage medium, said processor being arranged to communicate with other devices, said other devices including at least:

• a position sensor for sensing a position of said personal mobile device; • an environmental data sensor for sensing and storing environmental data relating to an environment of said personal mobile device; wherein the processor readable storage medium comprises instructions, when loaded on the processor allow the processor to:

• upon sensing a first event, collect said track data representative of the track of a person including at least position and time information of the personal mobile device along the track, as well as said environmental data of an environment along said track;

• storing the track data and environmental data in a memory, the memory being either an internal memory of said personal mobile device or a memory external to said personal mobile device. 31

16. Personal mobile device according to claim 15, said instructions, when loaded on the processor, allowing the processor, upon sensing a second event, to provide the stored data from said memory to an observation station

17. Personal mobile device according to claim 16, said instructions, when loaded on the processor, allowing the processor, after said second event, to collect and send to the observation station continuing track data representative of the actual track of the personal mobile device including at least position information, as well as continuing environmental data.

18. Personal mobile device according to claim 16 or 17, said instructions, when loaded on the processor, allowing the processor to generate a command as said second event if said personal mobile device has moved away more than a predetermined distance from one of a predetermined location, area or path.

19. Personal mobile device according to claim 16 or 17, said instructions, when loaded on the processor, allowing the processor to generate a command as said second event if said personal mobile device is not within a predetermined distance from one of a predetermined location, area and path in a predetermined time interval.

20. Personal mobile device according to claim 16 or 17, said instructions, when loaded on the processor, allowing the processor to generate a command as said second event if said personal mobile device is within a predetermined distance from one of a predetermined location, area and path.

21. Personal mobile device according to any of the claims 15-20, wherein said first event is a command received from a user of said personal mobile device.

22. Personal mobile device according to any of the claims 15-21, wherein said position sensor is a GPS receiver, and the environmental data sensor comprising at least one of a camera, a microphone, a laser scanner and a radar unit. 32

23. Telecommunication system comprising a personal mobile device according to any of the claims 15-22, and an observation station for receiving said track data and environmental data.

24. Telecommunication system according to claim 23, wherein said position sensor is a mobile network station providing a cell ID.

25. Telecommunication system according to claim 23 or 24, wherein said memory is an external memory of an external server which is also part of said telecommunication system.

26. Computer arrangement comprising a further processor arranged to communicate with a further processor readable storage medium storing a computer program comprising instructions and data that can be processed by the further processor, and storing track data and environmental data as collected by the personal mobile device according to any of the claim 15-22, the computer program being arranged to enable the further processor to reproduce the track data and to superimpose the track data over a map generated from a map database, wherein the map corresponds to position information in the track data.

27. A computer program product comprising instructions, which when loaded on a personal mobile device according to any of the claims 15-22, allow said personal mobile device to perform any one of the methods according to claims 1-14.

28. A processor readable medium carrying a computer program product according to claim 27.