EP2766702A2

EP2766702A2 - Miniaturized optoelectronic system for spectral analysis

Info

Publication number: EP2766702A2
Application number: EP12816009.0A
Authority: EP
Inventors: Nico Correns
Original assignee: Individual
Current assignee: Individual
Priority date: 2011-10-12
Filing date: 2012-10-12
Publication date: 2014-08-20
Also published as: DE102011084348A1; WO2013053876A3; WO2013053876A2

Abstract

The invention relates to a miniaturized optoelectronic system for producing static or moving images of scenes or individual objects (11) and for determining and evaluating the spectral properties of the objects (11) within a scene or of individually imaged objects (11). According to the invention, a system of this type comprises optical components and beam paths for producing static of moving images of a scene or of an individual object (11), optical components and beam paths for determining the spectral properties of one or more objects (11) contained in the scene or of the individual object (11), at least one image sensor as an optoelectronic converter, electronic components for processing the output signals of the image sensor, an information output unit designed to present the results in a sensory manner, preferably for presenting the spectral properties in association with the objects (11) in a visually perceptible manner, and means for supply power to electronic components, wherein a design of the system in the form of a hand-held device, also as a handheld, is provided for.

Description

title

Miniaturized opto-electronic system for spectral analysis

Field of the invention

The invention relates to a miniaturized optoelectronic system for generating static or moving images of scenes or individual objects and for determining and evaluating the spectral properties of the objects within a scene or of individually imaged objects.

State of the art

A study is known about commercial devices in the form of mobile phones, which are equipped with microscopic imaging properties as well as with the ability to spectral analysis of an object to be examined. This study is published under "Cell-Phone-Based Platform for Biomedical Device Development and Education Applications; Zachary J. Smith at all; Center for Biophotonics Science and Technology, University of California Davis, Sacramento, California, USA.

Here, a mobile phone was coupled with a single-channel spectrometer, which has a bandwidth of 300 nm and a spectral resolution of less than 5 nm. The determination of white light transmission spectra and fluorescence spectra is thus possible. The use of this device combination, referred to as a mobile phone spectrometer, is provided in connection with medically relevant applications. The device base used was a GSM mobile phone suitable for media playback, which has a video camera, mobile Internet access with e-mail and GPS cards and is currently known under the trade name iPhone.

Furthermore, US 2010/0309454 A1 describes a compact spectrometer which is coupled to a mobile telephone or integrated in a miniaturized version in a mobile telephone. imp- The power supply, data memory, signal processing and real-time display of the measurement results are also indicated. The results can be communicated immediately via wireless communication to a remote station or to other mobile phones or their users. For example, for diabetics, it is possible to perform blood glucose measurements in a non-invasive manner at home and to transmit the results to the doctor.

A disadvantage of the aforementioned arrangements is that a comprehensive and precise determination of the spectral properties of imaged scenes or objects in real time is not possible and the output of inferential information, for example in the sense of recommendations for action for the user, which relate to the determination and evaluation results , not intended or - measured against existing needs - only limited and therefore insufficiently possible.

Description of the invention

The invention described below is based on the object to remedy the deficiencies inherent in the prior art.

According to the invention, a miniaturized optoelectronic system of the type mentioned in the introduction comprises

optical components and beam paths for generating static or moving images of a scene or a single object,

optical components and beam paths for determining the spectral properties of one or more objects contained in the scene or a single object, - at least one image sensor as an optoelectronic transducer,

electronic components adapted for processing the output signals of the image sensor,

an information output unit, designed to display the results sensually, preferably visually perceptible representation of the spectral properties in association with the objects, and

Means for powering the electronic components,

wherein an embodiment of the system in the form of a hand-held device, also referred to as a handheld, is provided. A handheld or handheld device according to the present invention is a portable electronic device with its own power supply for different applications. It is so small and light that it can be held in use in only one hand, hence the term Handheld, held by English in the hand (cited from Wikipedia, 31.08.201 1). As beam paths in the sense of the present invention, the paths of the light through the optical part of the system are to be understood. In different embodiments of the system according to the invention, the beam paths for image generation and the beam paths for determining the spectral properties run at least in sections separately or together, and the image generation and the spectral analysis are provided simultaneously or in chronological succession.

In a preferred embodiment, the optical means for determining the spectral properties are designed as multichannel spectrometers and intended to determine spectrums for several wavelengths of the illumination and / or measurement light independently of one another, preferably simultaneously with the generation of images of the scenes or objects, whose spectral properties are to be determined. Included in the inventive concept is the training in the form of a hyperspectral multi-channel spectrometer, which is suitable for spectral analysis in the wavelength range from ultraviolet to long-wave infrared and is provided in each of the channels with up to 250 different, each corresponding to a wavelength or a wavelength range support points.

Optionally, it may be provided as an optoelectronic converter

a polychromatic silicon based image sensor,

a polychromatic indium gallium arsenite (InGaAs) based image sensor,

a plurality of monochromatic silicon or indium gallium arsenide based single sensors cooperatively covering a particular spectral wavelength range to be analyzed, or

Combinations of silicon-based and / or indium-gallium-arsenide-based polychromatic and monochromatic line or area sensors which also cover a specific wavelength range to be spectrally analyzed.

The system according to the invention is embodied in special embodiments for producing two- or three-dimensional images of scenes or objects, and the spectral analysis is carried out for one or more individual lines, for an area or for a volume of an object selected from a scene or individually recorded.

The generation of three-dimensional images is preferably carried out using the known TOF = Time Of Flight principle. This is particularly advantageous for applications that require a time-resolved spectral analysis. In this case, a photonic mixer, also referred to as a PMD sensor, is advantageously used as the image sensor. The means for imaging and the means for determining the spectral properties are either housed within a common compact housing or designed as separate modules. For both options embodiments are provided, which allow the one-handed holding and operating due to their size and weight.

In the case of execution in separate modules they are coupled to each other via interfaces. Advantageously, a dispersive optics, for example in the form of an optical grating, prism or filter, are arranged upstream of the optical elements for image generation, in particular an objective belonging to the imaging beam path.

In order to achieve a compensation of the image blurring, which arises due to the dispersion in the direction of the Z-axis in accordance with the Rowland circle, as well as to compensate for further image blurring in planes which are inclined to the plane spanned by the Z-axis and the dispersion direction , Image stacks are taken from different focal planes, evaluated the individual images of these stacks with respect to the image sharpness of preferably relevant pixels and extracted from the results images of the scene or objects whose image sharpness is corrected.

The focus variation is made, for example, by stepwise changing the distance between the lens and the focal plane of the dispersion element. With appropriate programming, an existing autofocusing device can be temporarily used for this purpose. Alternatively, the use of DOE ^' s or other optically correcting elements is conceivable in order to compensate the image blur.

For illuminating the scene or the object to be recorded and / or for spectral analysis, the respective natural ambient light or the light emitted by an artificial source is provided. The coupling of the measuring light coming from the scene or the object into the beam paths for the spectral analysis takes place, for example, by means of free-ray optics or optical waveguides. Within the scope of the invention is also the separate or integrated equipment of the opto-electronic system with an illumination light source that emits fluorescence exciting light.

If artificial light sources are used, they are preferably connected to means for the defined influencing of the intensity, the wavelength and / or the polarization of the emitted light. The use of polarization elements enables the processing of phase information, which is particularly useful in spectral ellipsometry for the investigation of Layer thicknesses and optical properties of different materials is advantageous. In this case, polarized light is irradiated onto the sample, and based on the amplitude and phase information or the ellipsometric parameters Y and D of the reflected beam is concluded that the polarizing properties of the sample.

In a further embodiment of the invention, the generation of instructions for action for the user is provided, which are generated in the form of conclusions from the results of the spectral analysis. Included in the concept of the invention is the additional linking of the results of the spectral analysis with further information relating to scenes and objects, such as metadata. These are either stored internally in the system, stored in an external data store assigned to the system, or made publicly accessible by communication and retrievable with the system.

With regard to the latter, the optoelectronic system according to the invention is equipped, for example, with means for wireless integration into a communications network, in particular for image and sound transmission, for transmission of measurement results to external data memories and / or for retrieving information from external data memories. Such information is for example reference values for the calibration of the means for image generation, but preferably for the calibration of the means for spectral analysis. In particular, the calibration of the intensity and / or the wavelength of the illumination light can be provided.

In a further embodiment, which is likewise within the scope of the invention, the optoelectronic system according to the invention is equipped with means for guiding the user in the search for such objects within arbitrary scenes which have certain spectral properties.

This route guidance is achieved by determining, for example by means of GPS and / or compass, the spatial orientation of the imaged scene or of the imaged object and correlating it with the temporal gradient of a change arbitrarily or deliberately caused by the user to the measured absolute or relative spatial spectral information becomes. The return information thus obtained to the user allows the route guidance of the system to areas that are relevant to the user in connection with the respective application.

Further information supporting the context of the measurement, such as temperature, is taken into account in the calculation of the gradient space information. The real-time feedback of the measurement information takes place, for example, in the display by means of gratings, arrows, false-color areas offering support points, or also by means of haptic instant information to the user, for example acoustically and / or in the form of vibrations. The feedback to the user does not necessarily have to be in the same place, that is to say the user can receive the information spatially separated via the communication interfaces of the system according to the invention, such as TCP / IP, Bluetooth etc.

In connection with the route guidance, for example, the linking of objects that have been determined to be interesting on account of their spectral properties, with image-covered terrain forms, is also provided for certain applications.

The information output unit has, for example, an LED or OLED display for visually perceptible presentation of results.

In a further advantageous embodiment, the means for image generation are designed as a light field camera system. Light field camera systems are known as plenoptic cameras. They detect the 4D light field of a scene, which not only knows the position and intensity of a light beam incident on the image sensor, but also its direction of incidence. For light field measurement, for example, a grid of a plurality of microlenses is arranged in front of the image sensor. Thus, even without autofocusing, very large depth of field shots are advantageously achieved with the least possible image blur.

The image-forming means, preferably embodied as a camera, advantageously have a sensor matrix, for example embodied in the form of a CCD image sensor, which is connected to an interface for temporarily combining adjacent pixels to form pixel blocks in order to achieve a higher photosensitivity and at the same time a higher read-out speed. This control can be made spatially and / or temporally staggered with respect to the pixels of the image sensor.

Within the scope of the invention, the equipment of the optoelectronic system with acceleration sensor, gyroscope and inclinometer as well as means for integration into the Global Positioning System (GPS) are optional as well. By means of the inclinometer and GPS, for example, the determination of the angle of incidence of the measuring and / or illumination light on an object to be imaged is provided in relation to the optical axis of the imaging optics.

In the context of the present invention, the term "illumination light" is intended to mean the light directed onto the scene or the object under the term measuring light that of the scene or the object coming and corresponding to be evaluated by the respective application light to be understood.

If artificial sources are provided for the illumination or measuring light, they can optionally be operated continuously or pulsed. If the system according to the invention is operated as a spectral video system, the sequence in which the moving images are recorded must be synchronized with the pulse frequency of the light source in such a way that the image is always taken when the scene or object is illuminated, so that periodically repetitive processes or periodically recurring events are made visible, for example, in order to be able to detect temporally progressive changes.

In a further advantageous embodiment, means are provided for classifying imaged objects on the basis of their spectral properties, the classification being carried out for example under chemically, medically or agronomically revealing aspects.

The miniaturized optoelectronic system according to the invention offers significant advantages in contrast to the systems available according to the prior art. Thus, according to the invention, a superimposition of the live image can be carried out with a spectral measurement in which the sequence of image acquisition is greater than the sequence with which the spectral analysis takes place. The image sensor is used in some areas for the spectral measurement and partially for the live image. In this regard, two cameras are preferably provided in the system, of which one of the spectral measurement and the second is reserved for the live image. In the live image - recognizable for the user - a marking of the image portion is made in which the spectral measurement takes place,

a superimposition of the live image are made with a spectral measurement in which the sequence of image acquisition corresponds to the sequence with which the spectral analysis is performed. Here, the image and spectral information recorded in real time are processed and processed in the same image sequences. This offers the advantage that time-critical or individually changing processes can be analyzed in detail and the processes can be adapted to the physiological conditions of humans, which can perceive a refresh rate of 16 Hz, a superimposition of the live image can be made with a spectral measurement in which the sequence of image acquisition is smaller than the sequence with which the spectral analysis is performed. This is advantageous if the measurement object is spectrally inhomogeneous.

A large number of individual measurements during a smaller number of image generations improves the accuracy of the overall measurement. The system according to the invention can be operated as a spectral video stroboscope system, wherein a synchronization to the operating frequency of the existing light sources for the purpose of time allocation of lighting and recording is provided. Alternatively, applications using the time sequence are useful, in which the dynamic history is to be visualized.

For applications in connection with time-resolved spectroscopy, the system according to the invention is additionally equipped with a shutter, which preferably has a reaction time in the nanosecond range. The shutter serves to link the beam path for image generation with the beam path for spectral measurement and makes it possible to perform spectral measurement and image generation sequentially in time with the shutter control, which is advantageous in terms of time-resolved spectroscopy, especially in the case of rapid temporal changes to the measurement objects. If the optoelectronic system according to the invention is equipped with a 3D camera, which is preferably provided with the time-of-flight function, the spectral measurements can be carried out with time resolution. If a spectroscopic intent is used for the spectral measurement, as explained in greater detail below on exemplary embodiments, the distance determination to the object is nevertheless possible. Furthermore, it is thereby possible to measure transit time differences for the measurement of information from the temporal signal behavior of certain substances and applications and to provide them for further processing, such as, for example, the so-called FLIM technique (Fluorescence Lifetime Imaging Microscopy), the ion-sensitive fluorescent dyes for measuring intracellular ion concentrations used. Here, the fact is used that the fluorescence lifetime, that is, the average residence time of the electrons in the excited state, changes with the ion concentration.

As already partially mentioned above, ambient light or artificial light can be used to illuminate the scenes or objects to be recorded. The source of artificial light may be internal light sources, with optional optics, or external light sources that may also be pulsed, for example stroboscopic flashlights or TOF based 3D equipment.

The following variants are within the scope of the invention:

Use of the display of the information output unit as lighting,

Use of one or more monochromatic light sources, which serve as support points for the spectral measurements, for parallel or serial illumination, optionally also in time with the recording of moving images. The spectral resolution of the Total system can be realized by the number of light sources the entire spectral wavelength range - even outside the RGB color space - covers, use of a light source for fluorescence excitation in planar illumination, use of natural light sources for fluorescence spectroscopy excitation, - temporally successive illumination of the scene or of the object with light of different optical properties, such as polarization, intensity, wavelength or spectral range,

Use of natural ambient light with the aid of one or more polarizing elements for illumination,

Modulation of the natural, internal or external illumination, e.g. in connection with time-resolved applications,

Control and / or regulation of the spectral and temporal intensity of the illumination light,

diffuse illumination, for example by means of an interposed microlens array or inserted plane mirror optics, in order to avoid energy or light losses,

individual programming of the existing light sources with regard to the intensity, wavelength and / or polarization of the emitted light,

Using adaptive optical elements in the beam paths of the illumination and / or measurement light to influence the wavefront and thereby correct, for example, aberrations of the participating optical assemblies.

In order to calibrate the intensity of the illumination light, it is provided, for example, to decouple a part of the light coming from the light source and to couple it as light with a reference spectrum directly into the spectroscopic beam path. This type of referencing can preferably be done in parallel, but also serially.

In the case of the use of a multi-channel spectrometer, the reference is "white" on one channel during parallel referencing, and the measurement information on the other channels.From a region of the image sensor to which no light falls, the reference "black" is obtained. Depending on the applicative task to be solved, however, the measurement of an additional dark signal can also be provided. The referencing preferably takes place simultaneously and with each image.

In the case of using a single-channel spectrometer, these processes are performed serially. In this regard, DE 195 28 855 A1, for example, describes a device in which renewed referencing between the measurements is possible with little effort. For this purpose, a separate reference beam path is used, and the measurement beam path is merged via a Y light guide with the reference beam path, wherein in each optical fiber branch a switchable shutter is arranged. The Y-light guide with shutter serves as an optical switch, and the cross section of the common beam path between the switch and the spectrometer is split between the reference beam and the beam path. This common beam path is coupled into the inlet opening of the spectrometer.

A second camera, for example with an installation position with respect to the detection or light entry direction offset by an angle of 180 ° to the first camera, can be used for reference measurement for the intensity of the ambient light, regardless of which light sources for object or Scene lighting are provided. Alternatively, however, a use of the optoelectronic system according to the invention is also possible without a white reference, which - although not detrimental for certain applications - reduces the accuracy of measurement, but at the same time advantageously reduces the production costs.

In order to calibrate the wavelength of the illumination light, it is provided, for example, to subject the means for determining the spectral properties to a one-time calibration immediately after the system has been manufactured. If necessary, or each time a predetermined period has elapsed, then e.g. be recalibrated with designed as line sources light sources. For this purpose, a selection is made manually or automatically from a data memory in which the parameters of a plurality of line sources are stored.

The transmission of information, such as spectral values, image data, video sequences u.a. takes place between the individual functional groups or modules via interfaces within the system or bidirectionally by wired or wireless communication with external units, including subordinate evaluation routines and taking into account the application-related use.

The feedback to the user is realized by the following procedures:

Spectral information with, for example, chemometrically relevant statements are coupled with other information and represented as "augmented reality", which is to be understood in the sense of the present invention as the computer-aided extension of the perception of reality, which includes all human sensory perceptions

The information can be used for machine vision, and in the context of the invention all forms of computer-aided solution of tasks can be considered which are based on the capabilities of the human visual

Systems, from communication to haptic perception eg for the blind. The information output to the user takes place in an application-related advantageous manner so that unwanted disturbances of the scene to be recorded or of the object to be recorded, for example by acoustic signals, are avoided. The communication between users of the optoelectronic systems according to the invention is optionally provided in addition to the usual visual communicative possibilities by means of bidirectional acoustic communication. The use of external sources of information via a server and the merger or networking of several of the optoelectronic systems according to the invention are also possible.

When operating the optoelectronic system, the light coupled out of the observation beam path of an imaging optics for the micro, macro, near and far ranges is transmitted into the beam path for determining the spectral properties of one or more objects contained in the scene to be imaged, with the aim of obtaining the spectral information to determine and evaluate.

Unknown spectral properties of the object are measured and classified. The spectral identification can be used for identical replication, that is to say for the storage of the data relating to the spectral properties obtained at several different locations and synchronization of these data sources.

The invention will be explained in more detail with reference to embodiments. The accompanying drawings show in

1 shows the optoelectronic system according to the invention in a first embodiment, consisting of a first, designed as a spectrometer intent assembly and a second module with the functions of a smartphone, in the camera of the smartphone only the coming of an object light passes, the Spectrometer intent has happened

2 shows the basic illustration of a second exemplary embodiment, comprising a spectrometer intent, a smartphone having two cameras, an internal light source for object illumination and means for referencing the illumination light,

Figure 3 shows the schematic diagram of a third embodiment, formed from a spectrometer-intent and a smartphone with a camera, which comes directly into the imaging optics of the smartphone light coming from the object and at the same time separate light that has passed the spectrometer intent, and wherein the light entering the imaging optics is separated within the smartphone or directed together to an image sensor,

4 shows the basic illustration of a fourth exemplary embodiment, consisting of two spectrometer attachments and a smartphone, which has two cameras, wherein an external natural or artificial light source for scene or object illumination is provided in the imaging optics of a first camera directly from the object coming light and light that has passed the first spectrometer intent, while in the second camera only light enters the second spectrometer intent has happened,

5 shows the schematic representation of a fifth embodiment, consisting of a spectrometer intent and a smartphone having two cameras, both cameras, in contrast to the examples of Figure 2 and Figure 4 have the same light incident direction, in the first camera directly light coming from the object passes, and only light which has passed the spectrometer intent is passed into the second camera.

6 is a schematic diagram of a sixth exemplary embodiment, consisting of a smartphone with a camera and an external spectrometer, with light coming directly from the object into the camera and into the spectrometer,

7 shows the basic representation of a further exemplary embodiment, consisting of a smartphone with a camera and an integrated spectrometer, with light coming directly from the object into the camera and into the spectrometer in each case.

1 shows the basic structure of a first embodiment of the optoelectronic system according to the invention. A multichannel prism spectrometer, designed in the form of a spectrometer intent 1, has three channels 2, 3 and 4 by way of example, with which three different spectra of the measuring light beam 5 coming from an object 11 can be measured simultaneously. Essentially, the spectrometer comprises light columns S2, S3 and S4 associated with the individual channels, lens groups with collimator function L1 .2, L1 .3 and L1 .4, lens groups L2.2, L2.3 and L2.4 with telescope function and a prism P as a dispersive optical element.

In Fig.1a, a plan view of the said and symbolically indicated functional elements of the spectrometer, the lying in the plane of adjacent channels 2, 3, 4 can be seen. 1b shows the same representation in a side view, in which the channels 2, 3, 4, concealing each other, lie behind one another.

When operating this arrangement, the measuring light enters the light gaps S2, S3, S4 at the same time, passes through the channels 2, 3, 4, exits via an interface 5 from the spectrometer attachment 1 and subsequently into a camera which, for example, substantially an objective O, a zoom optics Z and a spatially resolving image sensor B. The camera is preferably part of a smartphone 6.

For the purposes of the present invention, the term "smart phone" is to be understood as meaning devices which have the functions and functional components of a mobile telephone with a PDA (personnel Digital Assistant). They are equipped with at least one digital camera, GPS and WLAN receivers and internal power supply, and in addition to the processor have several data memories, for example a very fast but volatile main memory (RAM), a non-volatile NAND flash memory for the operating system and programs. me, and optional removable media, for example in the form of mass storage. In addition to the operating system, the user can install application programs. The synchronization of data with external data storage and data processing equipment is possible.

In this embodiment of the optoelectronic system according to the invention, only the light coming from the object 11 which has passed the spectrometer intent 1 passes into the camera of the smartphone 6 as measurement light. The image generation and the spectral analysis are carried out immediately in time. In contrast, the spectral analysis or the measurement of the three different spectra is carried out simultaneously using three different areas B1, B2 and B3 of the image sensor B. Alternatively, the detection of the different spectra with separate sensors is possible and is within the scope of the invention.

2 shows an exemplary embodiment of the optoelectronic system according to the invention, comprising a dual-channel spectrometer front attachment 1 and a smartphone 6. For putting, the smartphone 6 has two cameras 7 and 8. The cameras 7, 8 have 180 degrees offset from each other and thus opposite light incidence directions. Both cameras 7, 8 are commercially equipped with the required for generating static or moving images of a scene or a single object optical components and beam paths. Furthermore, in this exemplary embodiment, the optoelectronic system according to the invention is equipped with an internal light source 9.

The camera 7 is preceded by the spectrometer intent 1, the camera 8 is used for image generation. The light coming from the light source 9 is via a first beam path 10 as illuminating light to be imaged and spectrally analyzed object 1 1 and at the same time via a second beam path 12 as a reference light by means of mirror 13 in the spectrometer intent 1 and this subsequently in the camera. 7 directed. The light coming from the object 1 1 passes as measuring light over a beam path 14 in the spectrometer intent 1 and at the same time as imaging light over a beam path 15, for example by means of mirrors 16 and 17 deflected into the second camera 8. The image generation and spectral analysis are provided at the same time.

The optoelectronic system in one exemplary embodiment according to FIG. 3 consists of a spectrometer intent 1 and a smartphone 6 with a camera 7. Ambient light is used to illuminate the object 11. The measuring light coming from the object 1 1 passes via a beam path 18 in the spectrometer intent 1 and then into the camera 7 and at the same time as imaging light over a beam path 19 directly into the camera 7. Here, the beam paths for spectral analysis and the beam paths for image production run separately, and the image generation and spectral analysis are at the same time by means of a flat image sensor. The light entering the imaging optics is separated within the smartphone or directed together at the image sensor.

4 shows a further embodiment in connection with a smartphone 6, which in turn has two cameras 7 and 8 with opposite light incidence directions. The Kal o mera 7 is preceded by a first spectrometer intent 1, which is used for spectral analysis, while the camera 8, a second spectrometer intent 20 is disposed upstream, which is used for referencing the illumination light. To illuminate the spectrally analyzed object 1 1, a natural or artificial light source 21 is provided.

15 The illumination light is directed via a beam path 22 through the spectrometer intent 20 in the camera 8 and at the same time on a beam path 23 to the object 1 1. The measuring light coming from the object 11 is, as in FIG. 3, directed via a beam path 18 through the spectrometer intent 1 into the first camera 7 and at the same time directly into the camera 7 via a beam path 19. The beam paths for spectral analysis and the

20 beam paths for image generation run separately, the image generation and the spectral analysis are carried out simultaneously by means of a flat image sensor, of which separate surface sections are used for spectral analysis and image generation.

The basic illustration of the optoelectronic system according to FIG. 5 shows a further embodiment with a smartphone 6 which has two cameras 7 and 8. Both cameras 7, 8 have here - unlike in the examples of Figure 2 and Figure 4 - no opposite directions of light incidence. To illuminate the object 1 1 ambient light is used. The measuring light coming from the object 11 is directed via a beam path 18 through the spectrometer intent 1 into the first camera 7 and at the same time via a beam path 19 into the second camera 8. The beam paths for spectral analysis and the beam paths for image generation run separately, and the image generation and the spectral analysis are carried out simultaneously by means of a flat image sensor or two flat image sensors. In the case of using an image sensor, separate area sections are used for spectral analysis and image generation. If two image sensors are used, one of them serves for spectral analysis, the second for image generation.

FIG. 6 shows an exemplary embodiment in which the optoelectronic system according to the invention is formed from a smartphone 6 with a camera 7 and an external spectrometer 24. To illuminate the object 1 1, an artificial light source 25 is provided. The illumination light is directed onto the object 11 via a beam path 26. Via a beam path 18, light coming from the object passes into the spectrometer 24, light coming from the object passes into the camera 7 via a beam path 19. The coupling of the spectrometer 24 to the smartphone 6 takes place via cable 27.

7 shows the basic illustration of a further exemplary embodiment, consisting here of a smartphone 6 with a camera 7 and an integrated spectrometer 28. Via the beam path 18, light coming from the object passes into the spectrometer 28, light coming from the object arrives via the beam path 19 the camera 7. The beam paths for spectral analysis and the beam paths for image generation run separately, and the image generation and the spectral analysis are carried out simultaneously by means of a flat image sensor or two flat image sensors. In the case of using an image sensor, separate area sections are used for spectral analysis and image generation. If two image sensors are used, one of them serves the spectral analysis, the second of the image generation.

In all embodiments, the weight of the optoelectronic system according to the invention is a maximum of 1 kg, so that in this respect, the conditions of a hand-held device are met. The use of the optoelectronic system according to the invention takes place e.g. for the purpose of color determination in the context of

Color sensors for monitors, printing technology, video or digital projectors for the correction of color values, or

Color sensors for building applications for color determination of surfaces, objects, etc. and subsequent controlled color matching in various applications.

The data collected will continue to be used for ingredient determination in connection with

Reflection-based pulse oximeters,

- photometers for point-of-care (POC) applications,

wet-chemical coupling to photometers,

forensic applications,

remote-optical explorations coupled with spectral measurements,

micro-optical observations in conjunction with spectral measurement,

- military applications, eg visualization of warfare agents, camouflage, etc., collection of food information regarding safety, condition, nutritional value, etc., agronomic applications such as chlorophyll determination, plant status, rating, soil information, etc., where the reflection properties of derive information on vegetation indices and the health status of plants, eg

Skin protection detectors for the purpose of warning of health-damaging UV radiation instead of the known systems based on intensity measurements. The spectrometer-based measuring system is used to differentiate the UV-A, UV-B and UV-C components and separate evaluation of the individual and sum signals over the course of time, thermography detectors, also connected to thermometer applications,

Fire and smoke detectors, where the flames are detected due to their characteristic frequency and spectral radiation. Fire gases are detected by comparing a picture or spectrum of the same image with smoke. This can also be done in the NIR area so as not to affect the user. The alarm message is preferably carried out on site acoustically or by radio.

An essential component of the inventive idea is the link with metadata such as time, location, angle of view, the high degree of mobility and the coupling in the feedback as an augmented reality system.

A particularly advantageous application consists in the route guidance of the user when searching for objects with specific spectral properties within arbitrary scenes.

LIST OF REFERENCES

1 spectrometer attachment

2 channel

5 3 channel

4 channel

5 interface

6 smartphone

7 camera

10 8 camera

9 light source

10 ray path

1 1 object

12 ray path

15 13 mirrors

14 ray path

15 ray path

16 mirrors

17 mirrors

20 18 ray path

19 ray path

20 spectrometer attachment

21 light source

22 ray path

25 23 Beam path

24 external spectrometer

25 light source

26 ray path

27 cables

30 28 internal spectrometer

B2, B3, B4 areas on image sensor

L1 .2, L1 .3 L1 .4 lens groups

L2.2, L2.3; L2.4 lens groups

35 O lens

P prism

S2, S3, S4 light column

Z zoom optics

Claims

claims

1 . Miniaturized optoelectronic system comprising

optical components and beam paths for generating static or moving images of a scene or a single object (1 1),

Optical components and beam paths for determining the spectral properties of one or more objects contained in the scene (1 1) or the individual object (1 1),

at least one image sensor as optoelectronic transducer,

an information output unit for displaying results in association with the objects (1 1), and

Means for powering the electronic components

wherein an embodiment of the system is provided in the form of a hand-held device.

2. Miniaturized optoelectronic system according to claim 1, wherein

the image generation and the spectral analysis are provided simultaneously or temporally successively, wherein

the respective beam paths separated or run together.

3. Miniaturized optoelectronic system according to claim 1 or 2, wherein the means for determining the spectral properties are designed as multi-channel spectrometer, preferably for the determination of spectra simultaneously for several wavelengths of the measuring and / or illumination light simultaneously with the image formation.

4. Miniaturized optoelectronic system according to claim 3,

equipped with a hyperspectral multichannel spectrometer, and

designed for analyzing the image data obtained with up to 250 different color channels in the wavelength range from ultraviolet to long-wave infrared, wherein the use of a wavelength range detecting silicon based image sensor and / or the use of multiple monochromatic, a predetermined wavelength range detecting individual sensors is provided.

5. Miniaturized optoelectronic system according to claim 4, wherein the generation of two- or three-dimensional images of the scene or the object (11), and

the determination of the spectra based on one or more individual lines, on the surface or on the volume of the respective object (1 1) is provided.

6. Miniaturized optoelectronic system according to claim 5, wherein the generation of three-dimensional images on the basis of the TOF = Time Of Flight principle is provided.

7. Miniaturized optoelectronic system according to one of the preceding claims, wherein the means for image generation and spectral analysis

housed within a common housing, or

housed in separate modules and coupled to each other via interfaces, wherein the means for image forming a dispersive optics, preferably an optical grating (5), prism or filter, is arranged upstream.

8. Miniaturized optoelectronic system according to claim 7,

designed to use the natural light as a lighting and measuring light, or equipped with an artificial source for illumination and measuring light, wherein free-beam optics or optical waveguides are present for coupling the measuring light into the beam paths for determining the spectral properties.

9. Miniaturized optoelectronic system according to claim 8, with a fluorescence exciting light emitting light source.

10. Miniaturized optoelectronic system according to claim 8 or 9, wherein the artificial light sources are connected to a drive circuit for influencing the radiation intensity, the wavelength and / or the polarization of the emitted light.

1 1. Miniaturized optoelectronic system according to one of the preceding claims, equipped with means for generating instructions for the user to act as a result of the evaluation of the determined spectral properties, or

as a result of additional linking of these evaluation results with other information concerning the scenes and objects (11).

12. Miniaturized optoelectronic system according to claim 1 1, equipped with means for guidance of the user in the search for objects (1 1) with certain spectral properties within arbitrary scenes.

13. Miniaturized optoelectronic system according to one of the preceding claims, equipped with means for wireless integration into a communication network, in particular designed for image and sound transmission, for transmitting measurement results in external data storage and / or retrieving information from external data storage, preferably reference values for calibrations of at least the optical imaging or spectral analysis means.

14. Miniaturized optoelectronic system according to claim 13, wherein the calibration of the intensity and / or the wavelength of the illumination and / or measuring light is provided.

15. Miniaturized optoelectronic system according to one of the preceding claims, wherein the information output unit comprises a LED or OLED display.

16. Miniaturized optoelectronic system according to one of the preceding claims, wherein the means for image formation are designed as a light field camera system and both the detection of the position and intensity of the light incident on the image sensor and the detection of the direction of incidence is provided.

17. Miniaturized optoelectronic system according to one of the preceding claims, wherein a pulsed illumination or measuring light source provided and the sequence is synchronized in the recording of moving images with the turn-on frequency of the light source.

A miniaturized optoelectronic system according to any one of the preceding claims, wherein the image sensor is connected to a drive for temporarily merging adjacent pixels into pixel blocks to achieve higher photosensitivity.

19. Miniaturized optoelectronic system according to one of the preceding claims, equipped with acceleration sensor, gyroscope, inclinometer and designed for integration in the Global Positioning System (GPS).

20. Miniaturized optoelectronic system according to claim 19, wherein the determination of the angle of incidence of the measurement or illumination light is provided on an object to be imaged (1 1) in relation to the optical axis of the imaging optics by means of GPS and inclinometer. Miniaturized optoelectronic system according to one of the preceding claims, designed to classify imaged objects (1 1) on the basis of their spectral properties.