WO2016018250A1

WO2016018250A1 - Sensor module calibration

Info

Publication number: WO2016018250A1
Application number: PCT/US2014/048600
Authority: WO
Inventors: Jinman Kang; Santiago REYERO
Original assignee: Hewlett Packard Development Company, L.P.
Priority date: 2014-07-29
Filing date: 2014-07-29
Publication date: 2016-02-04

Abstract

Examples of a calibration system, calibration method, and machine-readable non-transitory storage medium including instructions executable by a processor are disclosed herein. An example of the machine-readable non-transitory storage medium includes instructions executable by a processor to normalize first differences between known white point red, green, and blue values of a white balance calibration target and red, green, and blue values of a recorded image of an illuminated white balance calibration target; determine an adjusted exposure setting based on second differences between known red, green, and blue values of each color in a color calibration target and red, green, and blue values of each color of a recorded image of an illuminated color calibration target; and create default calibrated sensor module settings to address unit-by-unit variations in different sensor modules.

Description

SENSOR MODULE CALIBRATION BACKGROUND

[0001] End users appreciate quality images and video. They also appreciate the ability to easily use devices that create such images and video. Designers and manufacturers may, therefore, endeavor to create and provide technology directed toward at least some of these objectives.

BRIEF DESCRIPTION OF THE DRAWINGS

[0002] The following detailed description references the drawings, wherein:

[0003] FIG. 1 is an example of calibration system in accordance with an implementation.

[0004] FIG. 2 is an example of additional elements of the calibration system of FIG. 1 in accordance with an implementation,

[0005] FIG. 3 is another example of additional elements of the calibration system of FIG . 1 in accordance with an implementation,

[0006] FIG. 4 is a further example of additional elements of the calibration system of FIG. 1 in accordance with an implementation,

[0007] FIG. 5 is an example of a calibration method in accordance with an implementation,

[0008] FIG. 6 is an example of additional elements of the calibration method of FIG. 5 in accordance with an implementation.

[0009] FIG. 7 is an example of a machine-readable non-transitory storage medium including instructions executable by a processor in accordance with an implementation. [0010] FIG. 8 is an example of additional instructions in accordance with an implementation that are executable by a processor that may be included on the machine-readable non-transitory storage medium of FIG. 7.

DETAILED DESCRIPTION

[0011] Sensor modules that record still images and/or video may have unit-by-unit variations in their individual components. For example, the illumination sources of sensor modules may vary in the intensity of their light output or the wavelength of the light they emit. As another example, the photosensitive members of sensor modules may differ in their responsiveness to different wavelengths of light. These sensor module unit-by-unit variations can result in visually perceptible differences in the still images and/or video they produce. For example, the colors in these still images and/or video may differ enough so as to be noticeable by and objectionable to at least some end users of the sensor modules or the devices in which they are used. As another example, the contrast of still images and/or video of the same subject taken by different sensor modules or the devices in which they are used may vary enough so as to be perceptible by and a concern to at least some end users of the sensor modules or the de vices in which they are utilized.

[0012] Addressing these technical challenges caused by such sensor module unit by unit variations may assist such end users by providing them with more consistent and visually pleasing images and/or video between different sensor modules and the devices in which these sensor modules may be used. This may be achieved by creating a set of default calibrated sensor module settings that are utilized with different sensor modules, as shown, for example, in FIGs. 1-8. [0013] As used herein, the term "sensor module" represents, but is not necessarily limited to, a photosensitive member and an illumination source that are utilized to record still images and/or video. Examples of a photosensitive member include, but are not limited to, a charge-coupled device (CCD), a complementary_' metal-oxide semiconductor (CMOS), a camera, film, a light- sensitive plate, light sensitive paper, or any combination of the foregoing, Examples of an illumination source include, but are not limited to, a light-emitting diode (LED), a bulb, a tube, a l aser, a reflector, a lens, ambient lighting, or any combination of the foregoing.

[0014] As used herein, the term "processor' represents, but is not necessarily limited to, an instruction execution system such as a computer-based system, an Application Specific

Integrated Circuit (ASIC), a computing device, a hardware and/or machine-readable instruction system, or any combination thereof, that can fetch or obtain the logic from a machine-readable non-transitory storage medium and execute the instructions contained thereon, "Processor" can also include any controller, state-machine, microprocessor, logic control circuitry, cloud-based utility, sendee or feature, any other analogue, digital and/or mechanical implementation thereof, or any combination of the forgoing, A processor may be a component of a distributed system.

[0015] As used herein, the term "distributed system" represents, but is not necessarily limited to, multiple processors and machine-readable non- transitory storage media in different locations or systems that communicate via a network, such as the cloud. As used herein, the term "cloud" represents, but is not necessarily limited to, computing resources (hardware and/or machine readable instructions) that are delivered as a sendee over a network (such as the internet). As used herein the term "memory" represents, but is not necessarily limited to, a device and/or process that allows data and information to be stored thereon for subsequent retrieval by, for example, a processor. [0016] As used herein, the term "machine-readable non-transitory storage medium" represents, but is not necessarily limited to, any medium that can contain, store, retain, or maintain programs, code, scripts, information, and/or data. A machine-readable non-transitory storage medium may include any one of many physical media such as, for example, electronic, magnetic, optical, electromagnetic, or semiconductor media, A machine-readable non-transitory storage medium may be a component of a distributed system. More specific examples of suitable machine-readable non-transitory storage media include, but are not limited to, a magnetic computer diskette such as floppy diskettes or hard drives, magnetic tape, a read-only memory_' (ROM), an erasable programmable read-only memory (EPROM), a flash drive or memory, a compact disc (CD), a digital video disk (DVD), or a memristor.

[0017] As used herein, the term "circuitry" represents, but is not necessarily limited to, an interconnection of elements such as a resistor, inductor, capacitor, voltage source, current source, transistor, diode, application specific integrated circuit (ASIC), processor, controller, switch, transformer, gate, timer, relay, multiplexor, connector, comparator, amplifier, filter, and/or module having these elements that allow operations to be performed alone or in combination with other elements or components. As used herein, the terms "include", "includes",

"including", "have", "has", "having" and variations thereof, mean the same as the terms

"comprise", "comprises", and "comprising" or appropriate variations thereof.

[0018] An example of calibration system 10 in accordance with an implementation is shown in FIG. 1. As can be seen in FIG. 1 , calibration system 10 includes a sensor module 12 to record images. These recorded images may be still images and/or video. Sensor module 12 includes an illumination source 14 to illuminate a white balance calibration target 16 and a color calibration target 1 8 with light 15. White balance calibration target 16 is utilized by calibration system 10 to remove color casts so that objects which appear white to the eye are rendered as white when recorded by sensor module 12. These color casts may appear, for example, to be blue, orange, yellow, green, or another color and may be caused by, for example, illumination source 14. Color calibration target 18 may include a plurality of different colored (e.g., red, blue, green, brown, violet, magenta, yellow, cyan, etc.) patches 20 and is utilized by calibration system 10 to adjust intensities of colors produced by sensor module 12 so that the colors of images of objects captured by sensor module 12 are rendered accurately (e.g., closely match the colors of patches 20 of color calibration target 18 ).

[0019] As can also be seen in FIG. 1 , calibration system 10 additionally includes a white balance estimation engine 22 to measure first differences between known white point red, green, and blue values of white balance calibration target 16 (e.g., the amount or extent of the combination of red color, green color, and blue color in calibration target 16) and red, green, and blue values of a recorded image by sensor module 12 of i lluminated white balance calibration target 16. White balance estimation engine 22 also normalizes these first differences to use with sensor module 12, as generally indicated by double-headed arrow 23.

[0020] As can additionally be seen in FIG. 1 , calibration system 10 also includes an exposure estimation engine 24 to measure second differences between known red, green, and blue values of each color in color calibration target 18 (e.g., color patches 20) and red, green, and blue values of each color of a recorded image by sensor module 12 of illuminated color calibration target 18. Exposure estimation engine 24 also determines an adjusted exposure setting based on the measured second differences to use with sensor module 12, as generally indicated by double- headed arrow 25, [0021] An example of additional elements of calibration system 10 in accordance with an implementation is shown in FIG. 2. As can be seen in FIG. 2, calibration system 10 may include a calibration engine 26 to create default calibrated sensor module settings, based on the normalized first differences from white balance estimation engine 22, as generally indicated by arrow 27, and the adjusted exposure setting from exposure estimation engine 24, as generally indicated by arrow 33, to use with different sensor modules that have unit-by-unit variations. This helps to provide more consistent and visually pleasing images and/or video between such different sensor modules and the devices in which they may be used. These default calibrated sensor module settings may include, but are not necessarily limited to, at least one of the following parameters or settings: white balance, contrast, resolution, sensitivity, color space, aperture, shutter speed, compression, noise reduction, frames per second, metering pattern, the determined adjusted exposure setting, and exposure compensation. Calibration engine 26 may represent any circuitry, processor, executable instructions, application programming interfaces (APIs), machine-readable non-transitory storage medium, or any combination thereof, that creates these default calibrated sensor module settings. For example, as can also be seen in FIG. 2, calibration engine 26 may include a processor 28 and a machine-readable non-transitory storage medium 30 including instructions executable by processor 28, as generally indicated by double-headed arrow 29, to create the default calibrated sensor module settings.

[0022] Another example of additional elements of calibration system 10 in accordance with an implementation is shown in FIG. 3. As can be seen in FIG. 3, calibration system 10 may include a memory 31 to store the normalized first differences from white balance estimation engine 22, as generally indicated by arrow 32, and the adjusted exposure setting from exposure estimation engine 24, as generally indicated by arrow 34. As can also be seen in FIG. 3, calibration system 10 may additionally include a capture engine 36 to record images with sensor module 12, as generally indicated by arrow 38, utilizing the stored normalized first differences from white balance estimation engine 22 and the stored adjusted exposure setting from exposure estimation engine 24 by retrieving them from memory 31 , as generally indicated by arrow 40. Capture engine 36 may represent any circuitry, processor, executable instructions, application programming interfaces (APIs), machine -readable non-transitory storage medium, or any combination thereof!, that records images with sensor module 12 utilizing these normalized first differences from white balance estimation engine 22 and the adjusted exposure setting from exposure estimation engine 24.

|0023] A further example of additional elements of calibration system 10 in accordance with an implementation is shown in FIG. 4. As can be seen in FIG. 4, calibration system 10 may include an iteration engine 42 to minimize the nonnalized first differences from white balance estimation engine 22 to use with sensor module 12, as generally indicated by double-headed arrow 44. Iteration engine 42 may represent any circuitry, processor, executable instructions, application programming interfaces (APIs), machine-readable non-transitory storage medium, or any combination thereof, that performs this minimization. This minimization may be accomplished, for example, as follows. The above-described first differences measured by white balance estimation engine 22 may be based on deviations between the known white point red, green, and blue valu es of white balance calibration target 16 and the red, green, and blue valu es of the image recorded by sensor module 12 of illuminated white balance target 16, Iteration engine 42 may determine whether these deviations have con verged to a minimum. If not, then iteration engine 42 may update any saved white balance setting for sensor module 12 and white balance estimation engine 22 may proceed to normalize these first differences. Iteration engine 42 may then actuate white balance estimation engine 22 to measure new first differences between known white point red, green, and blue values of white balance calibration target 16 and red, green, and blue values of a newly recorded image by sensor module 12 of illuminated white balance calibration target 16 until these first differences have converged to a minimum, as determined by iteration engine 42.

[0024] White balance estimation engine 22 may represent any circuitry, processor, executable instructions, application programming interfaces (APIs), machine -readable non-transitory storage medium, or any combination thereof that measures these first differences and performs this normalization. For example, as can also be seen in FIG. 4, white balance estimation engine 22 may include a processor 46 and a machine-readable non-transitory storage medium 48 including instructions executable by processor 46, as generally indicated by double-headed arrow 49, to measure the first differences and perform the nonnalization of the first differences to use with sensor module 12.

[0025] As can additionally be seen in FIG. 4, calibration system 10 may also or alternatively include an iteration engine 50 to minimize the measured second differences from exposure estimation engine 24, as generally indicated by double-headed arrow 52. In such cases, exposure estimation engine 24 determines the adjusted exposure setting based on the minimized measured second differences from iteration engine 50 to use with sensor module 12. Iteration engine 50 may represent any circuitry, processor, executable instructions, application programming interfaces (APIs), machine-readable non-transitory storage medium, or any combination thereof, that performs this minimization. This minimization may be accomplished, for example, as follows. The above-described second differences measured by exposure estimation engine 24 may be based on calculated deviations between the known red, green, and blue values of each color in color calibration target 18 (e.g., color patches 20) and the red, green, and blue values of each color the image recorded by sensor module 12 of illuminated color calibration target 18. Exposure estimation engine 24 may determine the adjusted exposure estimation setting based on these calculated deviations. Iteration engine 50 may determine whether these calculated deviations have converged to a minimum. If they have reached a minimum, then iteration engine 50 may actuate exposure estimation engine 24 to save the minimized adjusted exposure setting to use with sensor module 12, If they have not reached a minimum, then iteration engine 50 may actuate exposure estimation engine 24 to measure new second differences between known red, green, and blue values of each color in color calibration target 18 (e.g., color patches 20) and the red, green, and blue values of each color of a newly recorded image by sensor module 12 of the illuminated color calibration target 18 and determine a new adjusted exposure setting for use with sensor module 12 until these second differences have converged to a minimum, as determined by iteration engine 50.

[0026] Exposure estimation engine 24 may represent any circuitry, processor, executable instructions, application programming interfaces (APIs), machine-readable non-transitory storage medium, or any combination thereof that measures these second differences and determines this adjusted exposure setting. For example, as can further be seen in FIG. 4, exposure estimation engine 24 may include a processor 54 and a machine-readable non- transitory storage medium 56 including instructions executable by processor 46, as generally indicated by double-headed arrow 58, to measure the second differences and determine the adjusted exposure setting to use with sensor module 12.

[0027] An example of a calibration method 60 in accordance with an implementation is shown in. FIG. 5. As can be seen in FIG. 5, calibration method 60 begins or starts 62 by measuring first differences between known white point red, green, and blue values of a white balance calibration target (e.g., the amount or extent of the combination of red color, green color, and blue color in the calibration target), such as white balance calibration target 16, and red, green, and blue values of a captured image of an illuminated white balance calibration target, such as white balance calibration target 16, as indicated by block 64. Calibration method 60 continues by normalizing the measured first differences, as indicated by block 66, and measuring second differences between known red, green, and blue values of each color in a color calibration target, such as color calibration target 18, and red, green, and blue values of each color of a captured image of an illuminated color calibration target, such as color calibration target 18, as indicated by block 68. Calibration method 60 additionally continues by determining an adjusted exposure setting based on the measured second differences, as indicated by block 70, and creating default calibrated sensor module settings to use with different sensor modules that have unit-by-unit variations, as indicated by block 72, This may help to provide more consistent and visually pleasing images and/ or video between such different sensor modules and the devices in which they may be used. These default calibrated sensor module settings may include, but are not necessarily limited to, at least one of the following parameters or settings: white balance, contrast, resolution, sensitivity, color space, aperture, shutter speed, compression, noise reduction, frames per second, metering pattern, the determined adjusted exposure setting, and exposure compensation. Calibration method 60 may then finish or end 74.

[0028] An example of additional elements of calibration method 60 in accordance with an implementation is shown in FIG. 6. As can be seen in FIG. 6, calibration method 60 may include capturing an image with a sensor module, such as sensor module 12, utilizing the default calibrated sensor module settings, as indicated by block 76, and/ or minimizing the first differences, as indicated by block 78. As can also be seen in FIG. 6, calibration method 60 may additionally or alternatively include minimizing the measured second differences, as indicated by block 80, and determining the adjusted exposure setting based on the minimized measured second differences, as indicated by block 82. As can additionally be seen in FIG. 6, calibration method 60 may also or alternatively include executing a set of instructions on a machine- readable non-transitory storage medium via a processor to measure first differences between known white point red, green, and blue values of the white balance calibration target and red, green, and blue values of the captured image of the illuminated white balance calibration target; normalize the measured first differences; measure second differences between known red, green, and blue values of each color in the color calibration target and red, green, and blue values of each color of the captured image of the illuminated color calibration target; determine the adjusted exposure setting based on the measured second differences and/or create the default calibrated sensor module settings to use with different sensor modules that have unit-by-unit variations, as indicated by block 84.

[0029] An example of a machine -readable non-transitory storage medium 86 including instructions executable by a processor 88, as generally indicated by double headed arrow 90, in accordance with an implementation is shown in FIG. 7. As can be seen in FIG. 7, machine- readable non-transitory storage medium 86 includes instructions 92, executable by processor 88, to normalize first differences between known white point red, green, and blue values of a white balance calibration target and red, green, and blue values of a recorded image of an illuminated white balance calibration target. As can also be seen in FIG. 7, machine-readable non-transitory storage medium 86 includes instructions 94, executable by processor 88, to determine an adjusted exposure setting based on second differences between known red, green, and blue values of each color in a color calibration target and red, green, and blue values of each color of a recorded image of an illuminated color calibration target and instructions 96 to create default calibrated sensor module settings to address unit-by-unit variations in different sensor modules. This may help to pro vide more consistent and visually pleasing images and/or video between such different sensor modules and the devices in which they may be used. These default calibrated sensor module settings may include, but are not necessarily limited to, at least one of the following parameters or settings: white balance, contrast, resolution, sensitivity, color space, aperture, shutter speed, compression, noise reduction, frames per second, metering pattern, and exposure compensation. f0030] An example of additional instructions in accordance with an implementation that are executable by processor 88, as generally indicated by double-headed arrow 90, that may be included on machine-readable non-transitory storage medium 86 are shown in FIG. 8. As can be seen in FIG. 8, machine-readable non-transitory storage medium 86 may include instructions 98, executable by processor 88, to minimize the normalized first differences, instructions 100, executable by processor 88, to minimize the measured second differences, and instructions 102, executable by processor 88, to determine the adjusted exposure setting based on the minimized measured second differences.

[0031] Although several drawings have been described and illustrated in detail, it is to be understood that the same are intended by way of illustration and example. These examples are not intended to be exhaustive or to be limited to the precise form disclosed. Modifications, additions, and variations may well be apparent. For example, calibration system 10, calibration method 60 and/ or machine-readable non-transitory storage medium 86 may be utilized with, in or as a part of a device, apparatus, process, or system that includes at least one camera, display, input device, computer, computing system, printer, printing system, scanner, projector, touch- sensitive surface or speaker, or any combination of the forgoing,

[0032] Additionally, reference to an element in the singular is not intended to mean one, unless explicitly so stated, but rather means at least one. Furthermore, unless specifically stated, any method elements are not limited to the sequence or order described and illustrated. Moreover, no element or component is intended to be dedicated to the public regardless of whether the element or component is explicitly recited in the following claims.

Claims

CLAIMS What is claimed is :

1. A. calibration method, comprising: measuring first differences between known white point red, green, and blue values of a white balance calibration target and red, green, and blue values of a captured image of an illuminated white balance calibration target; normalizing the measured first differences; measuring second differences between known red, green, and blue values of each color in a color calibration target and red, green, and blue values of each color of a captured image of an illuminated color calibration target; determining an adjusted exposure setting based on the measured second differences; and creating default calibrated sensor module settings to use with different sensor modules that have unit-by-unit variations.

2. The calibration method of Claim 1 , further comprising capturing an image with a sensor module utilizing the default calibrated sensor module settings,

3. The calibration method of Claim 1, further comprising minimizing the first differences.

4. The calibration method of Claim 1 , further comprising: minimizing the measured second differences; and determining the adjusted exposure setting based on the minimized measured second differences.

5. The calibration method of Claim 1, further comprising executing a set of instructions on a machine-readable non-transitory storage medium via a processor to one of measure first differences between known white point red, green, and blue values of the white balance calibration target and red, green, and blue values of the captured image of the il luminated white balance calibration target; normalize the measured first differences; measure second differences between known red, green, and blue values of each color in the color calibration target and red, green, and blue values of each color of the captured image of the illuminated color calibration target; determine the adjusted exposure setting based on the measured second differences; and create the default calibrated sensor module settings to use with different sensor modules that have unit-by-unit variations,

6. A calibration system, comprising: a sensor module to record images, the sensor module including an illumination source to illuminate a white balance calibration target and a color calibration target; a white balance estimation engine to measure first differences between known white point red, green, and blue values of the white balance calibration target and red, green, and blue values of a recorded image of the illuminated white balance calibration target and to normalize the first differences to use with the sensor module; and an exposure estimation engine to measure second differences between known red, green, and blue values of each color in the color calibration target and red, green, and blue values of each color of a recorded image of the illuminated color calibration target and to determine an adjusted exposure setting based on the measured second differences to use with the sensor module.

7. The calibration system of Claim 6, further comprising a calibration engine to create default calibrated sensor module settings, based on the normalized first differences from the white balance estimation engine and the adjusted exposure setting from the exposure estimation engine, to use with different sensor modules that have unit-by-unit variations.

8. The calibration system of Claim 7, wherein the calibration engine includes a processor and a machine-readable non-transitory storage medium including instructions executable by the processor to create the default calibrated sensor module settings.

9. The calibration system of Claim 6, further comprising: a memory to store the normalized first differences from the white balance estimation engine and the adjusted exposure setting from the exposure estimation engine: and a capture engine to record images with the sensor module utilizing the stored normalized first differences from the white balance estimation engine and the adjusted exposure setting from the exposure estimation engine.

10. The calibration system of Claim 6, further comprising an iteration engine to minimize the normalized first differences from the white balance estimation engine to use with the sensor module.

1 1 . The calibration system of Claim 6, further comprising an iteration engine to minimize the measured second differences from the exposure estimation engine, and wherein the exposure estimation engine determines the adjusted exposure setting based on the minimized measured second differences from the iteration engine to use with the sensor module.

12. The calibration system of Claim 6, wherein the white balance estimation engine includes a processor and a machine-readable non-transitory storage medium including instructions executable by the processor to one of measure the first differences between known red, green, and blue values of each color in the color calibration target and red, green, and blue values of ea ch color of the recorded image of the illuminated color calibration target and normalize the first differences to use with the sensor module.

13. The calibration system of Claim 6, wherein the exposure estimation engine includes a processor and a machine-readable non-transitory storage medium including instructions executable by the processor to one of measure the second differences between known red, green, and blue values of each color in the color calibration target and red, green, and blue values of each color of the recorded ima ge of the illuminated color calibra tion target and determine the adjusted exposure setting based on the measured second differences to use with the sensor module.

14. A machine -readable non-transitory storage medium comprising instructions executable by a processor to: normalize first differences between known white point red, green, and blue values of a white balance calibration target and red, green, and blue values of a recorded image of an illuminated white balance calibration target; determine an adjusted exposure setting based on second differences between known red, green, and blue values of each color in a color calibration target and red, green, and blue values of each color of a recorded image of an illuminated color calibration target; and create default calibrated sensor module settings to address unit-by-unit variations in different sensor modules.

15. The machine-readable non-transitory storage medium of Claim 14, further comprising additional instructions executable by a processor to: minimize the normalized first differences; minimize the measured second differences; and determine the adjusted exposure setting based on the minimized measured second differences.