WO2006095274A1 - Camera pair using fluid based lenses - Google Patents

Camera pair using fluid based lenses Download PDF

Info

Publication number
WO2006095274A1
WO2006095274A1 PCT/IB2006/050327 IB2006050327W WO2006095274A1 WO 2006095274 A1 WO2006095274 A1 WO 2006095274A1 IB 2006050327 W IB2006050327 W IB 2006050327W WO 2006095274 A1 WO2006095274 A1 WO 2006095274A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
lens
interface
data processor
information
Prior art date
Application number
PCT/IB2006/050327
Other languages
French (fr)
Inventor
Stein Kuiper
Richard P. Kleihorst
Bernardus H. W. Hendriks
Harry Broers
Martijn Bennebroek
Original Assignee
Koninklijke Philips Electronics N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics N.V. filed Critical Koninklijke Philips Electronics N.V.
Publication of WO2006095274A1 publication Critical patent/WO2006095274A1/en

Links

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B3/00Simple or compound lenses
    • G02B3/12Fluid-filled or evacuated lenses
    • G02B3/14Fluid-filled or evacuated lenses of variable focal length
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B26/00Optical devices or arrangements for the control of light using movable or deformable optical elements
    • G02B26/004Optical devices or arrangements for the control of light using movable or deformable optical elements based on a displacement or a deformation of a fluid
    • G02B26/005Optical devices or arrangements for the control of light using movable or deformable optical elements based on a displacement or a deformation of a fluid based on electrowetting
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/204Image signal generators using stereoscopic image cameras
    • H04N13/239Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance

Definitions

  • the present invention relates to an electronic device having a first lens and a second lens for respectively capturing different parts of an image to extract information from the overlap.
  • the present invention further relates to an image information extraction method using a first lens and a second lens for capturing different parts of an image.
  • Electronic devices having a first lens and a second lens, or a first camera and a second camera, for instance for respectively capturing partially overlapping parts of an image to extract information from the overlap can be found in a number of application domains.
  • the extracted information may be based on the recognition of an object in the overlap, which can be used to estimate the depth of the object in the captured image, i.e. the distance of the object from the electronic device.
  • Such application domains comprise robotics and automotive, where accurate real-time depth estimation is crucial to the performance of the electronic device, e.g. a robot or a car incorporating collision prevention techniques that rely on such information.
  • the present invention seeks to provide an electronic device and a method according to the opening paragraph that allow for an improved tracking of objects.
  • an electronic device comprising a first lens for capturing a first part of an image and a second lens for capturing a second part of an image, each lens comprising a chamber enclosing a first fluid and an electrically susceptible second fluid, said fluids being immiscible, being separated by an interface and having different refractive indices, the chamber further carrying a plurality of electrodes oriented in parallel with an optical path through the chamber for controlling the shape of the interface; first image sensing means for sensing the first part of the image; second image sensing means for sensing the second part of the image; a data processor coupled to the first image sensing means and the second image sensing means for extracting image information from the first part and the second part of the image; first driving means coupled between the data processor and the first lens for individually controlling the electrodes of the first lens in response to the data processor; and second driving means coupled between the data processor and the second lens for individually controlling the electrodes of the second lens in response to the data processor.
  • the present invention is based on the realization that a new class of fluid based lenses, i.e. electrowetting lenses, is particularly suited for the fast response requirements of 3D image generation from multiple partially overlapping 2D images, depth estimation, object tracking and so on.
  • An example of a suitable lens for these applications is disclosed in WO2004/051323, and is shown in Fig. 1.
  • This lens comprises two immiscible fluids A and B, one of which is an electrically susceptible fluid, which means that the wetting of this fluid on the inner wall of the chamber 5 increases if an electrical field is applied.
  • the lens further comprises replica lenses 4 and 6 on either side of the chamber, which may operate as lids to the chamber or may be separate lenses.
  • the fluids A and B which preferably are of similar density to avoid gravitational influence on their orientation inside the chamber 5 and have different refractive indices to achieve the lens function, are separated by an interface 14 and are held by a chamber 5, which carries a first electrode 2a and a second electrode 2a' in parallel with the optical path 1 through the chamber 5, as well as an annular electrode 12 in contact with the electrically susceptible fluid. Consequently, by the application of different voltages V1 and V2 across the annular electrode 12 and the first electrode 2a and the second electrode 2a' respectively, the contact angles 61 and 62 of the interface 14 with the coating 10 on the inner wall of the chamber 5 have different values, thus yielding a lens having a tilted interface 14 with respect to the optical path 1.
  • the focal point of this variable focus lens not only can be varied in depth by a variation of the strength of the applied voltage, but the focal point can also be changed in a direction perpendicular to the optical axis by variation of the difference in the voltage applied to the first electrode 2a and the second electrode 2a'.
  • the response time of a change in focus for such a lens typically lies in the (sub)millisecond domain, which makes these lenses extremely suitable for use in object tracking application domains.
  • Various known combinations of fluids suitable for application in such a lens are readily available; as an example, a combination of an aqueous salt solution, e.g. aqueous KCI, and a silicon oil or an alkane may be used.
  • the electrodes may be made of known conductive material, e.g. IndiumTinOxide (ITO), whereas the coating 10 may be a teflon coating such as AF1600 supplied by the DuPont chemical company or a parylene stack; various alternative embodiments are readily available in the prior art.
  • ITO IndiumTinOxide
  • the number of wall electrodes such as wall electrodes 2a and 2a' is not limited to two; more electrodes may be present to allow for a more fine-grained control of the shape of the interface 14.
  • the present invention uses a combination of these lenses in a non-obvious way, because WO2004/051323 is completely silent about the use of more than one lens in an electronic device, and is also completely silent about the usefulness of these lenses in object tracking applications. Furthermore, the present invention is based on the realization that the mechanical tilting of complete cameras in the prior art arrangements can be replaced by liquid-based tilting inside the lens of such a camera, which is an improvement that is neither disclosed nor suggested in the prior art.
  • the electronic device of the present invention may utilize known algorithms for generating an image from a mosaic of non-overlapping image parts.
  • the first part of the image and the second part of the image may have an overlap, with the data processor being arranged to extract the image information, e.g. depth estimation of an object in the overlap, from the overlap.
  • the first image sensing means and the second image sensing means comprise separate areas of an image sensor. This has the advantage that only a single sensor having a large enough area to detect both parts of the image can be used, thus reducing cost.
  • the data processor is configured to control the first driving means and second driving means in response to a processing result of the image.
  • a processing result which may be a focus error evaluation or a detected object displacement in two contiguous images, can be used to estimate the optimal shape of the interface for the next image to be captured.
  • the data processor is equipped to generate a control signal for the driver circuitry of the first and second lens, which will be translated by the driver circuitry into a plurality of voltages for the individual electrodes of the lenses.
  • the electronic device further comprises first means coupled between the first lens and the data processor for measuring a parameter relating to the shape of the interface of the first lens and second means coupled between the second lens and the data processor for measuring a parameter relating to the shape of the interface of the second lens.
  • Such measuring means which may comprise a voltage meter or a capacitance meter to measure the capacitance formed by the wall electrode and the electrically susceptible fluid with the coating 10 acting as a dielectric layer, may be used to compare the actual shape of the interface with the intended shape of the interface. This way, deviations from the intended shape of the interface, which for instance may be caused by ageing effects of the fluid-based lenses, can be compensated for. Also, the deviations can be taken into consideration when calculating an image characteristic, e.g. an object distance.
  • a method of extracting information from an image using a plurality of image parts comprising providing a first lens and a second lens, each lens comprising a chamber enclosing a first fluid and an electrically susceptible second fluid, said fluids being immiscible, being separated by an interface and having different refractive indices, the chamber further carrying a plurality of electrodes oriented in parallel with an optical path through the chamber for controlling the shape of the interface; capturing a first part of the image with the first lens; capturing a second part of the image with the second lens; and extracting the image information from the first part and the second part.
  • the plurality of image parts may be partially overlapping, with the step of extracting the image information from the first part and the second part comprising extracting the image information from the overlap between the first part and the second part.
  • the method further comprises calculating a difference between the extracted information and the expected information; adjusting the shape of the interface of the first lens and/or the interface of the second lens responsive to the difference; capturing a first further part of the image with the first lens; capturing a second further part of the image with the second lens; and extracting the image information from the overlap of the first further part and the second further part.
  • this adjustment e.g. the correction of a focus error
  • the method further comprises measuring a parameter relating to the shape of the interface of the first lens; and measuring a parameter relating to the shape of the interface of the second lens. This facilitates the correction of deviations of the intended shapes of the interfaces.
  • the method further comprises including the determined parameters in the step of extracting the image information. This has the advantage that an even more accurate image characteristic can be obtained.
  • Fig. 1 schematically depicts a prior art variable focus lens
  • Fig. 2 depicts an embodiment of an electronic device according to the present invention
  • Fig. 3 depicts another embodiment of an electronic device according to the present invention.
  • Fig. 2 shows an electronic device 100 including a first lens 110 and a second lens 120.
  • the electronic device may be a camera, a robot, a mobile communications device including camera functionality, an automotive device, e.g. a car, and so on.
  • the first lens 110 and the second lens 120 are embodiments of the lens as shown in Fig. 1 and as previously described. A more detailed description of such a lens and its operation can be found in
  • the first lens 110 is placed in front of a first image sensor 132, and the second lens 120 is placed in front of a second image sensor 134.
  • image sensors maybe separate sensors, e.g. CMOS or CCD sensors, or may be discrete areas of a large image sensor 130.
  • the first image sensor 132 and the second image sensor 134 are arranged to provide an output signal, e.g. a CMY(G), YUV or a RGB signal, to a data processor 140.
  • the data processor 140 is arranged to extract information from the output signals of the first image sensor 132 and the second image sensor 134.
  • the data processor 140 may comprise of various processing stages, e.g. a stage for parallel processing of the pixelated information from the image sensor, which may be a single instruction multiple data (SIMD) architecture, and a stage for further processing the output of the SIMD architecture, which may comprise object recognition calculations followed by object depth estimations.
  • SIMD single instruction multiple data
  • the data processor 140 will have access to a memory (not shown) for storing and accessing temporary and previous calculation results.
  • the data processor 140 may have an output 142 for providing a calculation result to a display device (not shown) or to a further processor (not shown) in case of the calculation result being an intermediate result.
  • the data processor 140 is further arranged to provide a control signal such as a focus error signal to first driver circuitry 150 and a further control signal to second driver circuitry 160 for updating the shape of the interfaces 14 of the first lens 110 and the second lens 120 respectively.
  • the first driver circuitry 150 and the second driver circuitry 160 which may combined into a single driver circuit, are arranged to translate the control signals from the data processor 140 into a plurality of driving voltages for the individual electrodes 2a and 2a' of the first lens 110 and the second lens 120 to alter the shape of the interfaces 14 of these lenses.
  • a shape change of the interfaces 14 can be completed within 10 ms and switching frequencies of well over 100Hz can be achieved, thus yielding an electronic device 100 with tracking capabilities superior to that of human tracking capabilities.
  • Fig. 3 shows another embodiment of an electronic device 100 according to the present invention, in which the electronic device 100 as shown in Fig.2 is extended with a first circuit 170 coupled between the first lens 110 and the data processor 140 and a second circuit 180 coupled between the second lens 120 and the data processor 140.
  • the first circuit 170 and the second circuit 180 may be combined into a single circuit. The function of the first circuit 170 and the second circuit 180 will be explained using the first circuit 170.
  • the first circuit 170 is configured to extract information, e.g. a parameter, from the first lens 110 relating to the shape of the interface 14. To this end, the first circuit 170 is coupled to the individual electrodes of the first lens 110, in order to measure the voltage across the various electrode pairs, e.g. the pair formed by electrode 2a and electrode 12. This way, voltage losses from the driver circuit 150 to the first lens 110 can be determined.
  • the first circuit 170 is configured to measure the capacitance formed by a wall electrode, e.g. electrode 2a and the electrically susceptible fluid separated by the inner wall of the chamber of the lens, which acts as a dielectric.
  • the magnitude of the capacitance is determined by the area of the electrically susceptible fluid in contact with the layer 10 of the chamber, which can be translated to a contact angle ⁇ of the interface 14 with the layer 10 in that area. Consequently, the separate capacitance measurements provide detailed information about the various contact angles of the interface 14 with the layer 10, from which a shape of the interface 14 can be extracted. Obviously, the more individual electrodes 2 are present, the more fine-grained this information will be, and the more accurate the calculated shape will be.
  • the first circuit 170 will forward the measured values to the data processor 140, which can calculate a deviation of the shape of the interface 14 with its intended shape, e.g. by comparing the measured values with predefined values.
  • the predefined values may originate from calibration of the first lens 110 upon manufacturing of the electronic device 100 and may for instance be stored in general memory or in a dedicated look-up table.
  • the data processor 140 can use the shape of the interface 14, or the difference between intended and measured shape of the interface 14 as a parameter in the calculation of the characteristic of the image to be determined, and/or may use this difference to further refine the control signal provided to the respective driver circuitries 150 and 160.
  • the electronic device 100 implements the method of the present invention in the following way.
  • the first lens 110 and the second lens 120 are placed in the electronic device 100 in such a way that they can capture different parts of an image, with an overlap optionally existing between these different parts.
  • the parts of the image captured by the lenses 110 and 120 and the associated image sensors 132 and 134 is forwarded to the data processor 140 for extracting the image information from the first part and the second part, e.g. from the overlap between the first part and the second part.
  • This image information may be a depth estimation of an object recognized in the overlap between the first part and the second part of the image.
  • the image information may be the amount of overlap between the first part and the second part, which for instance can be used to create a panoramic photo, e.g. by removing the overlap from one of the two parts of the image and seaming the two parts together in a subsequent processing step.
  • Taking a panoramic photo with an electronic device 100 of the present invention has the advantage that more than one images can be taken at the same time, thus reducing the overall time it takes to generate the panoramic photo, which reduces the chance of unwanted inadvertent displacement of the electronic device 100, which would interfere with a successful capturing of the panoramic photo.
  • a plurality of non-overlapping image parts can be captured, from which an overall image is generated. This can be realized with known algorithms.
  • the method of the present invention may comprise the additional step of calculating a difference between the extracted information and the expected information, such as the sharpness of the object captured in the overlap of the first part and second part of the image, which can be executed by the data processor 140.
  • the calculated difference can be used to adjust the shape of the interface 14 of the first lens 110 and/or the interface 14 of the second lens 120, e.g. by providing the associated driver circuits 150 and 160 with a focus error signal, after which a first further part of the image can be captured with the first lens 110 and a second further part of the image can be captured with the second lens 120.
  • updated image information can be extracted from the overlap of the first further part and the second further part of the image.
  • the method may include steps to measure respective parameters relating to the shapes of the interfaces 14 of the first lens 110 and the second lens 120. These parameters may be included in the image characteristic extraction step.

Abstract

According to the present invention, there is provided an electronic device (100) comprising a first and second lens (110, 120) for capturing a first and second part of an image respectively. Each lens (110; 120) comprises a chamber (5) enclosing a first fluid (A) and an electrically susceptible second fluid (B), said fluids (A; B) being immiscible, being separated by an interface (14) and having different refractive indices, the chamber (5) further carrying a plurality of electrodes (2a; 2a') oriented in parallel with an optical path (1 ) through the chamber (5) for controlling the shape of the interface (14). First and second image sensing means (130, 132, 134) are present for respectively sensing the first and second part of the image, and a data processor (140) is coupled to the first and second image sensing means (130, 132, 134) for extracting information from the first part and the second part of the image, e.g. from the overlap between the first part and the second part. First and second driving means (150, 160) coupled between the data processor (140) and the first and second lens (110, 120) respectively are present for individually controlling the electrodes (2a; 2a'; 12) of the first and the second lens (110, 120) in response to the data processor (140). Consequently, an electronic device is obtained that can obtain image information such as depth estimation more quickly and more cheaply than known electronic devices.

Description

DESCRIPTION
CAMERA PAIR USING FLUID BASED LENSES
The present invention relates to an electronic device having a first lens and a second lens for respectively capturing different parts of an image to extract information from the overlap.
The present invention further relates to an image information extraction method using a first lens and a second lens for capturing different parts of an image.
Electronic devices having a first lens and a second lens, or a first camera and a second camera, for instance for respectively capturing partially overlapping parts of an image to extract information from the overlap can be found in a number of application domains. For example, in some application domains the extracted information may be based on the recognition of an object in the overlap, which can be used to estimate the depth of the object in the captured image, i.e. the distance of the object from the electronic device. Such application domains comprise robotics and automotive, where accurate real-time depth estimation is crucial to the performance of the electronic device, e.g. a robot or a car incorporating collision prevention techniques that rely on such information.
The design of such electronic devices is not without problems. For instance, in real-time depth-estimation, if the object of which the depth is to be determined is a moving object, the tracking of the object may be complicated because it may move out of focus. To this end, known electronic devices have been fitted with cameras that can be mechanically displaced, e.g. by coupling each of the cameras to a motor-driven rotating plateau, such as disclosed in European patent EP0642275. This way, the tilt angle of each camera can be altered by rotating the plateau, which facilitates the tracking of moving objects. However, this solution still suffers from drawbacks. For instance, the mechanical construction is relatively expensive and sensitive to wear and tear. Moreover, the translation speed of the camera fitted on the plateau is limited, which prohibits the accurate real-time tracking of fast moving objects.
The present invention seeks to provide an electronic device and a method according to the opening paragraph that allow for an improved tracking of objects.
According to a first aspect of the invention, there is provided an electronic device comprising a first lens for capturing a first part of an image and a second lens for capturing a second part of an image, each lens comprising a chamber enclosing a first fluid and an electrically susceptible second fluid, said fluids being immiscible, being separated by an interface and having different refractive indices, the chamber further carrying a plurality of electrodes oriented in parallel with an optical path through the chamber for controlling the shape of the interface; first image sensing means for sensing the first part of the image; second image sensing means for sensing the second part of the image; a data processor coupled to the first image sensing means and the second image sensing means for extracting image information from the first part and the second part of the image; first driving means coupled between the data processor and the first lens for individually controlling the electrodes of the first lens in response to the data processor; and second driving means coupled between the data processor and the second lens for individually controlling the electrodes of the second lens in response to the data processor.
The present invention is based on the realization that a new class of fluid based lenses, i.e. electrowetting lenses, is particularly suited for the fast response requirements of 3D image generation from multiple partially overlapping 2D images, depth estimation, object tracking and so on. An example of a suitable lens for these applications is disclosed in WO2004/051323, and is shown in Fig. 1. This lens comprises two immiscible fluids A and B, one of which is an electrically susceptible fluid, which means that the wetting of this fluid on the inner wall of the chamber 5 increases if an electrical field is applied. The lens further comprises replica lenses 4 and 6 on either side of the chamber, which may operate as lids to the chamber or may be separate lenses. The fluids A and B, which preferably are of similar density to avoid gravitational influence on their orientation inside the chamber 5 and have different refractive indices to achieve the lens function, are separated by an interface 14 and are held by a chamber 5, which carries a first electrode 2a and a second electrode 2a' in parallel with the optical path 1 through the chamber 5, as well as an annular electrode 12 in contact with the electrically susceptible fluid. Consequently, by the application of different voltages V1 and V2 across the annular electrode 12 and the first electrode 2a and the second electrode 2a' respectively, the contact angles 61 and 62 of the interface 14 with the coating 10 on the inner wall of the chamber 5 have different values, thus yielding a lens having a tilted interface 14 with respect to the optical path 1. As a consequence, the focal point of this variable focus lens not only can be varied in depth by a variation of the strength of the applied voltage, but the focal point can also be changed in a direction perpendicular to the optical axis by variation of the difference in the voltage applied to the first electrode 2a and the second electrode 2a'. The response time of a change in focus for such a lens typically lies in the (sub)millisecond domain, which makes these lenses extremely suitable for use in object tracking application domains. Various known combinations of fluids suitable for application in such a lens are readily available; as an example, a combination of an aqueous salt solution, e.g. aqueous KCI, and a silicon oil or an alkane may be used. The electrodes may be made of known conductive material, e.g. IndiumTinOxide (ITO), whereas the coating 10 may be a teflon coating such as AF1600 supplied by the DuPont chemical company or a parylene stack; various alternative embodiments are readily available in the prior art. Moreover, it will be obvious to the skilled person that the number of wall electrodes such as wall electrodes 2a and 2a' is not limited to two; more electrodes may be present to allow for a more fine-grained control of the shape of the interface 14.
It is emphasized that the present invention uses a combination of these lenses in a non-obvious way, because WO2004/051323 is completely silent about the use of more than one lens in an electronic device, and is also completely silent about the usefulness of these lenses in object tracking applications. Furthermore, the present invention is based on the realization that the mechanical tilting of complete cameras in the prior art arrangements can be replaced by liquid-based tilting inside the lens of such a camera, which is an improvement that is neither disclosed nor suggested in the prior art.
The electronic device of the present invention may utilize known algorithms for generating an image from a mosaic of non-overlapping image parts. Alternatively, the first part of the image and the second part of the image may have an overlap, with the data processor being arranged to extract the image information, e.g. depth estimation of an object in the overlap, from the overlap.
In an embodiment, the first image sensing means and the second image sensing means comprise separate areas of an image sensor. This has the advantage that only a single sensor having a large enough area to detect both parts of the image can be used, thus reducing cost.
Advantageously, the data processor is configured to control the first driving means and second driving means in response to a processing result of the image. Such a processing result, which may be a focus error evaluation or a detected object displacement in two contiguous images, can be used to estimate the optimal shape of the interface for the next image to be captured. To this end, the data processor is equipped to generate a control signal for the driver circuitry of the first and second lens, which will be translated by the driver circuitry into a plurality of voltages for the individual electrodes of the lenses.
In a further embodiment, the electronic device further comprises first means coupled between the first lens and the data processor for measuring a parameter relating to the shape of the interface of the first lens and second means coupled between the second lens and the data processor for measuring a parameter relating to the shape of the interface of the second lens. Such measuring means, which may comprise a voltage meter or a capacitance meter to measure the capacitance formed by the wall electrode and the electrically susceptible fluid with the coating 10 acting as a dielectric layer, may be used to compare the actual shape of the interface with the intended shape of the interface. This way, deviations from the intended shape of the interface, which for instance may be caused by ageing effects of the fluid-based lenses, can be compensated for. Also, the deviations can be taken into consideration when calculating an image characteristic, e.g. an object distance.
According to a further aspect of the invention, there is provided a method of extracting information from an image using a plurality of image parts, the method comprising providing a first lens and a second lens, each lens comprising a chamber enclosing a first fluid and an electrically susceptible second fluid, said fluids being immiscible, being separated by an interface and having different refractive indices, the chamber further carrying a plurality of electrodes oriented in parallel with an optical path through the chamber for controlling the shape of the interface; capturing a first part of the image with the first lens; capturing a second part of the image with the second lens; and extracting the image information from the first part and the second part. With this method, the image information can be generated more quickly and with lower cost than with known methods. The plurality of image parts may be partially overlapping, with the step of extracting the image information from the first part and the second part comprising extracting the image information from the overlap between the first part and the second part.
Advantageously, the method further comprises calculating a difference between the extracted information and the expected information; adjusting the shape of the interface of the first lens and/or the interface of the second lens responsive to the difference; capturing a first further part of the image with the first lens; capturing a second further part of the image with the second lens; and extracting the image information from the overlap of the first further part and the second further part. With the method of the present invention, this adjustment, e.g. the correction of a focus error, can be done much faster than with prior art methods. It is another advantage if the method further comprises measuring a parameter relating to the shape of the interface of the first lens; and measuring a parameter relating to the shape of the interface of the second lens. This facilitates the correction of deviations of the intended shapes of the interfaces. In an embodiment, the method further comprises including the determined parameters in the step of extracting the image information. This has the advantage that an even more accurate image characteristic can be obtained.
The invention is described in more detail and by way of non-limiting examples with reference to the accompanying drawings, wherein:
Fig. 1 schematically depicts a prior art variable focus lens; and Fig. 2 depicts an embodiment of an electronic device according to the present invention; and
Fig. 3 depicts another embodiment of an electronic device according to the present invention.
It should be understood that the Figures are merely schematic and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the Figures to indicate the same or similar parts.
Fig. 2 shows an electronic device 100 including a first lens 110 and a second lens 120. The electronic device may be a camera, a robot, a mobile communications device including camera functionality, an automotive device, e.g. a car, and so on. The first lens 110 and the second lens 120 are embodiments of the lens as shown in Fig. 1 and as previously described. A more detailed description of such a lens and its operation can be found in
WO2004/051323 from the same applicant as the present invention. The first lens 110 is placed in front of a first image sensor 132, and the second lens 120 is placed in front of a second image sensor 134. These image sensors maybe separate sensors, e.g. CMOS or CCD sensors, or may be discrete areas of a large image sensor 130.
The first image sensor 132 and the second image sensor 134 are arranged to provide an output signal, e.g. a CMY(G), YUV or a RGB signal, to a data processor 140. The data processor 140 is arranged to extract information from the output signals of the first image sensor 132 and the second image sensor 134. To this end, the data processor 140 may comprise of various processing stages, e.g. a stage for parallel processing of the pixelated information from the image sensor, which may be a single instruction multiple data (SIMD) architecture, and a stage for further processing the output of the SIMD architecture, which may comprise object recognition calculations followed by object depth estimations. Typically, the data processor 140 will have access to a memory (not shown) for storing and accessing temporary and previous calculation results. The data processor 140 may have an output 142 for providing a calculation result to a display device (not shown) or to a further processor (not shown) in case of the calculation result being an intermediate result.
The data processor 140 is further arranged to provide a control signal such as a focus error signal to first driver circuitry 150 and a further control signal to second driver circuitry 160 for updating the shape of the interfaces 14 of the first lens 110 and the second lens 120 respectively. The first driver circuitry 150 and the second driver circuitry 160, which may combined into a single driver circuit, are arranged to translate the control signals from the data processor 140 into a plurality of driving voltages for the individual electrodes 2a and 2a' of the first lens 110 and the second lens 120 to alter the shape of the interfaces 14 of these lenses. Due to the high switching speeds of the first lens 110 and the second lens 120, a shape change of the interfaces 14 can be completed within 10 ms and switching frequencies of well over 100Hz can be achieved, thus yielding an electronic device 100 with tracking capabilities superior to that of human tracking capabilities.
Fig. 3 shows another embodiment of an electronic device 100 according to the present invention, in which the electronic device 100 as shown in Fig.2 is extended with a first circuit 170 coupled between the first lens 110 and the data processor 140 and a second circuit 180 coupled between the second lens 120 and the data processor 140. The first circuit 170 and the second circuit 180 may be combined into a single circuit. The function of the first circuit 170 and the second circuit 180 will be explained using the first circuit 170. The first circuit 170 is configured to extract information, e.g. a parameter, from the first lens 110 relating to the shape of the interface 14. To this end, the first circuit 170 is coupled to the individual electrodes of the first lens 110, in order to measure the voltage across the various electrode pairs, e.g. the pair formed by electrode 2a and electrode 12. This way, voltage losses from the driver circuit 150 to the first lens 110 can be determined.
Alternatively, the first circuit 170 is configured to measure the capacitance formed by a wall electrode, e.g. electrode 2a and the electrically susceptible fluid separated by the inner wall of the chamber of the lens, which acts as a dielectric. The magnitude of the capacitance is determined by the area of the electrically susceptible fluid in contact with the layer 10 of the chamber, which can be translated to a contact angle Θ of the interface 14 with the layer 10 in that area. Consequently, the separate capacitance measurements provide detailed information about the various contact angles of the interface 14 with the layer 10, from which a shape of the interface 14 can be extracted. Obviously, the more individual electrodes 2 are present, the more fine-grained this information will be, and the more accurate the calculated shape will be. Typically, the first circuit 170 will forward the measured values to the data processor 140, which can calculate a deviation of the shape of the interface 14 with its intended shape, e.g. by comparing the measured values with predefined values. The predefined values may originate from calibration of the first lens 110 upon manufacturing of the electronic device 100 and may for instance be stored in general memory or in a dedicated look-up table. The data processor 140 can use the shape of the interface 14, or the difference between intended and measured shape of the interface 14 as a parameter in the calculation of the characteristic of the image to be determined, and/or may use this difference to further refine the control signal provided to the respective driver circuitries 150 and 160.
Thus, the electronic device 100 implements the method of the present invention in the following way. The first lens 110 and the second lens 120 are placed in the electronic device 100 in such a way that they can capture different parts of an image, with an overlap optionally existing between these different parts. The parts of the image captured by the lenses 110 and 120 and the associated image sensors 132 and 134 is forwarded to the data processor 140 for extracting the image information from the first part and the second part, e.g. from the overlap between the first part and the second part. This image information may be a depth estimation of an object recognized in the overlap between the first part and the second part of the image. However, other applications can be thought of; for instance, the image information may be the amount of overlap between the first part and the second part, which for instance can be used to create a panoramic photo, e.g. by removing the overlap from one of the two parts of the image and seaming the two parts together in a subsequent processing step. Taking a panoramic photo with an electronic device 100 of the present invention has the advantage that more than one images can be taken at the same time, thus reducing the overall time it takes to generate the panoramic photo, which reduces the chance of unwanted inadvertent displacement of the electronic device 100, which would interfere with a successful capturing of the panoramic photo.
Alternatively, a plurality of non-overlapping image parts can be captured, from which an overall image is generated. This can be realized with known algorithms.
The method of the present invention may comprise the additional step of calculating a difference between the extracted information and the expected information, such as the sharpness of the object captured in the overlap of the first part and second part of the image, which can be executed by the data processor 140. The calculated difference can be used to adjust the shape of the interface 14 of the first lens 110 and/or the interface 14 of the second lens 120, e.g. by providing the associated driver circuits 150 and 160 with a focus error signal, after which a first further part of the image can be captured with the first lens 110 and a second further part of the image can be captured with the second lens 120. Subsequently, updated image information can be extracted from the overlap of the first further part and the second further part of the image. Also, the method may include steps to measure respective parameters relating to the shapes of the interfaces 14 of the first lens 110 and the second lens 120. These parameters may be included in the image characteristic extraction step.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements. In the device claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. An electronic device (100) comprising: a first lens (110) for capturing a first part of an image and a second lens
(120) for capturing a second part of an image, each lens (110; 120) comprising a chamber (5) enclosing a first fluid (A) and an electrically susceptible second fluid (B), said fluids (A; B) being immiscible, being separated by an interface (14) and having different refractive indices, the chamber (5) further carrying a plurality of electrodes (2a; 2a') oriented in parallel with an optical path (1 ) through the chamber (5) for controlling the shape of the interface (14); first image sensing means (130, 132) for sensing the first part of the image; second image sensing means (130, 134) for sensing the second part of the image; a data processor (140) coupled to the first image sensing means (130, 132) and the second image sensing means (130, 134) for extracting image information from the first part and the second part of the image; first driving means (150) coupled between the data processor (140) and the first lens (110) for individually controlling the electrodes (2a; 2a'; 12) of the first lens (110) in response to the data processor (140); and second driving means (160) coupled between the data processor (140) and the second lens (120) for individually controlling the electrodes (2a; 2a'; 12) of the second lens (120) in response to the data processor (140).
2. An electronic device as claimed in claim 1 , wherein the first part of the image and the second part of the image have an overlap, and wherein the data processor is arranged to extract the image information from the overlap.
3. An electronic device (110) as claimed in claim 1 , wherein the first image sensing means and the second image sensing means comprise separate areas of an image sensor (130).
4. An electronic device (110) as claimed in claim 1 or 3, wherein the data processor (140) is configured to control the first driving means (150) and second driving means (160) in response to a processing result of the image.
5. An electronic device (100) as claimed in claim 1 or 3, further comprising: first means (170) coupled between the first lens (110) and the data processor (140) for measuring a parameter relating to the shape of the interface (14) of the first lens (110); and second means (180) coupled between the second lens (120) and the data processor (140) for measuring a parameter relating to the interface (14) of the second lens (120).
6. A method of extracting information from an image using a plurality of image parts, the method comprising: providing a first lens (110) and a second lens (120), each lens comprising a chamber (5) enclosing a first fluid (A) and an electrically susceptible second fluid (B), said fluids being immiscible, being separated by an interface (14) and having different refractive indices, the chamber (5) further carrying a plurality of electrodes (2a, 2a') oriented in parallel with an optical path (1) through the chamber (5) for controlling the shape of the interface (14); capturing a first part of the image with the first lens (110); capturing a second part of the image with the second lens (120); and extracting the image information from the first part and the second part of the image.
7. A method as claimed in claim 6, wherein the pluraility of image parts are partially overlapping, and wherein the step of extracting the image information from the first part and the second part of the image comprises extracting the information from the overlap between the first part and the second part.
8. A method as claimed in claim 6 or 7, further comprising: calculating a difference between the extracted information and the expected information; adjusting the shape of the interface (14) of the first lens (110) and/or the interface (14) of the second lens (120) responsive to the difference; capturing a first further part of the image with the first lens (110); capturing a second further part of the image with the second lens (120); and extracting further image information from the first further part and the second further part.
9. A method as claimed in claim 6 or 7, further comprising: measuring a parameter relating to the shape of the interface (14) of the first lens (110); and measuring a parameter relating to the shape of the interface (14) of the second lens (120).
10. A method as claimed in claim 9, further comprising including the determined parameters in the step of extracting the image information.
PCT/IB2006/050327 2005-02-02 2006-01-31 Camera pair using fluid based lenses WO2006095274A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP05100719.3 2005-02-02
EP05100719 2005-02-02

Publications (1)

Publication Number Publication Date
WO2006095274A1 true WO2006095274A1 (en) 2006-09-14

Family

ID=36570957

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2006/050327 WO2006095274A1 (en) 2005-02-02 2006-01-31 Camera pair using fluid based lenses

Country Status (1)

Country Link
WO (1) WO2006095274A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007080521A1 (en) * 2006-01-11 2007-07-19 Koninklijke Philips Electronics N. V. Control of electrowetting lenses
CN101929844A (en) * 2009-06-25 2010-12-29 (株)赛丽康 Distance measuring apparatus having dual stereo camera
KR20180029664A (en) * 2016-09-13 2018-03-21 엘지이노텍 주식회사 Dual camera module
KR20180050977A (en) * 2016-11-07 2018-05-16 엘지이노텍 주식회사 Camera module and movement method of camera module
CN109716746A (en) * 2016-09-13 2019-05-03 Lg伊诺特有限公司 Double camera module, Optical devices, camera model and the method for operating camera model
US20210239886A1 (en) * 2018-05-04 2021-08-05 Lg Innotek Co., Ltd. Dual camera module comprising liquid lenses

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06308427A (en) * 1993-04-26 1994-11-04 Canon Inc Stereoscopic image forming device
EP0779535A1 (en) * 1995-12-11 1997-06-18 THOMSON multimedia S.A. Camera with variable deflection
US5917657A (en) * 1996-02-22 1999-06-29 Denso Corp Imaging device having flexible liquid-filled film lens
WO2004051323A1 (en) * 2002-12-03 2004-06-17 Koninklijke Philips Electronics N.V. Apparatus for forming variable fluid meniscus configurations

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06308427A (en) * 1993-04-26 1994-11-04 Canon Inc Stereoscopic image forming device
EP0779535A1 (en) * 1995-12-11 1997-06-18 THOMSON multimedia S.A. Camera with variable deflection
US5917657A (en) * 1996-02-22 1999-06-29 Denso Corp Imaging device having flexible liquid-filled film lens
WO2004051323A1 (en) * 2002-12-03 2004-06-17 Koninklijke Philips Electronics N.V. Apparatus for forming variable fluid meniscus configurations

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
PATENT ABSTRACTS OF JAPAN vol. 1995, no. 02 31 March 1995 (1995-03-31) *

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007080521A1 (en) * 2006-01-11 2007-07-19 Koninklijke Philips Electronics N. V. Control of electrowetting lenses
CN101929844A (en) * 2009-06-25 2010-12-29 (株)赛丽康 Distance measuring apparatus having dual stereo camera
US11467322B2 (en) 2016-09-13 2022-10-11 Lg Innotek Co., Ltd. Dual camera module, optical device, camera module, and method for operating camera module
KR20180029664A (en) * 2016-09-13 2018-03-21 엘지이노텍 주식회사 Dual camera module
CN109716746A (en) * 2016-09-13 2019-05-03 Lg伊诺特有限公司 Double camera module, Optical devices, camera model and the method for operating camera model
EP3515058A4 (en) * 2016-09-13 2019-07-24 LG Innotek Co., Ltd. Dual camera module, optical device, camera module, and method for operating camera module
JP2019529994A (en) * 2016-09-13 2019-10-17 エルジー イノテック カンパニー リミテッド Dual camera module, optical device, camera module, and camera module operating method
US10768345B2 (en) 2016-09-13 2020-09-08 Lg Innotek Co., Ltd. Dual camera module, optical device, camera module, and method for operating camera module
CN109716746B (en) * 2016-09-13 2021-06-01 Lg伊诺特有限公司 Dual camera module, optical device, camera module and method for operating camera module
KR102609581B1 (en) * 2016-09-13 2023-12-05 엘지이노텍 주식회사 Dual camera module
JP7146740B2 (en) 2016-09-13 2022-10-04 エルジー イノテック カンパニー リミテッド Dual camera module, optical device, camera module and method of operating camera module
KR20180050977A (en) * 2016-11-07 2018-05-16 엘지이노텍 주식회사 Camera module and movement method of camera module
KR102637778B1 (en) * 2016-11-07 2024-02-19 엘지이노텍 주식회사 Camera module and movement method of camera module
US20210239886A1 (en) * 2018-05-04 2021-08-05 Lg Innotek Co., Ltd. Dual camera module comprising liquid lenses

Similar Documents

Publication Publication Date Title
CN105659580B (en) A kind of Atomatic focusing method, device and electronic equipment
JP5446797B2 (en) Imaging device
JP6887960B2 (en) Systems and methods for autofocus triggers
JP6245885B2 (en) Imaging apparatus and control method thereof
CN103988115B (en) Optical image stabilization
CN109922255B (en) Dual camera system for generating real-time depth maps
JP4673202B2 (en) Image input device
WO2006095274A1 (en) Camera pair using fluid based lenses
CN105657237B (en) Image acquiring device and its digital zooming method
US20080055425A1 (en) Measuring Device
US20150358531A1 (en) Apparatus and method for image acquisition
JP2011013778A5 (en) Stereoscopic image display device
EP3328056B1 (en) Focusing processing method and apparatus, and terminal device
KR20200071960A (en) Method and Apparatus for Vehicle Detection Using Lidar Sensor and Camera Convergence
WO2014041859A1 (en) Imaging control device, imaging device, and method for controlling imaging control device
CN108151657A (en) A kind of dimension of object measuring method and measuring device based on intelligent terminal
KR20160077666A (en) Method of detecting face, method of processing image, face detection device and electronic system including the same
CN106973199B (en) Multi-aperture camera system for improving depth accuracy by using focusing distance scanning
JP2011149931A (en) Distance image acquisition device
US9247125B2 (en) Auxiliary light projection apparatus, flash apparatus, and photographing apparatus
US20170070668A1 (en) Electronic devices for capturing images
WO2013069279A1 (en) Image capturing device
Gil et al. Monster: Awakening the mono in stereo
KR20200014204A (en) Control apparatus, imaging apparatus, and storage medium
JP2018074362A (en) Image processing apparatus, image processing method, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 06710793

Country of ref document: EP

Kind code of ref document: A1

WWW Wipo information: withdrawn in national office

Ref document number: 6710793

Country of ref document: EP