WO2014175484A1

WO2014175484A1 - Method for stabilizing image having jitter and image processing device to which same is applied

Info

Publication number: WO2014175484A1
Application number: PCT/KR2013/003542
Authority: WO
Inventors: 배주한; 황영배; 최병호; 김정호
Original assignee: 전자부품연구원
Priority date: 2013-04-24
Filing date: 2013-04-25
Publication date: 2014-10-30
Also published as: KR20140127049A; KR101458099B1

Abstract

Provided are a method for stabilizing an image having jitter and an image processing device to which the same is applied. The method for stabilizing an image having jitter, according to one embodiment of the present invention, extracts a motion between frames from a high fps image, stabilizes the extracted motion, and generates a stabilized low fps image on the basis of the stabilized motion by selecting only a portion of the frames. Accordingly, even in an image having much jitter, a motion between frames is accurately extracted, unnaturalness which may occur in a low fps image is minimized, and a natural image stabilization result can be obtained even in a sudden motion.

Description

[Name of invention]

Image stabilization method and image processing device using the same

The present invention relates to image stabilization, and more particularly, to a method for stabilizing shake caused in an image and an image processing apparatus using the same.

Background Art

Low fps video with low frame rate is less accurate in feature point matching due to image blur when camera shake is severe and inaccurate motion between frames.

Image stabilization using hardware, such as a gyro sensor, cannot correct the movement of a complex camera by correcting only horizontal and vertical changes, and causes a problem of raising a model price due to a sensor price.

The general image stabilization method of stabilizing an image by minimizing the motion difference between frames has a problem of generating an unnatural image when there is a sudden motion change in the future. [Initiation of invention]

[Technical problem]

The present invention been made in view of the above problems, an object of the present invention, after extracting the ^"motion between frames in High fps video, and stabilize the extraction motion, stabilized by selecting only a portion of the frame An image stabilization method for generating a stabilized low fps image based on motion and an image processing apparatus using the same are provided.

Technical Solution

In accordance with an aspect of the present invention, there is provided a method for stabilizing an image, comprising: selecting a part of frames constituting a first image at a first frame rate; Motion stabilizing the frames selected in the selecting step; And outputting a second image having a second frame rate composed of frames stabilized in the stabilizing step.

The image stabilization method according to an embodiment of the present invention may further include grouping the frames constituting the first image into a plurality of segments, wherein the selecting step may select a frame for each segment. have. . Also, in the selecting step, one frame may be selected for each segment.

And, the most stabilized frame, the actual number of frames included in the segment It may be a frame in which the difference between the cumulative motion and the stabilized cumulative motion is minimal.

In addition, the first frame rate may be higher than the second frame rate.

The number of frames included in the segments may be n, and n = 'first frame rate / second frame rate'.

In addition, the image stabilization method according to an embodiment of the present invention, the step of extracting the motion between the frames constituting the first image; And step 1 for generating stabilized motions from the motions extracted in the extracting step, wherein the stabilizing step multiplies the selected frame by the inverse of the cumulative motion up to a specific previous point, and the previous specific point in time. By multiplying the stabilized cumulative motion up to a corresponding point in time of the frame, the motion stabilization may be performed.

In addition, the stabilized motion generation step may generate a stabilized motion by a weighted sum using the motions of the previous time point and the motions of the after time point.

In addition, the weight may be determined based on motions from a frame output time point to subsequent time points.

On the other hand, the image processing apparatus according to another embodiment of the present invention, the acquisition unit for obtaining a first image of the first frame rate; And an image processor that selects some of the frames constituting the first image acquired by the acquisition unit, performs motion stabilization of the selected frames, and outputs a second image having a second frame rate composed of the stabilized frames. Include. Advantageous Effects

As described above, according to embodiments of the present invention, after extracting motion between frames as a high fps image and stabilizing the extracted motion, only a portion of the frames are selected to stabilize the low fps based on the stabilized motion. An image can be generated.

As a result, since the motion extraction between frames is accurate even in a severely shaken image by using a high fps image having less blurring than a low fps image, image stabilization performance can be improved.

In addition, since a low fps image is generated by selecting frames having a minimum difference between a real cumulative motion and a stabilized cumulative motion, unnaturalness that may occur in the low fps image may be minimized.

Also, since the motion stabilization is performed by referring to the motion of the frames of the previous time point and the subsequent time point based on the frame output time point, a natural image stabilization result can be obtained even with a sudden camera movement.

[Short description of drawing]

1 is a block diagram of an image processing apparatus to which the present invention is applicable;

2 is a flowchart provided to explain a method of stabilizing a shake image according to an exemplary embodiment of the present invention. 2 is a graph illustrating an example of actual motion and stabilized motion, and FIG. 4 is a diagram illustrating a specific method of selecting a frame for each segment. BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, with reference to the drawings will be described the present invention in more detail.

1 is a block diagram of an image processing apparatus to which the present invention is applicable. The image processing apparatus 100 to which the present invention is applicable includes an image capturing unit 110, an image processor 120, and an image application unit 130 as shown in FIG. 1.

The image capturing unit 110 photographs the front to generate a high fps image (an image having a high frame rate, for example, a 120 fps image), and transmits the generated image to the image processor 120.

Due to the external environment, the movement of the image capturing unit 110 itself, and the movement of a device / device in which the image capturing unit 110 is installed, the high fps image captured by the image capturing unit 110 is unstable because of the large shake. can do.

The image processor 120 converts the 'unstable' Highfps image received from the image capturing unit 110 into a 'stable' low fps image (an image having a low frame rate, for example, a 30 fps image). In this process, the image is stabilized by shaking the image by the image processor 120.

The image application unit 130 is a low fps stabilized by the image processor 120 Perform a variety of applications from the image. The grand performance performed by the image grand unit 130 may include image display and storage as well as image recognition, object tracking, and event detection.

The image stabilization processing by the image processor 120 will be described in detail below with reference to FIG. 2. 2 is a flowchart provided to explain a shake image stabilization method according to a preferred embodiment of the present invention.

As shown in FIG. 2, when an unstable high fps image is input from the image capturing unit 110 (S210), the image processor 120 extracts interframe motions constituting the high fps image (S220). ). Specifically, in step S220, the time between adjacent frames and ^

After removing the outlier, the feature point Daeung relationship is generated and the inter-frame motion ᅳ 1) is extracted based on this.

You can use a similarity motion model with D0F of 4 as a motion model, but you can use other motion models (e.g. Translational model with D0F of 2, Affine motion model with D0F of 6, Projective motion model with DOF of 8, etc.). Can also be used.

Since the previous inter-frame motions are extracted for all the frames constituting the high fps image, the relationship shown in Equation 1 below may be established.

[Equation 1]

气 o ^W i, o

^LvI t, ti _. Where ¾ is a reference key-frame and ^/ , 0 corresponds to the cumulative motion from ^J 0 to. (The product of the motions between the frames generated from to). In Figure 3, the dx component of is shown as a solid line, it can be confirmed that the shaking is a lot unstable.

Next, the image processor 120 stabilizes all the motions extracted in step S220 (S230). Specifically, in step S230, the motion w-i stabilized by the sum of the additive value using the smoothing KernelOO and the inter-frame motions is generated.

Smoothing Kernel (K) can be implemented with Gaussian kernel. When implemented with a Gaussian kernel, the highest weight can be enjoyed at 't', which is the frame output time. In this case, the input section of the smoothing kernel (K) is from 't-N' to 't + N'. 't + N' is the current time, after the frame output time, and 't-N' is before the frame output time. Therefore, for the motion stabilization at step S230, both the output time point and the motions before and after the time point are used. In the Gaussian kernel, as the sigma increases, the Gaussian distribution becomes flat and the stabilization increases.

Therefore, in order to satisfy both stabilization and rapid change of the screen, the absolute value average of motions from the frame output time 't' to the current time 't + N' is calculated. Σ can be determined based on Equation 2 below.

[Equation 2]

σ = σ.

init ... (d _t <d _t )

Where d _th is the threshold, c is a constant, and a _init is the initial value of σ, which is determined by the specifications of the system and the desired performance.

In FIG. 3, the dx components of the motions stabilized by step S230 of FIG. 2 are indicated by a dashed line, and it can be confirmed that the shaking is alleviated.

Next, the image processor 120 groups the frames constituting the high fps image into a plurality of segments (S240). The number of frames included in the segments is equal to n.

The number n of frames included in the segment may be determined by Equation 3 below.

[Equation 3]

JV ^S H

fps _L where fps _H is the frame rate of the high fps image input to the image processor 120, and fps _L is the frame of the low fps image to be output from the image processor 120. If f _pSH is 120 and ips _L is 30, then n = 4. That is, it will contain a ^"segments all four frames.

Thereafter, the image processor 120 selects one of the most stable frames for each segment (S250). The frame selected in step S250 is a frame having a minimum difference between the actual cumulative motion (Ά′ο) and the stabilized cumulative motion (^ o), and is specifically selected by using Equation 4 below.

[Equation 4]

D _t is the difference between the actual cumulative motion and the stabilized cumulative motion, where the cumulative motion is a cumulative motion from the output time point t to a subsequent time point t + n. And, f means ^ whose D _t is the minimum among ^ belonging to the segment ^.

4 schematically illustrates a specific method of selecting a frame for each segment. Next, the image processor 120 generates a motion-stabilized low fps image with the selected at step S250 (S260). The low fps image generation in step S260 is generated using Equation 5 below.

[Equation 5]

Here, denotes a frame constituting a low fps image, and may be generated by multiplying by and multiplying ^ o. Then, ^(Λ) is the inverse of, which is the actual cumulative motion from the key frame ^J o to, and is the stabilized cumulative motion in the same period.

Thereafter, the image processor 120 outputs the motion stabilized low fps image generated in operation S260 ^' (S270).

Up to now, the shake image stabilization method and the image processing apparatus to which the same has been described in detail with reference to a preferred embodiment.

In the above embodiment, it is assumed that the image capturing unit 110 stabilizes an image acquired through photographing, which is merely exemplary. The technical idea of the present invention is also applicable to stabilization of an image stored in a storage medium or an image received through a network.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the specific embodiments described above, the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

Claims

[Range of request]

[Claim 1]

Selecting some of the frames constituting the first image at the first frame rate;

Motion stabilizing the frames selected in the selecting step; And

And outputting a second image having a second frame rate composed of frames stabilized in the stabilizing step.

[Claim 2]

The method of claim 1,

Grouping the frames constituting the first image into a plurality of segments;

The selection step,

An image stabilization method comprising selecting a frame for each segment.

[Claim 3]

Tax in Clause 2

The selection step is'

An image stabilization method characterized by selecting the most stabilized frame for each segment.

[Claim 4]

The method of claim 3, wherein

The most stabilized frame is,

The image stabilization method of claim 1, wherein the frame is a frame having a minimum difference between the actual cumulative motion and the stabilized cumulative motion.

[Claim 5]

The method of claim 2,

The first frame rate is,

And the second frame rate is higher than the second frame rate.

[Claim 6]

The method of claim 5,

The number of frames included in the segments is n,

n = 'first frame rate I second frame rate'.

[Claim 7]

The method of claim 1, Extracting motions between frames constituting the first image; And generating stabilized motions from the motions extracted in the extraction step.

The stabilization step,

And performing a motion stabilization by multiplying the selected frame by the inverse of the cumulative motion up to a specific point in time, and multiplying the stabilized cumulative motion from the previous point in time to the corresponding point in time of the frame.

[Claim 8]

The method of claim 7,

The stabilized motion generation step,

An image stabilization method comprising generating a stabilized motion by a weighted sum using motions of a previous time and motions of a later time.

[Claim 9]

The method of claim 8,

The weight is,

The image stabilization method of claim 1, wherein the image is determined based on motions from the frame output time point to the subsequent time points. [Claim 10]

An acquirer configured to acquire a first image having a first frame rate; And

An image processor that selects some of the frames constituting the first image acquired by the acquisition unit, performs motion stabilization of the selected frames, and outputs a second image having a second frame rate composed of the stabilized frames; An image processing apparatus, characterized in that.