WO2015135477A1

WO2015135477A1 - Handheld terminal and screen anti-shake method and device for handheld terminal

Info

Publication number: WO2015135477A1
Application number: PCT/CN2015/074008
Authority: WO
Inventors: 张辉; 谭焕清
Original assignee: 努比亚技术有限公司
Priority date: 2014-03-14
Filing date: 2015-03-11
Publication date: 2015-09-17
Also published as: CN103885593A; CN103885593B

Abstract

The present invention relates to the technical field of terminals. Disclosed are a handheld terminal and a screen anti-shake method and device for the handheld terminal. The method comprises: when a screen is turned on, starting a motion sensor to monitor the moving state of the handheld terminal; if shaking of the handheld terminal is detected, starting a camera for shooting and carrying out face recognition, and calculating a displacement parameter of a face relative to the screen; and carrying out inverse compensation on a man-machine interface displayed on the screen according to the displacement parameter, so that the man-machine interface is static relative to the face. By means of embodiments of the present invention, by introducing the face recognition technology, the real relative displacement between the face and the screen is considered in screen compensation, so that the terminal carries out synchronous movement displacement compensation on the man-machine interface displayed on the screen according to the displacement parameter in a certain displacement range, the man-machine interface is kept static relative to the face as much as possible, and the stability of screen watching is improved.

Description

Handheld terminal and screen anti-shake method and device thereof

Technical field

The present invention relates to the field of communications technologies, and in particular, to a handheld terminal and a method and apparatus for preventing image stabilization thereof.

Background technique

At present, the screen anti-shake method of the handheld terminal is: monitoring the movement of the handheld terminal through the motion sensor built in the handheld terminal, and performing displacement compensation on the screen display according to the jitter parameter of the mobile terminal. This screen displacement compensation is only compensation for the jitter of the handheld terminal itself, and does not consider the jitter of the screen relative to the human face (human eyes). In fact, the human face (human eyes) is often also shaking, so the stability when viewing the screen. Sexual effects are not good.

Summary of the invention

In view of the above, the technical problem to be solved by the present invention is to provide a handheld terminal and a screen anti-shake method and device thereof, which can solve the screen jitter caused by the shaking of the face and further improve the stability when viewing the screen.

The technical solution adopted by the present invention to solve the above technical problems is as follows:

According to an aspect of the present invention, a screen anti-shake method of a handheld terminal is provided, including:

When the screen is lit, the motion sensor is turned on to monitor the motion state of the handheld terminal;

If the handheld terminal is detected to be shaken, the camera image is started and face recognition is performed, and the displacement parameter of the face relative to the screen is calculated; the human-machine interface displayed on the screen is inversely compensated according to the displacement parameter _, so that the person The machine interface is relatively stationary relative to the face.

Preferably, the calculating the displacement parameter of the face relative to the screen further comprises: calculating a lateral displacement and a longitudinal displacement of the face relative to the display screen, and a distance between the face and the screen and a displacement acceleration.

Preferably, compensating the human-machine interface displayed on the screen according to the jitter displacement parameter further comprises:

According to the lateral displacement and the longitudinal displacement of the jitter, the man-machine interface displayed on the screen is reversely compensated with respect to the direction of the terminal displacement;

The dithering or reducing the dithering human-machine interface is performed according to the distance between the dithering face and the screen and the displacement acceleration.

Preferably, before the shaking start camera captures the face recognition, the method further comprises: performing opposite compensation on the screen display position relative to the moving direction of the handheld terminal according to the jitter parameter of the motion sensor.

Preferably, the jitter parameters include a jitter direction, a jitter acceleration, and a jitter displacement.

Preferably, the inversely compensating the screen display position relative to the moving direction of the handheld terminal according to the jitter parameter of the motion sensor further comprises:

Determining a dithering direction and an amplitude in three dimensions of the screen coordinate system according to accelerations in three dimensions of the screen coordinate system; translating the screen display position according to the dithering direction and amplitude in the three dimensions of the screen coordinate system or Zoom.

Preferably, the method further includes:

When the displacement of the center point of the screen with respect to the center point of the human-machine interface in the horizontal or vertical direction is within a preset range, it is determined that the hand-held terminal has shaken.

Preferably, the calculating the displacement parameter of the face relative to the screen further comprises:

Acquiring preliminary motion data through motion sensors;

The foreground image is captured by the camera, and the captured image is subjected to face recognition;

The motion data is corrected and adjusted by image recognition displacement to obtain a displacement parameter.

Preferably, the photographing the foreground image by the camera further comprises: using the camera to capture the foreground image when the light is sufficient, and using the infrared light line to cooperate with the infrared camera to capture the foreground image when the light is relatively dark.

According to another aspect of the present invention, a screen anti-shake device for a handheld terminal is provided that includes the following modules:

The motion monitoring module is configured to: when the screen is lit, turn on the motion sensor to monitor the motion state of the handheld terminal;

The face recognition module is configured to: when detecting the jitter of the handheld terminal, start camera imaging and perform face recognition, and calculate a displacement parameter of the face relative to the screen;

The display processing module is configured to inversely compensate the human-machine interface displayed on the screen according to the displacement parameter, so that the human-machine interface is relatively stationary relative to the human face.

Preferably, the face recognition module is configured to: start camera imaging and perform face recognition, calculate lateral displacement and longitudinal displacement of the face relative to the display screen, and accelerate the distance and displacement of the face and the screen degree.

Preferably, the display processing module is configured to: inversely compensate the man-machine interface displayed on the screen relative to the direction of the terminal displacement according to the lateral displacement and the longitudinal displacement; and enlarge or reduce according to the distance between the shaking face and the screen and the displacement acceleration Jitter the human machine interface.

Preferably, the display processing module is further configured to: inversely compensate the screen display position for the direction of movement of the handheld terminal according to the jitter parameter of the motion sensor.

Preferably, the motion monitoring module is configured to determine that the handheld terminal has jittered when the center point of the screen is within a preset range in the horizontal or vertical direction with respect to the center point of the human machine interface.

Preferably, the face recognition module is set to:

Acquiring preliminary motion data by the motion sensor;

Preferably, the face recognition module is configured to: use the camera to capture the foreground image when the light is sufficient, and use the infrared light line to cooperate with the infrared camera to capture the foreground image when the light is relatively dark.

According to still another aspect of the present invention, there is also provided a handheld terminal comprising the above-described screen anti-shake device.

The handheld terminal and the screen anti-shake method and device thereof according to the embodiments of the present invention calculate a true relative displacement between a human face (human eye) and a screen by introducing a face recognition technology, and consider the true relative displacement between the face and the screen. In the screen compensation, in order to make the terminal within a certain range of displacement, according to the displacement parameter, the synchronous movement displacement compensation is performed on the man-machine interface displayed on the screen, so that the human-machine interface is kept stationary relative to the human face as much as possible, thereby improving the user's viewing of the screen. Stability, which improves the user's reading experience.

DRAWINGS

1 is a flowchart of a screen anti-shake method according to an embodiment of the present invention;

2 is a flowchart of another screen anti-shake method according to a preferred embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a module of a screen anti-shake device according to an embodiment of the present invention.

detailed description

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, in order to make the present invention. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Embodiment 1

FIG. 1 is a schematic diagram of a screen anti-shake method of a handheld terminal according to an embodiment of the present invention, where the method includes:

S101. When the screen of the handheld terminal is lit, the motion sensor is turned on to monitor the motion state of the handheld terminal.

Among them, the motion sensor can be an acceleration sensor or a gyroscope.

S102. Determine whether the handheld terminal jitter is detected. If yes, execute step S103, otherwise continue monitoring.

Specifically, the jitter means that the displacement of the center point of the screen relative to the center point of the human-machine interface in the horizontal or vertical direction is greater than 1 mm and less than 50% of the width or length of the screen size, that is, the compensation displacement of the present invention is constant. The displacement range is performed. The maximum displacement depends on the screen size (such as the size of the mobile phone and the flat screen). It is better to see half of the human-machine interface. Otherwise, the displacement is too large. The synchronization point is beyond the screen range. In other words, when the displacement of the center point of the screen with respect to the center point of the human-machine interface in the horizontal or vertical direction is within a preset range, it is determined that the hand-held terminal has shaken.

S103. Start camera imaging and perform face recognition, and calculate displacement parameters of the face relative to the screen.

Preferably, calculating the displacement parameter of the face relative to the screen comprises calculating a lateral displacement and a longitudinal displacement of the face relative to the display screen, and a distance between the face and the screen and a displacement acceleration.

Specifically, the foreground image is captured by the camera, and the camera is used when the light is sufficient. When the light is dark, the infrared light is used together with the infrared camera to perform face recognition on the captured image (the face recognition technology can adopt the existing one) Technology, not described in detail here). However, it is not simply relying on image recognition technology to judge displacement, but based on motion sensor data, and further combined with image recognition technology, this can make displacement judgment faster and more accurate. For example, the motion data is first obtained according to the motion sensor, and then the motion of the screen is compared by the contrast of the screen content (for example, the position change of the face in the image before and after the motion is up, down or in which direction) Based on the data, further correct and adjust by image recognition displacement, and calculate the lateral displacement and longitudinal displacement (ie, the amount of compensation data) of the output human-machine interface through the combination of two data, so that the subsequent displacement distance is within the preset range. In the inner direction, the displacement of the human-machine interface relative to the direction of the terminal displacement is reversely compensated. Can also To calculate the distance between the face and the screen and the displacement acceleration, so as to subsequently enlarge or reduce the human-machine interface, as far as possible, the image display is relatively stationary with respect to the human face and the human eye.

S104. Compensate the human-machine interface displayed on the screen according to the displacement parameter.

Preferably, the step S104 further comprises: inversely compensating the human-machine interface displayed on the screen with respect to the moving direction of the terminal according to the lateral displacement and the longitudinal displacement; and enlarging or reducing the human-machine interface according to the distance and acceleration of the human face from the screen . Specifically, in the case where the handheld terminal is not shaken, the human-machine interface is located at the center of the display screen.

When the handheld terminal is shaken to the right parallel to the human face and generates displacement, the human-machine interface moves to the left side of the display screen, and the specific movement amplitude is determined according to the displacement of the handheld terminal.

When the handheld terminal is shaken to the left parallel to the face, the human-machine interface moves to the right side of the display screen, and the specific movement amplitude is determined according to the displacement of the handheld terminal.

When the handheld terminal is parallel to the face and shakes, and the displacement occurs, the human-machine interface moves to the upper side of the display screen, and the specific movement amplitude is determined according to the displacement of the handheld terminal.

When the handheld terminal is shaken parallel to the human face to the upper left and generates displacement, the human-machine interface moves to the lower left side of the display screen, and the specific movement amplitude is determined according to the displacement of the handheld terminal.

When the handheld terminal is away from or close to the human face, the human-machine interface will assume an enlarged or reduced state, and the specific enlargement or reduction ratio is proportional to the acceleration. Specifically, the variation coefficient of the output reduction or amplification is calculated according to the distance change value and the displacement acceleration change value, and the specific reduction or amplification factor (such as 10%, 15% or 20%) can be given by experiments, so that after scaling The size of the human-machine interface is always the same as the original initial state. For example, when the back is 1cm, the interface and the font become smaller due to the distance from the visual angle, and a magnification factor is given according to the amount of change of 1cm backward (for example, the interface is enlarged by 10%), so that the back and forth 1cm displacement is relative to the human eye. The human-machine interface does not seem to change and is still.

The screen anti-shake method and device of the handheld terminal according to the embodiment of the present invention calculates a true relative displacement between a human face (human eye) and a screen by introducing a face recognition technology, and takes into account the true relative displacement between the face and the screen. Within the screen compensation, so that the terminal can perform synchronous displacement compensation on the man-machine interface displayed on the screen according to the displacement parameter within a certain range of displacement, so as to keep the human-machine interface stationary relative to the human face as much as possible, thereby improving the stability of the viewing screen. Thereby improving the user's reading experience.

Embodiment 2

FIG. 2 is a schematic diagram of a screen anti-shake method of a handheld terminal according to a preferred embodiment of the present invention. The method includes:

S201. When the screen of the handheld terminal is lit, the motion sensor is turned on to monitor the motion state of the handheld terminal.

The motion sensor may be an acceleration sensor or a gyroscope, and the motion sensor detects a jitter parameter of the handheld terminal, and the jitter parameter includes a jitter direction, a jitter acceleration, and a jitter displacement. The dithering direction may be included in three dimensions of the handheld terminal screen coordinate system, and the jitter displacement and the jitter acceleration also correspond to the displacement amplitude and the jitter acceleration included in the three dimensional directions.

S202. Determine whether the handheld terminal jitter is detected. If yes, execute step S203, otherwise continue monitoring.

Specifically, when the jitter is detected, the jitter parameter is transmitted to the display processor, and when no jitter is detected, the display is performed in the normal mode while continuing to monitor whether the jitter is present.

S203. Perform opposite compensation on the screen display position relative to the moving direction of the handheld terminal according to the jitter parameter of the motion sensor.

Preferably, the jitter direction and the amplitude in three dimensions of the screen coordinate system are determined according to the accelerations in the three dimensions of the screen coordinate system; and the screen display position is displayed according to the shaking direction and the amplitude in the three dimensions of the screen coordinate system. Pan or zoom.

Specifically, the screen display position is relative to the display screen of the mobile phone (ie, the physical part between the borders of the mobile phone), generally the default full screen display, and the human machine interface is the displayed screen relative to the screen, and generally the default is The picture is full of screens. If the handheld terminal is dithered, firstly, according to the jitter parameter of the motion sensor, the screen display position is displacement-compensated with respect to the opposite movement direction of the handheld terminal, thereby compensating for the jitter of the handheld terminal itself, trying to keep the screen as relatively static as possible, and then further compensating in the screen. HMI.

S204. Start camera imaging and perform face recognition, and calculate displacement parameters of the face relative to the screen.

This step S204 is the same as step S103 of the first embodiment, and will not be repeated here.

S205. Compensate the human-machine interface displayed on the screen according to the displacement parameter.

This step S205 is the same as step S104 of the first embodiment, and will not be repeated here.

The screen anti-shake method of the handheld terminal according to the embodiment of the present invention introduces face recognition technology to calculate the sway displacement of the face relative to the screen, based on the displacement compensation of the display screen by the sensor in the handheld terminal. The man-machine interface further compensates for the displacement and is stabilized by double compensation. Sexual effects are better.

In summary, in the prior art, the position of components inside the video recording module is usually compensated. The present invention compensates for the human-machine interface, and mainly includes two compensation modes: first, according to the lateral displacement and the longitudinal displacement, the human-machine interface displayed on the screen is inversely compensated with respect to the moving direction of the terminal; , enlarge or reduce the human-machine interface according to the distance and acceleration of the face from the screen. Thereby, the human-machine interface is kept as static as possible with respect to the human face, and the stability of the viewing screen is improved.

Embodiment 3

As shown in FIG. 3, a screen anti-shake device of a handheld terminal according to an embodiment of the present invention includes:

The motion monitoring module 10 is configured to: when the handheld terminal screen is lit, turn on the motion sensor to monitor the motion state of the handheld terminal;

The face recognition module 20 is configured to: when detecting the jitter of the handheld terminal, start camera imaging and perform face recognition, and calculate a displacement parameter of the face relative to the screen;

Preferably, the face recognition module 20 is specifically configured to: start camera imaging and perform face recognition, calculate lateral displacement and longitudinal displacement of the face relative to the display screen, and distance and displacement acceleration of the face from the screen.

The display processing module 30 is configured to compensate the human-machine interface displayed on the screen according to the displacement parameter.

Preferably, the display processing module 30 is specifically configured to: inversely compensate the human-machine interface displayed on the screen according to the lateral displacement and the longitudinal displacement; and enlarge or reduce the human-machine interface according to the distance between the face and the screen and the displacement acceleration.

As another preferred embodiment of the present embodiment, the display processing module 30 is further configured to: inversely compensate the screen display position relative to the direction of movement of the handheld terminal according to the jitter parameter of the motion sensor to make the display screen relatively stationary. Specifically, the jitter direction and the amplitude in the three dimensions of the screen coordinate system may be determined according to the accelerations in the three dimensions of the screen coordinate system; according to the jitter direction and the amplitude in the three dimensions of the screen coordinate system Display position for panning or zooming.

It should be noted that the technical features in the first embodiment and the second embodiment of the foregoing method are applicable to the device, and are not repeated herein.

Correspondingly, an embodiment of the present invention further provides a handheld terminal, where the handheld terminal includes the above-mentioned screen anti-shake device. Handheld terminals include, but are not limited to, mobile phones, IPADs, and tablets.

The handheld terminal and the screen anti-shake device of the embodiment of the invention adopt the face recognition technology to consider the real displacement of the face relative to the screen in the screen compensation, so that the terminal is within a certain displacement range, according to the displacement parameter pair The human-machine interface displayed on the screen performs synchronous displacement compensation to keep the human-machine interface still with respect to the human face as much as possible, thereby improving the stability of the viewing screen and thereby improving the user experience.

The preferred embodiments of the present invention have been described above with reference to the drawings, and are not intended to limit the scope of the invention. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and spirit of the invention are intended to be included within the scope of the invention.

Industrial applicability

The handheld terminal and the screen anti-shake method and device thereof according to the embodiments of the present invention calculate a true relative displacement between a human face (human eye) and a screen by introducing a face recognition technology, and consider the true relative displacement between the face and the screen. In the screen compensation, the terminal can perform synchronous motion displacement compensation on the man-machine interface displayed on the screen according to the displacement parameter within a certain range of displacement, and keep the human-machine interface stationary relative to the human face as much as possible, thereby improving the user's viewing screen. The stability improves the user's reading experience.

Claims

A screen anti-shake method for a handheld terminal, comprising:

When the screen of the handheld terminal is lit, the motion sensor is turned on to monitor the motion state of the handheld terminal;

If the handheld terminal is detected to be shaken, the camera image is started and face recognition is performed, and the displacement parameter of the face relative to the screen is calculated;

The human-machine interface displayed on the screen is inversely compensated according to the displacement parameter, so that the human-machine interface is relatively stationary relative to the human face.
The screen anti-shake method according to claim 1, wherein the calculating the displacement parameter of the face relative to the screen further comprises: calculating a lateral displacement and a longitudinal displacement of the face with respect to the display screen, and a distance between the face and the screen and Displacement acceleration.
The screen anti-shake method according to claim 2, wherein compensating the human-machine interface displayed on the screen according to the displacement parameter further comprises:

According to the lateral displacement and the longitudinal displacement, the man-machine interface displayed on the screen is reversely compensated with respect to the direction of the terminal displacement;

The human-machine interface is enlarged or reduced according to the distance between the face and the screen and the displacement acceleration.
The screen anti-shake method according to claim 1, wherein the initiating camera imaging before performing face recognition further comprises:

The screen display position is inversely compensated for the direction of movement of the handheld terminal according to the jitter parameter of the motion sensor.
The screen stabilization method according to claim 4, wherein the jitter parameter comprises a jitter direction, a jitter acceleration, and a jitter displacement.
The screen anti-shake method of claim 4, wherein the counter-compensating the screen display position relative to the direction of movement of the handheld terminal according to the jitter parameter of the motion sensor further comprises:

Determining a dithering direction and an amplitude in three dimensions of the screen coordinate system according to accelerations in three dimensions of the screen coordinate system; translating the screen display position according to the dithering direction and amplitude in the three dimensions of the screen coordinate system or Zoom.
The screen anti-shake method according to claim 1, wherein the method further comprises:

When the center point of the screen is displaced in the lateral or longitudinal direction relative to the center point of the human-machine interface, When the range is within, it is determined that the handheld terminal is shaken.
The screen anti-shake method according to claim 1, wherein the calculating the displacement parameter of the face relative to the screen further comprises:

Acquiring preliminary motion data through motion sensors;

The foreground image is captured by the camera, and the captured image is subjected to face recognition;

The motion data is corrected and adjusted by image recognition displacement to obtain a displacement parameter.
The screen anti-shake method according to claim 8, wherein the photographing the foreground image by the camera further comprises: using the camera to capture the foreground image when the light is sufficient, and using the infrared light line to cooperate with the infrared camera to capture the foreground image when the light is relatively dark .
A screen anti-shake device for a handheld terminal, comprising:

The motion monitoring module is configured to: when the screen of the handheld terminal is illuminated, turn on the motion sensor to monitor the motion state of the handheld terminal;

The face recognition module is configured to: when detecting the jitter of the handheld terminal, start camera imaging and perform face recognition, and calculate a displacement parameter of the face relative to the screen;

The display processing module is configured to inversely compensate the human-machine interface displayed on the screen according to the displacement parameter, so that the human-machine interface is relatively stationary relative to the human face.
The screen anti-shake device according to claim 10, wherein the face recognition module is configured to: start camera imaging and perform face recognition, calculate lateral displacement and longitudinal displacement of the face relative to the display screen, and face and face The distance and displacement acceleration of the screen.
The screen anti-shake device according to claim 11, wherein the display processing module is configured to: inversely compensate a human-machine interface displayed on the screen with respect to a direction in which the terminal is displaced according to the lateral displacement and the longitudinal displacement; The distance between the face and the screen and the displacement acceleration are used to enlarge or reduce the human-machine interface.
The screen anti-shake device of claim 10, wherein the display processing module is further configured to inversely compensate the screen display position relative to the direction of movement of the handheld terminal based on the jitter parameter of the motion sensor.
The screen anti-shake device of claim 13, wherein the display processing module is configured to:

Determining the three dimensions of the screen coordinate system according to the accelerations in three dimensions of the screen coordinate system Dithering direction and amplitude; panning or scaling the screen display position according to the dithering direction and amplitude in the three dimensions of the screen coordinate system.
The screen anti-shake device according to claim 10, wherein the motion monitoring module is configured to: when a displacement of a center point of the screen with respect to a center point of the human-machine interface in a horizontal or vertical direction is within a preset range, Then it is determined that the handheld terminal has shaken.
The screen anti-shake device of claim 10, wherein the face recognition module is configured to:

Acquiring preliminary motion data by the motion sensor;

The foreground image is captured by the camera, and the captured image is subjected to face recognition;

The motion data is corrected and adjusted by image recognition displacement to obtain a displacement parameter.
The screen anti-shake device according to claim 16, wherein the face recognition module is configured to: use a camera to capture a foreground image when the light is sufficient, and use an infrared light line to cooperate with the infrared camera to capture the foreground image when the light is relatively dark.
A handheld terminal includes a screen anti-shake device, and the screen anti-shake device includes:

The motion monitoring module is configured to: when the screen of the handheld terminal is illuminated, turn on the motion sensor to monitor the motion state of the handheld terminal;

The face recognition module is configured to: when detecting the jitter of the handheld terminal, start camera imaging and perform face recognition, and calculate a displacement parameter of the face relative to the screen;

The display processing module is configured to compensate the human-machine interface displayed on the screen according to the displacement parameter.
The handheld terminal according to claim 18, wherein the face recognition module is configured to: start camera imaging and perform face recognition, calculate lateral displacement and longitudinal displacement of the face relative to the display screen, and face and screen Distance and displacement acceleration.
The handheld terminal according to claim 19, wherein the display processing module is configured to: inversely compensate a human-machine interface displayed on the screen with respect to a direction in which the terminal is displaced according to the lateral displacement and the longitudinal displacement; The distance between the face and the screen and the displacement acceleration are used to enlarge or reduce the human-machine interface.