WO2009109047A1

WO2009109047A1 - Automatic video method and system

Info

Publication number: WO2009109047A1
Application number: PCT/CA2009/000272
Authority: WO
Inventors: Marc Berube; Dominic Lessard
Original assignee: Marc Berube; Dominic Lessard
Priority date: 2008-03-05
Filing date: 2009-03-03
Publication date: 2009-09-11

Abstract

A system and a method for people to shoot and record videos, in a booth, to select a pre-existing video as a background on which to seat the recorded scenes, the combination of the scenes and the selected background being performed automatically, and to sent the resulting video on the spot, via e-mail for example, using an interface in the booth, such as a touch screen or a keyboard, all this within a few minutes, depending on the length of the video.

Description

TITLE OF THE INVENTION Automatic video method and system

FIELD OF THE INVENTION

[0001] The present invention relates to a video method and system.

More specifically, the present invention is concerned with a method and a system for recording personalized videos and sending them.

SUMMARY OF THE INVENTION

[0002] There is provided a method for creating customised videos, comprising shooting a video against a plain background; extracting the plain background from the video; dynamically combining the video with an alternate background; and encoding the resulting video.

[0003] There is further provided a system for recording videos in a booth, comprising a user layer, by which a user enters selections; a processing layer receiving the user's selections from the user layer; and a transmission layer interfaced with the processing layer; wherein the processing layer activates shooting of a video against a plain background, extracting the plain background from the video, dynamically combining the video with an alternate background; and encoding the resulting video into a format suitable for numerical transmission by the transmission layer, according to the user's selections.

[0004] There is further provided a booth for recording and sending videos, comprising a camera, a background screen, a lighting system including lamps positioned behind the background screen and projectors illuminating a subject positioned in front of the background screen, and a user interface for a user to select parameters for a desired video by the user interface; wherein a monitor drives shooting of a first video against the background screen in the booth by the camera, and processing of the first video into a desired second video.

[0005] Other objects, advantages and features of the present invention will become more apparent upon reading of the following non- restrictive description of embodiments thereof, given by way of example only with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0006] In the appended drawings:

[0007] Figure 1 is a flowchart of a method according to an embodiment of an aspect of the present invention; and

[0008] Figure 2 illustrates a system according to an embodiment of another aspect of the present invention; and

[0009] Figures 3 illustrate a booth according to an embodiment of still another aspect of the present invention.

DESCRIPTION OF EMBODIMENTS OF THE INVENTION

[0010] Generally stated, there is provided a method and a system for recording a video, combining the recorded video with a pre-existing video and encoding the resulting combined video into a format for transmission over the internet, in an automated way, within an enclosure of reduced size.

[0011] The present invention provides filming a subject against a background consisting of a single color or a relatively narrow range of colors, usually blue or green because these colors are considered to be the furthest away from skin tone, and then replacing the portions of the video which match the preselected color by an alternate background video, to yield a customized video, and send it over the Internet.

[0012] This is done, automatically, and in an enclosure such as a photo booth for example,

[0013] As known in the art of color keying, blue is generally used for both weather maps and special effects because blue is a color complementary to the human skin tone. Green is also used because digital cameras retain more details in the green range: at least in the early digital formats, the green channel was sampled twice as often as the blue. Moreover, as green has a higher luminance value than blue, it requires less light than blue, making it easier to work with. Apart from green and blue, any color can be used. Although red for example is usually avoided due to its prevalence in normal human skin pigments, red may be adequate for objects and scenes which do not involve people. Occasionally, a magenta background may also used.

[0014] The method, according to an embodiment illustrated in Figure

1 for example, basically comprises shooting a video of a subject against a plain background screen in a booth (Step 12), extracting the plain background from this video (Step 14), dynamically combining this video devoid of its plain background with an alternate background (Step 16). Then, the resulting video may be encoded into a format suitable for numerical transmission (Step 18). Alternatively, it may be saved on the spot on a storage medium for the user to take away.

[0015] For step 12, a camera, a background screen and lighting sources are selected, as well as a microphone for sound recording.

[0016] As far as cameras are concerned, it is found that a minimum of 30 images per second would yield a satisfactory result.

[0017] Thus, analog video surveillance cameras may have the expected performances in terms of the number of images per second. However, with analog video surveillance cameras, the colors may not be accurately rendered, due to the physical restriction of the present system, which causes a difficulty in white balance adjustments.

[0018] A standard video camera may be used providing manually setting the white balance using the proper functionalities of the camera. Reduction or automation of such adjustments are incorporated in the present method. Those adjustments may include setting a proper white balance, fixing the aperture of the lens and setting the shutter speed.

[0019] A personal digital video camera such as a standard Mini-DV camera may also be used. Alternatively, a stand alone digital video camera with built-in white balance adjustments functionality in addition to manual aperture and shutter adjustments may also be used.

[0020] Once positioned for shooting, the camera may be made to rotate to better match the filmed surface, since typically the plain background is taller than wide and a standard video camera image is typically wider than tall. The image is rotated back during the processing step.

[0021] To capture the video feed from the camera, a video recording card is selected that easily integrates into the system comprising the camera and the processor. For example, an application program interfaces such as the Microsoft Direct X API (application programming interface) may be used to ease configuration of video recording cards. Modules such as DirectShow and DirectSound may be used to develop a central configuration interface and to isolate peripheral equipments from the user interfaces of the system. This allows the replacement of the implementation of a module of the software being developed with another implementation with no impacts on other modules of the system, since the interface of each module remains unchanged.

[0022] Alternatively, the capture of the video feed from the camera may be done using a Firewire™ interface (IEEE1394) and capture software also developed with the Direct X API for example.

[0023] In the present invention, the space of the enclosure of the shooting is typically reduced, and therefore the use of an opaque green screen lighted from the front may involve that the subject being filmed needs be too far from the green screen in order to give sufficient space for the light sources to evenly illuminate the green screen without throwing ray of lights onto the subject. To alleviate this restriction, a transparent Plexiglas screen, covered with a green film for example, may be used, such as a painted film or a vinyl film printed with green. This Plexiglas screen is then lightened from the back, allowing the subject to be relatively close to the green screen in front thereof. It is to be noted that a painted may yield a non-uniform color, whereas a color printed on a vinyl film for example provides an adequately uniform color. However, color printed on a vinyl film may cause glare problems. To alleviate these glare problems, a stretched sheet of green spandex fabric may be used. A mat color further allows avoiding stray lights that may adversely affect extraction of the background in step 14.

[0024] The light sources are selected so as to yield an even lighting and thereby avoid shadow, because a color range as narrow as possible facilitates the further step of extracting the plain background (Step 14) for replacement by an alternate background. Indeed, shadows would be perceived as a darker color by the camera and therefore they may not be registered for replacement. Overall, a similar light intensity on both the subject and the background screen allows eliminating such shadow effects. On the other hand, the background screen illumination needs to be limited so as to prevent spill of the background on the subject, which may occur as a result of contamination of the subject by the light coming from the background screen.

[0025] In step 14, the plain background of the recorded video is extracted numerically as known in the art. A commercial software or a homemade video keyer may be used to perform this task.

[0026] Step 16 comprises automatically inlaying the recorded video, deprived of its plain background, within a selected background.

[0027] A number of commercial softwares are available for combining backgrounds with videos shot against a green or blue screen, by numerical incrustation, such as, for example, Serious Magic Ultra 2™ (now an Adobe ™ product), FxHome Composite Lab Pro ™, SetupFlixPro PC Demo ™, Veescope ™ key Chroma Keying plug-in (for Sony Vegas), BorisFx Video key And Matte ™ plug-in (for Adobe Premiere Pro). Usually, these are heavy applications or extension modules adaptable to image processing applications, and they require manual intervention of an expert person for parameter adjustments and selection of the zones to be treated. These applications are known as video keying softwares.

[0028] In the present method and system, an application allowing incrustation of two video sequences is developed under the form of an autonomous module so as to allow integration thereof into the overall processing and automation.

[0029] Moreover, in order to interact with a video keying software, a control module is developed to interact programmatically with the external application, as is done in automated tests softwares used for regression tests for example, by sending keyboard and mouse events under the form of windows messages to the event loop of the controlled software, which is, in the present case, the video keying software.

[0030] Messages and parameters to be transmitted to the internal processing loop of the video keying software are pre-identified. Sending those messages directly to the message loop of the video keying software simulates a user going through the different steps requested by the software.

[0031] As a result, the incrustation step is totally automated.

[0032] Encoding the resulting video into a format suitable for numerical transmission (Step 18) is achieved as known in the art. A good compression ratio is important to get a good balance between the quality and the size of the resulting video. For the video file to be sent by email, a maximum file size that suits all current major email providers (such as Hotmail, GoogleMail or Yahoo) is found to be about 10 Mb.

[0033] Figure 2 illustrates a system according to an embodiment of the present invention. The system 200 generally comprises a user layer 210, a processing layer 310 and a transmission layer 410.

[0034] The user layer 210 comprises a payment unit 214, such as a magnetic card reader, for the user to pay an amount corresponding with the selected video message he/she has selected via a user interface 212, such as a touch screen for example, which may be located within the booth as described hereinabove. The user also selects, via the user interface 212, the length of the video, recipient's addresses for transmission etc... These selections are transmitted to the processing layer 310.

[0035] The processing layer 310 generally comprises a processor

312, a video capture unit 314, a library of background videos 316, an image processing unit 318, an interface 320 to the user layer 210 and an interface 322 to the transmission layer 410.

[0036] The video capture unit 314 includes a camera, a microphone for sound, a background screen, a lighting system as described hereinabove, in an enclosure. Typically, the enclosure is a reduced space of the type of a coin operated photo boot for example, with a typical surface of about 4' X 8' (1 ,20 m X 2,40 m) and a height of about T (2, 13 m).

[0037] It is found that the quality of extraction in step 14 depends on a number of parameters in step 12, including the color of the background screen, the reflectance of the background screen and the illumination (direct, indirect, halogen incandescent or fluorescent) of the background screen; the coloration of the subject (colors on the recorded video that are close to the color of the plain background will be eliminated ); and the distance between the subject and the background screen for example.

[0038] Indeed, in step 12, in conditions of a reduced space, the subject is placed very close, for example at a distance between 12 to 18 inches, to the background screen, a distance of about 6 feet typically separates the background screen from the camera, and the background screen is typically about 4 feet wide. As mentioned hereinabove, such relative positions of the subject and the background screen may create unwanted shadows. Therefore, the background screen is illuminated from behind to reduce those unwanted shadows. A translucent material may be selected, such as a vinyl film printed with green, or a stretched sheet of green spandex fabric for example, as described above.

[0039] Also, for step 12, as mentioned hereinabove, the lighting system is selected to provide an even lighting in the enclosure. For example, fluorescent lamps may be positioned behind the background screen to allow leveling the brightness of the screen. An optic density meter (photometer) may also be used to balance illumination of the subject from the front with the power of such fluorescent lamps positioned behind the background screen.

[0040] In step 14, the processor 312 coordinates the selection of an alternate background in the backgrounds library 316 according to the user's selection, video recording by the video capture unit 314 for a duration requested and paid by the user, processing by the image processing unit 318 and transmission of the resulting video message to the recipient(s) listed by the user.

[0041] The backgrounds library 316 comprises pre-recorded videos or static images, among which the user may select a desired background for his/her own video.

[0042] The image-processing unit 318 comprises a video color keyer and a video encoder as described hereinabove.

[0043] A watchdog service 500 may monitor the activity of the whole system. In case of a lock or crash, such watchdog service is able to restart the software automatically without human intervention, thereby providing autonomy to the booth.

[0044] Booths, such as illustrated for example in Figures 3, for recording and sending videos that are customised by a user, may be placed in a number of locations to which people go often or generally especially for rest or recreation, as on vacation, such as but not limited to resorts, museums, tourist sites, sports events etc... The user may select backgrounds such as scenic vistas of the places he/she is. Monitoring and control of the booths may be remotely achieved via an Internet link. Each booth can accommodate at least one person.

[0045] Figures 3 illustrate an example of a booth 20, as an enclosure in which typically at least one user 22 can stand (see Figure 3a). The enclosure houses a video capture unit including a camera 24, a microphone, a background screen 28, a lighting system including lamps 26 positioned behind the background screen 28 as mentioned hereinabove, projectors 30 illuminating the user, and a user interface 212, 214. The user 22 positions himself/herself in front of the background screen 28 as described hereinabove, once his/her selections are made through the user interface 212, 214.

[0046] For a 20 second-long video for example, it may take, from the moment the user starts the process within the booth to the moment the video is ready for transmission after the user has had the opportunity to see it once, about 4 minutes 20 seconds. A 45 seconds video may take about 6 minutes for example.

[0047] The user may then exit the booth and leave, while the transmission is completed.

[0048] The system and method of the present invention therefore allow people to shoot and record videos of themselves, in the booths, to select a pre-existing video as a background on which to seat the recorded scenes, the combination of the scenes and the selected background being performed automatically, and to sent the resulting video on the spot, via e-mail for example, using an interface in the booths, such as a touch screen or a keyboard, all this within a few minutes, depending on the length of the video.

[0049] There is provided an automated system and method for creating customised videos in a space like a photo booth and sending them under a numerical format.

[0050] The present method and system allow high-quality image and control of the extraction of the background screen, using a chromatic key technology to remove solid colored backgrounds, or usually green or blue screens, and replace them with transparency to facilitate 'background replacement'. A proper calibration of the shades is performed so that the extraction does not reduce the quality of the shot scenes. Furthermore, the image quality is enhanced by the elimination of the blue or green spill, which may cause a halo surrounding the subject scene.

[0051] Although the present invention has been described hereinabove by way of embodiments thereof, it may be modified, without departing from the nature and teachings of the subject invention as defined in the appended claims.

Claims

WHAT IS CLAIMED IS:

1. A method for creating customised videos, comprising: a) shooting a video against a plain background; b) extracting the plain background from the video; c) dynamically combining the video with an alternate background; and d) encoding the resulting video.

2. The method of claim 1 , wherein said step a) comprises shooting a video using one of: i) a personal digital video camera and ii) a standalone digital video camera, against a plain background screen.

3. The method of claim 1 , wherein said step a) comprises shooting a video against a screen lighted from the back.

4. The method of claim 1 , wherein said step c) comprises automatically inlaying the video, deprived of the plain background, within the alternate background.

5. A system for recording videos in a booth, comprising: a user layer, by which a user enters selections; a processing layer receiving the user's selections from the user layer; and a transmission layer interfaced with said processing layer; wherein said processing layer activates shooting of a video against a plain background, extracting the plain background from the video, dynamically combining the video with an alternate background; and encoding the resulting video into a format suitable for numerical transmission by said transmission layer, according to the user's selections.

6. The system of claim 5, wherein said user layer comprises a user interface, the user selecting, via said user interface, parameters for the resulting video.

7. The system of claim 6, wherein said user layer further comprises a payment unit, the user paying an amount corresponding with said parameters for the resulting video.

8. The system of claim 5, said processing layer comprising: an interface to the user layer, said interface receiving the user's selections; a video capture unit comprising a plain background; a library of background videos; an image processing unit; and a processor; wherein said processor activates said video capture unit into shooting the video against the plain background, drives said image processing unit into extracting the plain background from the video once the video is shot, into dynamically combining the video with the alternate background; and into encoding the resulting video into the format suitable for numerical transmission by said transmission layer.

9. The system of claim 5, said processing layer comprising: a video capture unit; a library of background videos; and an image processing unit; wherein said processing layer coordinates selection in said library of background videos according to the user's selections, video recording by said video capture unit for a duration requested and paid by the user, processing by the image processing unit and transmission of the resulting video message to at least one recipient listed by the user by said transmission layer.

10. The system of any one of claims 8 and 9, said video capture unit comprising a background screen and a lighting system.

11. The system of claim 10, said background screen being one of: i) a vinyl film and ii) a stretched sheet of spandex fabric.

12. The system of any one of claims 10 and 11 , said lighting system comprising lamps positioned behind the background screen and projectors illuminating the user.

13. The system of any one of claims 8 to 12, said library of background videos comprising at least one of: i) pre-recorded videos and ii) static images, among which the user selects a background for the resulting video.

14. The system of any one of claims 8 to 13, the image- processing unit comprising a video encoder.

15. Booth for recording and sending videos, comprising a camera, a background screen, a lighting system including lamps positioned behind said background screen and projectors illuminating a subject positioned in front of said background screen, and a user interface for a user to select parameters for a desired video by said user interface; wherein a monitor drives shooting of a first video against said background screen in said booth by said camera, and processing of said first video into a desired second video.

16. The booth of claim 15, wherein said booth is an enclosure able to accommodate at least one person.

17. The booth of any one of claims 15 and 16, wherein said booth is placed in one of: resorts, museums, tourist sites, sports events sites, cultural events sites and entertainments sites.

18. The booth of any one of claims 15 to 17, monitored via an internet link.

19. The booth of any one of claims 15 to 18, further comprising a microphone for recording sound along with the video.