US20110180666A1 - System and method for displaying aerial refueling symbology - Google Patents
System and method for displaying aerial refueling symbology Download PDFInfo
- Publication number
- US20110180666A1 US20110180666A1 US12/139,032 US13903208A US2011180666A1 US 20110180666 A1 US20110180666 A1 US 20110180666A1 US 13903208 A US13903208 A US 13903208A US 2011180666 A1 US2011180666 A1 US 2011180666A1
- Authority
- US
- United States
- Prior art keywords
- boom
- refueling device
- aerial refueling
- component
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENTS OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- B64D39/00—Refuelling during flight
-
- C—CHEMISTRY; METALLURGY
- C07—ORGANIC CHEMISTRY
- C07D—HETEROCYCLIC COMPOUNDS
- C07D271/00—Heterocyclic compounds containing five-membered rings having two nitrogen atoms and one oxygen atom as the only ring hetero atoms
- C07D271/02—Heterocyclic compounds containing five-membered rings having two nitrogen atoms and one oxygen atom as the only ring hetero atoms not condensed with other rings
- C07D271/10—1,3,4-Oxadiazoles; Hydrogenated 1,3,4-oxadiazoles
- C07D271/113—1,3,4-Oxadiazoles; Hydrogenated 1,3,4-oxadiazoles with oxygen, sulfur or nitrogen atoms, directly attached to ring carbon atoms, the nitrogen atoms not forming part of a nitro radical
Definitions
- the present disclosure relates generally to aerial refueling and, more particularly, to displaying aerial refueling symbology.
- in-flight refueling extends the range and endurance of aircraft without requiring the aircraft to land.
- the aircraft to be refueled e.g., the receiver aircraft
- the aircraft to be refueled must be precisely positioned relative to the tanker aircraft to provide safe engagement while the fuel is dispensed to the receiver aircraft.
- the precise relative spatial positioning of the two rapidly moving aircraft makes in-flight refueling a challenging operation.
- One type of in-flight refueling system includes a boom refueling system that includes a rigid boom extending from the tanker aircraft, with a probe and nozzle at its distal end.
- the boom includes airfoils (e.g., rudders) controlled by a boom operator stationed on the refueling aircraft.
- the airfoils allow the boom operator to actively maneuver the boom with respect to the receiver aircraft, which usually flies in a fixed refueling position below and aft of the tanker aircraft.
- the boom operator typically monitors the aerial refueling boom with video displays to assist with guiding the boom nozzle into the receptacle of the receiver aircraft.
- Conventional video displays provide essential information to the boom operator for readily cross-checking the status of the refueling system including boom elevation, azimuth, telescoping length, boom state, and fuel delivery without obscuring or otherwise losing sight of the receiver aircraft.
- Warning indicators are positioned so as to attract the boom operator's attention when concentrating on positioning the boom nozzle in the receiver aircraft's receptacle, without compromising safety by obscuring essential areas of the scene.
- Some conventional video displays provide visual overlays of this essential information in the form of linear scales, numeric fields, and warning indicators that are displayed at fixed locations in the displayed images.
- the visual overlays displayed over the images of the receiver aircraft obscure the boom operator's view of the receiver aircraft during the aerial refueling process, which is typically unsafe.
- conventional overlays may obscure dangerous situations in aerial refueling, which may increase the occurrence of damage to the aerial refueling boom and receiver.
- Systems and methods disclosed herein provide for overlaying or superimposing graphic display symbology on a displayed image, such as a photographic scene of an aerial refueling operation between a tanker aircraft and a receiver aircraft.
- the graphic display symbology provides an operator with information on position, status, and cautions/warnings of an aerial refueling device.
- the graphic display symbology presents information to an operator in an easily viewable form, reduces interference with viewing receiver aircraft, and provides warning indications within the operator's field of view (e.g., primary field of view).
- a system for in-flight fuel delivery includes an aerial refueling device adapted to provide fuel to a receiver aircraft, an optical component adapted to capture images of the aerial refueling device and the receiver aircraft, an operator input component adapted to interface with an operator and capture control signals as input from the operator, and a display component adapted to display images.
- the system includes a controller adapted to receive the captured images and the captured control signals, process the control signals by generating graphic display symbology, process the images by generating a combined image having the generated graphic display symbology superimposed on the images, and display the combined image on the display component for viewing by the operator.
- the controller is adapted to superimpose the graphic display symbology on a portion of the images obscured by the aerial refueling device.
- the controller is adapted to process the control signals by determining a position of the aerial refueling device.
- the controller is adapted to dynamically position the graphic display symbology over the portion of the images obscured by the aerial refueling device as the aerial refueling device moves in the images.
- the aerial refueling device includes a boom component and at least one rudder component having physical markings on a portion thereof.
- the controller superimposes the graphic display symbology on a portion of the image proximate to the physical markings
- the optical components include stereoscopic cameras adapted to have a field of view of at least one of the aerial refueling device and the receiver aircraft, and the graphic display symbology is superimposed over the image in the field of view.
- the graphic display symbology includes a plurality of chevron symbols that define a boom envelope in the image.
- the chevron symbols may be positioned at corners of the boom envelope.
- the chevron symbols may be sized and/or positioned for a specific type of receiver aircraft and boom extension.
- the graphic display symbology includes a center symbol positioned at a center of the boom envelope.
- the graphic display symbology includes a plurality of position indicators that provide position information related to the aerial refueling device including at least one of an elevation position, an azimuth position, and a telescoping position.
- the graphic display symbology includes warning indicators and status indicators that are displayed in the image with a semi-transparent pixel mask to avoid obscuring the aerial refueling device or the receiver aircraft.
- the graphic display symbology includes a force cue, such as an arrow superimposed on a portion of the aerial refueling device. The arrow defines a direction of projected force on a nozzle component of the aerial refueling device.
- the graphic display symbology includes a master caution indicator and a master warning indicator superimposed on a portion of the aerial refueling device.
- the master caution indicator comprises a yellow color to indicate a master caution state
- the master warning indicator comprises a red color to indicate a master warning state.
- the graphic display symbology includes graphic position gauges and refueling status lights that are displayed over and track the position of at least one of a boom component and a rudder component in the images of the aerial refueling device.
- the graphic position gauges provide position information related to the aerial refueling device including at least one of an elevation position of the boom component, an azimuth position of the boom component, and an extension position of the boom tip component.
- One or more refueling status indicators provide an indication of a refueling status including but not limited to the terms Ready, Contact, and Disconnect.
- a method for in-flight fuel delivery from a tanker aircraft to a receiver aircraft via an aerial refueling device includes receiving images from an optical component, the images having a field of view of at least one of the aerial refueling device and the receiver aircraft and receiving control signals from an operator input component.
- the method includes processing the control signals by generating graphic display symbology and processing the images by generating a combined image having the generated graphic display symbology superimposed on the images.
- the method includes displaying the combined image on a display component.
- the graphic display symbology is superimposed on a portion of the images obscured by the aerial refueling device.
- FIG. 1A shows a tanker aircraft refueling a receiver aircraft, in accordance with an embodiment of the present disclosure.
- FIG. 1B shows a system for controlling an aerial refueling device, in accordance with an embodiment of the present disclosure.
- FIG. 1C shows a field of view (FOV) of a tanker aircraft refueling a receiver aircraft, in accordance with an embodiment of the present disclosure.
- FOV field of view
- FIG. 1D shows a receiver aircraft, in accordance with an embodiment of the present disclosure.
- FIGS. 2A-2B show various FOV displays having graphic display symbology, in accordance with one or more embodiments of the present disclosure.
- FIGS. 3A-3E show various exemplary embodiments of an aerial refueling device in various FOV displays having graphic display symbology, in accordance with one or more embodiments of the present disclosure.
- FIGS. 4A-4B show various other exemplary embodiments of an aerial refueling device in various FOV displays having graphic display symbology, in accordance with one or more embodiments of the present disclosure.
- FIG. 5 shows one embodiment of a process for facilitating FOV displays having graphic display symbology, in accordance with an embodiment of the present disclosure.
- FIG. 6 is a block diagram of a computer/controller system suitable for implementing one or more embodiments of the present disclosure.
- systems and methods disclosed herein provide for overlaying graphic display symbology on a displayed image, such as a photographic scene, to provide an operator with information on position, status, and cautions/warnings of an aerial refueling device.
- the graphic display symbology includes a plurality of graphical symbols related to the aerial refueling operation and presents information to an operator in an easily viewable form, reduces interference with viewing receiver aircraft, and provides warning indications within the operator's field of view (e.g., primary field of view).
- the symbology may be superimposed on portions of the scene.
- tracking of the aerial refueling device may operate in conjunction with physical markings on the underside of the boom to avoid obscuring the operator's view of receiver aircraft.
- the graphic display symbology provides an operator with essential information for monitoring and controlling aerial refueling.
- the graphic display symbology displayed over one or more components of a boom provides easily viewable information close to a center of the operator's field of view during pre-contact and contact phases of air refueling while maximizing the available unobstructed view of the receiver aircraft.
- systems and methods disclosed herein are capable of displaying boom position (e.g., elevation angle, azimuth angle, and telescoping length), fuel system state (e.g., ready/contact/disconnect), and warnings, cautions, and alerts superimposed on images of an aerial refueling display.
- boom position e.g., elevation angle, azimuth angle, and telescoping length
- fuel system state e.g., ready/contact/disconnect
- warnings, cautions, and alerts superimposed on images of an aerial refueling display.
- the position of data and information in the displayed field of view is varied continuously so as to superimpose the symbols on the displayed image of the aerial refueling boom in the field of view.
- essential data and information are provided in an operator's field of view without obscuring the operator's view of the receiver aircraft.
- the overlays operate in concert with markings on the aerial refueling boom to provide the operator with analog indication of the boom's angular position with respect to the contact, disconnect
- FIG. 1A shows one embodiment of an aerial refueling system 160 comprising a tanker aircraft 100 refueling a receiver aircraft 110 a with an aerial refueling device 120 .
- the tanker aircraft 100 includes a fuselage 101 , one or more wings 102 , and one or more engines 103 .
- the aircraft 100 may include various other types of configurations.
- the aerial refueling device 120 may include a refueling boom 121 having one or more actuatable control components 122 (e.g., rudders). The positions of the control components 122 may be adjusted to steer the boom 121 into engagement with the receiver aircraft 110 a. The position of the boom 121 may be controlled based on inputs received via one or more sensors and/or inputs received from a human operator.
- the aerial refueling device 120 may be configured to allow the operator to manually control the boom 121 and/or may allow the system 160 to automatically control the boom 121 , either separately or simultaneously. In one implementation, the operator may control the location of the boom 121 with assistance from the automatic portion of the system 160 , while allowing the operator to override the automatic portion of the system 160 at any point.
- FIG. 1B shows one embodiment of a plurality of components of the system 160 in reference to FIG. 1A .
- the system 160 includes a controller 130 adapted to control the motion of the aerial refueling device 120 , based on inputs received from an operator input device 150 and one or more sensor 140 .
- the operator input device 150 may include a stick 151 or various other types of manual input control device (e.g., a yoke).
- Each sensor 140 in one embodiment, may include an optical sensor, such as one or more photosensitive detectors 141 (e.g., a pair of cameras) that provide a stereoscopic image to the controller 130 .
- the controller 130 may receive input signals from the one or more sensors 140 and the operator input device 150 and, based upon a combination of the input signals, direct a command signal to the aerial refueling device 120 to change and/or alter a position of the aerial refueling device 120 .
- the controller 130 is adapted to direct actuation of the control components 122 to provide for lateral motion of the boom 121 , as indicated by arrow A, and vertical or pitch motion of the boom 121 , as indicated by arrow B.
- the controller 130 is adapted to control axial actuation (e.g., extension, such as telescoping extension) of a deployable boom tip 123 , as indicated by arrow C.
- an actuator 152 may be coupled between the controller 130 and the operator input device 150 to provide feedback to the operator.
- a corresponding command signal may be directed to the actuator 152 to move the operator input device 150 in a corresponding position.
- the operator may sense the stick 151 move in a manner that tracks the motion of the boom 121 and/or in a manner that is commanded by the controller 130 based on input from the one or more sensors 140 .
- the operator may influence the position of the boom 121 .
- the rate at which the controller 130 directs a motion of the boom 121 based on the operator input may vary proportionately to the force applied by the operator to the operator input device 150 .
- the controller 130 in one embodiment, is adapted to display images from the one or more sensors 140 (e.g., one or more image or video cameras) on a display component 153 for viewing by the operator.
- the controller 130 may be adapted to display a field of view that includes the receiver aircraft 110 a and the aerial refueling device 120 on the display component 153 .
- the controller 130 is adapted to graphically display operator input from the operator input device 150 .
- the controller 130 may be adapted to superimpose graphics on the displayed images for viewing by the operator.
- the graphics may include control graphics that indicate positions of the aerial refueling device 120 , including the boom 121 , the control components 122 and various other components thereof.
- FIG. 1C shows one embodiment of a partially cut away side view of the tanker aircraft 100 refueling the receiver aircraft 110 a.
- a sensor pod 142 houses the one or more sensors 140 of FIG. 1B at a location where the one or more sensors 140 may detect the position of the boom 121 and the receiver aircraft 110 a.
- the one or more sensors 140 is adapted to have a sensor field of view 143 that extends downwardly and aft from the sensor pod 142 .
- the one or more sensors 140 within the sensor pod 142 is adapted to detect a position of at least one of the receiver aircraft 110 a and the boom 121 so as to determine a position error E.
- the position error E may include the vector distance between the tip of the boom 121 and a refueling receptacle 111 of the aircraft and may include position components in a plurality of coordinate directions (e.g., azimuth, elevation, and extension such as telescoping position).
- the controller 130 in one embodiment, is adapted to display images from the field of view 124 of the one or more sensors 140 (e.g., one or more image detectors) on the display component 153 for viewing by the operator.
- the displayed images of the field of view 124 may include display graphics that provide information related to the position of the aerial refueling device 120 and the various components thereof including the boom 121 , the control components 122 and the telescoping boom tip 123 .
- the display graphics including various aspects thereof are described in greater detail herein.
- FIG. 1D shows one embodiment of a receiver aircraft 110 b with one suitable position for a fuel receptacle 181 .
- the fuel receptacle 181 may include a distinctive visual appearance and/or may include visual cue markings 182 that assist the one or more sensors 140 in precisely identifying the location of the fuel receptacle 181 .
- the one or more sensors 140 may be used to optically determine the distance between the boom tip 111 and the receptacle 182 .
- the boom 121 may include a separate position sensor (e.g., an accelerometer with a processor), and the position detected by the boom position sensor may be combined with the visual image of the fuel receptacle 181 to determine the distance between the boom tip 111 and the fuel receptacle 181 .
- a separate position sensor e.g., an accelerometer with a processor
- FIG. 2A shows one embodiment of a field of view (FOY) display 200 having graphic display symbology 210 including a plurality of chevron symbols 212 , a center symbol 214 , a master warning/caution indicator 216 , a force cue 218 , a plurality of status indicators 220 , and a plurality of position indicators 230 .
- the chevron symbols 212 mark four corners of a disconnect envelope, which identify a boom envelope 208 (e.g., frame of control). In various implementations, the chevron symbols 212 may be sized and/or positioned for a specific type of receiver aircraft and boom extension.
- the center symbol 214 in one embodiment, identifies the central region of the boom envelope 208 as identified by the chevrons 212 .
- the master warning/caution indicator 216 in one embodiment, is positioned proximate to the central region of the boom envelope 208 . In this position, the master warning/caution indicator 216 is easily viewable by an operator for immediate indication of an alert, a caution or a warning. In various implementations, the master warning/caution indicator 216 comprises a red color to indicate a master warning state and a yellow color to indicate a master caution state.
- the force cue 218 in one embodiment, identifies the direction of force on the aerial refueling device 120 so that the operator may steer or maneuver the aerial refueling device 120 in a corrective or compensating manner.
- the status indicators 220 identify various information related to the “state of the fuel system” of the aerial refueling device 120 and include a fail status indicator 222 , a ready status indicator 224 , a contact status indicator 226 , and a disconnect status indicator 228 .
- the status indicators 230 comprise a semi-transparent rectangular graphic with alphanumeric characters that provide the operator with easily viewable information about the status (e.g., fuel system state) of the aerial refueling device 120 .
- the warning and status indicators are displayed with a pixel mask that allows the underlying image to show through, which may avoid obscuration of the visual scene.
- the fail status indicator 222 indicates to the operator that the aerial refueling device 120 is positioned outside the boom envelope 208 , as indicated by the chevrons 212 .
- the ready status indicator 224 indicates to the operator that the aerial refueling device 120 is ready for operation including steering and maneuvering.
- the contact status indicator 226 indicates to the operator that the aerial refueling device 120 is in contact with the receiver aircraft 110 a, for example, as shown in FIG. 1A .
- the disconnect status indicator 228 indicates to the operator that the aerial refueling device 120 is disconnected from the receiver aircraft 110 a, which may be indicated after refueling of the receiver aircraft 110 a has been completed.
- the position indicators 230 identify positions of the aerial refueling device 120 and include an elevation position indicator 232 , an azimuth position indicator 234 , and an extension (e.g., telescoping) position indicator 236 .
- the position indicators 230 include a circular arrangement of multi-dot circles and a data component (e.g., alphanumeric component) to identify a particular degree of orientation in elevation (el) 232 , azimuth (az) 234 , and telescope (tel) 236 .
- the multi-dot circles e.g., 10-dot circles
- the position indicators 230 may include a hand that rotates clockwise and/or counter-clockwise to visually indicate the degree of orientation in reference to the data component.
- the elevation position indicator 232 identifies the degree of elevation movement of the boom 121 of the aerial refueling device 120 , as described in reference to the vertical or pitch motion of the boom 121 by arrow B in FIG. 1B .
- the azimuth position indicator 234 identifies the degree of azimuth movement of the boom 121 of the aerial refueling device 120 , as described in reference to the lateral motion of the boom by arrow A of FIG. 1B .
- the extension or telescoping position indicator 236 identifies the degree of axial movement of the boom tip 123 from the boom 121 of the aerial refueling device 120 , as described in reference to the extending or telescoping motion of the boom tip 123 by arrow C of FIG. 1B .
- FIG. 2B shows one embodiment of another field of view display 250 having graphic display symbology 210 including the chevrons 212 , the center symbol 214 , the ready status indicator 224 , and the position indicators 232 , 234 , 236 .
- the field of view 250 includes an image of the aerial refueling device 120 with portions of the boom 121 and the control components 122 (e.g., rudders) within the boom envelope 208 , as provided by the chevrons 212 .
- the ready status indicator 224 informs an operator that the aerial refueling device 120 is properly positioned and ready for use.
- the field of view display 250 includes a stripe 260 superimposed on the image of the boom 121 that parallels an angular orientation of the boom 121 .
- the stripe 260 is adapted to track the angle of the boom 121 , along its length, to provide an operator with an easily viewable graphic for immediate indication of the angular orientation of the boom 121 .
- the field of view display 250 includes one or more elevation indicators 270 superimposed on the image of the boom 121 .
- a first elevation indicator 272 identities a first elevation of the boom 121 , such as a 35° el
- a second elevation indicator 274 identities a second elevation of the boom 121 , such as a 40° el
- a third elevation indicator may identity a third elevation of the boom 121 , such as a 50° el., which is the mechanical boundary of the boom 121 .
- physical markings on the boom 121 may include one or more transverse stripes 240 on the boom 121 and one or more fiducial marks 242 on the housing of the control components 122 .
- Examples of fixed symbols displayed on the image include a dot at the center (e.g., center symbol 214 ) and the chevrons 212 marking the corners of the disconnect envelope (e.g., boom envelope 208 ).
- the symbols are positioned on the stereoscopic images such that they appear to be at the same distance as the length of the structural tube of the boom 121 .
- the boom 121 when the bottom of the fiducial mark on the boom 121 is coincident with the center symbol 214 , the boom 121 is positioned at the center of the boom envelope 208 (e.g., 30° el and 0° az). As the boom 121 approaches a boundary of the boom envelope 208 , one or more chevrons 212 enter into the operator's area of concentration, which provides, for example, an immediate and intuitive indication of how far the boom 121 may move before reaching the boundary.
- the field of view display 250 includes quantitative boom position information, as provided by the multi-dot circles of the position status indicators 232 , 234 , 236 at the bottom of the display.
- trend information are readily viewable by the operator's peripheral vision, and the indicators 232 , 234 , 236 are readily viewable such that all three may be perceived at a glance.
- an under-running receiver aircraft 110 a, 110 b may bring the elevation status indicator 232 within the operator's area of concentration, thereby reinforcing the need to disconnect immediately.
- the field of view display 250 provides boom status and warning indicators 222 , 224 , 226 , 228 ( FIG. 2A ), which are displayed in a semi-transparent manner so as to not overly obscure the operator's view of the receiver aircraft 110 a, 110 b when the receiver aircraft 110 a, 110 b is in an extreme position relative to the tanker aircraft 100 .
- the warning indicator 216 FIG. 2A
- This warning indicator 216 is positioned to immediately alert the operator to view the status area for more information as to the cause of the warning indication.
- positioning the warning indicator 216 at an end of the boom 121 which is in the operator's area of concentration, reduces the probability of missing the onset of the alert. Since the boom 121 may physically obscure the view of the receiver aircraft 121 , this does not interfere with the operator's ability to position the boom 121 in the receptacle 181 of the receiver aircraft 110 b of FIG. 1D or maintain a proper clearance from the receive aircraft 110 b.
- FIGS. 3A-3E show various exemplary embodiments of the aerial refueling device 120 in the fields of view 200 , 250 of the display component 153 .
- FIGS. 3A-3E show combinations of symbols having fixed locations, symbols that are superimposed on a boom image, and reference markings painted on the boom itself for tracking positions of the boom as it moves in the displayed images.
- FIG. 3A shows one embodiment of a field of view 300 displaying a photographic image of the aerial refueling device 120 , as displayed by the system 160 and seen from an operator in-flight.
- the field of view 300 includes an image of the aerial refueling device 120 with portions of the boom 121 and the control components 122 within the boom envelope 208 , as provided by the chevrons 212 .
- the boom 121 is positioned at the center of the refueling envelope (e.g., boom envelope 208 ) with the telescoping boom tip 123 retracted.
- the ready status indicator 224 informs an operator that the aerial refueling device 120 is properly positioned and ready for use.
- the aerial refueling boom 121 is an object of visual interest for an operator as the operator maneuvers the boom 121 in the vicinity of a receiver aircraft. Therefore, indicators that appear to be mounted to the boom 121 are provided in the operator's area of attention. Since the boom 121 may obscure part of the view of the receiver aircraft, superimposing the symbols on the boom 121 may not subtract from the view of the receiver aircraft by the operator. Physical markings on the boom 121 are adapted to operate in conjunction with symbols fixed in the field of view of the vision system 160 to provide an intuitive, analog indication of boom angular position. Symbols are not fixed in the field of view, but are adapted to move dynamically with the aerial refueling boom 121 .
- Symbols are not displayed at a fixed apparent distance or viewing plane, but are adapted to be located at a radius of the boom 121 , which may obviate any need for the operator to change a focus of depth perception. Markings may be added to the boom 121 that provide the boom with an indicator, as part of the display system 160 .
- FIG. 3B shows one embodiment of another field of view 310 displaying another photographic image of the aerial refueling device 120 , as displayed by the system 160 and seen from an operator in-flight.
- the field of view 310 includes an image of a receiver aircraft 110 c and an image of the aerial refueling device 120 in contact with the receiver aircraft 110 c, as indicated by the contact status indicator 226 .
- the contact status indicator 224 informs an operator that the aerial refueling device 120 is properly connected to the receiver aircraft 110 c and ready for refueling.
- the boom 121 is coupled to the receiver aircraft 110 c.
- the force cue 218 comprises an arrow superimposed on the image of the boom 121 and indicates a direction of force being applied to the boom nozzle, as measured by one or more sensors, such as an Automatic Load Alleviation System (ALAS) sensors.
- ALAS Automatic Load Alleviation System
- Moving the boom 121 in the direction of the force cue arrow 218 will manually alleviate the nozzle load.
- the force cue arrow 218 is green when the load is within a normal tolerance (e.g., ⁇ 500 lb), yellow when the load is elevated, and red when excessive.
- the force cue arrow 218 is not shown when the boom 121 is disconnected from the receiver aircraft 110 c, which would provide a redundant indication of boom status.
- FIG. 3C shows one embodiment of another field of view 320 displaying another photographic image of the aerial refueling device 120 , as displayed by the system 160 and seen from an operator in-flight.
- the field of view 320 includes an image of the aerial refueling device 120 with portions of the boom 121 and the control components 122 outside the boom envelope 208 , as indicated by the fail status indicator 222 and the warning indicator 216 , which is positioned at a lower part of the boom 121 .
- the boom 121 is positioned at a side boundary of the disconnect envelope (e.g., boom envelope 208 ).
- the warning indicator 218 is red (e.g., indicating a master warning) and superimposed on an end of the boom 121 with a corresponding warning message in the warning status indicator 222 .
- the azimuth position indicator is red because the boom 121 is outside the boom envelope 208 .
- the fail status indicator 222 informs an operator that the aerial refueling device 120 is not properly positioned within the boom envelope 208 and, as such, is not ready for use.
- the “flight fail” text informs an operator that the control components 122 of the aerial refueling device 120 have steered the boom 121 to a position outside the boom envelope 208 .
- the operator should counter steer (e.g., fly) the boom 121 to a proper position within the boom envelope 208 (e.g., as shown in FIG. 3A ).
- FIG. 3D shows one embodiment of another field of view 330 displaying another photographic image of the aerial refueling device 120 , as displayed by the system 160 and seen from an operator in-flight.
- the field of view 330 includes an image of the aerial refueling device 120 with portions of the boom 121 and the control components 122 outside the boom envelope 208 , as indicated by the fail status indicator 222 and the warning indicator 216 , which is positioned at a lower part of the boom 121 .
- the warning indicator 216 is yellow (e.g., indicating a master caution) and superimposed on an end of the boom 121 when the boom 121 is at an upper boundary of the disconnect envelope (e.g., boom envelope 208 ).
- the position of the warning indicator 216 easily gets the operator's attention when the boom 121 is away from the other indicators.
- the warning indicator 216 informs the operator when the boom has reached the upper boundary of the boom envelope 208 without having to shift their attention down to the bottom of the display.
- the disconnect status indicator 228 indicates that the aerial refueling device was properly disconnected, but the fail status indicator 222 informs an operator that the aerial refueling device 120 is not properly positioned within the boom envelope 208 and, as such, is not ready for use.
- the “alias fail” text informs an operator that the control components 122 of the aerial refueling device 120 have steered the boom 121 to a position outside the boom envelope 208 .
- the operator should counter steer (e.g., fly) the boom 121 to a proper position within the boom envelope 208 (e.g., as shown in FIG. 3A ).
- FIG. 3E shows one embodiment of another field of view 340 displaying another photographic image of the aerial refueling device 120 , as displayed by the system 160 and seen from an operator in-flight.
- the field of view 310 includes an image of the receiver aircraft 110 c and an image of the aerial refueling device 120 in contact with the receiver aircraft 110 c, as indicated by the contact status indicator 226 .
- FIG. 3 E shows how the second elevation indicator 274 (e.g., 40° el band) on the boom 121 lines up with the center symbol 214 , and the stripe 260 on the boom 121 lines up with the chevrons 212 at the lower portion of the boom envelope 208 .
- the second elevation indicator 274 e.g. 40° el band
- the position status indicators 232 , 234 , 236 at the lower portion of the display do not interfere excessively with the operator's view of the receiver aircraft 110 c, when the receiver aircraft 110 c is low in the display of the field of view 340 .
- FIG. 4A shows another embodiment of a field of view display 400 having graphic display symbology 410 including a plurality of position indicators 430 .
- FIG. 4B shows still another embodiment of a field of view display 450 having graphic display symbology 410 including the boom envelope 208 , the chevrons 212 , the center symbol 214 , the stripe 260 , the one or more elevation indicators 270 , the plurality of position indicators 430 , and one or more refueling status indicators 438 .
- the position indicators 430 are adapted to be superimposed on portions of the aerial refueling device 120 by tracking the position of the aerial refueling device 121 in the displayed image. As such, the position indicators 430 are adapted to move with the movement of the image of the aerial refueling device 120 as displayed in the field of view 400 .
- the position indicators 430 identify positions of the aerial refueling device 120 and include an elevation position indicator 432 , an azimuth position indicator 434 , and a telescope position indicator 436 .
- the position indicators 430 are displayed in appearance as analog gauges with a data component (e.g., alphanumeric component) to identify particular degrees of orientation in elevation (el) 232 , azimuth (az) 234 , and telescope (tel) 236 .
- the position indicators 430 include an indicator arm (e.g., pointer, dial, or hand) that rotates clockwise and/or counter-clockwise to visually indicate the degree of orientation in reference to the data component.
- the boom envelope boundaries are represented graphically around the indicator arms and may be dynamically adjusted for a type of receiver aircraft and boom telescoping. This arrangement provides an easily viewable indication and image of critical position data and information for steering and maneuvering the aerial refueling device 120 for proper placement relative to the receiver aircraft 110 d.
- the elevation position indicator 432 identifies the degree of elevation movement of the boom 121 of the aerial refueling device 120 , as described in reference to the vertical or pitch motion of the boom 121 by arrow B in FIG. 1B .
- the elevation position indicator 432 may be adapted to track the displayed position of at least one of the control components (e.g., wings) 122 of the aerial refueling device 120 .
- the azimuth position indicator 434 identifies the degree of azimuth movement of the boom 121 of the aerial refueling device 120 , as described in reference to the lateral motion of the boom by arrow A of FIG. 1B .
- the azimuth position indicator 434 may be adapted to track the displayed position of at least one of the control components 122 of the aerial refueling device 120 .
- the telescoping position indicator 436 identifies the degree of axial movement of the boom tip 123 from the boom 121 of the aerial refueling device 120 , as described in reference to the telescoping motion of the boom tip 123 by arrow C of FIG. 1B .
- the telescoping position indicator 436 may be adapted to track the displayed position of an end portion of the boom 121 of the aerial refueling device 120 .
- the one or more refueling status indicators 438 provide an indication of a refueling status of the receiver aircraft 110 d.
- the refueling status indicator 438 may be positioned in the field of view 400 , such as proximate to a lower portion of the boom 121 , e.g., near the boom tip 123 and/or near one or more of the position indicators 432 , 434 , 436 .
- the refueling status indicator 438 may include an indication of a ready (r) refueling status (e.g., as shown in FIG. 4A ), a contact (c) refueling status, and/or a disconnect (d) refueling status.
- the refueling status indicator 438 may be color-coded in appearance such that a blue background with a white colored character (r) may indicate a ready status, a green background with a black colored character (c) may indicate a connect status, and/or a yellow background with a black colored character (d) may indicate a disconnect status.
- a fuel status box (not shown) may be displayed in a portion of the field of view 400 .
- the fuel status box may be displayed within a 30 degree field of view centered on the housing of the control components 122 at the intersection of the boom 121 and control component hinge lines.
- caution and warning display indications such as the warning indicator 216 and status indicators 220 of FIG. 2A , may be positioned in the field of view 400 with the intent of facilitating operator recognition of events.
- boom envelope azimuth, elevation, and telescoping boundaries for the receiver aircraft 11 d are dynamically adjusted for that receiver aircraft type and boom telescoping.
- the background color of the gauges 432 , 434 , 436 coincide with position of the boom 121 relative to the boom envelope 208 .
- Status indicators are overlaid on the housing of the control components 122 , and in this case, they include routine boom state information. All of the indicators move with the boom 121 . Filters may be used to reduce jitter or jumpiness of the indicators as a result of noise in the boom position measurements, and to ensure that the symbols remain readable during rapid boom movement. As the boom moves to a boundary of the visual field of view of the display, the overlaid symbology remains visible at the edge of the display, ghosting the position of the boom 121 .
- boom markings reference overlays as described in FIG. 2B , which are combined with the symbology anchored to the boom 121 .
- This concept provides intuitive analog boom position references with the concentrated information superimposed on the boom 121 .
- the chevrons 212 provide an operator with known references in the field of view that aid in judging the distance to the receiver aircraft 110 c as the receiver aircraft 110 c approaches the tanker aircraft 100 .
- FIG. 5 shows one embodiment of a process 500 for facilitating the display of a field of view (FOV) having graphic display symbology.
- the description of FIG. 5 references the visual display system 160 of FIGS. 1A-1D and the FOV displays of FIGS. 2A-4B .
- the controller 130 receives FOV image data from the optical sensors 140 including stereoscopic cameras 141 (block 510 ), and the controller 130 receives control signals from an operator via the operator input device 150 (block 510 ).
- the controller 130 processes the received FOV image data and control data (block 514 ).
- the controller 130 generates graphic display symbology based, at least in part, on the control data received from the operator input device 150 (block 518 ).
- the controller 130 further processes the FOV image data by combining the generated symbology with the FOV image data (block 522 ). In one aspect, this processing is adapted to display the symbology as overlaid or superimposed on the FOV image data.
- the controller 130 displays the FOV image with the symbology in the display component 153 for viewing by an operator (block 526 ).
- the controller 130 updates the FOV image data from the optical sensors 140 and the control signals from the operator input device 150 (block 530 ).
- the process 500 returns (or loops back) to block 514 so that the controller 130 may process the updated FOV image data and the control data. The process 500 continues therefrom.
- the controller 130 may be adapted to store optical sensor data, FOV image data, and control signal data in a storage component, without departing from the scope of the present disclosure.
- the controller 130 may also be adapted to retrieve these various types of data and information related thereto from the storage component.
- FIG. 6 is a block diagram of a computer system and/or controller 600 suitable for implementing embodiments of the present disclosure.
- Computer system 600 includes a bus 602 or other communication mechanism for communicating information, which interconnects subsystems and components, such as processor 604 , system memory component 606 (e.g., RAM), static storage component 608 (e.g., ROM), removable memory component 610 (e.g., removable ROM memory, such as EEPROM, smart card, flash memory, etc.), wired or wireless communication interface 612 (e.g., transceiver, modem or Ethernet card), display component 614 (e.g., LCD, CRT, etc.), input component 616 (e.g., sensors, such as optical sensors including stereoscopic cameras, keyboard, microphone, touch screen on display), and cursor control component 618 (e.g., mouse button).
- system memory component 606 e.g., RAM
- static storage component 608 e.g., ROM
- removable memory component 610 e.g
- computer system 600 performs specific operations by processor 604 executing one or more sequences of one or more instructions contained in system memory component 606 .
- Such instructions may be read into system memory component 606 from another computer readable medium, such as static storage component 608 or removable memory component 610 .
- static storage component 608 or removable memory component 610 may be used in place of or in combination with software instructions to implement the invention.
- Non-volatile media includes removable storage media, such as removable memory component 610
- volatile media includes dynamic memory, such as system memory component 606
- transmission media including wireless transceivers.
- transmission media may take the form of radio waves, such as those generated during radio wave and infrared data communications.
- Computer readable media includes, for example, floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, carrier wave, or any other medium from which a computer is adapted to read.
- execution of instruction sequences to practice the invention may be performed by computer system 600 .
- a plurality of computer systems 600 coupled by communication link 620 e.g., wireless cell phone network, wireless or wired LAN, PTSN, or various other wireless networks
- Computer system 600 may transmit and receive messages, data, information and instructions, including one or more programs (i.e., application code) through communication link 620 and communication interface 612 .
- Received program code may be executed by processor 604 as received and/or stored in removable memory component 610 or some other non-volatile storage component for execution.
- various embodiments of the invention may be implemented using hardware, software, or various combinations of hardware and software.
- various hardware components and/or software components set forth herein may be combined into composite components comprising software, hardware, and/or both without departing from the scope and functionality of the present disclosure.
- various hardware components and/or software components set forth herein may be separated into subcomponents having software, hardware, and/or both without departing from the scope and functionality of the present disclosure.
- software components may be implemented as hardware components and vice-versa.
- Software in accordance with the present disclosure, such as program code and/or data, may be stored on one or more computer readable mediums. It is also contemplated that software identified herein may be implemented using one or more general purpose or specific purpose computers and/or computer systems, networked and/or otherwise. Where applicable, ordering of various steps described herein may be changed, combined into composite steps, and/or separated into sub-steps to provide features described herein.
Abstract
Description
- The present disclosure relates generally to aerial refueling and, more particularly, to displaying aerial refueling symbology.
- In the aviation industry, in-flight refueling extends the range and endurance of aircraft without requiring the aircraft to land. Although in-flight refueling is a relatively common operation, especially for military aircraft, the aircraft to be refueled (e.g., the receiver aircraft) must be precisely positioned relative to the tanker aircraft to provide safe engagement while the fuel is dispensed to the receiver aircraft. The precise relative spatial positioning of the two rapidly moving aircraft makes in-flight refueling a challenging operation.
- One type of in-flight refueling system includes a boom refueling system that includes a rigid boom extending from the tanker aircraft, with a probe and nozzle at its distal end. The boom includes airfoils (e.g., rudders) controlled by a boom operator stationed on the refueling aircraft. The airfoils allow the boom operator to actively maneuver the boom with respect to the receiver aircraft, which usually flies in a fixed refueling position below and aft of the tanker aircraft. The boom operator typically monitors the aerial refueling boom with video displays to assist with guiding the boom nozzle into the receptacle of the receiver aircraft.
- Conventional video displays provide essential information to the boom operator for readily cross-checking the status of the refueling system including boom elevation, azimuth, telescoping length, boom state, and fuel delivery without obscuring or otherwise losing sight of the receiver aircraft. Warning indicators are positioned so as to attract the boom operator's attention when concentrating on positioning the boom nozzle in the receiver aircraft's receptacle, without compromising safety by obscuring essential areas of the scene.
- Some conventional video displays provide visual overlays of this essential information in the form of linear scales, numeric fields, and warning indicators that are displayed at fixed locations in the displayed images. Generally, the visual overlays displayed over the images of the receiver aircraft obscure the boom operator's view of the receiver aircraft during the aerial refueling process, which is typically unsafe. In some instances, conventional overlays may obscure dangerous situations in aerial refueling, which may increase the occurrence of damage to the aerial refueling boom and receiver. As a result, there is a need for an improved system and method for providing visual overlays in displayed images of aerial refueling operations that improve safety.
- Systems and methods disclosed herein, in accordance with one or more embodiments, provide for overlaying or superimposing graphic display symbology on a displayed image, such as a photographic scene of an aerial refueling operation between a tanker aircraft and a receiver aircraft. The graphic display symbology provides an operator with information on position, status, and cautions/warnings of an aerial refueling device. The graphic display symbology presents information to an operator in an easily viewable form, reduces interference with viewing receiver aircraft, and provides warning indications within the operator's field of view (e.g., primary field of view).
- In accordance with one or more embodiments, a system for in-flight fuel delivery includes an aerial refueling device adapted to provide fuel to a receiver aircraft, an optical component adapted to capture images of the aerial refueling device and the receiver aircraft, an operator input component adapted to interface with an operator and capture control signals as input from the operator, and a display component adapted to display images. The system includes a controller adapted to receive the captured images and the captured control signals, process the control signals by generating graphic display symbology, process the images by generating a combined image having the generated graphic display symbology superimposed on the images, and display the combined image on the display component for viewing by the operator. The controller is adapted to superimpose the graphic display symbology on a portion of the images obscured by the aerial refueling device. The controller is adapted to process the control signals by determining a position of the aerial refueling device.
- In various implementations, the controller is adapted to dynamically position the graphic display symbology over the portion of the images obscured by the aerial refueling device as the aerial refueling device moves in the images. The aerial refueling device includes a boom component and at least one rudder component having physical markings on a portion thereof. The controller superimposes the graphic display symbology on a portion of the image proximate to the physical markings The optical components include stereoscopic cameras adapted to have a field of view of at least one of the aerial refueling device and the receiver aircraft, and the graphic display symbology is superimposed over the image in the field of view.
- In various implementations, the graphic display symbology includes a plurality of chevron symbols that define a boom envelope in the image. The chevron symbols may be positioned at corners of the boom envelope. The chevron symbols may be sized and/or positioned for a specific type of receiver aircraft and boom extension. The graphic display symbology includes a center symbol positioned at a center of the boom envelope. The graphic display symbology includes a plurality of position indicators that provide position information related to the aerial refueling device including at least one of an elevation position, an azimuth position, and a telescoping position. The graphic display symbology includes warning indicators and status indicators that are displayed in the image with a semi-transparent pixel mask to avoid obscuring the aerial refueling device or the receiver aircraft. The graphic display symbology includes a force cue, such as an arrow superimposed on a portion of the aerial refueling device. The arrow defines a direction of projected force on a nozzle component of the aerial refueling device. The graphic display symbology includes a master caution indicator and a master warning indicator superimposed on a portion of the aerial refueling device. The master caution indicator comprises a yellow color to indicate a master caution state, and the master warning indicator comprises a red color to indicate a master warning state.
- In various implementations, the graphic display symbology includes graphic position gauges and refueling status lights that are displayed over and track the position of at least one of a boom component and a rudder component in the images of the aerial refueling device. The graphic position gauges provide position information related to the aerial refueling device including at least one of an elevation position of the boom component, an azimuth position of the boom component, and an extension position of the boom tip component. One or more refueling status indicators provide an indication of a refueling status including but not limited to the terms Ready, Contact, and Disconnect.
- In accordance with one or more embodiments, a method for in-flight fuel delivery from a tanker aircraft to a receiver aircraft via an aerial refueling device includes receiving images from an optical component, the images having a field of view of at least one of the aerial refueling device and the receiver aircraft and receiving control signals from an operator input component. The method includes processing the control signals by generating graphic display symbology and processing the images by generating a combined image having the generated graphic display symbology superimposed on the images. The method includes displaying the combined image on a display component. The graphic display symbology is superimposed on a portion of the images obscured by the aerial refueling device.
- The scope of the disclosure is defined by the claims, which are incorporated into this section by reference. A more complete understanding of embodiments is afforded to those skilled in the art, as well as a realization of additional advantages thereof, by a consideration of the following detailed description of one or more embodiments. Reference is made to the appended sheets of drawings as described in greater detail herein.
-
FIG. 1A shows a tanker aircraft refueling a receiver aircraft, in accordance with an embodiment of the present disclosure. -
FIG. 1B shows a system for controlling an aerial refueling device, in accordance with an embodiment of the present disclosure. -
FIG. 1C shows a field of view (FOV) of a tanker aircraft refueling a receiver aircraft, in accordance with an embodiment of the present disclosure. -
FIG. 1D shows a receiver aircraft, in accordance with an embodiment of the present disclosure. -
FIGS. 2A-2B show various FOV displays having graphic display symbology, in accordance with one or more embodiments of the present disclosure. -
FIGS. 3A-3E show various exemplary embodiments of an aerial refueling device in various FOV displays having graphic display symbology, in accordance with one or more embodiments of the present disclosure. -
FIGS. 4A-4B show various other exemplary embodiments of an aerial refueling device in various FOV displays having graphic display symbology, in accordance with one or more embodiments of the present disclosure. -
FIG. 5 shows one embodiment of a process for facilitating FOV displays having graphic display symbology, in accordance with an embodiment of the present disclosure. -
FIG. 6 is a block diagram of a computer/controller system suitable for implementing one or more embodiments of the present disclosure. - Embodiments and their advantages are best understood by referring to the detailed description that follows. It should be appreciated that like reference numerals are used to identify like elements illustrated in one or more of the figures.
- In accordance with one or more embodiments of the present disclosure, systems and methods disclosed herein provide for overlaying graphic display symbology on a displayed image, such as a photographic scene, to provide an operator with information on position, status, and cautions/warnings of an aerial refueling device. The graphic display symbology includes a plurality of graphical symbols related to the aerial refueling operation and presents information to an operator in an easily viewable form, reduces interference with viewing receiver aircraft, and provides warning indications within the operator's field of view (e.g., primary field of view). In one implementation, as described in greater detail herein, the symbology may be superimposed on portions of the scene. In other implementations, tracking of the aerial refueling device may operate in conjunction with physical markings on the underside of the boom to avoid obscuring the operator's view of receiver aircraft. As such, the graphic display symbology provides an operator with essential information for monitoring and controlling aerial refueling. For example, the graphic display symbology displayed over one or more components of a boom provides easily viewable information close to a center of the operator's field of view during pre-contact and contact phases of air refueling while maximizing the available unobstructed view of the receiver aircraft.
- In accordance with one or more embodiments of the present disclosure, systems and methods disclosed herein are capable of displaying boom position (e.g., elevation angle, azimuth angle, and telescoping length), fuel system state (e.g., ready/contact/disconnect), and warnings, cautions, and alerts superimposed on images of an aerial refueling display. In various implementations, the position of data and information in the displayed field of view is varied continuously so as to superimpose the symbols on the displayed image of the aerial refueling boom in the field of view. As such, essential data and information are provided in an operator's field of view without obscuring the operator's view of the receiver aircraft. In one aspect, the overlays operate in concert with markings on the aerial refueling boom to provide the operator with analog indication of the boom's angular position with respect to the contact, disconnect and mechanical envelopes.
-
FIG. 1A shows one embodiment of anaerial refueling system 160 comprising atanker aircraft 100 refueling areceiver aircraft 110 a with anaerial refueling device 120. As shown inFIG. 1A , thetanker aircraft 100 includes afuselage 101, one ormore wings 102, and one ormore engines 103. In various other embodiments, theaircraft 100 may include various other types of configurations. - The
aerial refueling device 120, in one embodiment, may include arefueling boom 121 having one or more actuatable control components 122 (e.g., rudders). The positions of thecontrol components 122 may be adjusted to steer theboom 121 into engagement with thereceiver aircraft 110 a. The position of theboom 121 may be controlled based on inputs received via one or more sensors and/or inputs received from a human operator. Theaerial refueling device 120 may be configured to allow the operator to manually control theboom 121 and/or may allow thesystem 160 to automatically control theboom 121, either separately or simultaneously. In one implementation, the operator may control the location of theboom 121 with assistance from the automatic portion of thesystem 160, while allowing the operator to override the automatic portion of thesystem 160 at any point. -
FIG. 1B shows one embodiment of a plurality of components of thesystem 160 in reference toFIG. 1A . As shown inFIG. 1B , thesystem 160 includes acontroller 130 adapted to control the motion of theaerial refueling device 120, based on inputs received from anoperator input device 150 and one ormore sensor 140. Theoperator input device 150, in one embodiment, may include astick 151 or various other types of manual input control device (e.g., a yoke). Eachsensor 140, in one embodiment, may include an optical sensor, such as one or more photosensitive detectors 141 (e.g., a pair of cameras) that provide a stereoscopic image to thecontroller 130. Thecontroller 130, in one embodiment, may receive input signals from the one ormore sensors 140 and theoperator input device 150 and, based upon a combination of the input signals, direct a command signal to theaerial refueling device 120 to change and/or alter a position of theaerial refueling device 120. - Referring to
FIG. 1B , thecontroller 130 is adapted to direct actuation of thecontrol components 122 to provide for lateral motion of theboom 121, as indicated by arrow A, and vertical or pitch motion of theboom 121, as indicated by arrow B. Thecontroller 130 is adapted to control axial actuation (e.g., extension, such as telescoping extension) of adeployable boom tip 123, as indicated by arrow C. In one implementation, anactuator 152 may be coupled between thecontroller 130 and theoperator input device 150 to provide feedback to the operator. In one example, when thecontroller 130 directs a command signal to move theboom 121 to a particular position, a corresponding command signal may be directed to theactuator 152 to move theoperator input device 150 in a corresponding position. When placing a hand lightly on thestick 151, the operator may sense thestick 151 move in a manner that tracks the motion of theboom 121 and/or in a manner that is commanded by thecontroller 130 based on input from the one ormore sensors 140. As such, if the operator places any force (e.g., greater than a threshold level of force) on thestick 151, the operator input to thestick 151 may influence the position of theboom 121. Hence, in one aspect, the rate at which thecontroller 130 directs a motion of theboom 121 based on the operator input, may vary proportionately to the force applied by the operator to theoperator input device 150. - Referring to
FIG. 1B , thecontroller 130, in one embodiment, is adapted to display images from the one or more sensors 140 (e.g., one or more image or video cameras) on adisplay component 153 for viewing by the operator. Thecontroller 130 may be adapted to display a field of view that includes thereceiver aircraft 110 a and theaerial refueling device 120 on thedisplay component 153. In another embodiment, thecontroller 130 is adapted to graphically display operator input from theoperator input device 150. For example, thecontroller 130 may be adapted to superimpose graphics on the displayed images for viewing by the operator. In various implementations, the graphics may include control graphics that indicate positions of theaerial refueling device 120, including theboom 121, thecontrol components 122 and various other components thereof. These display graphics including the control graphics are described in greater detail herein. Further scope and operation related to controlling aircraft refueling may be found in co-pending U.S. patent application entitled, “SYSTEMS AND METHODS FOR AUTOMATICALLY CONTROLLING AIRCRAFT REFUELING,” Ser. No. 11/090,347, filed Mar. 24, 2005, which is hereby incorporated by reference in its entirety. -
FIG. 1C shows one embodiment of a partially cut away side view of thetanker aircraft 100 refueling thereceiver aircraft 110 a. As shown inFIG. 1C , asensor pod 142 houses the one ormore sensors 140 ofFIG. 1B at a location where the one ormore sensors 140 may detect the position of theboom 121 and thereceiver aircraft 110 a. In one implementation, the one ormore sensors 140 is adapted to have a sensor field ofview 143 that extends downwardly and aft from thesensor pod 142. The one ormore sensors 140 within thesensor pod 142 is adapted to detect a position of at least one of thereceiver aircraft 110 a and theboom 121 so as to determine a position error E. For example, the position error E may include the vector distance between the tip of theboom 121 and arefueling receptacle 111 of the aircraft and may include position components in a plurality of coordinate directions (e.g., azimuth, elevation, and extension such as telescoping position). Thecontroller 130, in one embodiment, is adapted to display images from the field ofview 124 of the one or more sensors 140 (e.g., one or more image detectors) on thedisplay component 153 for viewing by the operator. In various implementations, the displayed images of the field ofview 124 may include display graphics that provide information related to the position of theaerial refueling device 120 and the various components thereof including theboom 121, thecontrol components 122 and thetelescoping boom tip 123. The display graphics including various aspects thereof are described in greater detail herein. -
FIG. 1D shows one embodiment of areceiver aircraft 110 b with one suitable position for afuel receptacle 181. Thefuel receptacle 181 may include a distinctive visual appearance and/or may includevisual cue markings 182 that assist the one ormore sensors 140 in precisely identifying the location of thefuel receptacle 181. In one implementation, the one ormore sensors 140 may be used to optically determine the distance between theboom tip 111 and thereceptacle 182. In another embodiment, theboom 121 may include a separate position sensor (e.g., an accelerometer with a processor), and the position detected by the boom position sensor may be combined with the visual image of thefuel receptacle 181 to determine the distance between theboom tip 111 and thefuel receptacle 181. -
FIG. 2A shows one embodiment of a field of view (FOY)display 200 havinggraphic display symbology 210 including a plurality ofchevron symbols 212, acenter symbol 214, a master warning/caution indicator 216, aforce cue 218, a plurality ofstatus indicators 220, and a plurality ofposition indicators 230. Thechevron symbols 212, in one embodiment, mark four corners of a disconnect envelope, which identify a boom envelope 208 (e.g., frame of control). In various implementations, thechevron symbols 212 may be sized and/or positioned for a specific type of receiver aircraft and boom extension. Thecenter symbol 214, in one embodiment, identifies the central region of theboom envelope 208 as identified by thechevrons 212. - The master warning/
caution indicator 216, in one embodiment, is positioned proximate to the central region of theboom envelope 208. In this position, the master warning/caution indicator 216 is easily viewable by an operator for immediate indication of an alert, a caution or a warning. In various implementations, the master warning/caution indicator 216 comprises a red color to indicate a master warning state and a yellow color to indicate a master caution state. Theforce cue 218, in one embodiment, identifies the direction of force on theaerial refueling device 120 so that the operator may steer or maneuver theaerial refueling device 120 in a corrective or compensating manner. - The
status indicators 220, in one embodiment, identify various information related to the “state of the fuel system” of theaerial refueling device 120 and include afail status indicator 222, aready status indicator 224, acontact status indicator 226, and adisconnect status indicator 228. In various implementations, as shown inFIG. 2A , thestatus indicators 230 comprise a semi-transparent rectangular graphic with alphanumeric characters that provide the operator with easily viewable information about the status (e.g., fuel system state) of theaerial refueling device 120. In one example, the warning and status indicators are displayed with a pixel mask that allows the underlying image to show through, which may avoid obscuration of the visual scene. - The
fail status indicator 222, in one embodiment, indicates to the operator that theaerial refueling device 120 is positioned outside theboom envelope 208, as indicated by thechevrons 212. Theready status indicator 224 indicates to the operator that theaerial refueling device 120 is ready for operation including steering and maneuvering. Thecontact status indicator 226 indicates to the operator that theaerial refueling device 120 is in contact with thereceiver aircraft 110 a, for example, as shown inFIG. 1A . Thedisconnect status indicator 228 indicates to the operator that theaerial refueling device 120 is disconnected from thereceiver aircraft 110 a, which may be indicated after refueling of thereceiver aircraft 110 a has been completed. - The
position indicators 230, in one embodiment, identify positions of theaerial refueling device 120 and include anelevation position indicator 232, anazimuth position indicator 234, and an extension (e.g., telescoping)position indicator 236. In one implementation, as shown inFIG. 2A , theposition indicators 230 include a circular arrangement of multi-dot circles and a data component (e.g., alphanumeric component) to identify a particular degree of orientation in elevation (el) 232, azimuth (az) 234, and telescope (tel) 236. The multi-dot circles (e.g., 10-dot circles) are used for numeric scales to reduce their area in the field of view and to provide improved trend information. Theposition indicators 230 may include a hand that rotates clockwise and/or counter-clockwise to visually indicate the degree of orientation in reference to the data component. - The
elevation position indicator 232 identifies the degree of elevation movement of theboom 121 of theaerial refueling device 120, as described in reference to the vertical or pitch motion of theboom 121 by arrow B inFIG. 1B . Theazimuth position indicator 234 identifies the degree of azimuth movement of theboom 121 of theaerial refueling device 120, as described in reference to the lateral motion of the boom by arrow A ofFIG. 1B . The extension ortelescoping position indicator 236 identifies the degree of axial movement of theboom tip 123 from theboom 121 of theaerial refueling device 120, as described in reference to the extending or telescoping motion of theboom tip 123 by arrow C ofFIG. 1B . -
FIG. 2B shows one embodiment of another field of view display 250 havinggraphic display symbology 210 including thechevrons 212, thecenter symbol 214, theready status indicator 224, and theposition indicators FIG. 2B , the field of view 250 includes an image of theaerial refueling device 120 with portions of theboom 121 and the control components 122 (e.g., rudders) within theboom envelope 208, as provided by thechevrons 212. In one aspect, theready status indicator 224 informs an operator that theaerial refueling device 120 is properly positioned and ready for use. - The field of view display 250, in one embodiment, includes a
stripe 260 superimposed on the image of theboom 121 that parallels an angular orientation of theboom 121. In one implementation, as shown inFIG. 2B , thestripe 260 is adapted to track the angle of theboom 121, along its length, to provide an operator with an easily viewable graphic for immediate indication of the angular orientation of theboom 121. - The field of view display 250, in one embodiment, includes one or
more elevation indicators 270 superimposed on the image of theboom 121. In various implementations, afirst elevation indicator 272 identities a first elevation of theboom 121, such as a 35° el, and asecond elevation indicator 274 identities a second elevation of theboom 121, such as a 40° el, and a third elevation indicator may identity a third elevation of theboom 121, such as a 50° el., which is the mechanical boundary of theboom 121. - Referring to
FIG. 2B , physical markings on theboom 121 may include one or moretransverse stripes 240 on theboom 121 and one or morefiducial marks 242 on the housing of thecontrol components 122. Examples of fixed symbols displayed on the image include a dot at the center (e.g., center symbol 214) and thechevrons 212 marking the corners of the disconnect envelope (e.g., boom envelope 208). In various implementations, the symbols are positioned on the stereoscopic images such that they appear to be at the same distance as the length of the structural tube of theboom 121. For example, when the bottom of the fiducial mark on theboom 121 is coincident with thecenter symbol 214, theboom 121 is positioned at the center of the boom envelope 208 (e.g., 30° el and 0° az). As theboom 121 approaches a boundary of theboom envelope 208, one ormore chevrons 212 enter into the operator's area of concentration, which provides, for example, an immediate and intuitive indication of how far theboom 121 may move before reaching the boundary. - The field of view display 250, in one embodiment, includes quantitative boom position information, as provided by the multi-dot circles of the
position status indicators indicators receiver aircraft elevation status indicator 232 within the operator's area of concentration, thereby reinforcing the need to disconnect immediately. - The field of view display 250, in one embodiment, provides boom status and
warning indicators FIG. 2A ), which are displayed in a semi-transparent manner so as to not overly obscure the operator's view of thereceiver aircraft receiver aircraft tanker aircraft 100. As such, when a new warning, caution or alert occurs, the warning indicator 216 (FIG. 2A ) may be displayed superimposed on a portion of theboom 121. Thiswarning indicator 216 is positioned to immediately alert the operator to view the status area for more information as to the cause of the warning indication. For example, positioning thewarning indicator 216 at an end of theboom 121, which is in the operator's area of concentration, reduces the probability of missing the onset of the alert. Since theboom 121 may physically obscure the view of thereceiver aircraft 121, this does not interfere with the operator's ability to position theboom 121 in thereceptacle 181 of thereceiver aircraft 110 b ofFIG. 1D or maintain a proper clearance from the receiveaircraft 110 b. -
FIGS. 3A-3E show various exemplary embodiments of theaerial refueling device 120 in the fields ofview 200, 250 of thedisplay component 153. In various implementations,FIGS. 3A-3E show combinations of symbols having fixed locations, symbols that are superimposed on a boom image, and reference markings painted on the boom itself for tracking positions of the boom as it moves in the displayed images. -
FIG. 3A shows one embodiment of a field ofview 300 displaying a photographic image of theaerial refueling device 120, as displayed by thesystem 160 and seen from an operator in-flight. The field ofview 300 includes an image of theaerial refueling device 120 with portions of theboom 121 and thecontrol components 122 within theboom envelope 208, as provided by thechevrons 212. As shown inFIG. 3A , theboom 121 is positioned at the center of the refueling envelope (e.g., boom envelope 208) with thetelescoping boom tip 123 retracted. In one implementation, theready status indicator 224 informs an operator that theaerial refueling device 120 is properly positioned and ready for use. - In various embodiments, the
aerial refueling boom 121 is an object of visual interest for an operator as the operator maneuvers theboom 121 in the vicinity of a receiver aircraft. Therefore, indicators that appear to be mounted to theboom 121 are provided in the operator's area of attention. Since theboom 121 may obscure part of the view of the receiver aircraft, superimposing the symbols on theboom 121 may not subtract from the view of the receiver aircraft by the operator. Physical markings on theboom 121 are adapted to operate in conjunction with symbols fixed in the field of view of thevision system 160 to provide an intuitive, analog indication of boom angular position. Symbols are not fixed in the field of view, but are adapted to move dynamically with theaerial refueling boom 121. Symbols are not displayed at a fixed apparent distance or viewing plane, but are adapted to be located at a radius of theboom 121, which may obviate any need for the operator to change a focus of depth perception. Markings may be added to theboom 121 that provide the boom with an indicator, as part of thedisplay system 160. These and other aspects of the present disclosure are described in greater detail herein -
FIG. 3B shows one embodiment of another field of view 310 displaying another photographic image of theaerial refueling device 120, as displayed by thesystem 160 and seen from an operator in-flight. As shown inFIG. 3B , the field of view 310 includes an image of areceiver aircraft 110 c and an image of theaerial refueling device 120 in contact with thereceiver aircraft 110 c, as indicated by thecontact status indicator 226. In one aspect, thecontact status indicator 224 informs an operator that theaerial refueling device 120 is properly connected to thereceiver aircraft 110 c and ready for refueling. - In one implementation, referring to
FIG. 3B , theboom 121 is coupled to thereceiver aircraft 110 c. When in contact, theforce cue 218 comprises an arrow superimposed on the image of theboom 121 and indicates a direction of force being applied to the boom nozzle, as measured by one or more sensors, such as an Automatic Load Alleviation System (ALAS) sensors. This allows the operator to monitor ALAS performance. Moving theboom 121 in the direction of theforce cue arrow 218 will manually alleviate the nozzle load. In one example, theforce cue arrow 218 is green when the load is within a normal tolerance (e.g., <500 lb), yellow when the load is elevated, and red when excessive. In another example, theforce cue arrow 218 is not shown when theboom 121 is disconnected from thereceiver aircraft 110 c, which would provide a redundant indication of boom status. -
FIG. 3C shows one embodiment of another field ofview 320 displaying another photographic image of theaerial refueling device 120, as displayed by thesystem 160 and seen from an operator in-flight. As shown inFIG. 3C , the field ofview 320 includes an image of theaerial refueling device 120 with portions of theboom 121 and thecontrol components 122 outside theboom envelope 208, as indicated by thefail status indicator 222 and thewarning indicator 216, which is positioned at a lower part of theboom 121. - In one implementation, referring to
FIG. 3C , theboom 121 is positioned at a side boundary of the disconnect envelope (e.g., boom envelope 208). Thewarning indicator 218 is red (e.g., indicating a master warning) and superimposed on an end of theboom 121 with a corresponding warning message in thewarning status indicator 222. Also, the azimuth position indicator is red because theboom 121 is outside theboom envelope 208. - In one aspect, even though the
ready status indicator 224 indicates ready for use, thefail status indicator 222 informs an operator that theaerial refueling device 120 is not properly positioned within theboom envelope 208 and, as such, is not ready for use. In another aspect, the “flight fail” text, as displayed by thefail status indicator 222, informs an operator that thecontrol components 122 of theaerial refueling device 120 have steered theboom 121 to a position outside theboom envelope 208. Thus, for corrective maneuvering, the operator should counter steer (e.g., fly) theboom 121 to a proper position within the boom envelope 208 (e.g., as shown inFIG. 3A ). -
FIG. 3D shows one embodiment of another field ofview 330 displaying another photographic image of theaerial refueling device 120, as displayed by thesystem 160 and seen from an operator in-flight. As shown inFIG. 3D , the field ofview 330 includes an image of theaerial refueling device 120 with portions of theboom 121 and thecontrol components 122 outside theboom envelope 208, as indicated by thefail status indicator 222 and thewarning indicator 216, which is positioned at a lower part of theboom 121. - In one implementation, referring to
FIG. 3D , thewarning indicator 216 is yellow (e.g., indicating a master caution) and superimposed on an end of theboom 121 when theboom 121 is at an upper boundary of the disconnect envelope (e.g., boom envelope 208). The position of thewarning indicator 216 easily gets the operator's attention when theboom 121 is away from the other indicators. Thewarning indicator 216 informs the operator when the boom has reached the upper boundary of theboom envelope 208 without having to shift their attention down to the bottom of the display. - In one aspect, the
disconnect status indicator 228 indicates that the aerial refueling device was properly disconnected, but thefail status indicator 222 informs an operator that theaerial refueling device 120 is not properly positioned within theboom envelope 208 and, as such, is not ready for use. In another aspect, the “alias fail” text, as displayed by thefail status indicator 222, informs an operator that thecontrol components 122 of theaerial refueling device 120 have steered theboom 121 to a position outside theboom envelope 208. Thus, the operator should counter steer (e.g., fly) theboom 121 to a proper position within the boom envelope 208 (e.g., as shown inFIG. 3A ). -
FIG. 3E shows one embodiment of another field ofview 340 displaying another photographic image of theaerial refueling device 120, as displayed by thesystem 160 and seen from an operator in-flight. As shown inFIG. 3E , the field of view 310 includes an image of thereceiver aircraft 110 c and an image of theaerial refueling device 120 in contact with thereceiver aircraft 110 c, as indicated by thecontact status indicator 226. In one aspect, FIG. 3E shows how the second elevation indicator 274 (e.g., 40° el band) on theboom 121 lines up with thecenter symbol 214, and thestripe 260 on theboom 121 lines up with thechevrons 212 at the lower portion of theboom envelope 208. As shown inFIG. 3E , theposition status indicators receiver aircraft 110 c, when thereceiver aircraft 110 c is low in the display of the field ofview 340. -
FIG. 4A shows another embodiment of a field ofview display 400 havinggraphic display symbology 410 including a plurality ofposition indicators 430.FIG. 4B shows still another embodiment of a field ofview display 450 havinggraphic display symbology 410 including theboom envelope 208, thechevrons 212, thecenter symbol 214, thestripe 260, the one ormore elevation indicators 270, the plurality ofposition indicators 430, and one or morerefueling status indicators 438. - In one implementation, as shown on
FIGS. 4A and 4B , theposition indicators 430 are adapted to be superimposed on portions of theaerial refueling device 120 by tracking the position of theaerial refueling device 121 in the displayed image. As such, theposition indicators 430 are adapted to move with the movement of the image of theaerial refueling device 120 as displayed in the field ofview 400. - The
position indicators 430, as previous described, identify positions of theaerial refueling device 120 and include anelevation position indicator 432, anazimuth position indicator 434, and atelescope position indicator 436. In one implementation, as shown inFIG. 4A , theposition indicators 430 are displayed in appearance as analog gauges with a data component (e.g., alphanumeric component) to identify particular degrees of orientation in elevation (el) 232, azimuth (az) 234, and telescope (tel) 236. As shown inFIG. 4A , theposition indicators 430 include an indicator arm (e.g., pointer, dial, or hand) that rotates clockwise and/or counter-clockwise to visually indicate the degree of orientation in reference to the data component. In one aspect, the boom envelope boundaries are represented graphically around the indicator arms and may be dynamically adjusted for a type of receiver aircraft and boom telescoping. This arrangement provides an easily viewable indication and image of critical position data and information for steering and maneuvering theaerial refueling device 120 for proper placement relative to thereceiver aircraft 110 d. - The
elevation position indicator 432, in one embodiment, identifies the degree of elevation movement of theboom 121 of theaerial refueling device 120, as described in reference to the vertical or pitch motion of theboom 121 by arrow B inFIG. 1B . In one implementation, as shown inFIG. 4A , theelevation position indicator 432 may be adapted to track the displayed position of at least one of the control components (e.g., wings) 122 of theaerial refueling device 120. - The
azimuth position indicator 434, in one embodiment, identifies the degree of azimuth movement of theboom 121 of theaerial refueling device 120, as described in reference to the lateral motion of the boom by arrow A ofFIG. 1B . In one implementation, as shown inFIG. 4A , theazimuth position indicator 434 may be adapted to track the displayed position of at least one of thecontrol components 122 of theaerial refueling device 120. - The
telescoping position indicator 436, in one embodiment, identifies the degree of axial movement of theboom tip 123 from theboom 121 of theaerial refueling device 120, as described in reference to the telescoping motion of theboom tip 123 by arrow C ofFIG. 1B . In one implementation, as shown inFIG. 4A , thetelescoping position indicator 436 may be adapted to track the displayed position of an end portion of theboom 121 of theaerial refueling device 120. - The one or more
refueling status indicators 438, in various embodiments, provide an indication of a refueling status of thereceiver aircraft 110 d. As shown inFIG. 4A , therefueling status indicator 438 may be positioned in the field ofview 400, such as proximate to a lower portion of theboom 121, e.g., near theboom tip 123 and/or near one or more of theposition indicators refueling status indicator 438 may include an indication of a ready (r) refueling status (e.g., as shown inFIG. 4A ), a contact (c) refueling status, and/or a disconnect (d) refueling status. In various implementations, therefueling status indicator 438 may be color-coded in appearance such that a blue background with a white colored character (r) may indicate a ready status, a green background with a black colored character (c) may indicate a connect status, and/or a yellow background with a black colored character (d) may indicate a disconnect status. - In one implementation, a fuel status box (not shown) may be displayed in a portion of the field of
view 400. For example, the fuel status box may be displayed within a 30 degree field of view centered on the housing of thecontrol components 122 at the intersection of theboom 121 and control component hinge lines. Also, caution and warning display indications, such as thewarning indicator 216 andstatus indicators 220 ofFIG. 2A , may be positioned in the field ofview 400 with the intent of facilitating operator recognition of events. - In various implementations, referring to
FIGS. 4A and 4B , boom envelope azimuth, elevation, and telescoping boundaries for the receiver aircraft 11 d are dynamically adjusted for that receiver aircraft type and boom telescoping. The background color of thegauges boom 121 relative to theboom envelope 208. - Status indicators are overlaid on the housing of the
control components 122, and in this case, they include routine boom state information. All of the indicators move with theboom 121. Filters may be used to reduce jitter or jumpiness of the indicators as a result of noise in the boom position measurements, and to ensure that the symbols remain readable during rapid boom movement. As the boom moves to a boundary of the visual field of view of the display, the overlaid symbology remains visible at the edge of the display, ghosting the position of theboom 121. - Referring to
FIG. 4B , boom markings reference overlays as described inFIG. 2B , which are combined with the symbology anchored to theboom 121. This concept provides intuitive analog boom position references with the concentrated information superimposed on theboom 121. Besides providing a reference for boom position, thechevrons 212 provide an operator with known references in the field of view that aid in judging the distance to thereceiver aircraft 110 c as thereceiver aircraft 110 c approaches thetanker aircraft 100. -
FIG. 5 shows one embodiment of aprocess 500 for facilitating the display of a field of view (FOV) having graphic display symbology. The description ofFIG. 5 references thevisual display system 160 ofFIGS. 1A-1D and the FOV displays ofFIGS. 2A-4B . - In one implementation, the
controller 130 receives FOV image data from theoptical sensors 140 including stereoscopic cameras 141 (block 510), and thecontroller 130 receives control signals from an operator via the operator input device 150 (block 510). Next, thecontroller 130 processes the received FOV image data and control data (block 514). Next, thecontroller 130 generates graphic display symbology based, at least in part, on the control data received from the operator input device 150 (block 518). Next, thecontroller 130 further processes the FOV image data by combining the generated symbology with the FOV image data (block 522). In one aspect, this processing is adapted to display the symbology as overlaid or superimposed on the FOV image data. Next, thecontroller 130 displays the FOV image with the symbology in thedisplay component 153 for viewing by an operator (block 526). Next, thecontroller 130 updates the FOV image data from theoptical sensors 140 and the control signals from the operator input device 150 (block 530). Next, theprocess 500 returns (or loops back) to block 514 so that thecontroller 130 may process the updated FOV image data and the control data. Theprocess 500 continues therefrom. - In various implementations, the
controller 130 may be adapted to store optical sensor data, FOV image data, and control signal data in a storage component, without departing from the scope of the present disclosure. Thecontroller 130 may also be adapted to retrieve these various types of data and information related thereto from the storage component. -
FIG. 6 is a block diagram of a computer system and/orcontroller 600 suitable for implementing embodiments of the present disclosure.Computer system 600 includes abus 602 or other communication mechanism for communicating information, which interconnects subsystems and components, such asprocessor 604, system memory component 606 (e.g., RAM), static storage component 608 (e.g., ROM), removable memory component 610 (e.g., removable ROM memory, such as EEPROM, smart card, flash memory, etc.), wired or wireless communication interface 612 (e.g., transceiver, modem or Ethernet card), display component 614 (e.g., LCD, CRT, etc.), input component 616 (e.g., sensors, such as optical sensors including stereoscopic cameras, keyboard, microphone, touch screen on display), and cursor control component 618 (e.g., mouse button). - In accordance with embodiments of the invention,
computer system 600 performs specific operations byprocessor 604 executing one or more sequences of one or more instructions contained insystem memory component 606. Such instructions may be read intosystem memory component 606 from another computer readable medium, such asstatic storage component 608 orremovable memory component 610. In other embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. - Logic may be encoded in a computer readable medium, which may refer to any medium that participates in providing instructions to
processor 604 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. In various implementations, non-volatile media includes removable storage media, such asremovable memory component 610, volatile media includes dynamic memory, such assystem memory component 606, and transmission media including wireless transceivers. In one example, transmission media may take the form of radio waves, such as those generated during radio wave and infrared data communications. - Some common forms of computer readable media includes, for example, floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, carrier wave, or any other medium from which a computer is adapted to read.
- In various embodiments of the invention, execution of instruction sequences to practice the invention may be performed by
computer system 600. In various other embodiments of the invention, a plurality ofcomputer systems 600 coupled by communication link 620 (e.g., wireless cell phone network, wireless or wired LAN, PTSN, or various other wireless networks) may perform instruction sequences to practice the invention in coordination with one another. -
Computer system 600 may transmit and receive messages, data, information and instructions, including one or more programs (i.e., application code) throughcommunication link 620 andcommunication interface 612. Received program code may be executed byprocessor 604 as received and/or stored inremovable memory component 610 or some other non-volatile storage component for execution. - Where applicable, various embodiments of the invention may be implemented using hardware, software, or various combinations of hardware and software. Where applicable, various hardware components and/or software components set forth herein may be combined into composite components comprising software, hardware, and/or both without departing from the scope and functionality of the present disclosure. Where applicable, various hardware components and/or software components set forth herein may be separated into subcomponents having software, hardware, and/or both without departing from the scope and functionality of the present disclosure. Where applicable, it is contemplated that software components may be implemented as hardware components and vice-versa.
- Software, in accordance with the present disclosure, such as program code and/or data, may be stored on one or more computer readable mediums. It is also contemplated that software identified herein may be implemented using one or more general purpose or specific purpose computers and/or computer systems, networked and/or otherwise. Where applicable, ordering of various steps described herein may be changed, combined into composite steps, and/or separated into sub-steps to provide features described herein.
- Embodiments described above illustrate but do not limit the disclosure. It should also be understood that numerous modifications and variations are possible in accordance with the principles of the present disclosure. Accordingly, the scope of the disclosure is defined only by the following claims.
Claims (25)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/139,032 US7980512B1 (en) | 2008-06-13 | 2008-06-13 | System and method for displaying aerial refueling symbology |
EP20090763382 EP2300321B1 (en) | 2008-06-13 | 2009-06-08 | System and method for displaying aerial refueling symbology |
PCT/US2009/046608 WO2009152091A2 (en) | 2008-06-13 | 2009-06-08 | System and method for displaying aerial refueling symbology |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/139,032 US7980512B1 (en) | 2008-06-13 | 2008-06-13 | System and method for displaying aerial refueling symbology |
Publications (2)
Publication Number | Publication Date |
---|---|
US7980512B1 US7980512B1 (en) | 2011-07-19 |
US20110180666A1 true US20110180666A1 (en) | 2011-07-28 |
Family
ID=40999876
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/139,032 Active 2030-04-03 US7980512B1 (en) | 2008-06-13 | 2008-06-13 | System and method for displaying aerial refueling symbology |
Country Status (3)
Country | Link |
---|---|
US (1) | US7980512B1 (en) |
EP (1) | EP2300321B1 (en) |
WO (1) | WO2009152091A2 (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100332055A1 (en) * | 2009-06-24 | 2010-12-30 | Airbus Operations (Sas) | Assistance process and device for managing an in-flight refueling |
US20140353430A1 (en) * | 2012-01-04 | 2014-12-04 | Israel Aerospace Industries Ltd. | Devices, systems and methods for refueling air vehicles |
US10343788B2 (en) * | 2017-09-26 | 2019-07-09 | The Boeing Company | Telescoping refueling boom control systems and methods |
US11465768B2 (en) | 2017-07-10 | 2022-10-11 | Israel Aerospace Industries Ltd. | Refueling device |
US11599323B2 (en) | 2020-09-28 | 2023-03-07 | Rockwell Collins, Inc. | Touchscreen boom pod |
USD1006822S1 (en) | 2020-10-30 | 2023-12-05 | Rockwell Collins, Inc. | Touchscreen display boom graphical user interface |
US11919655B2 (en) | 2017-06-18 | 2024-03-05 | Israel Aerospace Industries Ltd. | System and method for refueling air vehicles |
Families Citing this family (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9090354B2 (en) * | 2006-03-02 | 2015-07-28 | The Boeing Company | System and method for identifying a receiving aircraft during airborne fueling |
FR2935792B1 (en) * | 2008-09-05 | 2010-09-03 | Thales Sa | VISUALIZATION DEVICE FOR AIRCRAFT COMPRISING RADIONAVIGATION TAG DISPLAY MEANS AND ASSOCIATED METHOD |
EP2182387A1 (en) * | 2008-10-30 | 2010-05-05 | EADS Construcciones Aeronauticas, S.A. | Indication system and method for refuelling operations. |
EP2280359A1 (en) * | 2009-07-31 | 2011-02-02 | EADS Construcciones Aeronauticas, S.A. | Training method and system using augmented reality |
US9150310B1 (en) * | 2010-09-10 | 2015-10-06 | The Boeing Company | Methods and systems for directing remote aerial refueling operations |
US9085370B2 (en) * | 2013-06-03 | 2015-07-21 | General Electric Company | Systems and methods for wireless data transfer during in-flight refueling of an aircraft |
US10012667B2 (en) * | 2014-10-16 | 2018-07-03 | The Boeing Company | Methods and systems for airborne vehicle wind depiction |
ES2584231B2 (en) * | 2015-10-09 | 2017-06-02 | Defensya Ingeniería Internacional, S.L. | LOCALIZATION SYSTEM FOR THE END OF THE BOOM, THE MOUTH OF THE RECOVERY RECEPTACLE AND THE TANKER |
US10654584B2 (en) * | 2016-08-13 | 2020-05-19 | Modern Technology Solutions, Inc. | Refueling system and method |
US11745894B2 (en) * | 2016-08-20 | 2023-09-05 | Modern Technology Solutions, Inc. | Vehicle refueling and recharging |
US11505330B2 (en) * | 2016-08-20 | 2022-11-22 | Modern Technology Solutions, Inc. | Refueling system and systems with end effectors |
US10882635B2 (en) * | 2018-05-09 | 2021-01-05 | The Boeing Company | Aiding maneuvering of obscured objects |
US11459116B2 (en) * | 2020-06-24 | 2022-10-04 | The Boeing Company | Virtual laser pointer as a point of contact indicator for machine learning assisted aerial refueling or other targeting |
CA3136258A1 (en) * | 2021-01-05 | 2022-07-05 | The Boeing Company | Fuel receptacle and boom tip position and pose estimation for aerial refueling |
US20220306311A1 (en) * | 2021-03-23 | 2022-09-29 | The Boeing Company | Segmentation-based fuel receptacle localization for air-to-air refueling (a3r) |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4093347A (en) * | 1976-05-10 | 1978-06-06 | Farrand Optical Co., Inc. | Optical simulation apparatus using controllable real-life element |
US5175616A (en) * | 1989-08-04 | 1992-12-29 | Her Majesty The Queen In Right Of Canada, As Represented By The Minister Of National Defence Of Canada | Stereoscopic video-graphic coordinate specification system |
US5296854A (en) * | 1991-04-22 | 1994-03-22 | United Technologies Corporation | Helicopter virtual image display system incorporating structural outlines |
US5499784A (en) * | 1993-05-12 | 1996-03-19 | Aerospatiale Societe Nationale Industrielle | Flight refuelling system |
US6111582A (en) * | 1996-12-20 | 2000-08-29 | Jenkins; Barry L. | System and method of image generation and encoding using primitive reprojection |
US20050012684A1 (en) * | 2003-07-16 | 2005-01-20 | Hewitt Gordon S. | Methods and apparatus for reducing frame violations in a stereoscopic display |
US20060055628A1 (en) * | 2004-09-14 | 2006-03-16 | The Boeing Company | Situational awareness components of an enhanced vision system |
US20070242066A1 (en) * | 2006-04-14 | 2007-10-18 | Patrick Levy Rosenthal | Virtual video camera device with three-dimensional tracking and virtual object insertion |
US20080100570A1 (en) * | 1999-03-02 | 2008-05-01 | Wolfgang Friedrich | Augmented-Reality System for Situation-Related Support of the Interaction between a User and an Engineering Apparatus |
US7437221B2 (en) * | 2004-12-16 | 2008-10-14 | Raytheon Company | Interactive device for legacy cockpit environments |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE3440812A1 (en) | 1984-11-08 | 1986-05-22 | Messerschmitt-Bölkow-Blohm GmbH, 8012 Ottobrunn | Control station for an aircraft flight-refuelling device |
US5530650A (en) | 1992-10-28 | 1996-06-25 | Mcdonnell Douglas Corp. | Computer imaging system and method for remote in-flight aircraft refueling |
US6152357A (en) | 1999-01-06 | 2000-11-28 | The Standard Register Company | Shipping envelope |
US7469863B1 (en) | 2005-03-24 | 2008-12-30 | The Boeing Company | Systems and methods for automatically and semiautomatically controlling aircraft refueling |
US7309048B2 (en) | 2005-07-29 | 2007-12-18 | The Boeing Company | Vision system and method incorporating graphics symbology for use in a tanker refueling system |
-
2008
- 2008-06-13 US US12/139,032 patent/US7980512B1/en active Active
-
2009
- 2009-06-08 WO PCT/US2009/046608 patent/WO2009152091A2/en active Application Filing
- 2009-06-08 EP EP20090763382 patent/EP2300321B1/en active Active
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4093347A (en) * | 1976-05-10 | 1978-06-06 | Farrand Optical Co., Inc. | Optical simulation apparatus using controllable real-life element |
US5175616A (en) * | 1989-08-04 | 1992-12-29 | Her Majesty The Queen In Right Of Canada, As Represented By The Minister Of National Defence Of Canada | Stereoscopic video-graphic coordinate specification system |
US5296854A (en) * | 1991-04-22 | 1994-03-22 | United Technologies Corporation | Helicopter virtual image display system incorporating structural outlines |
US5499784A (en) * | 1993-05-12 | 1996-03-19 | Aerospatiale Societe Nationale Industrielle | Flight refuelling system |
US6111582A (en) * | 1996-12-20 | 2000-08-29 | Jenkins; Barry L. | System and method of image generation and encoding using primitive reprojection |
US20080100570A1 (en) * | 1999-03-02 | 2008-05-01 | Wolfgang Friedrich | Augmented-Reality System for Situation-Related Support of the Interaction between a User and an Engineering Apparatus |
US20050012684A1 (en) * | 2003-07-16 | 2005-01-20 | Hewitt Gordon S. | Methods and apparatus for reducing frame violations in a stereoscopic display |
US20060055628A1 (en) * | 2004-09-14 | 2006-03-16 | The Boeing Company | Situational awareness components of an enhanced vision system |
US7437221B2 (en) * | 2004-12-16 | 2008-10-14 | Raytheon Company | Interactive device for legacy cockpit environments |
US20070242066A1 (en) * | 2006-04-14 | 2007-10-18 | Patrick Levy Rosenthal | Virtual video camera device with three-dimensional tracking and virtual object insertion |
Cited By (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100332055A1 (en) * | 2009-06-24 | 2010-12-30 | Airbus Operations (Sas) | Assistance process and device for managing an in-flight refueling |
US8463538B2 (en) * | 2009-06-24 | 2013-06-11 | Airbus Operations (Sas) | Assistance process and device for managing an in-flight refueling |
US10479523B2 (en) | 2012-01-04 | 2019-11-19 | Israel Aerospace Industries Ltd. | Systems and methods for air vehicles |
US10543929B2 (en) | 2012-01-04 | 2020-01-28 | Israel Aerospace Industries Ltd. | Systems and method for air vehicles |
US9573696B2 (en) | 2012-01-04 | 2017-02-21 | Israel Aerospace Industries Ltd. | Systems and methods for air vehicles |
US11834192B2 (en) | 2012-01-04 | 2023-12-05 | Israel Aerospace Industries Ltd. | Devices, systems and methods for refueling air vehicles |
US10421556B2 (en) * | 2012-01-04 | 2019-09-24 | Israel Aerospace Industries Ltd. | Devices, systems and methods for refueling air vehicles |
US10427801B2 (en) | 2012-01-04 | 2019-10-01 | Israel Aerospace Industries Ltd. | Devices, systems and methods for refueling air vehicles |
US20140353430A1 (en) * | 2012-01-04 | 2014-12-04 | Israel Aerospace Industries Ltd. | Devices, systems and methods for refueling air vehicles |
US9457912B2 (en) | 2012-01-04 | 2016-10-04 | Israel Aerospace Industries Ltd. | Systems and methods for air vehicles |
US11167860B2 (en) | 2012-01-04 | 2021-11-09 | Israel Aerospace Industries Ltd. | Devices, systems and methods for refueling air vehicles |
US11180262B2 (en) | 2012-01-04 | 2021-11-23 | Israel Aerospace Industries Ltd. | Devices, systems and methods for refueling air vehicles |
US11919655B2 (en) | 2017-06-18 | 2024-03-05 | Israel Aerospace Industries Ltd. | System and method for refueling air vehicles |
US11465768B2 (en) | 2017-07-10 | 2022-10-11 | Israel Aerospace Industries Ltd. | Refueling device |
US10343788B2 (en) * | 2017-09-26 | 2019-07-09 | The Boeing Company | Telescoping refueling boom control systems and methods |
US11599323B2 (en) | 2020-09-28 | 2023-03-07 | Rockwell Collins, Inc. | Touchscreen boom pod |
USD1006822S1 (en) | 2020-10-30 | 2023-12-05 | Rockwell Collins, Inc. | Touchscreen display boom graphical user interface |
Also Published As
Publication number | Publication date |
---|---|
US7980512B1 (en) | 2011-07-19 |
WO2009152091A2 (en) | 2009-12-17 |
EP2300321A2 (en) | 2011-03-30 |
WO2009152091A3 (en) | 2010-04-08 |
EP2300321B1 (en) | 2015-05-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7980512B1 (en) | System and method for displaying aerial refueling symbology | |
US8400330B2 (en) | System for displaying multiple overlaid images to a pilot of an aircraft during flight | |
CN109552650B (en) | Method for assessing misalignment when landing a local machine and cockpit display system for use on a local machine | |
US6121899A (en) | Impending aircraft tail strike warning display symbology | |
EP1747992B1 (en) | Vision system and method incorporating graphics symbology for use in a tanker refueling system | |
US7928927B1 (en) | Head worn head up display system | |
JP5207177B2 (en) | Visual recognition support device and visual recognition support method | |
CN104249813B (en) | Aerocraft system for showing runway light information and method | |
EP2128732B1 (en) | Method for displaying crosswind correction for approach to a runway | |
US9150310B1 (en) | Methods and systems for directing remote aerial refueling operations | |
EP3637058B1 (en) | Vision guidance systems for aircraft | |
US5357263A (en) | Display instrument for aircraft for showing the aircraft orientation, particularly the rolling and pitching position or the flight path angle | |
US7463954B1 (en) | Terrain augmented display symbology | |
JP5185141B2 (en) | Method and apparatus for automatically adjusting aircraft navigation screen images | |
CN107487449B (en) | Display system and method for an aircraft | |
US20150019048A1 (en) | Display systems and methods for providing displays having an adaptive combined vision system | |
US11802780B2 (en) | Display systems and methods for aircraft | |
EP2466264B1 (en) | Method for presenting spatial attitude and heading information of a vehicle | |
US7508322B1 (en) | Large error to desired path correction indicator | |
US11699349B2 (en) | Electronic exocentric symbol display device and associated display method and computer program product | |
US20200020238A1 (en) | Aircraft, and a system for signalling a non-prepared landing area | |
EP1835260B1 (en) | Terrain augmented display symbology | |
EP3246905B1 (en) | Displaying data by a display system | |
US20170168560A1 (en) | Method of displaying an "attitude director indicator" in a head viewing system for aircraft | |
JP7367922B2 (en) | Pilot support system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: THE BOEING COMPANY, ILLINOIS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SPEER, THOMAS E;DOWNS, JOSHUA L;REEL/FRAME:021095/0242 Effective date: 20080612 |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 12TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1553); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 12 |