Improving Aviation Safety using Synthetic Vision System integrated with Eye-tracking Devices

By collecting the data of eyeball movement of pilots, it is possible to monitor pilot's operation in the future flight in order to detect potential accidents. In this paper, we designed a novel SVS system that is integrated with an eye tracking device, and is able to achieve the following functions:1) A novel method that is able to learn from the eyeball movements of pilots and preload or render the terrain data in various resolutions, in order to improve the quality of terrain display by comprehending the interested regions of the pilot. 2) A warning mechanism that may detect the risky operation via analyzing the aviation information from the SVS and the eyeball movement from the eye tracking device, in order to prevent the maloperations or human factor accidents. The user study and experiments show that the proposed SVS-Eyetracking system works efficiently and is capable of avoiding potential risked caused by fatigue in the flight simulation.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 4

page 6

11/14/2021

An Assessment of the Eye Tracking Signal Quality Captured in the HoloLens 2

We present an analysis of the eye tracking signal quality of the HoloLen...
09/02/2020

Addressing the eye-fixation problem in gaze tracking for human computer interface using the Vestibulo-ocular Reflex

A custom head-mounted system to track smooth eye movements for control o...
08/18/2020

Hierarchical HMM for Eye Movement Classification

In this work, we tackle the problem of ternary eye movement classificati...
05/07/2020

Sunny Pointer: Designing a mouse pointer for people with peripheral vision loss

We present a new mouse cursor designed to facilitate the use of the mous...
07/17/2017

Hybrid PS-V Technique: A Novel Sensor Fusion Approach for Fast Mobile Eye-Tracking with Sensor-Shift Aware Correction

This paper introduces and evaluates a hybrid technique that fuses effici...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Synthetic Vision Systems (SVS) are designed to provide 3D terrain images of the aircraft’s future routine, in order to enhance the awareness of potential Controlled Flight into Terrain (CFIT) accidents for the pilots[5][6][7]. The images are composed by a virtual environment of the outside terrains and various symbols representing the relevant information that may impact the aircraft in the flight path, such as a real-time navigation route, the buildings and water that are beyond the sight [7, 12], and the depiction of traffic in the airspace ahead of the pilot [13]. Through the virtual tunnel shown on the SVS displays, the pilots may detect the hidden obstacles before it is too late to react, or be aware of the potential risk of their operations that may endanger the flight.

In recent times, with the development of eye tracking technology that has been widely used in the flight training simulations, the original SVS techniques are also benefited via introducing the eye tracking devices into the real time aviation environment. In [15] and [1], eye-tracking data were collected from pilots in order to investigate their concentration during the flight. On the other hand, by collecting the data of eyeball movement of pilots, it is possible to monitor pilot’s operation in the future flight in order to detect potential accidents. In this paper, we designed a novel SVS system that is integrated with an eye tracking device, and is able to achieve the following functions:

  • A novel method that is able to learn from the eyeball movements of pilots and preload or render the terrain data in various resolutions, in order to improve the quality of terrain display by comprehending the interested regions of the pilot.

  • A warning mechanism that may detect the risky operation via analyzing the aviation information from the SVS and the eyeball movement from the eye tracking device, in order to prevent the maloperations or human factor accidents.

The novel SVS system integrated an eye tracking device into an ordinary SVS system. A novel SVS-Eyetracking (S-E) architecture is proposed in this paper. Based on the proposed architecture, a novel algorithm of preloading and rendering the terrain data of multiple resolutions concerning the eyeball movements of pilots is illustrated. The proposed algorithm is able to transmit minimum terrain data during the data preloading period and render the polygonal tessellation considering the real-time eyeball movements. According to the questionnaires of user study, the proposed algorithm is capable of improving the driving experience of pilots.

The warning mechanism of fatigue/maloperation detection is another major feature of the proposed system. The fatigue detection via eye tracking in vehicle driving has been widely investigated by the community of smart transportation and human machine interface. However, the fatigue detection on an aircraft, especially the airline flights, is different from the detections on ordinary vehicles. The aviating operations based on the flight status are much more complicated than driving the vehicle on the road, and the maloperations are much more difficult to be recognized. In this case, we propose a novel warning mechanism based on the proposed SVS-Eyetracking architecture that is able to detect the potential risky operations on the current flight status in a non-intrusive way with good accuracy. The novel mechanism may also perceive pilots’ fatigue and prevent human-factor accidents by warning them.

The paper is organized as follows: the related work of SVS systems and eye tracking devices are in Section 2; the proposed S-E architecture is illustrated in Section 3; the optimal data preload algorithm is proposed in Section 4; the new warning mechanism is illustrated in Section 5; the experiences and user study are in Section 6; the conclusion of this paper is in Section 7.

Ii Related work

Ii-a Terrain Representation

The measuring and representing terrain textures has been widely studied during the last decades [7, 18, 27, 30, 31, 32, 8, 9, 14, 23, 4, 28, 29]. The terrain generated on a SVS display device is a continuous mesh of polygons stored in a binary tree. The general solution of terrain LOD is using multi-resolution to control the data pre-loading and rendering process. There are many related surveys and references in [7, 18, 27, 30, 31, 32]. The terrain data are updated in real time, and the amount of pre-loaded data is incrementally changed according to the height and velocity of the aircraft. The first applied method of dealing with massive terrain data is the restricted quadtree triangulation (RQT) algorithm. In [13], the method of triangle stripping cost models is presented. However, the earlier terrain triangulation techniques depended on greedy algorithms do not support realtime or view-dependent scenarios [15, 21]. In [12], the data meshes are subdivided, and the problem of irregular meshes are converted into smoother surfaces. Based on these former works, the method of wavelet analysis to terrain LOD are proposed in [24]. This method supports good coherence for a movable and overlooking point of view, but if does not gurarantee the error bounds on fine-grained vertex deletion.

The recent researches on the terrain visualization are focused on dealing with the hierarchical triangulated meshes. In article [18], the method of triangle-bintree mesh is chosen. The mesh of their method is very like the same mesh in our method. The simple bintree structure is not recognized nor supported the split and merge operations. Therefore, special care must be taken to handle the continuity and coherence of maintain meshes. Meanwhile, with the requirement of better display performance, the frame rate is increased tremendously, which greatly increased the amount of pre-loaded data meshes. In [22, 14], a hierarchical triangulated-irregular network (TIN) data structure with near/far annotations for vertex morphing is described. These methods also include a queue-driven top-down refinement procedure for building the triangle mesh for a frame. However, The first version of TIN does not consider the automatic morphing procedures, and the memory requirements are still high for each multi-resolution meshes. The better version of a quadtree sturcture is proposed in [12], which preprocesses the height of the meshes in unified grids. The pre-processing phase computes the vertices at each quadree level, and the vertices are fitted to an approximate least-squares of the below level. The method also applies a priority queue in order to refine the quadtree from top levels to the root levels. The disadvantage of the proposed method is that it does not consider frame-to-frame coherence, while only one type of error metric is applied in the structure.

The recent development of fine-grained LOD representation techniques can handle the ordinary TIN data meshes[17, 2, 19, 20]. The new methods are able to preprocess the progressive mesh representation based on view dependent refinement, and the algorithms are allowed error metrics which are designed for flexible point of view. Moreover, detail reduction based on nested Gauss-map normal bounds are shown in their works. In [24], thin triangles are removed so that the error rate can be significantly reduced. New metrics for avoiding edge-collapse operations are proposed in [21]

, however, the heuristic estimation cannot justify the geometric distortions well. In

[25, 27], a novel feedback technique is proposed in order to process the rough rate regulations. However, the frame-to-frame consistency are not fully considered and the time complexity and cost are still depending on the size of meshes.

Fig. 1: Synthetic Vision Systems Integrated with eye tracking device. It preserves the original SVS functions and supports customized multi-resolution based on eye tracking as well as fatigue detection mechanism.

Ii-B Fatigue Detection

Fatigue detection via tracking the driver’s eye has been proved to be an efficient method[26]. The expressions on the face such as eyelid motions, yawning, staring, or gaze are visual cues of fatigue. Eye tracking devices are installed in order to accumulate the eyeball movement information of the pilots. According to researches [9, 23], most of the fatigues-realated information can be obtained from the driver’s eyeball movements. There are already plenty of metrics of fatigues related to eyeball movement information being developed. Many commercial fatigue detection systems have been developed for vehicles, such as AntiSleep developed by Smart Eye AB and Driver State Sensor (DSS) developed by Seeing Machines[23]. By analyzing the timely records of the eyeball movements in a short time period, as well as the driving information collected from the SVS system, the fatigue status of the pilots can be easily justified.

Iii System Overview

The integrated SVS-Eyetracking system architecture contains two modules: the SVS module, including the aviation status analyzer, the terrain data server, the display devices, and etc; the eye tracking module, including the eyeball movement tracking device, the human status analyzer, and the warning device. The architecture of the system is shown in fig 1.

The SVS module of the system contains the components of an ordinary SVS system. According to NASA SVS concept(2009), the ordinary SVS systems are composed of the sensors and database servers, the embedded computation server, and the display devices. The sensors and database servers include the weather radar, the radar altimeter, forward looking infrared sensor, and on-board synthetic vision databases. The embedded computation server analyzes the status of the aircraft in real time. It is also responsible for the image object detection and fusion, image enhancement, and terrain data rendering. The display devices are head-mounted or screen-based. In our proposed architecture, several new components are added to the embedded computation servers and the terrain data servers. New embedded hardware chips and software interfaces are installed in the ordinary SVS system in order to communicated with the eye tracking module.

The eye tracking device records three kinds of eyeball movements of the pilots: fixations, saccades, and pursuit movements. It provides the trail of the gazing spots on the terrain image to the SVS computation server, and by analyzing it with the flight status information from the SVS module, predicts the high resolution area of the terrain on the screen. On the other hand, by comparing the eyeball movements of the previous records with the present ones, the warning device may find out whether pilots are in fatigue, and remind them when the SVS module indicates the flight is in abnormal status.

Fig. 2: The SVS-Eyetracking Architecture

Iv Data Preloading and rendering Algorithms

In this section, we present a data preload algorithm that is designed to preload and render minimum terrain data from terrain data server to the computation server according to the eye-tracking information of the pilot. The procedure of the whole system is shown in figure 2.

Fig. 3: Procedure of the SVS and Eyetracking modules

Iv-a Terrain Data Structure

We propose a tree structure for the terrain data in order to integrate various types of geometry meshes into one hierarchical model. The improved terrain data structure is capable of emphasizing the interested area of the pilot.

First, we assume be the data point sets within the -plane. Suppose for each data set , the bounding box of is within the area surrounded by the extremal points and . The domain of is defined as .

Let be a texture mesh which consists terrain patches of . The extremal points of the axis parallel bounding box are denoted by and . The domain of the texture is defined by .

We define a geometry patch in a tree structure as for a terrain data set , where there are at most points of terrain data within the boundary, and child nodes in the tree structure. Each child node represents a rectangular region . The bounding box of is the domain . The way of calculating the exact number of points for each node depends on the data preload strategy.

For every node , there are at most child nodes contained. We construct the child nodes as follows: Let be the terrain data of a node N of which the domain . The geometric approximation error are defined as the maximal vertical distance between the terrains D and , i.e. [5], . For each subdomain , a node will be constructed which approximates the terrain in that subdomain, and will be added as child node to the parent node N. If the error between and exceeds a certain threshold , i.e., , then the domain of is subdivided into a set of at most rectangular subdomains such that with . The domain of the root node of the tree is covering the whole domain of the terrain T.

Iv-B Data Preloading Algorithm

In this section, we introduce the data preload algorithm that is utilized in the proposed integrated SVS system. Although the modern graphics stations may be capable of rendering thousands of shaded or textured polygons at interactive rates, the geometry complexity of the terrain data is still far more exceeded the capability of computation servers on the aircraft. Therefore, many algorithms of Level of Detailed (LOD) rendering have been proposed in the previous work of large terrain display on airbone SVS systems. The bottleneck of ordinary SVS systems is that the terrain data are too large to be stored in the computation servers. Only the data that may be required in the future will be loaded from the terrain data server to the computation server. In order to minimize the unnecessary transmitted data, we propose the data preload algorithm that is able to select the least required data to be preloaded.

The data preload algorithm considers the navigation information, the aircraft status, the operation of the pilot, and the important spots by tracking the eyeball movement of the pilot. In ordinary SVS systems, the terrain data are stored in a format of hierarchical bintree or quadtree. The vertexes of each triangle/rectangle are arranged in layers. Generally, when the SVS module needs to render the current view of the terrain, it renders the terrain near the flight in high resolutions, and the distant view in low resolutions. Since the capability of the computation server and the terrain data server is limited, at each time step, the computation server only loads the necessary terrain data from the terrain data server.

Fig. 4: Preloaded data according to the fusum

The navigation information of the aircraft is obtained from the navigation server of the SVS module. The navigation information contains the route from the source to the destination. The computation server only preload the terrain data in the sight scope of the route.

The flight status of the aircraft mainly consists of three kind of information: the altitude, the vocality, and the direction of the flight. In Figure 4, the area of preloaded terrain data of current time step is illustrated. We suppose that left angle and right angle to the forwarding direction is and .

is the extended terrain area of sight. In order to simplify the computation, we suppose the aircraft is the origin of axis, the aircraft heading is the Y axis, the vector connected the center of the earth and the aircraft is the Z axis, and the vector perpendicular to the YOZ plane is the X axis. The height of the aircraft is

. The velocity of the aircraft is . At the initiation, the preloaded terrain area . and are defaulted angle that represent the possible area of sight. . The preloaded terrain area is . The real time extended area of sight and the angles and are computed in formulation 1.

(1)

In formulation 1, is the angle between the velocity vector and the Z axis. The real-time preloaded terrain area is . For each frame in the time step, is defined as the geometry patch of this terrain area at time . Based on the tree structure defined previously, we preload the nodes of for each frame.

Iv-C Marking Interested Areas based on Eye Tracking

The greedy data preload algorithm is capable of minimize the amount of unnecessary terrain data transmission between the terrain data server and the computation server. However, based on the survey of pilots encountering the abnormal events during the flight [15], each pilot requires different spots on the screen for higher resolution. Since there is no obvious way of satisfying all pilots’ needs, we propose a method that discovering the important spots via the eyeball movements of pilot.

The novel method consists of two steps. First, the flight route and the eyeball movements of the pilot are recorded during the whole trip with time stamps. Each eyeball movement is re-mapped to the terrain data of the route by the time stamps.

According to the study of [16], the interested area can be acquired by learning the trial of eyeball movements. Since we use the time stamp to re-map the eyeball movements and the terrain vertexes, it is easy to find out the spots that the pilots gazed or become watchful in the route. It is also possible to use HMM model or other machine learning method to find out the interested area in a more precise way.

After the interested spots are located, the terrain data server preload these data to the computation server in the next flight trip. Each time, the interested area are alternated by the last record of the same trip. The mechanism may maintain a limited list of spots descendent by priority. The spots of least priority are removed from the list, and new spots of higher priority are added into the list. The whole algorithm is present in Algorithm 4.1.

0:   The terrain data points of trip , the record of eyeball movement during the trip , the priority list of previous time step , the time stamps of the terrain data in the route , the time stamp of the eyeball movement during the trip
0:   A new priority list of spots .
1:  Remap to based on and .
2:  Detect interested area of vertexes by the gazing time or the glance time. The interested spots are given a priority based on the attention time of the pilot.
3:  Update the priority list by comparing the priority of new spots to the ones in the list. The spots of least priority are removed from the list.
4:  Generate the new priority list by moving the top N spots of to the .
Algorithm 1 The Data Preload Algorithm
Fig. 5: The historical interested area (in red circles) of the pilots are displayed with higher resolutions.
Fig. 6: The rendering result of the terrain area

V Fatigue Detection and Warning Mechanism

In this section, we introduce the novel warning mechanism based on eye tracking. The warning mechanism, named as Eye Tracking Fatigue Alert (ETFA), is illustrated. The Proposed Algorithm can be divided into two steps. First, a fatigue detection mechanism is generated. Second, the fatigue generating mechanism is integrated with the SVS system.

In our implementation, every tenth frame from the eye tracking video is processed. We utilize the important spots that are acquired in the optimal data preload algorithm to activate the warning mechanism. When the flight is within the warning range of the spot, the fatigue alarm is activated. As long as the fatigue status of the pilot is above the normal status, an alert will be sent to the pilot.

When the system starts, frames are continuously fed from the camera to the eye tracking analyzer. We use the initial frame in order to localize the eye-positions. Once the eyes are localized, we start the tracking process by using information in previous frames in order to achieve localization in subsequent frames. During the tracking process, error detection is performed in order to recover from possible tracking failure. When a tracking failure is detected, the eyes are relocalized.

The fatigue detection and warning mechanism are activated during the flight simultaneously. When preloading the terrain data, the SVS system detects the potential risks on the terrain map, like high mountains, towers, waters, or other static obstacles. The sensors of the SVS system detects the flight status in real time in order to analyze the current risk level of the flight. If the pilots are in fatigue when the flight is near the risky terrain area, or the risk level of the flight is too high, the warning device will send signals immediately.

Vi Experiments and User Study

The evaluations of our S-E system are conducted in three parts. First, we invited pilots using our S-E system in an aviating simulation environment in order to evaluate the display performance of the system. Second, we invited twenty pilots in 8 experimental scenarios to test the accuracy of fatigue detection. Third, we conduct a user study for the pilots for the evaluation of the improvement of driving experience. A questionnaire is given to the pilots who drive a long distance simulation using the proposed S-E system.

Vi-a Performance Evaluation

The pilots’s eyeball movement is recorded at the first time when they are participating the simulation. The multiresolution display performance is shown in figure 8.

The simulation is an one hour flight period, including the taking off period, the smooth flight period in the middle, and the landing period. We demonstrate the data transmission amount between the terrain data server and the computation server during the simulation in figure 9. In the table of the figure 9, the X axis is the time line and the Y axis is the amount of data transmitted from the data server to the computation server. The black line at the above is the amount of data transmitted without data prediction algorithm. The histogram is the amount of data transmitted utilizing the proposed data prediction algorithm. The “Taking off” title shows the time period of aircraft rising from the airport to a certain height. The “Smooth flight in the middle” title shows the time period that the aircraft cruises at certain altitude. The “Landing” title shows the time period of the aircraft descends to the ground.

Fig. 7: Data throughput of the simulation

The evaluation shows that, without an optimized data preload algorithm, the throughput of the network and the computation load on the servers are remaining at a high status during the whole simulation. After utilizing the data prediction algorithm, the data transmission amount is reduced to an acceptable degree.

Vi-B Fatigue Detection Evaluation

Twenty participants are invited for this evaluation. The experiment was conducted in a cockpit equipped with the proposed integrated SVS-Eyetracking system.

Participants flew 8 experimental scenarios of 80-100 minutes each, involving a curved step-down approach through a terrain challenged region to a simulated airport. Pilots were instructed to use the eye tracking device during the whole experiment period. While flying, pilots were instructed to detect any important/interest terrain spot that became visible on the SVS display, and also to report any changes to traffic altitude that they noticed on either the SVS display or the navigation display.

During the evaluation, some accidents are designed to occur at random time. If the pilot failed to react, the simulation will judge the pilot was at fatigue status. The eye tracking device also evaluates the fatigue degree of the pilot in real time. In all experimental scenarios, the eye tracking system did not warn the pilots when it finds out the pilots are in fatigue. The experimental results are shown in Table 1.

No. of S-E Total No. Accuracy
Scenario 1 55 97 56.7%
Scenario 2 68 90 75.6%
Scenario 3 34 37 91.9%
Scenario 4 60 75 80%
Scenario 5 23 24 95.8%
Scenario 6 17 17 100%
Scenario 7 25 28 89.3%
Scenario 8 9 10 90%
TABLE I: Experimental results of 8 scenarios

The first four scenarios are comparable challenging, and the possibilities of the occurring of risky scenarios are high. However, since the eyetracking device only record the fatigue warning signals, the number of fatigue detected by the S-E system is always smaller than the number of fatigue the simulation records. The experimental results of the last four scenarios shows that the number of fatigue that the S-E system detected are very similar to the number of fatigue the simulation detected.

Questions ratings from 1 to 5,
5 is the score for perfect experience
Is the SVS system showing the terrain consistent with
the simulation environment?
Is the SVS system correctly showing the symbols of
particular spots on the map?
Is there any delay on the display screen of SVS system
during the simulation?
Is there any ambiguous symbols on the screen?
Is there any display error occurred during
the simulation?
How intrusive when you drive with the eye
tracking device?
Is there any discomfort (e.g. dazzle) caused
by the system during the simulation?
Is the warning signal noticeable?
Is there any potential risk are noticed by the alert?
Is the fatigue detection accurate according
to the scenario?
Is the eye tracking device working correctly during
the simulation?
Is the interested area on the screen has better
resolution?
Is the interested area on the screen are the
spots you have been paying attention during
last simulation?
hline Is there any fake alert during the simulation?
Is the terrain warned on the screen displays correctly?
Is the fatigue detection helpful for avoiding
potential risks?
TABLE II: Rating questions for driving experience

Vi-C User Study for the Driving Experience

The study in [23] categorized the human factors issues related to SVS systems into three research areas: Image quality, information integration, and operational concepts. Based on their study, a questionnaire is given to the participants for the evaluation of their driving experience using the proposed S-E system.

The questionnaire consists up to thirty questions, which contains fifteen rating questions for the user’s evaluation of their driving experience, and fifteen questions for their opinions of system setting and suggestions. The evaluation questions includes the evaluation of the performance of the SVS system, the comfort of using the system, the accuracy of fatigue detection and warning signal, and other question related to human factor. The questions of setting and opinions inquired the setting parameters of the system and suggestions of the participants. The questionnaire is shown in Table II and Table III.

Most of the pilots show that the system works smoothly with nearly no error on the display screen. Some participants indicates that the symbol of certain terrain spots are too simple, and expected us to update the textures of building and other obstacles. Some participants reported some fake alarms during the flight, but they also indicated that they didn’t cause any maloperations. The overall feedback indicates the S-E system works efficiently and is helpful for avoiding the potential risks during the simulation.

The suggestions of the participants are carefully collected and the system setting is altered according to their opinions. The suggestions also indicates that the S-E system may require more consideration of human factors in the future.

Vii Conclusion

In this paper, we propose an SVS-Eyetracking architecture which integrated the eye tracking device into an SVS system. Based on the proposed architecture, we proposed a data prediction algorithm which reduces the amount of data transmitted from the terrain data server to the computation server. The data prediction algorithm considers the flight status provided by the SVS system and the eyeball movement of the pilot. The evaluation shows that the proposed algorithm may reduce the data transmitted significantly. We also implemented an fatigue warning mechanism in the proposed S-E system. The evaluation shows that the S-E system works efficiently and is capable of avoiding potential risked caused by fatigue in the flight simulation.

Acknowledgment

The authors would like to thank…

References

  • [1] K. Baumann, J. Doellner, K. Hinrichs, and O. Kersting. A hybrid, hierarchical data structure for real-time terrain visualization. In Computer Graphics International, 1999. Proceedings, pp. 85–92, 1999.
  • [2] D. B. Beringer. Development of highway-in-the-sky displays for flight c path guidance: History, performance results, guidelines. In Human Factors & Ergonomics Society Annual Meeting Proceedings, 44(13):21–24, 2000.
  • [3] O. Bimber, F. Coriand, A. Kleppe, E. Bruns, S. Zollmann, and T. Langlotz. Superimposing pictorial artwork with projected imagery. In ACM SIGGRAPH 2004 Sketches, p. 78, 2004.
  • [4] C.L. Philip Chen and C. Zhang, Data-intensive applications, challenges, techniques and technologies: A survey on Big Data. In Information Sciences, (275):314–347, 2014.
  • [5] H. P. Chiu, A. Das, P. Miller, S. Samarasekera, and R. Kumar. Precise vision-aided aerial navigation. In Ieee/rsj International Conference on Intelligent Robots and Systems, pp. 688–695, 2014.
  • [6] P. Cignoni, F. Ganovelli, E. Gobbetti, F. Marton, F. Ponchio, and R. Scopigno. Bdam — batched dynamic adaptive meshes for high performance terrain visualization. In Computer Graphics Forum, pp. 505–514, 2003.
  • [7] X. F. Dai, H. J. Xiong, and X. W. Zheng. A New Three-Dimensional Spherical Terrain Rendering Method Based on Network Environment. Springer Berlin Heidelberg, 2012.
  • [8] E. Yusov and M. Shevtsov. G. Mcintyre and K. Hintz Comprehensive Approach to Sensor Management, Part I: A Survey of Modern Sensor Management Systems. IEEE Transactions on Smc, 1999.
  • [9] R. M. Haralick, K. Shanmugam and I. Dinstein. Textural Features for Image Classification. IEEE Transactions on Systems, Man, and Cybernetics, SMC-3(6):610–621, 1973.
  • [10] T. O. Olasupo and C. E. Otero. The Impacts of Node Orientation on Radio Propagation Models for Airborne-Deployed Sensor Networks in Large-Scale Tree Vegetation Terrains. IEEE Transactions on Systems, Man, and Cybernetics: Systems, PP(99):1–14, 2017.
  • [11] J. S. Weszka, C. R. Dyer and A. Rosenfeld. A Comparative Study of Texture Measures for Terrain Classification. IEEE Transactions on Systems, Man, and Cybernetics, SMC-6(4):269–285, 1976.
  • [12] M. Duchaineau, M. Wolinsky, D. E. Sigeti, M. C. Miller, C. Aldrich, and M. B. Mineevweinstein. Roaming terrain: Real-time optimally adapting meshes. In Visualization ’97., Proceedings, pp. 81 – 88, 1997.
  • [13] H. Hoppe. Smooth view-dependent level-of-detail control and its application to terrain rendering. In Visualization ’98. Proceedings, pp. 35–42, 1998.
  • [14] L. J. P. III, J. J. Raymond Comstock, L. J. Glaab, L. J. Kramer, J. J. Arthur, and J. S. Barry. The efficacy of head-down and head-up synthetic vision display concepts for retro- and forward-fit of commercial aircraft. International Journal of Aviation Psychology, 14(1):53–77, 2004.
  • [15] O. Koichi and N. Tomoyuki. An interactive deformation system for granular material. pp. 51–60, 2005.
  • [16] L. J. Kramer, J. J. A. Iii, R. E. Bailey, and L. J. P. Iii. Flight testing an integrated synthetic vision system. 2005.
  • [17] X. Y. Li, M. Li, and W. Cai. The research of 3d terrain generation and interactivity realization techniques in virtual battlefield environment. In International Conference on Image & Graphics, pp. 668–671, 2009.
  • [18] Y. Liu, H. Chang, and S. L. Dai. Large scale terrain tessellation in flight simulator visual system. In Guidance, Navigation and Control Conference, pp. 1028–1033, 2014.
  • [19] J. Luo, G. Hu, and G. Ni. Dual-space ray casting for height field rendering. Computer Animation & Virtual Worlds, 25(1):45 C56, 2014.
  • [20] A. I. Mourikis and S. I. Roumeliotis.

    A multi-state constraint kalman filter for vision-aided inertial navigation.

    22:3565–3572, 2007.
  • [21] R. Pajarola. Large scale terrain visualization using the restricted quadtree triangulation. In Visualization ’98. Proceedings, pp. 19–26, 1998.
  • [22] R. Pajarola and E. Gobbetti. Survey of semi-regular multiresolution models for interactive terrain rendering. Visual Computer, 23(8):583–605, 2007.
  • [23] T. Schnell, Y. Kwon, S. Merchant, and T. Etherington. Improved flight technical performance in flight decks equipped with synthetic vision information system displays. International Journal of Aviation Psychology, 14(1):79–102, 2009.
  • [24] M. Sebastian, D. Patrick, W. Rol, and K. Reinhard. Context aware terrain visualization for wayfinding and navigation. Computer Graphics Forum, 27(7):1853–1860, 2008.
  • [25] T. Wiesemann, J. Schiefele, and W. Kubbat. Multi-resolution terrain depiction on an embedded 2d/3d synthetic vision system. Aerospace Science & Technology, 9(9):517–524, 2005.
  • [26] D. M. Williams, M. C. Waller, J. H. Koelling, D. W. Burdette, W. R. Capron, J. S. Barry, R. B. Gifford, and T. M. Doyle. Concept of operations for commercial and business aircraft synthetic vision systems. American Institute of Aeronautics & Astronautics, 2001.
  • [27] E. Yusov and M. Shevtsov. High-performance terrain rendering using hardware tessellation. Journal of Wscg, 19(1):85–92, 2011.
  • [28] M. Xu, M. Li, W. Xu, Zhi. Deng, Y. Yang, K. Zhou. Interactive Mechanism Modeling from Multi-view Images. ACM Transactions on Graphics, 35(6): Article 236, 2016.
  • [29] M. Xu, J. Zhu, P. Lv, B. Zhou, M. Tappen, R. Ji. Learning-based Shadow Recognition and Removal from Monochromatic Natural Images. IEEE Transactions on Image Processing, 26(12):5811-5824, 2017.
  • [30] J. Niu, J. Lua, M. Xu*, P. Lv, X. Zhao.

    Robust Lane Detection using Two-stage Feature Extraction with Curve Fitting.

    Pattern Recognition, 59: 225-233, 2016.
  • [31] M. Xu, N. Gu, W. Xu, M. Li, J. Xue, B. Zhou. Mechanical Assembly Packing Problem Using Joint Constraints. Journal of Computer Science and Technology, 32(6): 1162 C1171, 2017.
  • [32] M. Xu, P. Lv, M. Li, H. Fang, H. Zhao, B. Zhou, Y. Lin, L. Zhou. Medical image denoising by parallel non-local means. Neurocomputing, 195:117-122, 2016.