docs/use_cases/simone_a320_cockpit_simulator.md


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

---
title: SimOne A320 cockpit simulator
---

SimOne A320 cockpit simulator
=============================

The [ACHIL platform](http://achil.recherche.enac.fr) have a A320 glass cockpit simulator usually operated by ENAC’s Air Transportation department for system engineering courses to students. It is also used during MCTA training to give them an overview of the pilot’s counterpart. As this cockpit is no longer certified, it can be modified for research purposes and prototyping. It can also be connected to any simulation ran on the platform and integrate the rest of traffic.

In order to track pilots gaze during experimentation, a set of ArUco markers have been positioned to cover most of cockpit workspace.

![Cockpit](../img/simone_cockpit.png)

Then, in order to build AR environment from such complex geometry workspace, a 3D LIDAR scanner have been used to get a 3D scan of cockpit.

![Cockpit 3D](../img/simone_cockpit_3d.png)

The 3D scan have been loaded in a 3D editor to help in ArUco markers and AOI poses reporting.

![ArUco scene](../img/simone_aruco_scene.png) ![AOI scene](../img/simone_aoi_scene.png)

Finally, a python script connect Tobii eyetracker glasses to ArGaze toolkit. The 3D AR environment is loaded then, ArUco markers are detected from Tobii eyetracker field camera stream allowing to estimate pilote head pose. The AOI are projected into camera frame then, gaze positions are analyzed to identify fixations and saccades to finally check if fixations matched any projected AOI.

![AOI and gaze projection](../img/simone_projection.png)

A demonstration movie to see ArUco detection and AOI projection in real time

![type:video](http://achil.recherche.enac.fr/videos/marker_detection_and_aoi_projection.mp4)