1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
|
Load and execute pipeline
=========================
Once [ArUco markers are placed into a scene](aruco_markers_description.md), they can be detected thanks to [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class.
As [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class also benefits from all the services described in [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md).
![ArUco camera frame](../../img/aruco_camera_frame.png)
## Load JSON configuration file
The [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) internal pipeline loads from a JSON configuration file thanks to [ArUcoCamera.from_json](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera.from_json) class method.
Here is a simple JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration file example:
```json
{
"name": "My FullHD camera",
"size": [1920, 1080],
"aruco_detector": {
"dictionary": "DICT_APRILTAG_16h5",
"marker_size": 5
},
"gaze_movement_identifier": {
"DispersionThresholdIdentification": {
"deviation_max_threshold": 25,
"duration_min_threshold": 150
}
},
"image_parameters": {
"background_weight": 1,
"draw_detected_markers": {
"color": [0, 255, 0],
"draw_axes": {
"thickness": 3
}
},
"draw_gaze_positions": {
"color": [0, 255, 255],
"size": 2
},
"draw_fixations": {
"deviation_circle_color": [255, 0, 255],
"duration_border_color": [127, 0, 127],
"duration_factor": 1e-2
},
"draw_saccades": {
"line_color": [255, 0, 255]
}
}
}
```
Then, here is how to load the JSON file:
```python
from argaze.ArUcoMarkers import ArUcoCamera
# Load ArUcoCamera
aruco_camera = ArUcoCamera.ArUcoCamera.from_json('./configuration.json')
```
Now, let's understand the meaning of each JSON entry.
### *name - inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)*
The name of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. Basically useful for visualisation purpose.
### *size - inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)*
The size of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame in pixels. Be aware that gaze positions have to be in the same range of value to be projected in.
### *aruco_detector*
The first [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) pipeline step is to detect ArUco markers inside input image.
![ArUco markers detection](../../img/aruco_camera_markers_detection.png)
The [ArUcoDetector](../../argaze.md/#argaze.ArUcoMarkers.ArUcoDetector) is in charge to detect all markers from a specific dictionary with a given size in centimeters.
!!! warning "Mandatory"
JSON *aruco_detector* entry is mandatory.
### *gaze_movement_identifier - inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)*
The first [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) pipeline step dedicated to identify fixations or saccades from consecutive timestamped gaze positions.
![Gaze movement identification](../../img/aruco_camera_gaze_movement_identification.png)
### *image_parameters - inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)*
The usual [ArFrame visualisation parameters](../gaze_analysis_pipeline/visualisation.md) plus one additional *draw_detected_markers* field.
## Pipeline execution
### Detect ArUco markers, estimate scene pose and project 3D AOI
Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole pipeline dedicated to ArUco markers detection, scene pose estimation and 3D AOI projection.
```python
# Assuming that Full HD (1920x1080) timestamped images are available
...:
# Detect ArUco markers, estimate scene pose then, project 3D AOI into camera frame
aruco_camera.watch(timestamp, image)
# Display ArUcoCamera frame image to display detected ArUco markers, scene pose, 2D AOI projection and ArFrame visualisation.
... aruco_camera.image()
```
### Analyse timestamped gaze positions into camera frame
As mentioned above, [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) and so, benefits from all the services described in [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md).
Particularly, timestamped gaze positions can be passed one by one to [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method to execute the whole pipeline dedicated to gaze analysis.
```python
# Assuming that timestamped gaze positions are available
...
# Look ArUcoCamera frame at a timestamped gaze position
aruco_camera.look(timestamp, gaze_position)
```
!!! note ""
At this point, the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method only detects ArUco markers and the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method only process gaze movement identification without any AOI support as no scene description is provided into the JSON configuration file.
Read the next chapters to learn [how to estimate scene pose](pose_estimation.md), [how to describe 3D scene's AOI](aoi_3d_description.md) and [how to project them into camera frame](aoi_3d_projection.md).
|