1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
|
Load and execute pipeline
=========================
Once [ArUco markers are placed into a scene](aruco_markers_description.md), they can be detected thanks to [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class.
As [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class also benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md).
## Load JSON configuration file
The [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) internal pipeline loads from a JSON configuration file thanks to [ArUcoCamera.from_json](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera.from_json) class method.
Here is a simple JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration file example:
```json
{
"name": "My FullHD camera",
"size": [1920, 1080],
"aruco_detector": {
"dictionary": "DICT_APRILTAG_16h5",
"marker_size": 5
},
"gaze_movement_identifier": {
"DispersionThresholdIdentification": {
"deviation_max_threshold": 25,
"duration_min_threshold": 150
}
},
"image_parameters": {
"background_weight": 1,
"draw_gaze_positions": {
"color": [0, 255, 255],
"size": 2
},
"draw_fixations": {
"deviation_circle_color": [255, 0, 255],
"duration_border_color": [127, 0, 127],
"duration_factor": 1e-2
},
"draw_saccades": {
"line_color": [255, 0, 255]
},
"draw_detected_markers": {
"color": [0, 255, 0],
"draw_axes": {
"thickness": 3
}
}
}
}
```
Then, here is how to load the JSON file:
```python
from argaze.ArUcoMarkers import ArUcoCamera
# Load ArUcoCamera
aruco_camera = ArUcoCamera.ArUcoCamera.from_json('./configuration.json')
```
Now, let's understand the meaning of each JSON entry.
### Name - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)*
The name of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. Basically useful for visualisation purpose.
### Size - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)*
The size of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame in pixels. Be aware that gaze positions have to be in the same range of value to be projected in.
### ArUco Detector
The first [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) pipeline step is to detect ArUco markers inside input image and estimate their poses.
The [ArUcoDetector](../../argaze.md/#argaze.ArUcoMarkers.ArUcoDetector) is in charge to detect all markers from a specific dictionary with a given size in centimeters.
!!! warning "Mandatory"
JSON *aruco_detector* entry is mandatory.
### Gaze Movement Identifier - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)*
The first [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) pipeline step dedicated to identify fixations or saccades from consecutive timestamped gaze positions.
### Image parameters - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)*
The usual [ArFrame visualisation parameters](./user_guide/gaze_analysis_pipeline/visualisation.md) plus one additional *draw_detected_markers* field.
## Pipeline execution
### Detect ArUco markers, estimate scene pose and project AOI
Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole pipeline dedicated to ArUco markers detection, scene pose estimation and AOI projection.
```python
# Assuming that Full HD (1920x1080) video stream or file is opened
...
# Assuming that the video reading is handled in a looping code block
...:
# Capture image from video stream of file
image = video_capture.read()
# Detect ArUco markers, estimate scene pose then, project AOI into camera frame
aruco_camera.watch(image)
# Display ArUcoCamera frame image to display detected ArUco markers, scene pose, AOI projection and ArFrame visualisation.
... aruco_camera.image()
```
!!! warning "Pose estimation error"
ArUco markers pose estimation algorithm can lead to errors due to geometric ambiguities as explain in [this article](https://ieeexplore.ieee.org/document/1717461). To discard such ambiguous cases, markers should **as less as possible be parallel to camera plan**.
### Analyse timestamped gaze positions into camera frame
As mentioned above, [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) and so, benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md).
Particularly, timestamped gaze positions can be passed one by one to [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method to execute the whole pipeline dedicated to gaze analysis.
```python
# Assuming that timestamped gaze positions are available
...
# Look ArUcoCamera frame at a timestamped gaze position
aruco_camera.look(timestamp, gaze_position)
```
!!! note ""
At this point, the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method only detects ArUco markers and the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method only process gaze movement identification without any AOI support as no scene description is provided into the JSON configuration file.
Read the next chapters to learn [how to estimate scene pose](pose_estimation.md) and [how to project AOI](aoi_projection.md).
|
