1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
|
Configure and execute ArUcoCamera
=================================
Once [ArUco markers are placed into a scene](aruco_markers_description.md) and [areas of interest are described](aoi_description.md), everything is ready to setup an ArUco marker pipeline thanks to [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class.
As [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md).
## Load JSON configuration file
The [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) internal pipeline loads from a JSON configuration file thanks to [ArUcoCamera.from_json](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera.from_json) class method.
Here is a simple JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration file example:
```json
{
"name": "My FullHD camera",
"size": [1920, 1080],
"aruco_detector": {
"dictionary": "DICT_APRILTAG_16h5",
"marker_size": 5
},
"image_parameters": {
"background_weight": 1,
...
"draw_detected_markers": {
"color": [0, 255, 0],
"draw_axes": {
"thickness": 3
}
}
}
}
```
Then, here is how to load the JSON file:
```python
from argaze.ArUcoMarkers import ArUcoCamera
# Load ArUcoCamera
aruco_camera = ArUcoCamera.ArUcoCamera.from_json('./configuration.json')
```
Now, let's understand the meaning of each JSON entry.
### Name - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)*
The name of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. Basically useful for visualisation purpose.
### Size - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)*
The size of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame in pixels. Be aware that gaze positions have to be in the same range of value to be projected in.
### ArUco Detector
The first [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) pipeline step is to detect ArUco markers inside input image and estimate their poses.
The [ArUcoDetector](../../argaze.md/#argaze.ArUcoMarkers.ArUcoDetector) is in charge to detect all markers from a specific dictionary with a given size in centimeters.
!!! warning
JSON *aruco_detector* entry is mandatory.
### Image parameters - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)*
The usual [ArFrame visualisation parameters](./user_guide/gaze_analysis_pipeline/visualisation.md) plus one additional *draw_detected_markers* field.
## Pipeline execution
### Detect ArUco markers, estimate scene pose and project scene
Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole pipeline dedicated to ArUco markers detection, scene pose estimation and projection.
```python
# Assuming that Full HD (1920x1080) video stream or file is opened
...
# Assuming that the video reading is handled in a looping code block
...:
# Capture image from video stream of file
image = video_capture.read()
# Detect ArUco markers, estimate scene pose then, project scene into camera frame
aruco_camera.watch(image)
# Display ArUcoCamera frame image to check that ArUco markers are well detected and scene is well projected
... aruco_camera.image()
```
!!! warning
ArUco markers pose estimation algorithm can lead to errors due to geometric ambiguities as explain in [this article](https://ieeexplore.ieee.org/document/1717461). To discard such ambiguous cases, markers should **not be parallel to camera plan**.
### Analyse timestamped gaze positions into camera frame
As mentioned above, [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) and so, benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md).
Particularly, timestamped gaze positions can be passed one by one to [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method to execute the whole pipeline dedicated to gaze analysis.
```python
# Assuming that timestamped gaze positions are available
...
# Look ArUcoCamera frame at a timestamped gaze position
aruco_camera.look(timestamp, gaze_position)
```
!!! warning ""
At this point, the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method only detects ArUco markers and the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method is not able to analyze gaze positions as no scene description is provided into the JSON configuration file.
Read the next chapters to learn [how to enable scene pose estimation and its projection](aruco_scene.md).
|
