1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
|
Define a 3D AOI as a frame
==========================
When an 3D AOI of the scene contains others coplanar 3D AOI, like a screen with GUI elements displayed on, it is better to described them as 2D AOI inside 2D coordinates system related to the containing 3D AOI.
## Add ArFrame to ArUcoScene
The [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) class defines a rectangular area where timestamped gaze positions are projected in and inside which they need to be analyzed.
Here is the previous extract where "Left_Screen" and "Right_Screen" AOI are defined as a frame into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration:
```json
{
"name": "My FullHD camera",
"size": [1920, 1080],
...
"scenes": {
"MyScene" : {
"aruco_markers_group": {
...
},
"layers": {
"MyLayer": {
"aoi_scene": {
"Left_Screen": [[0, 0, 0], [15, 0, 0], [0, 18.963333, -6.355470], [15, 18.963333, -6.355470]],
"Right_Screen": [[20, 0, 0], [35, 0, 0], [20, 18.963337 ,-6.355472], [35, 18.963337, -6.355472]],
"Control_Panel": [[49.5, 30, 18.333333], [55.5, 30, 18.333333], [49.5, 38, 18.333333], [55.5, 38, 18.333333]],
"Window": [[-57.8, 5.5, -33.5], [46, 15.5, -35], [1.5, 53, -1], [50.2, 61, 6], [-35.85, 35, -15]]
}
}
},
"frames": {
"Left_Screen": {
"size": [768, 1024],
"layers": {
"MyLayer": {
"aoi_scene": {
"LeftPanel": {
"Rectangle": {
"x": 0,
"y": 0,
"width": 768,
"height": 180
}
},
"CircularWidget": {
"Circle": {
"cx": 384,
"cy": 600,
"radius": 180
}
}
}
}
}
},
"Right_Screen": {
"size": [768, 1024],
"layers": {
"MyLayer": {
"aoi_scene": {
"GeoSector": [[724, 421], [537, 658], [577, 812], [230, 784], [70, 700], [44, 533], [190, 254], [537, 212]]
}
}
}
}
}
}
}
...
}
```
Now, let's understand the meaning of each JSON entry.
### *frames*
An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArFrames](../../argaze.md/#argaze.ArFeatures.ArFrame) stored by name.
### Left_Screen & Right_Screen
The names of 3D AOI **and** their related [ArFrames](../../argaze.md/#argaze.ArFeatures.ArFrame). Basically useful for visualisation purpose.
!!! warning "AOI / Frame names policy"
An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer 3D AOI is defined as an [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame, **provided they have the same name**.
!!! warning "Layer name policy"
An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame layer is projected into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer, **provided they have the same name**.
!!! note
[ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame layers are projected into their dedicated [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers when the JSON configuration file is loaded.
## Pipeline execution
### Map ArUcoCamera image into ArUcoScenes frames
After camera image is passed to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method, it is possible to apply a perpective transformation in order to project watched image into each [ArUcoScenes](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) [frames background](../../argaze.md/#argaze.ArFeatures.ArFrame) image.
```python
# Assuming that Full HD (1920x1080) timestamped images are available
...:
# Detect ArUco markers, estimate scene pose then, project 3D AOI into camera frame
aruco_camera.watch(image, timestamp=timestamp)
# Map watched image into ArUcoScenes frames background
aruco_camera.map(timestamp=timestamp)
```
### Analyse timestamped gaze positions into ArUcoScenes frames
[ArUcoScenes](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frames benefits from all the services described in [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md).
!!! note
Timestamped gaze positions passed to [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method are projected into [ArUcoScenes](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frames if applicable.
### Display each ArUcoScenes frames
All [ArUcoScenes](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frames image can be displayed as any [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame).
```python
...
# Display all ArUcoScenes frames
for frame in aruco_camera.scene_frames:
... frame.image()
```
|
