1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
|
Define a 3D AOI as a frame
==========================
When an 3D AOI of the scene contains other coplanar 3D AOI, like a screen with GUI elements displayed on it, it is better to describe them as 2D AOI inside a 2D coordinates system related to the containing 3D AOI.
## Add ArFrame to ArUcoScene
The [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) class defines a rectangular area where timestamped gaze positions are projected and inside which they need to be analyzed.
Here is the previous extract where "Left_Screen" and "Right_Screen" AOI are defined as a frame into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) configuration:
```json
{
"argaze.ArUcoMarker.ArUcoCamera.ArUcoCamera": {
"name": "My FullHD camera",
"size": [1920, 1080],
...
"scenes": {
"MyScene" : {
"aruco_markers_group": {
...
},
"layers": {
"MyLayer": {
"aoi_scene": {
"Left_Screen": [[0, 0, 0], [15, 0, 0], [0, 18.963333, -6.355470], [15, 18.963333, -6.355470]],
"Right_Screen": [[20, 0, 0], [35, 0, 0], [20, 18.963337 ,-6.355472], [35, 18.963337, -6.355472]],
"Control_Panel": [[49.5, 30, 18.333333], [55.5, 30, 18.333333], [49.5, 38, 18.333333], [55.5, 38, 18.333333]],
"Window": [[-57.8, 5.5, -33.5], [46, 15.5, -35], [1.5, 53, -1], [50.2, 61, 6], [-35.85, 35, -15]]
}
}
},
"frames": {
"Left_Screen": {
"size": [768, 1024],
"layers": {
"MyLayer": {
"aoi_scene": {
"LeftPanel": {
"Rectangle": {
"x": 0,
"y": 0,
"width": 768,
"height": 180
}
},
"CircularWidget": {
"Circle": {
"cx": 384,
"cy": 600,
"radius": 180
}
}
}
}
}
},
"Right_Screen": {
"size": [768, 1024],
"layers": {
"MyLayer": {
"aoi_scene": {
"GeoSector": [[724, 421], [537, 658], [577, 812], [230, 784], [70, 700], [44, 533], [190, 254], [537, 212]]
}
}
}
}
}
}
}
...
}
}
```
Now, let's understand the meaning of each JSON entry.
### *frames*
An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) instance can contain multiples [ArFrames](../../argaze.md/#argaze.ArFeatures.ArFrame) stored by name.
### Left_Screen & Right_Screen
The names of 3D AOI **and** their related [ArFrames](../../argaze.md/#argaze.ArFeatures.ArFrame). Basically, it is useful for visualization purposes.
!!! warning "AOI / Frame names policy"
An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) layer 3D AOI is defined as an [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frame, **provided they have the same name**.
!!! warning "Layer name policy"
An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frame layer is projected into an [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) layer, **provided they have the same name**.
!!! note
[ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frame layers are projected into their dedicated [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) layers when the JSON configuration file is loaded.
## Pipeline execution
### Map ArUcoCamera image into ArUcoScenes frames
After the timestamped camera image is passed to the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method, it is possible to apply a perspective transformation in order to project the watched image into each [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) [frame's background](../../argaze.md/#argaze.ArFeatures.ArFrame) image.
```python
# Assuming that Full HD (1920x1080) timestamped images are available
...:
# Detect ArUco markers, estimate scene pose then, project 3D AOI into camera frame
aruco_camera.watch(timestamped_image)
# Map watched image into ArUcoScene frames background
aruco_camera.map(timestamp=timestamp)
```
### Analyze timestamped gaze positions into ArUcoScene frames
[ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames benefits from all the services described in the [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md).
!!! note
Timestamped [GazePositions](../../argaze.md/#argaze.GazeFeatures.GazePosition) passed to the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method are projected into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames if applicable.
### Display each ArUcoScene frames
All [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames image can be displayed as any [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame).
```python
...
# Display all ArUcoScene frames
for frame in aruco_camera.scene_frames:
... frame.image()
```
|
