aboutsummaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorThéo de la Hogue2023-09-20 09:54:31 +0200
committerThéo de la Hogue2023-09-20 09:54:31 +0200
commitb2343f1cd1848e7e96f50088050a649aea38e6cf (patch)
tree87d340b0a8b75edfabdda5470469b5bb5c938bd9 /docs
parent83907d74866076c31201367d1f5194dcbcbea004 (diff)
downloadargaze-b2343f1cd1848e7e96f50088050a649aea38e6cf.zip
argaze-b2343f1cd1848e7e96f50088050a649aea38e6cf.tar.gz
argaze-b2343f1cd1848e7e96f50088050a649aea38e6cf.tar.bz2
argaze-b2343f1cd1848e7e96f50088050a649aea38e6cf.tar.xz
more work on aruco markers pipeline.
Diffstat (limited to 'docs')
-rw-r--r--docs/user_guide/aruco_markers_pipeline/aoi_description.md4
-rw-r--r--docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md36
-rw-r--r--docs/user_guide/aruco_markers_pipeline/aruco_scene.md113
3 files changed, 138 insertions, 15 deletions
diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_description.md b/docs/user_guide/aruco_markers_pipeline/aoi_description.md
index 8c57cd1..80ad858 100644
--- a/docs/user_guide/aruco_markers_pipeline/aoi_description.md
+++ b/docs/user_guide/aruco_markers_pipeline/aoi_description.md
@@ -55,8 +55,8 @@ JSON file format allows to describe AOIs vertices.
``` json
{
- "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [6.2, 1.275252, 1.753843], [31.2, 1.275252, 1.753843]],
- "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [2.5, 27.5, -0.5], [37.5, 27.5, -0.5]],
+ "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]],
+ "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]],
"BlueTriangle": [[12.5, 7.5, -0.5], [27.5, 7.5, -0.5], [20, 22.5, -0.5]]
}
```
diff --git a/docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md b/docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md
index 824e466..ba19e45 100644
--- a/docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md
+++ b/docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md
@@ -3,9 +3,9 @@ Configure and execute ArUcoCamera
Once [ArUco markers are placed into a scene](aruco_markers_description.md) and [areas of interest are described](aoi_description.md), everything is ready to setup an ArUco marker pipeline thanks to [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class.
-As it inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md).
+As [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md).
-![ArUco camera markers detection](../../img/aruco_camera_markers_detection.png)
+![ArUco camera frame](../../img/aruco_camera_frame.png)
## Load JSON configuration file
@@ -68,19 +68,21 @@ The usual [ArFrame visualisation parameters](./user_guide/gaze_analysis_pipeline
## Pipeline execution
-Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole intanciated pipeline.
+### Detect ArUco markers, estimate scene pose and project scene
+
+Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole pipeline dedicated to ArUco markers detection, scene pose estimation and projection.
```python
# Assuming that Full HD (1920x1080) video stream or file is opened
...
-# Assuming there is a way to escape the while loop
-while ...:
+# Assuming that the video reading is handled in a looping code block
+...:
# Capture image from video stream of file
image = video_capture.read()
- # Detect ArUco markers and more...
+ # Detect ArUco markers, estimate scene pose then, project scene into camera frame
aruco_camera.watch(image)
# Display ArUcoCamera frame image to check that ArUco markers are well detected and scene is well projected
@@ -90,4 +92,24 @@ while ...:
!!! warning
ArUco markers pose estimation algorithm can lead to errors due to geometric ambiguities as explain in [this article](https://ieeexplore.ieee.org/document/1717461). To discard such ambiguous cases, markers should **not be parallel to camera plan**.
-At this point, the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) only detects ArUco markers as no scene description is provided. \ No newline at end of file
+
+
+### Analyse timestamped gaze positions into camera frame
+
+As mentioned above, [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) and so, benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md).
+
+Particularly, timestamped gaze positions can be passed one by one to [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method to execute the whole pipeline dedicated to gaze analysis.
+
+```python
+# Assuming that timestamped gaze positions are available
+...
+
+ # Look ArUcoCamera frame at a timestamped gaze position
+ aruco_camera.look(timestamp, gaze_position)
+```
+
+!!! warning ""
+
+ At this point, the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method only detects ArUco markers and the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method is not able to analyze gaze positions as no scene description is provided into the JSON configuration file.
+
+ Read the next chapters to learn [how to enable scene pose estimation and its projection](aruco_scene.md). \ No newline at end of file
diff --git a/docs/user_guide/aruco_markers_pipeline/aruco_scene.md b/docs/user_guide/aruco_markers_pipeline/aruco_scene.md
index b47fefb..91d2702 100644
--- a/docs/user_guide/aruco_markers_pipeline/aruco_scene.md
+++ b/docs/user_guide/aruco_markers_pipeline/aruco_scene.md
@@ -58,9 +58,7 @@ The 3D places of ArUco markers into the scene as defined at [ArUco markers descr
## Add ArLayer to ArUcoScene to load AOI
-The [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer) class allows to load areas of interest description.
-
-An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArLayers](../../argaze.md/#argaze.ArFeatures.ArLayer).
+The [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer) class allows to load areas of interest description. An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArLayers](../../argaze.md/#argaze.ArFeatures.ArLayer).
Here is the previous extract where one layer is added to the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene):
@@ -77,8 +75,8 @@ Here is the previous extract where one layer is added to the [ArUcoScene](../../
"layers": {
"MyLayer": {
"aoi_scene": {
- "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [6.2, 1.275252, 1.753843], [31.2, 1.275252, 1.753843]],
- "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [2.5, 27.5, -0.5], [37.5, 27.5, -0.5]],
+ "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]],
+ "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]],
"BlueTriangle": [[12.5, 7.5, -0.5], [27.5, 7.5, -0.5], [20, 22.5, -0.5]]
}
}
@@ -123,7 +121,17 @@ Here is the previous extract where one layer is added to the [ArUcoCamera](../..
}
},
"layers": {
- "MyLayer": {}
+ "MyLayer": {
+ "aoi_matcher": {
+ ...
+ },
+ "aoi_scan_path": {
+ "duration_max": 30000
+ },
+ "aoi_scan_path_analyzers": {
+ ...
+ }
+ }
}
...
}
@@ -142,3 +150,96 @@ The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically
!!! note
[ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers are projected into their dedicated [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layers when calling the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method.
+
+## Add GazeMovementIdentifier to ArUcocamera to enable gaze movement identification
+
+Here is the previous extract where gaze movement identifier is added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera):
+
+```json
+{
+ "name": "My FullHD camera",
+ "size": [1920, 1080],
+ ...
+ "scenes": {
+ "MyScene" : {
+ "aruco_markers_group": {
+ ...
+ },
+ "layers": {
+ "MyLayer": {
+ "aoi_scene": {
+ ...
+ }
+ }
+ }
+ }
+ },
+ "layers": {
+ "MyLayer": {}
+ },
+ "gaze_movement_identifier": {
+ "DispersionThresholdIdentification": {
+ "deviation_max_threshold": 50,
+ "duration_min_threshold": 200
+ }
+ }
+ ...
+}
+```
+!!! note
+
+ Timestamped gaze positions are [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layers execution is done when calling the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method.
+
+## Add AOIMatcher, AOIScanPath and AOIScanPathAnalyzers to ArUcoCamera layer to enable gaze analysis
+
+Here is the previous extract where AOI matcher, AOI scan path and AOI scan path analyzers are added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer:
+
+```json
+{
+ "name": "My FullHD camera",
+ "size": [1920, 1080],
+ ...
+ "scenes": {
+ "MyScene" : {
+ "aruco_markers_group": {
+ ...
+ },
+ "layers": {
+ "MyLayer": {
+ "aoi_scene": {
+ ...
+ }
+ }
+ }
+ }
+ },
+ "layers": {
+ "MyLayer": {
+ "aoi_matcher": {
+ "DeviationCircleCoverage": {
+ "coverage_threshold": 0.5
+ }
+ },
+ "aoi_scan_path": {
+ "duration_max": 30000
+ },
+ "aoi_scan_path_analyzers": {
+ "Basic": {},
+ "TransitionMatrix": {},
+ "NGram": {
+ "n_min": 3,
+ "n_max": 5
+ }
+ }
+ }
+ },
+ "gaze_movement_identifier": {
+ ...
+ }
+ ...
+}
+```
+
+!!! warning
+
+ Adding scan path and scan path analyzers to [an ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer doesn't make sense if the camera is moving. \ No newline at end of file