aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorThéo de la Hogue2024-07-03 18:51:14 +0200
committerThéo de la Hogue2024-07-03 18:51:14 +0200
commit52e64be5bdbbb63ec81d1ba285c6f974845a434c (patch)
treeef318724be00fab373afd338cf78dc25b3fe3097
parentcfb377158da1b16b80e9b42a68519dbef657d8e5 (diff)
downloadargaze-52e64be5bdbbb63ec81d1ba285c6f974845a434c.zip
argaze-52e64be5bdbbb63ec81d1ba285c6f974845a434c.tar.gz
argaze-52e64be5bdbbb63ec81d1ba285c6f974845a434c.tar.bz2
argaze-52e64be5bdbbb63ec81d1ba285c6f974845a434c.tar.xz
Refactoring aruco marker pipeline documentation.
-rw-r--r--docs/img/argaze_load_gui_opencv_frame.pngbin0 -> 339281 bytes
-rw-r--r--docs/img/argaze_load_gui_opencv_pipeline.pngbin0 -> 517856 bytes
-rw-r--r--docs/index.md6
-rw-r--r--docs/user_guide/aruco_marker_pipeline/advanced_topics/scripting.md77
-rw-r--r--docs/user_guide/aruco_marker_pipeline/aoi_3d_frame.md37
-rw-r--r--docs/user_guide/aruco_marker_pipeline/configuration_and_execution.md88
6 files changed, 105 insertions, 103 deletions
diff --git a/docs/img/argaze_load_gui_opencv_frame.png b/docs/img/argaze_load_gui_opencv_frame.png
new file mode 100644
index 0000000..3ab3b5e
--- /dev/null
+++ b/docs/img/argaze_load_gui_opencv_frame.png
Binary files differ
diff --git a/docs/img/argaze_load_gui_opencv_pipeline.png b/docs/img/argaze_load_gui_opencv_pipeline.png
new file mode 100644
index 0000000..227a91d
--- /dev/null
+++ b/docs/img/argaze_load_gui_opencv_pipeline.png
Binary files differ
diff --git a/docs/index.md b/docs/index.md
index 00b8ed7..2e5a99d 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -43,3 +43,9 @@ This ArUco marker pipeline can be combined with any wearable eye tracking device
!!! note
*ArUco marker pipeline is greatly inspired by [Andrew T. Duchowski, Vsevolod Peysakhovich and Krzysztof Krejtz article](https://git.recherche.enac.fr/attachments/download/1990/Using_Pose_Estimation_to_Map_Gaze_to_Detected_Fidu.pdf) about using pose estimation to map gaze to detected fiducial markers.*
+
+## Demonstration
+
+![type:video](http://achil.recherche.enac.fr/videos/argaze_features.mp4)
+
+[Test **ArGaze** by reading the dedicated user guide section](./user_guide/utils/demonstrations_scripts.md). \ No newline at end of file
diff --git a/docs/user_guide/aruco_marker_pipeline/advanced_topics/scripting.md b/docs/user_guide/aruco_marker_pipeline/advanced_topics/scripting.md
index c81d57d..d3a340b 100644
--- a/docs/user_guide/aruco_marker_pipeline/advanced_topics/scripting.md
+++ b/docs/user_guide/aruco_marker_pipeline/advanced_topics/scripting.md
@@ -74,35 +74,80 @@ from argaze import ArFeatures
...
```
-## Pipeline execution outputs
+## Pipeline execution
-The [ArUcoCamera.watch](../../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method returns data about pipeline execution.
+### Detect ArUco markers, estimate scene pose and project 3D AOI
+
+Pass each camera image with timestamp information to the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole pipeline dedicated to ArUco marker detection, scene pose estimation and 3D AOI projection.
+
+!!! warning "Mandatory"
+
+ The [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method must be called from a *try* block to catch pipeline exceptions.
```python
-# Assuming that timestamped images are available
+# Assuming that Full HD (1920x1080) images are available with timestamp values
...:
+ # Edit timestamped image
+ timestamped_image = DataFeatures.TimestampedImage(image, timestamp=timestamp)
+
try:
- # Watch image with ArUco camera
- aruco_camera.watch(image, timestamp=timestamp)
+ # Detect ArUco markers, estimate scene pose then, project 3D AOI into camera frame
+ aruco_camera.watch(timestamped_image)
# Do something with pipeline exception
except Exception as e:
...
- # Do something with detected_markers
- ... aruco_camera.aruco_detector.detected_markers()
+ # Display ArUcoCamera frame image to display detected ArUco markers, scene pose, 2D AOI projection and ArFrame visualization.
+ ... aruco_camera.image()
+```
+
+### Detection outputs
+
+The [ArUcoCamera.watch](../../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method returns data about pipeline execution.
+
+```python
+# Assuming that watch method has been called
+
+# Do something with detected_markers
+... aruco_camera.aruco_detector.detected_markers()
```
Let's understand the meaning of each returned data.
-### *aruco_camera.aruco_detector.detected_markers()*
+#### *aruco_camera.aruco_detector.detected_markers()*
A dictionary containing all detected markers is provided by [ArUcoDetector](../../../argaze.md/#argaze.ArUcoMarker.ArUcoDetector) class.
+### Analyse timestamped gaze positions into the camera frame
+
+As mentioned above, [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) and, so, benefits from all the services described in the [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md).
+
+Particularly, timestamped gaze positions can be passed one by one to the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method to execute the whole pipeline dedicated to gaze analysis.
+
+!!! warning "Mandatory"
+
+ The [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method must be called from a *try* block to catch pipeline exceptions.
+
+```python
+# Assuming that timestamped gaze positions are available
+...
+
+ try:
+
+ # Look ArUcoCamera frame at a timestamped gaze position
+ aruco_camera.look(timestamped_gaze_position)
+
+ # Do something with pipeline exception
+ except Exception as e:
+
+ ...
+```
+
## Setup ArUcoCamera image parameters
Specific [ArUcoCamera.image](../../../argaze.md/#argaze.ArFeatures.ArFrame.image) method parameters can be configured thanks to a Python dictionary.
@@ -133,4 +178,18 @@ aruco_camera_image = aruco_camera.image(**image_parameters)
```
!!! note
- [ArUcoCamera](../../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) inherits from [ArFrame](../../../argaze.md/#argaze.ArFeatures.ArFrame) and, so, benefits from all image parameters described in [gaze analysis pipeline visualization section](../../gaze_analysis_pipeline/visualization.md). \ No newline at end of file
+ [ArUcoCamera](../../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) inherits from [ArFrame](../../../argaze.md/#argaze.ArFeatures.ArFrame) and, so, benefits from all image parameters described in [gaze analysis pipeline visualization section](../../gaze_analysis_pipeline/visualization.md).
+
+
+## Display ArUcoScene frames
+
+All [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames image can be displayed as any [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame).
+
+```python
+ ...
+
+ # Display all ArUcoScene frames
+ for frame in aruco_camera.scene_frames():
+
+ ... frame.image()
+``` \ No newline at end of file
diff --git a/docs/user_guide/aruco_marker_pipeline/aoi_3d_frame.md b/docs/user_guide/aruco_marker_pipeline/aoi_3d_frame.md
index 7323f2e..3a029b0 100644
--- a/docs/user_guide/aruco_marker_pipeline/aoi_3d_frame.md
+++ b/docs/user_guide/aruco_marker_pipeline/aoi_3d_frame.md
@@ -69,7 +69,8 @@ Here is the previous extract where "Left_Screen" and "Right_Screen" AOI are defi
}
}
}
- }
+ },
+ "copy_background_into_scenes_frames": true
...
}
}
@@ -96,40 +97,18 @@ The names of 3D AOI **and** their related [ArFrames](../../argaze.md/#argaze.ArF
[ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frame layers are projected into their dedicated [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) layers when the JSON configuration file is loaded.
-## Pipeline execution
-
-### Map ArUcoCamera image into ArUcoScenes frames
-
-After the timestamped camera image is passed to the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method, it is possible to apply a perspective transformation in order to project the watched image into each [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) [frame's background](../../argaze.md/#argaze.ArFeatures.ArFrame) image.
-
-```python
-# Assuming that Full HD (1920x1080) timestamped images are available
-...:
-
- # Detect ArUco markers, estimate scene pose then, project 3D AOI into camera frame
- aruco_camera.watch(timestamped_image)
+### *copy_background_into_scenes_frames*
- # Map watched image into ArUcoScene frames background
- aruco_camera.map(timestamp=timestamp)
-```
+When the timestamped camera image is passed to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) method, it is possible to apply a perspective transformation in order to project the watched image into each [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) [frame's background](../../argaze.md/#argaze.ArFeatures.ArFrame) image.
-### Analyze timestamped gaze positions into ArUcoScene frames
+## Pipeline execution
[ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames benefits from all the services described in the [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md).
!!! note
- Timestamped [GazePositions](../../argaze.md/#argaze.GazeFeatures.GazePosition) passed to the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method are projected into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames if applicable.
-
-### Display each ArUcoScene frames
-
-All [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames image can be displayed as any [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame).
-
-```python
- ...
+ Timestamped [GazePositions](../../argaze.md/#argaze.GazeFeatures.GazePosition) passed to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) are automatically projected into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames if applicable.
- # Display all ArUcoScene frames
- for frame in aruco_camera.scene_frames():
+Each [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames image is displayed in a separate window.
- ... frame.image()
-``` \ No newline at end of file
+![ArGaze load GUI](../../img/argaze_load_gui_opencv_frame.png) \ No newline at end of file
diff --git a/docs/user_guide/aruco_marker_pipeline/configuration_and_execution.md b/docs/user_guide/aruco_marker_pipeline/configuration_and_execution.md
index f4bd2d4..c2ee1b9 100644
--- a/docs/user_guide/aruco_marker_pipeline/configuration_and_execution.md
+++ b/docs/user_guide/aruco_marker_pipeline/configuration_and_execution.md
@@ -1,17 +1,17 @@
-Load and execute pipeline
+Edit and execute pipeline
=========================
Once [ArUco markers are placed into a scene](aruco_marker_description.md), they can be detected thanks to [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) class.
As [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) class also benefits from all the services described in the [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md).
-![ArUco camera frame](../../img/aruco_camera_frame.png)
+Once defined, an ArUco marker pipeline needs to embedded inside a context that will provides it both gaze positions and camera images to process.
-## Load JSON configuration file
+![ArUco camera frame](../../img/aruco_camera_frame.png)
-An [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) pipeline can be loaded from a JSON configuration file thanks to [argaze.load](../../argaze.md/#argaze.load) package method.
+## Edit JSON configuration
-Here is a simple JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) configuration file example:
+Here is a simple JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) configuration example:
```json
{
@@ -52,19 +52,7 @@ Here is a simple JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCam
}
```
-Then, here is how to load the JSON file:
-
-```python
-import argaze
-
-# Load ArUcoCamera
-with argaze.load('./configuration.json') as aruco_camera:
-
- # Do something with ArUcoCamera
- ...
-```
-
-Now, let's understand the meaning of each JSON entry.
+Let's understand the meaning of each JSON entry.
### argaze.ArUcoMarker.ArUcoCamera.ArUcoCamera
@@ -101,62 +89,32 @@ The usual [ArFrame visualization parameters](../gaze_analysis_pipeline/visualiza
## Pipeline execution
-### Detect ArUco markers, estimate scene pose and project 3D AOI
-
-Pass each camera image with timestamp information to the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole pipeline dedicated to ArUco marker detection, scene pose estimation and 3D AOI projection.
-
-!!! warning "Mandatory"
-
- The [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method must be called from a *try* block to catch pipeline exceptions.
-
-```python
-# Assuming that Full HD (1920x1080) images are available with timestamp values
-...:
-
- # Edit timestamped image
- timestamped_image = DataFeatures.TimestampedImage(image, timestamp=timestamp)
-
- try:
+A pipeline needs to be embedded into a context to be executed.
- # Detect ArUco markers, estimate scene pose then, project 3D AOI into camera frame
- aruco_camera.watch(timestamped_image)
+Copy the gaze analysis pipeline configuration defined above inside the following context configuration.
- # Do something with pipeline exception
- except Exception as e:
-
- ...
-
- # Display ArUcoCamera frame image to display detected ArUco markers, scene pose, 2D AOI projection and ArFrame visualization.
- ... aruco_camera.image()
+```json
+{
+ "argaze.utils.contexts.OpenCV.Movie": {
+ "name": "Movie player",
+ "path": "./src/argaze/utils/demo/tobii_record/segments/1/fullstream.mp4",
+ "pipeline": JSON CONFIGURATION
+ }
+}
```
-### Analyse timestamped gaze positions into the camera frame
-
-As mentioned above, [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) and, so, benefits from all the services described in the [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md).
-
-Particularly, timestamped gaze positions can be passed one by one to the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method to execute the whole pipeline dedicated to gaze analysis.
+Then, use the [*load* command](../utils/main_commands.md) to execute the context.
-!!! warning "Mandatory"
-
- The [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method must be called from a *try* block to catch pipeline exceptions.
-
-```python
-# Assuming that timestamped gaze positions are available
-...
-
- try:
+```shell
+python -m argaze load CONFIGURATION
+```
- # Look ArUcoCamera frame at a timestamped gaze position
- aruco_camera.look(timestamped_gaze_position)
+This command should open a GUI window with the detected markers and identified cursor fixations circles when the mouse moves over the window.
- # Do something with pipeline exception
- except Exception as e:
-
- ...
-```
+![ArGaze load GUI](../../img/argaze_load_gui_opencv_pipeline.png)
!!! note ""
- At this point, the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method only detects ArUco marker and the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method only processes gaze movement identification without any AOI support as no scene description is provided into the JSON configuration file.
+ At this point, the pipeline only processes gaze movement identification without any AOI support as no scene description is provided into the JSON configuration file.
Read the next chapters to learn [how to estimate scene pose](pose_estimation.md), [how to describe a 3D scene's AOI](aoi_3d_description.md) and [how to project them into the camera frame](aoi_3d_projection.md). \ No newline at end of file