diff options
Diffstat (limited to 'docs/user_guide/aruco_marker_pipeline')
7 files changed, 108 insertions, 109 deletions
diff --git a/docs/user_guide/aruco_marker_pipeline/advanced_topics/aruco_detector_configuration.md b/docs/user_guide/aruco_marker_pipeline/advanced_topics/aruco_detector_configuration.md index 975f278..311916b 100644 --- a/docs/user_guide/aruco_marker_pipeline/advanced_topics/aruco_detector_configuration.md +++ b/docs/user_guide/aruco_marker_pipeline/advanced_topics/aruco_detector_configuration.md @@ -5,7 +5,7 @@ As explain in the [OpenCV ArUco documentation](https://docs.opencv.org/4.x/d1/dc ## Load ArUcoDetector parameters -[ArUcoCamera.detector.parameters](../../../argaze.md/#argaze.ArUcoMarker.ArUcoDetector.Parameters) can be loaded thanks to a dedicated JSON entry. +[ArUcoCamera.detector.parameters](../../../argaze.md/#argaze.ArUcoMarker.ArUcoDetector.Parameters) can be loaded with a dedicated JSON entry. Here is an extract from the JSON [ArUcoCamera](../../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) configuration file with ArUco detector parameters: @@ -18,7 +18,7 @@ Here is an extract from the JSON [ArUcoCamera](../../../argaze.md/#argaze.ArUcoM "dictionary": "DICT_APRILTAG_16h5", "parameters": { "adaptiveThreshConstant": 10, - "useAruco3Detection": 1 + "useAruco3Detection": true } }, ... diff --git a/docs/user_guide/aruco_marker_pipeline/advanced_topics/optic_parameters_calibration.md b/docs/user_guide/aruco_marker_pipeline/advanced_topics/optic_parameters_calibration.md index 625f257..e9ce740 100644 --- a/docs/user_guide/aruco_marker_pipeline/advanced_topics/optic_parameters_calibration.md +++ b/docs/user_guide/aruco_marker_pipeline/advanced_topics/optic_parameters_calibration.md @@ -134,7 +134,7 @@ Below, an optic_parameters JSON file example: ## Load and display optic parameters -[ArUcoCamera.detector.optic_parameters](../../../argaze.md/#argaze.ArUcoMarker.ArUcoOpticCalibrator.OpticParameters) can be enabled thanks to a dedicated JSON entry. +[ArUcoCamera.detector.optic_parameters](../../../argaze.md/#argaze.ArUcoMarker.ArUcoOpticCalibrator.OpticParameters) can be enabled with a dedicated JSON entry. Here is an extract from the JSON [ArUcoCamera](../../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) configuration file where optic parameters are loaded and displayed: diff --git a/docs/user_guide/aruco_marker_pipeline/advanced_topics/scripting.md b/docs/user_guide/aruco_marker_pipeline/advanced_topics/scripting.md index c81d57d..f258e04 100644 --- a/docs/user_guide/aruco_marker_pipeline/advanced_topics/scripting.md +++ b/docs/user_guide/aruco_marker_pipeline/advanced_topics/scripting.md @@ -74,38 +74,83 @@ from argaze import ArFeatures ... ``` -## Pipeline execution outputs +## Pipeline execution -The [ArUcoCamera.watch](../../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method returns data about pipeline execution. +### Detect ArUco markers, estimate scene pose and project 3D AOI + +Pass each camera image with timestamp information to the [ArUcoCamera.watch](../../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole pipeline dedicated to ArUco marker detection, scene pose estimation and 3D AOI projection. + +!!! warning "Mandatory" + + The [ArUcoCamera.watch](../../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method must be called from a *try* block to catch pipeline exceptions. ```python -# Assuming that timestamped images are available +# Assuming that Full HD (1920x1080) images are available with timestamp values ...: + # Edit timestamped image + timestamped_image = DataFeatures.TimestampedImage(image, timestamp=timestamp) + try: - # Watch image with ArUco camera - aruco_camera.watch(image, timestamp=timestamp) + # Detect ArUco markers, estimate scene pose then, project 3D AOI into camera frame + aruco_camera.watch(timestamped_image) # Do something with pipeline exception except Exception as e: ... - # Do something with detected_markers - ... aruco_camera.aruco_detector.detected_markers() + # Display ArUcoCamera frame image to display detected ArUco markers, scene pose, 2D AOI projection and ArFrame visualization. + ... aruco_camera.image() +``` + +### Detection outputs + +The [ArUcoCamera.watch](../../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method returns data about pipeline execution. + +```python +# Assuming that watch method has been called + +# Do something with detected_markers +... aruco_camera.aruco_detector.detected_markers() ``` Let's understand the meaning of each returned data. -### *aruco_camera.aruco_detector.detected_markers()* +#### *aruco_camera.aruco_detector.detected_markers()* A dictionary containing all detected markers is provided by [ArUcoDetector](../../../argaze.md/#argaze.ArUcoMarker.ArUcoDetector) class. +### Analyse timestamped gaze positions into the camera frame + +As mentioned above, [ArUcoCamera](../../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) inherits from [ArFrame](../../../argaze.md/#argaze.ArFeatures.ArFrame) and, so, benefits from all the services described in the [gaze analysis pipeline section](../../gaze_analysis_pipeline/introduction.md). + +Particularly, timestamped gaze positions can be passed one by one to the [ArUcoCamera.look](../../../argaze.md/#argaze.ArFeatures.ArFrame.look) method to execute the whole pipeline dedicated to gaze analysis. + +!!! warning "Mandatory" + + The [ArUcoCamera.look](../../../argaze.md/#argaze.ArFeatures.ArFrame.look) method must be called from a *try* block to catch pipeline exceptions. + +```python +# Assuming that timestamped gaze positions are available +... + + try: + + # Look ArUcoCamera frame at a timestamped gaze position + aruco_camera.look(timestamped_gaze_position) + + # Do something with pipeline exception + except Exception as e: + + ... +``` + ## Setup ArUcoCamera image parameters -Specific [ArUcoCamera.image](../../../argaze.md/#argaze.ArFeatures.ArFrame.image) method parameters can be configured thanks to a Python dictionary. +Specific [ArUcoCamera.image](../../../argaze.md/#argaze.ArFeatures.ArFrame.image) method parameters can be configured with a Python dictionary. ```python # Assuming ArUcoCamera is loaded @@ -133,4 +178,18 @@ aruco_camera_image = aruco_camera.image(**image_parameters) ``` !!! note - [ArUcoCamera](../../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) inherits from [ArFrame](../../../argaze.md/#argaze.ArFeatures.ArFrame) and, so, benefits from all image parameters described in [gaze analysis pipeline visualization section](../../gaze_analysis_pipeline/visualization.md).
\ No newline at end of file + [ArUcoCamera](../../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) inherits from [ArFrame](../../../argaze.md/#argaze.ArFeatures.ArFrame) and, so, benefits from all image parameters described in [gaze analysis pipeline visualization section](../../gaze_analysis_pipeline/visualization.md). + + +## Display ArUcoScene frames + +All [ArUcoScene](../../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames image can be displayed as any [ArFrame](../../../argaze.md/#argaze.ArFeatures.ArFrame). + +```python + ... + + # Display all ArUcoScene frames + for frame in aruco_camera.scene_frames(): + + ... frame.image() +```
\ No newline at end of file diff --git a/docs/user_guide/aruco_marker_pipeline/aoi_3d_description.md b/docs/user_guide/aruco_marker_pipeline/aoi_3d_description.md index 46422b8..78a513a 100644 --- a/docs/user_guide/aruco_marker_pipeline/aoi_3d_description.md +++ b/docs/user_guide/aruco_marker_pipeline/aoi_3d_description.md @@ -1,7 +1,7 @@ Describe 3D AOI =============== -Now that the [scene pose is estimated](aruco_marker_description.md) thanks to ArUco markers description, [areas of interest (AOI)](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AreaOfInterest) need to be described into the same 3D referential. +Now that the [scene pose is estimated](aruco_marker_description.md) considering the ArUco markers description, [areas of interest (AOI)](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AreaOfInterest) need to be described into the same 3D referential. In the example scene, the two screens—the control panel and the window—are considered to be areas of interest. diff --git a/docs/user_guide/aruco_marker_pipeline/aoi_3d_frame.md b/docs/user_guide/aruco_marker_pipeline/aoi_3d_frame.md index 7323f2e..3a029b0 100644 --- a/docs/user_guide/aruco_marker_pipeline/aoi_3d_frame.md +++ b/docs/user_guide/aruco_marker_pipeline/aoi_3d_frame.md @@ -69,7 +69,8 @@ Here is the previous extract where "Left_Screen" and "Right_Screen" AOI are defi } } } - } + }, + "copy_background_into_scenes_frames": true ... } } @@ -96,40 +97,18 @@ The names of 3D AOI **and** their related [ArFrames](../../argaze.md/#argaze.ArF [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frame layers are projected into their dedicated [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) layers when the JSON configuration file is loaded. -## Pipeline execution - -### Map ArUcoCamera image into ArUcoScenes frames - -After the timestamped camera image is passed to the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method, it is possible to apply a perspective transformation in order to project the watched image into each [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) [frame's background](../../argaze.md/#argaze.ArFeatures.ArFrame) image. - -```python -# Assuming that Full HD (1920x1080) timestamped images are available -...: - - # Detect ArUco markers, estimate scene pose then, project 3D AOI into camera frame - aruco_camera.watch(timestamped_image) +### *copy_background_into_scenes_frames* - # Map watched image into ArUcoScene frames background - aruco_camera.map(timestamp=timestamp) -``` +When the timestamped camera image is passed to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) method, it is possible to apply a perspective transformation in order to project the watched image into each [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) [frame's background](../../argaze.md/#argaze.ArFeatures.ArFrame) image. -### Analyze timestamped gaze positions into ArUcoScene frames +## Pipeline execution [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames benefits from all the services described in the [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md). !!! note - Timestamped [GazePositions](../../argaze.md/#argaze.GazeFeatures.GazePosition) passed to the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method are projected into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames if applicable. - -### Display each ArUcoScene frames - -All [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames image can be displayed as any [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame). - -```python - ... + Timestamped [GazePositions](../../argaze.md/#argaze.GazeFeatures.GazePosition) passed to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) are automatically projected into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames if applicable. - # Display all ArUcoScene frames - for frame in aruco_camera.scene_frames(): +Each [ArUcoScene](../../argaze.md/#argaze.ArUcoMarker.ArUcoScene) frames image is displayed in a separate window. - ... frame.image() -```
\ No newline at end of file +
\ No newline at end of file diff --git a/docs/user_guide/aruco_marker_pipeline/configuration_and_execution.md b/docs/user_guide/aruco_marker_pipeline/configuration_and_execution.md index f4bd2d4..56846e2 100644 --- a/docs/user_guide/aruco_marker_pipeline/configuration_and_execution.md +++ b/docs/user_guide/aruco_marker_pipeline/configuration_and_execution.md @@ -1,17 +1,17 @@ -Load and execute pipeline +Edit and execute pipeline ========================= -Once [ArUco markers are placed into a scene](aruco_marker_description.md), they can be detected thanks to [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) class. +Once [ArUco markers are placed into a scene](aruco_marker_description.md), they can be detected by the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) class. As [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) class also benefits from all the services described in the [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md). - +Once defined, an ArUco marker pipeline needs to embedded inside a context that will provides it both gaze positions and camera images to process. -## Load JSON configuration file + -An [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) pipeline can be loaded from a JSON configuration file thanks to [argaze.load](../../argaze.md/#argaze.load) package method. +## Edit JSON configuration -Here is a simple JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) configuration file example: +Here is a simple JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) configuration example: ```json { @@ -52,19 +52,7 @@ Here is a simple JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCam } ``` -Then, here is how to load the JSON file: - -```python -import argaze - -# Load ArUcoCamera -with argaze.load('./configuration.json') as aruco_camera: - - # Do something with ArUcoCamera - ... -``` - -Now, let's understand the meaning of each JSON entry. +Let's understand the meaning of each JSON entry. ### argaze.ArUcoMarker.ArUcoCamera.ArUcoCamera @@ -101,62 +89,32 @@ The usual [ArFrame visualization parameters](../gaze_analysis_pipeline/visualiza ## Pipeline execution -### Detect ArUco markers, estimate scene pose and project 3D AOI - -Pass each camera image with timestamp information to the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole pipeline dedicated to ArUco marker detection, scene pose estimation and 3D AOI projection. - -!!! warning "Mandatory" - - The [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method must be called from a *try* block to catch pipeline exceptions. - -```python -# Assuming that Full HD (1920x1080) images are available with timestamp values -...: - - # Edit timestamped image - timestamped_image = DataFeatures.TimestampedImage(image, timestamp=timestamp) - - try: +A pipeline needs to be embedded into a context to be executed. - # Detect ArUco markers, estimate scene pose then, project 3D AOI into camera frame - aruco_camera.watch(timestamped_image) +Copy the gaze analysis pipeline configuration defined above inside the following context configuration. - # Do something with pipeline exception - except Exception as e: - - ... - - # Display ArUcoCamera frame image to display detected ArUco markers, scene pose, 2D AOI projection and ArFrame visualization. - ... aruco_camera.image() +```json +{ + "argaze.utils.contexts.OpenCV.Movie": { + "name": "Movie player", + "path": "./src/argaze/utils/demo/tobii_record/segments/1/fullstream.mp4", + "pipeline": JSON CONFIGURATION + } +} ``` -### Analyse timestamped gaze positions into the camera frame - -As mentioned above, [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarker.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) and, so, benefits from all the services described in the [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md). - -Particularly, timestamped gaze positions can be passed one by one to the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method to execute the whole pipeline dedicated to gaze analysis. +Then, use the [*load* command](../utils/main_commands.md) to execute the context. -!!! warning "Mandatory" - - The [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method must be called from a *try* block to catch pipeline exceptions. - -```python -# Assuming that timestamped gaze positions are available -... - - try: +```shell +python -m argaze load CONFIGURATION +``` - # Look ArUcoCamera frame at a timestamped gaze position - aruco_camera.look(timestamped_gaze_position) +This command should open a GUI window with the detected markers and identified cursor fixations circles when the mouse moves over the window. - # Do something with pipeline exception - except Exception as e: - - ... -``` + !!! note "" - At this point, the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method only detects ArUco marker and the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method only processes gaze movement identification without any AOI support as no scene description is provided into the JSON configuration file. + At this point, the pipeline only processes gaze movement identification without any AOI support as no scene description is provided into the JSON configuration file. Read the next chapters to learn [how to estimate scene pose](pose_estimation.md), [how to describe a 3D scene's AOI](aoi_3d_description.md) and [how to project them into the camera frame](aoi_3d_projection.md).
\ No newline at end of file diff --git a/docs/user_guide/aruco_marker_pipeline/introduction.md b/docs/user_guide/aruco_marker_pipeline/introduction.md index ef2e4da..54e1a1f 100644 --- a/docs/user_guide/aruco_marker_pipeline/introduction.md +++ b/docs/user_guide/aruco_marker_pipeline/introduction.md @@ -9,6 +9,9 @@ The OpenCV library provides a module to detect fiducial markers in a picture and The ArGaze [ArUcoMarker submodule](../../argaze.md/#argaze.ArUcoMarker) eases markers creation, markers detection, and 3D scene pose estimation through a set of high-level classes. +!!! warning "Read eye tracking context and gaze analysis pipeline sections before" + This section assumes that the incoming gaze positions are provided by an [eye tracking context](../eye_tracking_context/introduction.md) and also assumes that the way a [gaze analysis pipeline](../gaze_analysis_pipeline/introduction.md) works is understood. + First, let's look at the schema below. It gives an overview of the main notions involved in the following chapters.  |