diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/img/aruco_camera_aoi_projection.png | bin | 0 -> 57019 bytes | |||
| -rw-r--r-- | docs/img/aruco_camera_frame.png | bin | 54853 -> 49201 bytes | |||
| -rw-r--r-- | docs/img/aruco_camera_gaze_movement_identification.png | bin | 0 -> 56059 bytes | |||
| -rw-r--r-- | docs/img/aruco_camera_markers_detection.png | bin | 51729 -> 52844 bytes | |||
| -rw-r--r-- | docs/img/aruco_camera_pose_estimation.png | bin | 0 -> 51798 bytes | |||
| -rw-r--r-- | docs/user_guide/aruco_markers_pipeline/aoi_analysis.md | 58 | ||||
| -rw-r--r-- | docs/user_guide/aruco_markers_pipeline/aoi_description.md | 4 | ||||
| -rw-r--r-- | docs/user_guide/aruco_markers_pipeline/aoi_projection.md (renamed from docs/user_guide/aruco_markers_pipeline/aruco_scene.md) | 134 | ||||
| -rw-r--r-- | docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md (renamed from docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md) | 57 | ||||
| -rw-r--r-- | docs/user_guide/aruco_markers_pipeline/introduction.md | 7 | ||||
| -rw-r--r-- | docs/user_guide/aruco_markers_pipeline/pose_estimation.md | 82 |
11 files changed, 210 insertions, 132 deletions
diff --git a/docs/img/aruco_camera_aoi_projection.png b/docs/img/aruco_camera_aoi_projection.png Binary files differnew file mode 100644 index 0000000..59a8ab0 --- /dev/null +++ b/docs/img/aruco_camera_aoi_projection.png diff --git a/docs/img/aruco_camera_frame.png b/docs/img/aruco_camera_frame.png Binary files differindex 4b2a9a2..443285f 100644 --- a/docs/img/aruco_camera_frame.png +++ b/docs/img/aruco_camera_frame.png diff --git a/docs/img/aruco_camera_gaze_movement_identification.png b/docs/img/aruco_camera_gaze_movement_identification.png Binary files differnew file mode 100644 index 0000000..fc9ff39 --- /dev/null +++ b/docs/img/aruco_camera_gaze_movement_identification.png diff --git a/docs/img/aruco_camera_markers_detection.png b/docs/img/aruco_camera_markers_detection.png Binary files differindex 0452728..6192e09 100644 --- a/docs/img/aruco_camera_markers_detection.png +++ b/docs/img/aruco_camera_markers_detection.png diff --git a/docs/img/aruco_camera_pose_estimation.png b/docs/img/aruco_camera_pose_estimation.png Binary files differnew file mode 100644 index 0000000..b6c2675 --- /dev/null +++ b/docs/img/aruco_camera_pose_estimation.png diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_analysis.md b/docs/user_guide/aruco_markers_pipeline/aoi_analysis.md new file mode 100644 index 0000000..05afcd3 --- /dev/null +++ b/docs/user_guide/aruco_markers_pipeline/aoi_analysis.md @@ -0,0 +1,58 @@ +Add AOI analysis +================ + + + +## Add AOIMatcher, AOIScanPath and AOIScanPathAnalyzers to ArUcoCamera layer to enable gaze analysis + +Here is the previous extract where AOI matcher, AOI scan path and AOI scan path analyzers are added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer: + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + ... + "scenes": { + "MyScene" : { + "aruco_markers_group": { + ... + }, + "layers": { + "MyLayer": { + "aoi_scene": { + ... + } + } + } + } + }, + "layers": { + "MyLayer": { + "aoi_matcher": { + "DeviationCircleCoverage": { + "coverage_threshold": 0.5 + } + }, + "aoi_scan_path": { + "duration_max": 30000 + }, + "aoi_scan_path_analyzers": { + "Basic": {}, + "TransitionMatrix": {}, + "NGram": { + "n_min": 3, + "n_max": 5 + } + } + } + }, + "gaze_movement_identifier": { + ... + } + ... +} +``` + +!!! warning + + Adding scan path and scan path analyzers to [an ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer doesn't make sense if the camera is moving.
\ No newline at end of file diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_description.md b/docs/user_guide/aruco_markers_pipeline/aoi_description.md index 80ad858..101ec9f 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_description.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_description.md @@ -1,5 +1,5 @@ -Describe AOI scene -================== +Describe AOI +============ Once [ArUco markers are placed into a scene](aruco_markers_description.md), areas of interest need to be described into the same 3D referential. diff --git a/docs/user_guide/aruco_markers_pipeline/aruco_scene.md b/docs/user_guide/aruco_markers_pipeline/aoi_projection.md index 91d2702..027f805 100644 --- a/docs/user_guide/aruco_markers_pipeline/aruco_scene.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_projection.md @@ -1,66 +1,15 @@ -Add an ArUcoScene -================= +Project AOI into camera frame +============================= -An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) class defines a space where [areas of interest description](aoi_description.md) is mapped onto [ArUco markers description](aruco_markers_description.md). +Once [ArUcoScene pose is estimated](pose_estimation.md) and [AOI are described](aoi_description.md), AOI can be projected into [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. - - -## Add ArUcoScene to ArUcoCamera JSON configuration file - -An [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) instance can contains multiples [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene). - -Here is an extract from the JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration file with a sample where one scene is added: - -```json -{ - "name": "My FullHD camera", - "size": [1920, 1080], - ... - "scenes": { - "MyScene" : { - "aruco_markers_group": { - "dictionary": "DICT_APRILTAG_16h5", - "marker_size": 5, - "places": { - "0": { - "translation": [2.5, 2.5, 0], - "rotation": [0, 0, 0] - }, - "1": { - "translation": [0, 25.5, 2], - "rotation": [0, 45, 0] - }, - "2": { - "translation": [35.5, 0, 3], - "rotation": [-45, 0, 0] - } - } - } - } - } - ... -} -``` - -Now, let's understand the meaning of each JSON entry. - -### "MyScene" - -The name of the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene). Basically useful for visualisation purpose. - -### ArUco markers group - -The 3D places of ArUco markers into the scene as defined at [ArUco markers description chapter](aruco_markers_description.md). Thanks to this description, it is possible to estimate the pose of [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) in [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. - -!!! note - - [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) pose estimation is done when calling the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method. + ## Add ArLayer to ArUcoScene to load AOI The [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer) class allows to load areas of interest description. An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArLayers](../../argaze.md/#argaze.ArFeatures.ArLayer). -Here is the previous extract where one layer is added to the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene): +Here is the previous extract where one layer is added to the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration: ```json { @@ -99,7 +48,7 @@ The [AOIScene](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AOIScene) defi ## Add ArLayer to ArUcoCamera to project AOI -Here is the previous extract where one layer is added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera): +Here is the previous extract where one layer is added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) and displayed: ```json { @@ -121,19 +70,22 @@ Here is the previous extract where one layer is added to the [ArUcoCamera](../.. } }, "layers": { - "MyLayer": { - "aoi_matcher": { - ... - }, - "aoi_scan_path": { - "duration_max": 30000 - }, - "aoi_scan_path_analyzers": { - ... + "MyLayer": {} + } + ... + "image_parameters": { + ... + "draw_layers": { + "MyLayer": { + "draw_aoi_scene": { + "draw_aoi": { + "color": [255, 255, 255], + "border_size": 1 + } + } } } } - ... } ``` @@ -143,7 +95,7 @@ Now, let's understand the meaning of each JSON entry. The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically useful for visualisation purpose. -!!! warning +!!! warning "Layer name policy" An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer is projected into [an ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer, **provided they have the same name**. @@ -151,46 +103,11 @@ The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers are projected into their dedicated [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layers when calling the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method. -## Add GazeMovementIdentifier to ArUcocamera to enable gaze movement identification +## Add AOI analysis -Here is the previous extract where gaze movement identifier is added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera): +When a scene layer is projected into a camera layer, it means that the 3D [ArLayer.aoi_scene](../../argaze.md/#argaze.ArFeatures.ArLayer.aoi_scene) description of the scene becomes the 2D camera's [ArLayer.aoi_scene](../../argaze.md/#argaze.ArFeatures.ArLayer.aoi_scene) description of the camera. -```json -{ - "name": "My FullHD camera", - "size": [1920, 1080], - ... - "scenes": { - "MyScene" : { - "aruco_markers_group": { - ... - }, - "layers": { - "MyLayer": { - "aoi_scene": { - ... - } - } - } - } - }, - "layers": { - "MyLayer": {} - }, - "gaze_movement_identifier": { - "DispersionThresholdIdentification": { - "deviation_max_threshold": 50, - "duration_min_threshold": 200 - } - } - ... -} -``` -!!! note - - Timestamped gaze positions are [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layers execution is done when calling the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method. - -## Add AOIMatcher, AOIScanPath and AOIScanPathAnalyzers to ArUcoCamera layer to enable gaze analysis +Therefore, it means that [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) benefits from all the services described in [AOI analysis pipeline section](../gaze_analysis_pipeline/aoi_analysis.md). Here is the previous extract where AOI matcher, AOI scan path and AOI scan path analyzers are added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer: @@ -232,9 +149,6 @@ Here is the previous extract where AOI matcher, AOI scan path and AOI scan path } } } - }, - "gaze_movement_identifier": { - ... } ... } @@ -242,4 +156,4 @@ Here is the previous extract where AOI matcher, AOI scan path and AOI scan path !!! warning - Adding scan path and scan path analyzers to [an ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer doesn't make sense if the camera is moving.
\ No newline at end of file + Adding scan path and scan path analyzers to an [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer doesn't make sense if the camera is moving. diff --git a/docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md index ba19e45..81c577f 100644 --- a/docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md +++ b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md @@ -1,9 +1,9 @@ -Configure and execute ArUcoCamera -================================= +Load and execute pipeline +========================= -Once [ArUco markers are placed into a scene](aruco_markers_description.md) and [areas of interest are described](aoi_description.md), everything is ready to setup an ArUco marker pipeline thanks to [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class. +Once [ArUco markers are placed into a scene](aruco_markers_description.md), they can be detected thanks to [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class. -As [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md). +As [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class also benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md).  @@ -21,9 +21,26 @@ Here is a simple JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCa "dictionary": "DICT_APRILTAG_16h5", "marker_size": 5 }, + "gaze_movement_identifier": { + "DispersionThresholdIdentification": { + "deviation_max_threshold": 25, + "duration_min_threshold": 150 + } + }, "image_parameters": { "background_weight": 1, - ... + "draw_gaze_positions": { + "color": [0, 255, 255], + "size": 2 + }, + "draw_fixations": { + "deviation_circle_color": [255, 0, 255], + "duration_border_color": [127, 0, 127], + "duration_factor": 1e-2 + }, + "draw_saccades": { + "line_color": [255, 0, 255] + }, "draw_detected_markers": { "color": [0, 255, 0], "draw_axes": { @@ -57,20 +74,28 @@ The size of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) The first [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) pipeline step is to detect ArUco markers inside input image and estimate their poses. + + The [ArUcoDetector](../../argaze.md/#argaze.ArUcoMarkers.ArUcoDetector) is in charge to detect all markers from a specific dictionary with a given size in centimeters. -!!! warning +!!! warning "Mandatory" JSON *aruco_detector* entry is mandatory. +### Gaze Movement Identifier - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* + +The first [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) pipeline step dedicated to identify fixations or saccades from consecutive timestamped gaze positions. + + + ### Image parameters - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* The usual [ArFrame visualisation parameters](./user_guide/gaze_analysis_pipeline/visualisation.md) plus one additional *draw_detected_markers* field. ## Pipeline execution -### Detect ArUco markers, estimate scene pose and project scene +### Detect ArUco markers, estimate scene pose and project AOI -Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole pipeline dedicated to ArUco markers detection, scene pose estimation and projection. +Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole pipeline dedicated to ArUco markers detection, scene pose estimation and AOI projection. ```python # Assuming that Full HD (1920x1080) video stream or file is opened @@ -82,17 +107,15 @@ Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures # Capture image from video stream of file image = video_capture.read() - # Detect ArUco markers, estimate scene pose then, project scene into camera frame + # Detect ArUco markers, estimate scene pose then, project AOI into camera frame aruco_camera.watch(image) - # Display ArUcoCamera frame image to check that ArUco markers are well detected and scene is well projected + # Display ArUcoCamera frame image to display detected ArUco markers, scene pose, AOI projection and ArFrame visualisation. ... aruco_camera.image() ``` -!!! warning - ArUco markers pose estimation algorithm can lead to errors due to geometric ambiguities as explain in [this article](https://ieeexplore.ieee.org/document/1717461). To discard such ambiguous cases, markers should **not be parallel to camera plan**. - - +!!! warning "Pose estimation error" + ArUco markers pose estimation algorithm can lead to errors due to geometric ambiguities as explain in [this article](https://ieeexplore.ieee.org/document/1717461). To discard such ambiguous cases, markers should **as less as possible be parallel to camera plan**. ### Analyse timestamped gaze positions into camera frame @@ -108,8 +131,8 @@ Particularly, timestamped gaze positions can be passed one by one to [ArUcoCamer aruco_camera.look(timestamp, gaze_position) ``` -!!! warning "" +!!! note "" - At this point, the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method only detects ArUco markers and the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method is not able to analyze gaze positions as no scene description is provided into the JSON configuration file. + At this point, the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method only detects ArUco markers and the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method only process gaze movement identification without any AOI support as no scene description is provided into the JSON configuration file. - Read the next chapters to learn [how to enable scene pose estimation and its projection](aruco_scene.md).
\ No newline at end of file + Read the next chapters to learn [how to estimate scene pose](pose_estimation.md) and [how to project AOI](aoi_projection.md).
\ No newline at end of file diff --git a/docs/user_guide/aruco_markers_pipeline/introduction.md b/docs/user_guide/aruco_markers_pipeline/introduction.md index a10ca28..836569a 100644 --- a/docs/user_guide/aruco_markers_pipeline/introduction.md +++ b/docs/user_guide/aruco_markers_pipeline/introduction.md @@ -16,9 +16,10 @@ First, let's look at the schema below: it gives an overview of the main notions To build your own ArUco markers pipeline, you need to know: * [How to setup ArUco markers into a scene](aruco_markers_description.md), -* [How to describe areas of interest in the same scene](aoi_description.md), -* [How to deal with an ArUcoCamera instance](aruco_camera_configuration_and_execution.md), -* [How to add ArUcoScene instance](aruco_scene.md), +* [How to describe scene's AOI](aoi_description.md), +* [How to load and execute ArUco markers pipeline](configuration_and_execution.md), +* [How to estimate scene pose](pose_estimation.md), +* [How to project AOI into camera frame](aoi_projection.md), * [How to visualize ArUcoCamera and ArUcoScenes](visualisation.md) More advanced features are also explained like: diff --git a/docs/user_guide/aruco_markers_pipeline/pose_estimation.md b/docs/user_guide/aruco_markers_pipeline/pose_estimation.md new file mode 100644 index 0000000..6acafee --- /dev/null +++ b/docs/user_guide/aruco_markers_pipeline/pose_estimation.md @@ -0,0 +1,82 @@ +Estimate scene pose +=================== + +An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) class defines a space with [ArUco markers inside](aruco_markers_description.md) helping to estimate scene pose when they are watched by [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera). + + + +## Add ArUcoScene to ArUcoCamera JSON configuration file + +An [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) instance can contains multiples [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene). + +Here is an extract from the JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration file with a sample where one scene is added and displayed: + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + ... + "scenes": { + "MyScene" : { + "aruco_markers_group": { + "dictionary": "DICT_APRILTAG_16h5", + "marker_size": 5, + "places": { + "0": { + "translation": [2.5, 2.5, 0], + "rotation": [0, 0, 0] + }, + "1": { + "translation": [0, 25.5, 2], + "rotation": [0, 45, 0] + }, + "2": { + "translation": [35.5, 0, 3], + "rotation": [-45, 0, 0] + } + } + } + } + } + ... + "image_parameters": { + ... + "draw_scenes": { + "MyScene": { + "draw_aruco_markers_group": { + "draw_axes": { + "thickness": 3, + "length": 10 + }, + "draw_places": { + "color": [0, 0, 0], + "border_size": 1 + }, + "draw_places_axes": { + "thickness": 1, + "length": 2.5 + } + } + } + } + } +} +``` + +Now, let's understand the meaning of each JSON entry. + +### "MyScene" + +The name of the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene). Basically useful for visualisation purpose. + +### ArUco markers group + +The 3D places of ArUco markers into the scene as defined at [ArUco markers description chapter](aruco_markers_description.md). Thanks to this description, it is possible to estimate the pose of [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) in [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. + +!!! note + + [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) pose estimation is done when calling the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method. + +### Draw scenes + +The drawing parameters of each loaded [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) in [ArUcoCamera.image](../../argaze.md/#argaze.ArFeatures.ArFrame.image). |
