From 23fa1a7835b3c7cfd976b1d160878289b1f0657c Mon Sep 17 00:00:00 2001 From: Theo De La Hogue Date: Sat, 23 Sep 2023 07:22:23 +0200 Subject: Fixing code annotation. Removing useless documentation section. Fixing documentation cross reference. --- .../advanced_topics/optic_parameters_calibration.md | 8 ++++---- .../aruco_markers_pipeline/configuration_and_execution.md | 6 +++--- docs/user_guide/aruco_markers_pipeline/introduction.md | 12 ++++++------ 3 files changed, 13 insertions(+), 13 deletions(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md b/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md index 455d95a..fbe06d1 100644 --- a/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md +++ b/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md @@ -3,11 +3,11 @@ Calibrate optic parameters A camera device have to be calibrated to compensate its optical distorsion. -![Optic parameters calibration](../../img/optic_calibration.png) +![Optic parameters calibration](../../../img/optic_calibration.png) ## Print calibration board -The first step to calibrate a camera is to create an [ArUcoBoard](../../argaze.md/#argaze.ArUcoMarkers.ArUcoBoard) like in the code below: +The first step to calibrate a camera is to create an [ArUcoBoard](../../../argaze.md/#argaze.ArUcoMarkers.ArUcoBoard) like in the code below: ``` python from argaze.ArUcoMarkers import ArUcoMarkersDictionary, ArUcoBoard @@ -29,9 +29,9 @@ Let's print the calibration board before to go further. ## Capture board pictures -Then, the calibration process needs to make many different captures of an [ArUcoBoard](../../argaze.md/#argaze.ArUcoMarkers.ArUcoBoard) through the camera and then, pass them to an [ArUcoDetector](../../argaze.md/#argaze.ArUcoMarkers.ArUcoDetector.ArUcoDetector) instance to detect board corners and store them as calibration data into an [ArUcoOpticCalibrator](../../argaze.md/#argaze.ArUcoMarkers.ArUcoOpticCalibrator) for final calibration process. +Then, the calibration process needs to make many different captures of an [ArUcoBoard](../../../argaze.md/#argaze.ArUcoMarkers.ArUcoBoard) through the camera and then, pass them to an [ArUcoDetector](../../../argaze.md/#argaze.ArUcoMarkers.ArUcoDetector.ArUcoDetector) instance to detect board corners and store them as calibration data into an [ArUcoOpticCalibrator](../../../argaze.md/#argaze.ArUcoMarkers.ArUcoOpticCalibrator) for final calibration process. -![Calibration step](../../img/optic_calibration_step.png) +![Calibration step](../../../img/optic_calibration_step.png) The sample of code below illustrates how to: diff --git a/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md index 81c577f..35b64f7 100644 --- a/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md +++ b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md @@ -3,7 +3,7 @@ Load and execute pipeline Once [ArUco markers are placed into a scene](aruco_markers_description.md), they can be detected thanks to [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class. -As [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class also benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md). +As [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class also benefits from all the services described in [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md). ![ArUco camera frame](../../img/aruco_camera_frame.png) @@ -89,7 +89,7 @@ The first [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) pipeline step de ### Image parameters - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* -The usual [ArFrame visualisation parameters](./user_guide/gaze_analysis_pipeline/visualisation.md) plus one additional *draw_detected_markers* field. +The usual [ArFrame visualisation parameters](../gaze_analysis_pipeline/visualisation.md) plus one additional *draw_detected_markers* field. ## Pipeline execution @@ -119,7 +119,7 @@ Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures ### Analyse timestamped gaze positions into camera frame -As mentioned above, [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) and so, benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md). +As mentioned above, [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) and so, benefits from all the services described in [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md). Particularly, timestamped gaze positions can be passed one by one to [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method to execute the whole pipeline dedicated to gaze analysis. diff --git a/docs/user_guide/aruco_markers_pipeline/introduction.md b/docs/user_guide/aruco_markers_pipeline/introduction.md index 836569a..f781fe8 100644 --- a/docs/user_guide/aruco_markers_pipeline/introduction.md +++ b/docs/user_guide/aruco_markers_pipeline/introduction.md @@ -11,7 +11,7 @@ The ArGaze [ArUcoMarkers submodule](../../argaze.md/#argaze.ArUcoMarkers) eases First, let's look at the schema below: it gives an overview of the main notions involved in the following chapters. -![ArUco markers pipeline](../../img/aruco_markers_pipeline.png) + To build your own ArUco markers pipeline, you need to know: @@ -19,11 +19,11 @@ To build your own ArUco markers pipeline, you need to know: * [How to describe scene's AOI](aoi_description.md), * [How to load and execute ArUco markers pipeline](configuration_and_execution.md), * [How to estimate scene pose](pose_estimation.md), -* [How to project AOI into camera frame](aoi_projection.md), -* [How to visualize ArUcoCamera and ArUcoScenes](visualisation.md) +* [How to project AOI into camera frame](aoi_projection.md) + More advanced features are also explained like: -* [How to script ArUco markers pipeline](advanced_topics/scripting.md) -* [How to calibrate optic parameters](optic_parameters_calibration.md) -* [How to improve ArUco markers detection](advanced_topics/aruco_detector_configuration.md) + +* [How to calibrate optic parameters](advanced_topics/optic_parameters_calibration.md) + -- cgit v1.1 From 61df29ded4a1ffef3f9677f0d13f07821f07fcb7 Mon Sep 17 00:00:00 2001 From: Theo De La Hogue Date: Mon, 25 Sep 2023 10:31:22 +0200 Subject: Changing layer name policy warning. --- docs/user_guide/aruco_markers_pipeline/aoi_projection.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_projection.md b/docs/user_guide/aruco_markers_pipeline/aoi_projection.md index 027f805..2f764f8 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_projection.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_projection.md @@ -97,7 +97,7 @@ The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically !!! warning "Layer name policy" - An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer is projected into [an ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer, **provided they have the same name**. + An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer is projected into an [ ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer, **provided they have the same name**. !!! note -- cgit v1.1 From a6561584fa398b2626389f558ce59ce8d73e9dab Mon Sep 17 00:00:00 2001 From: Theo De La Hogue Date: Mon, 25 Sep 2023 10:32:10 +0200 Subject: Adding AOI frame chapter link. --- docs/user_guide/aruco_markers_pipeline/introduction.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/introduction.md b/docs/user_guide/aruco_markers_pipeline/introduction.md index f781fe8..dd82900 100644 --- a/docs/user_guide/aruco_markers_pipeline/introduction.md +++ b/docs/user_guide/aruco_markers_pipeline/introduction.md @@ -7,7 +7,7 @@ The OpenCV library provides a module to detect fiducial markers into a picture a ![OpenCV ArUco markers](https://pyimagesearch.com/wp-content/uploads/2020/12/aruco_generate_tags_header.png) -The ArGaze [ArUcoMarkers submodule](../../argaze.md/#argaze.ArUcoMarkers) eases markers creation, optic calibration, markers detection and 3D scene pose estimation through a set of high level classes. +The ArGaze [ArUcoMarkers submodule](../../argaze.md/#argaze.ArUcoMarkers) eases markers creation, markers detection and 3D scene pose estimation through a set of high level classes. First, let's look at the schema below: it gives an overview of the main notions involved in the following chapters. @@ -19,8 +19,8 @@ To build your own ArUco markers pipeline, you need to know: * [How to describe scene's AOI](aoi_description.md), * [How to load and execute ArUco markers pipeline](configuration_and_execution.md), * [How to estimate scene pose](pose_estimation.md), -* [How to project AOI into camera frame](aoi_projection.md) - +* [How to project AOI into camera frame](aoi_projection.md), +* [How to define an AOI as a frame](aoi_frame.md) More advanced features are also explained like: -- cgit v1.1 From f26f79113febf8a8dcccf18dabfcdd4bef04ce86 Mon Sep 17 00:00:00 2001 From: Theo De La Hogue Date: Mon, 25 Sep 2023 10:32:43 +0200 Subject: REmoving a uselss comma --- docs/user_guide/aruco_markers_pipeline/pose_estimation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/pose_estimation.md b/docs/user_guide/aruco_markers_pipeline/pose_estimation.md index 6acafee..d7da336 100644 --- a/docs/user_guide/aruco_markers_pipeline/pose_estimation.md +++ b/docs/user_guide/aruco_markers_pipeline/pose_estimation.md @@ -37,7 +37,7 @@ Here is an extract from the JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMark } } } - } + }, ... "image_parameters": { ... -- cgit v1.1 From 8f2b87bfec622dd32e90d9bfa17dfcda42add4fe Mon Sep 17 00:00:00 2001 From: Theo De La Hogue Date: Mon, 25 Sep 2023 14:45:52 +0200 Subject: Improving AOI description documentation. --- .../aruco_markers_pipeline/aoi_3d_description.md | 62 ++++++++ .../aruco_markers_pipeline/aoi_3d_projection.md | 159 +++++++++++++++++++++ .../aruco_markers_pipeline/aoi_description.md | 62 -------- .../user_guide/aruco_markers_pipeline/aoi_frame.md | 64 +++++++++ .../aruco_markers_pipeline/aoi_projection.md | 159 --------------------- .../configuration_and_execution.md | 2 +- .../aruco_markers_pipeline/introduction.md | 4 +- 7 files changed, 288 insertions(+), 224 deletions(-) create mode 100644 docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md create mode 100644 docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md delete mode 100644 docs/user_guide/aruco_markers_pipeline/aoi_description.md create mode 100644 docs/user_guide/aruco_markers_pipeline/aoi_frame.md delete mode 100644 docs/user_guide/aruco_markers_pipeline/aoi_projection.md (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md new file mode 100644 index 0000000..13f9c86 --- /dev/null +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md @@ -0,0 +1,62 @@ +Describe 3D AOI +=============== + +Once [ArUco markers are placed into a scene](aruco_markers_description.md), areas of interest need to be described into the same 3D referential. + +In the example scene, each screen is considered as an area of interest more the blue triangle area inside the top screen. + +![3D AOI description](../../img/aoi_3d_description.png) + +All AOIs need to be described from same origin than markers in a [right-handed 3D axis](https://robotacademy.net.au/lesson/right-handed-3d-coordinate-frame/) where: + +* +X is pointing to the right, +* +Y is pointing to the top, +* +Z is pointing to the backward. + +!!! warning + All AOIs spatial values must be given in **centimeters**. + +### Edit OBJ file description + +OBJ file format could be exported from most 3D editors. + +``` obj +o YellowSquare +v 6.200003 -7.275252 25.246159 +v 31.200003 -7.275252 25.246159 +v 6.200003 1.275252 1.753843 +v 31.200003 1.275252 1.753843 +s off +f 1 2 4 3 +o GrayRectangle +v 2.500000 2.500000 -0.500000 +v 37.500000 2.500000 -0.500000 +v 2.500000 27.500000 -0.500000 +v 37.500000 27.500000 -0.500000 +s off +f 5 6 8 7 +o BlueTriangle +v 12.500002 7.500000 -0.500000 +v 27.500002 7.500000 -0.500000 +v 20.000002 22.500000 -0.500000 +s off +f 9 10 11 +``` + +Here are common OBJ file features needed to describe AOIs: + +* Object lines (starting with *o* key) indicate AOI name. +* Vertice lines (starting with *v* key) indicate AOI vertices. +* Face (starting with *f* key) link vertices together. + +### Edit JSON file description + +JSON file format allows to describe AOIs vertices. + +``` json +{ + "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], + "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]], + "BlueTriangle": [[12.5, 7.5, -0.5], [27.5, 7.5, -0.5], [20, 22.5, -0.5]] +} +``` diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md new file mode 100644 index 0000000..bdebd6c --- /dev/null +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md @@ -0,0 +1,159 @@ +Project AOI into camera frame +============================= + +Once [ArUcoScene pose is estimated](pose_estimation.md) and [3D AOIs are described](aoi_3d_description.md), AOIs can be projected into [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. + +![3D AOI projection](../../img/aruco_camera_aoi_projection.png) + +## Add ArLayer to ArUcoScene to load AOI + +The [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer) class allows to load areas of interest description. An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArLayers](../../argaze.md/#argaze.ArFeatures.ArLayer). + +Here is the previous extract where one layer is added to the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration: + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + ... + "scenes": { + "MyScene" : { + "aruco_markers_group": { + ... + }, + "layers": { + "MyLayer": { + "aoi_scene": { + "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], + "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]], + "BlueTriangle": [[12.5, 7.5, -0.5], [27.5, 7.5, -0.5], [20, 22.5, -0.5]] + } + } + } + } + } + ... +} +``` + +Now, let's understand the meaning of each JSON entry. + +### "MyLayer" + +The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically useful for visualisation purpose. + +### AOI Scene + +The [AOIScene](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AOIScene) defines a set of 3D [AreaOfInterest](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AreaOfInterest) registered by name. + +## Add ArLayer to ArUcoCamera to project 3D AOIs + +Here is the previous extract where one layer is added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) and displayed: + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + ... + "scenes": { + "MyScene" : { + "aruco_markers_group": { + ... + }, + "layers": { + "MyLayer": { + "aoi_scene": { + ... + } + } + } + } + }, + "layers": { + "MyLayer": {} + } + ... + "image_parameters": { + ... + "draw_layers": { + "MyLayer": { + "draw_aoi_scene": { + "draw_aoi": { + "color": [255, 255, 255], + "border_size": 1 + } + } + } + } + } +} +``` + +Now, let's understand the meaning of each JSON entry. + +### "MyLayer" + +The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically useful for visualisation purpose. + +!!! warning "Layer name policy" + + An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer is projected into an [ ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer, **provided they have the same name**. + +!!! note + + [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers are projected into their dedicated [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layers when calling the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method. + +## Add 2D AOIs analysis + +When a scene layer is projected into a camera layer, it means that the 3D [ArLayer.aoi_scene](../../argaze.md/#argaze.ArFeatures.ArLayer.aoi_scene) description of the scene becomes the 2D camera's [ArLayer.aoi_scene](../../argaze.md/#argaze.ArFeatures.ArLayer.aoi_scene) description of the camera. + +Therefore, it means that [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) benefits from all the services described in [2D AOIs analysis pipeline section](../gaze_analysis_pipeline/aoi_2d_analysis.md). + +Here is the previous extract where AOI matcher, AOI scan path and AOI scan path analyzers are added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer: + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + ... + "scenes": { + "MyScene" : { + "aruco_markers_group": { + ... + }, + "layers": { + "MyLayer": { + "aoi_scene": { + ... + } + } + } + } + }, + "layers": { + "MyLayer": { + "aoi_matcher": { + "DeviationCircleCoverage": { + "coverage_threshold": 0.5 + } + }, + "aoi_scan_path": { + "duration_max": 30000 + }, + "aoi_scan_path_analyzers": { + "Basic": {}, + "TransitionMatrix": {}, + "NGram": { + "n_min": 3, + "n_max": 5 + } + } + } + } + ... +} +``` + +!!! warning + + Adding scan path and scan path analyzers to an [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer doesn't make sense if the camera is moving. diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_description.md b/docs/user_guide/aruco_markers_pipeline/aoi_description.md deleted file mode 100644 index 101ec9f..0000000 --- a/docs/user_guide/aruco_markers_pipeline/aoi_description.md +++ /dev/null @@ -1,62 +0,0 @@ -Describe AOI -============ - -Once [ArUco markers are placed into a scene](aruco_markers_description.md), areas of interest need to be described into the same 3D referential. - -In the example scene, each screen is considered as an area of interest more the blue triangle area inside the top screen. - -![AOI description](../../img/aoi_description.png) - -All AOIs need to be described from same origin than markers in a [right-handed 3D axis](https://robotacademy.net.au/lesson/right-handed-3d-coordinate-frame/) where: - -* +X is pointing to the right, -* +Y is pointing to the top, -* +Z is pointing to the backward. - -!!! warning - All AOIs spatial values must be given in **centimeters**. - -### Edit OBJ file description - -OBJ file format could be exported from most 3D editors. - -``` obj -o YellowSquare -v 6.200003 -7.275252 25.246159 -v 31.200003 -7.275252 25.246159 -v 6.200003 1.275252 1.753843 -v 31.200003 1.275252 1.753843 -s off -f 1 2 4 3 -o GrayRectangle -v 2.500000 2.500000 -0.500000 -v 37.500000 2.500000 -0.500000 -v 2.500000 27.500000 -0.500000 -v 37.500000 27.500000 -0.500000 -s off -f 5 6 8 7 -o BlueTriangle -v 12.500002 7.500000 -0.500000 -v 27.500002 7.500000 -0.500000 -v 20.000002 22.500000 -0.500000 -s off -f 9 10 11 -``` - -Here are common OBJ file features needed to describe AOIs: - -* Object lines (starting with *o* key) indicate AOI name. -* Vertice lines (starting with *v* key) indicate AOI vertices. -* Face (starting with *f* key) link vertices together. - -### Edit JSON file description - -JSON file format allows to describe AOIs vertices. - -``` json -{ - "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], - "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]], - "BlueTriangle": [[12.5, 7.5, -0.5], [27.5, 7.5, -0.5], [20, 22.5, -0.5]] -} -``` diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_frame.md b/docs/user_guide/aruco_markers_pipeline/aoi_frame.md new file mode 100644 index 0000000..6b87d52 --- /dev/null +++ b/docs/user_guide/aruco_markers_pipeline/aoi_frame.md @@ -0,0 +1,64 @@ +Define an AOI as a frame +======================== + + + + +## Add ArFrame to ArUcoScene + +The [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) class defines a rectangular area where timestamped gaze positions are projected in and inside which they need to be analyzed. An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArFrames](../../argaze.md/#argaze.ArFeatures.ArFrame). + +Here is the previous extract where the "GrayRectangle" AOI is defined as a frame into the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration: + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + ... + "scenes": { + "MyScene" : { + "aruco_markers_group": { + ... + }, + "layers": { + "MyLayer": { + "aoi_scene": { + "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], + "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]] + } + } + }, + "frames": { + "GrayRectangle": { + "size": [350, 250], + "layers": { + "MyLayer": { + "aoi_scene": { + "BlueTriangle": [[100, 50], [250, 50], [175, 200]] + } + } + } + } + } + } + } + ... +} +``` +Now, let's understand the meaning of each JSON entry. + +### "GrayRectangle" + +The name of the AOI and the [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame). Basically useful for visualisation purpose. + +!!! warning "Frame name policy" + + An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer AOI is defined as an [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame, **provided they have the same name**. + +!!! warning "Layer name policy" + + An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame layer is projected into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer, **provided they have the same name**. + +!!! note + + [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame layers are projected into their dedicated [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers when the JSON configuration file is loaded. \ No newline at end of file diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_projection.md b/docs/user_guide/aruco_markers_pipeline/aoi_projection.md deleted file mode 100644 index 2f764f8..0000000 --- a/docs/user_guide/aruco_markers_pipeline/aoi_projection.md +++ /dev/null @@ -1,159 +0,0 @@ -Project AOI into camera frame -============================= - -Once [ArUcoScene pose is estimated](pose_estimation.md) and [AOI are described](aoi_description.md), AOI can be projected into [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. - -![AOI projection](../../img/aruco_camera_aoi_projection.png) - -## Add ArLayer to ArUcoScene to load AOI - -The [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer) class allows to load areas of interest description. An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArLayers](../../argaze.md/#argaze.ArFeatures.ArLayer). - -Here is the previous extract where one layer is added to the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration: - -```json -{ - "name": "My FullHD camera", - "size": [1920, 1080], - ... - "scenes": { - "MyScene" : { - "aruco_markers_group": { - ... - }, - "layers": { - "MyLayer": { - "aoi_scene": { - "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], - "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]], - "BlueTriangle": [[12.5, 7.5, -0.5], [27.5, 7.5, -0.5], [20, 22.5, -0.5]] - } - } - } - } - } - ... -} -``` - -Now, let's understand the meaning of each JSON entry. - -### "MyLayer" - -The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically useful for visualisation purpose. - -### AOI Scene - -The [AOIScene](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AOIScene) defines a set of 3D [AreaOfInterest](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AreaOfInterest) registered by name. - -## Add ArLayer to ArUcoCamera to project AOI - -Here is the previous extract where one layer is added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) and displayed: - -```json -{ - "name": "My FullHD camera", - "size": [1920, 1080], - ... - "scenes": { - "MyScene" : { - "aruco_markers_group": { - ... - }, - "layers": { - "MyLayer": { - "aoi_scene": { - ... - } - } - } - } - }, - "layers": { - "MyLayer": {} - } - ... - "image_parameters": { - ... - "draw_layers": { - "MyLayer": { - "draw_aoi_scene": { - "draw_aoi": { - "color": [255, 255, 255], - "border_size": 1 - } - } - } - } - } -} -``` - -Now, let's understand the meaning of each JSON entry. - -### "MyLayer" - -The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically useful for visualisation purpose. - -!!! warning "Layer name policy" - - An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer is projected into an [ ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer, **provided they have the same name**. - -!!! note - - [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers are projected into their dedicated [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layers when calling the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method. - -## Add AOI analysis - -When a scene layer is projected into a camera layer, it means that the 3D [ArLayer.aoi_scene](../../argaze.md/#argaze.ArFeatures.ArLayer.aoi_scene) description of the scene becomes the 2D camera's [ArLayer.aoi_scene](../../argaze.md/#argaze.ArFeatures.ArLayer.aoi_scene) description of the camera. - -Therefore, it means that [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) benefits from all the services described in [AOI analysis pipeline section](../gaze_analysis_pipeline/aoi_analysis.md). - -Here is the previous extract where AOI matcher, AOI scan path and AOI scan path analyzers are added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer: - -```json -{ - "name": "My FullHD camera", - "size": [1920, 1080], - ... - "scenes": { - "MyScene" : { - "aruco_markers_group": { - ... - }, - "layers": { - "MyLayer": { - "aoi_scene": { - ... - } - } - } - } - }, - "layers": { - "MyLayer": { - "aoi_matcher": { - "DeviationCircleCoverage": { - "coverage_threshold": 0.5 - } - }, - "aoi_scan_path": { - "duration_max": 30000 - }, - "aoi_scan_path_analyzers": { - "Basic": {}, - "TransitionMatrix": {}, - "NGram": { - "n_min": 3, - "n_max": 5 - } - } - } - } - ... -} -``` - -!!! warning - - Adding scan path and scan path analyzers to an [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer doesn't make sense if the camera is moving. diff --git a/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md index 35b64f7..6bf84a9 100644 --- a/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md +++ b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md @@ -135,4 +135,4 @@ Particularly, timestamped gaze positions can be passed one by one to [ArUcoCamer At this point, the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method only detects ArUco markers and the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method only process gaze movement identification without any AOI support as no scene description is provided into the JSON configuration file. - Read the next chapters to learn [how to estimate scene pose](pose_estimation.md) and [how to project AOI](aoi_projection.md). \ No newline at end of file + Read the next chapters to learn [how to estimate scene pose](pose_estimation.md) and [how to project AOI](aoi_3d_projection.md). \ No newline at end of file diff --git a/docs/user_guide/aruco_markers_pipeline/introduction.md b/docs/user_guide/aruco_markers_pipeline/introduction.md index dd82900..917245d 100644 --- a/docs/user_guide/aruco_markers_pipeline/introduction.md +++ b/docs/user_guide/aruco_markers_pipeline/introduction.md @@ -16,10 +16,10 @@ First, let's look at the schema below: it gives an overview of the main notions To build your own ArUco markers pipeline, you need to know: * [How to setup ArUco markers into a scene](aruco_markers_description.md), -* [How to describe scene's AOI](aoi_description.md), +* [How to describe scene's AOI](aoi_3d_description.md), * [How to load and execute ArUco markers pipeline](configuration_and_execution.md), * [How to estimate scene pose](pose_estimation.md), -* [How to project AOI into camera frame](aoi_projection.md), +* [How to project AOIs into camera frame](aoi_3d_projection.md), * [How to define an AOI as a frame](aoi_frame.md) More advanced features are also explained like: -- cgit v1.1 From 12ae7e20aba323624d360567ea424ac2d315fbc7 Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Tue, 26 Sep 2023 10:47:43 +0200 Subject: Harmonizing AOI/aoi without s at the end. --- docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md | 8 ++++---- docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md | 8 ++++---- docs/user_guide/aruco_markers_pipeline/introduction.md | 2 +- 3 files changed, 9 insertions(+), 9 deletions(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md index 13f9c86..a2bb8d7 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md @@ -7,14 +7,14 @@ In the example scene, each screen is considered as an area of interest more the ![3D AOI description](../../img/aoi_3d_description.png) -All AOIs need to be described from same origin than markers in a [right-handed 3D axis](https://robotacademy.net.au/lesson/right-handed-3d-coordinate-frame/) where: +All AOI need to be described from same origin than markers in a [right-handed 3D axis](https://robotacademy.net.au/lesson/right-handed-3d-coordinate-frame/) where: * +X is pointing to the right, * +Y is pointing to the top, * +Z is pointing to the backward. !!! warning - All AOIs spatial values must be given in **centimeters**. + All AOI spatial values must be given in **centimeters**. ### Edit OBJ file description @@ -43,7 +43,7 @@ s off f 9 10 11 ``` -Here are common OBJ file features needed to describe AOIs: +Here are common OBJ file features needed to describe AOI: * Object lines (starting with *o* key) indicate AOI name. * Vertice lines (starting with *v* key) indicate AOI vertices. @@ -51,7 +51,7 @@ Here are common OBJ file features needed to describe AOIs: ### Edit JSON file description -JSON file format allows to describe AOIs vertices. +JSON file format allows to describe AOI vertices. ``` json { diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md index bdebd6c..d7df765 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md @@ -1,7 +1,7 @@ Project AOI into camera frame ============================= -Once [ArUcoScene pose is estimated](pose_estimation.md) and [3D AOIs are described](aoi_3d_description.md), AOIs can be projected into [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. +Once [ArUcoScene pose is estimated](pose_estimation.md) and [3D AOI are described](aoi_3d_description.md), AOI can be projected into [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. ![3D AOI projection](../../img/aruco_camera_aoi_projection.png) @@ -46,7 +46,7 @@ The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically The [AOIScene](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AOIScene) defines a set of 3D [AreaOfInterest](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AreaOfInterest) registered by name. -## Add ArLayer to ArUcoCamera to project 3D AOIs +## Add ArLayer to ArUcoCamera to project 3D AOI Here is the previous extract where one layer is added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) and displayed: @@ -103,11 +103,11 @@ The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers are projected into their dedicated [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layers when calling the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method. -## Add 2D AOIs analysis +## Add 2D AOI analysis When a scene layer is projected into a camera layer, it means that the 3D [ArLayer.aoi_scene](../../argaze.md/#argaze.ArFeatures.ArLayer.aoi_scene) description of the scene becomes the 2D camera's [ArLayer.aoi_scene](../../argaze.md/#argaze.ArFeatures.ArLayer.aoi_scene) description of the camera. -Therefore, it means that [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) benefits from all the services described in [2D AOIs analysis pipeline section](../gaze_analysis_pipeline/aoi_2d_analysis.md). +Therefore, it means that [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) benefits from all the services described in [2D AOI analysis pipeline section](../gaze_analysis_pipeline/aoi_analysis.md). Here is the previous extract where AOI matcher, AOI scan path and AOI scan path analyzers are added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer: diff --git a/docs/user_guide/aruco_markers_pipeline/introduction.md b/docs/user_guide/aruco_markers_pipeline/introduction.md index 917245d..dc3aa4a 100644 --- a/docs/user_guide/aruco_markers_pipeline/introduction.md +++ b/docs/user_guide/aruco_markers_pipeline/introduction.md @@ -19,7 +19,7 @@ To build your own ArUco markers pipeline, you need to know: * [How to describe scene's AOI](aoi_3d_description.md), * [How to load and execute ArUco markers pipeline](configuration_and_execution.md), * [How to estimate scene pose](pose_estimation.md), -* [How to project AOIs into camera frame](aoi_3d_projection.md), +* [How to project AOI into camera frame](aoi_3d_projection.md), * [How to define an AOI as a frame](aoi_frame.md) More advanced features are also explained like: -- cgit v1.1 From 128ca5fb2d5124784e217f3137bcde8eb0e7446c Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Tue, 26 Sep 2023 11:49:55 +0200 Subject: Harmonizing documentation concepts. --- docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md | 2 +- docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md index a2bb8d7..5a1a16e 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md @@ -1,7 +1,7 @@ Describe 3D AOI =============== -Once [ArUco markers are placed into a scene](aruco_markers_description.md), areas of interest need to be described into the same 3D referential. +Once [ArUco markers are placed into a scene](aruco_markers_description.md), [areas of interest (AOI)](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AreaOfInterest) need to be described into the same 3D referential. In the example scene, each screen is considered as an area of interest more the blue triangle area inside the top screen. diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md index d7df765..e96730a 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md @@ -44,7 +44,7 @@ The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically ### AOI Scene -The [AOIScene](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AOIScene) defines a set of 3D [AreaOfInterest](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AreaOfInterest) registered by name. +The set of 3D AOI into the layer as defined at [3D AOI description chapter](aoi_3d_description.md). ## Add ArLayer to ArUcoCamera to project 3D AOI -- cgit v1.1 From d06c56411a6a72fdd59d7da13308c20e8fbd77a2 Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Tue, 26 Sep 2023 13:04:13 +0200 Subject: Minor documenation change. --- docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md b/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md index 1c13013..b3ea2bb 100644 --- a/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md +++ b/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md @@ -7,7 +7,7 @@ Here is an example scene where markers are surrounding a multi-screen workspace ![Scene](../../img/scene.png) -## Print ArUco markers from a ArUco dictionary +## Print ArUco markers from an ArUco dictionary ArUco markers always belongs to a set of markers called ArUco markers dictionary. -- cgit v1.1 From 99a9baa109723f34af3372ed9fefc9557a592ff9 Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Tue, 26 Sep 2023 14:11:42 +0200 Subject: Indicating how to load and display optic parameters. --- .../optic_parameters_calibration.md | 58 ++++++++++++++++++++++ 1 file changed, 58 insertions(+) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md b/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md index fbe06d1..db9cb73 100644 --- a/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md +++ b/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md @@ -131,3 +131,61 @@ Below, an optic_parameters JSON file example: ] } ``` + +## Load and display optic parameters + +[ArUcoCamera.detector.optic_parameters](../../argaze.md/#argaze.ArUcoMarkers.ArUcoOpticCalibrator.OpticParameters) can be enabled thanks to a dedicated JSON entry. + +Here is an extract from a JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration file where optic parameters are loaded and displayed: + +```json +{ + "name": "My Full HD Camera", + "size": [1920, 1080], + "aruco_detector": { + "dictionary": "DICT_APRILTAG_16h5", + "marker_size": 5, + "optic_parameters": { + "rms": 0.6688921504088245, + "dimensions": [ + 1920, + 1080 + ], + "K": [ + [ + 1135.6524381415752, + 0.0, + 956.0685325355497 + ], + [ + 0.0, + 1135.9272506869524, + 560.059099810324 + ], + [ + 0.0, + 0.0, + 1.0 + ] + ], + "D": [ + 0.01655492265003404, + 0.1985524264972037, + 0.002129965902489484, + -0.0019528582922179365, + -0.5792910353639452 + ] + } + }, + ... + "image_parameters": { + ... + "draw_optic_parameters_grid": { + "width": 192, + "height": 108, + "z": 100, + "point_size": 1, + "point_color": [0, 0, 255] + } + } +``` \ No newline at end of file -- cgit v1.1 From 6598dc8ec081d56f8614d19f916731deb0bcf09f Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Tue, 26 Sep 2023 14:55:57 +0200 Subject: Hormonizing JSON entry description format. Improving ArUco markers pipeline documentation. --- .../optic_parameters_calibration.md | 4 +-- .../aruco_markers_pipeline/aoi_3d_projection.md | 36 +++++++++++++--------- .../configuration_and_execution.md | 20 ++++++------ .../aruco_markers_pipeline/pose_estimation.md | 16 ++++++---- 4 files changed, 44 insertions(+), 32 deletions(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md b/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md index db9cb73..0a7ec4d 100644 --- a/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md +++ b/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md @@ -136,11 +136,11 @@ Below, an optic_parameters JSON file example: [ArUcoCamera.detector.optic_parameters](../../argaze.md/#argaze.ArUcoMarkers.ArUcoOpticCalibrator.OpticParameters) can be enabled thanks to a dedicated JSON entry. -Here is an extract from a JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration file where optic parameters are loaded and displayed: +Here is an extract from the JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration file where optic parameters are loaded and displayed: ```json { - "name": "My Full HD Camera", + "name": "My FullHD Camera", "size": [1920, 1080], "aruco_detector": { "dictionary": "DICT_APRILTAG_16h5", diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md index e96730a..7f98aa2 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md @@ -1,13 +1,13 @@ -Project AOI into camera frame -============================= +Project 3D AOI into camera frame +================================ Once [ArUcoScene pose is estimated](pose_estimation.md) and [3D AOI are described](aoi_3d_description.md), AOI can be projected into [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. ![3D AOI projection](../../img/aruco_camera_aoi_projection.png) -## Add ArLayer to ArUcoScene to load AOI +## Add ArLayer to ArUcoScene to load 3D AOI -The [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer) class allows to load areas of interest description. An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArLayers](../../argaze.md/#argaze.ArFeatures.ArLayer). +The [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer) class allows to load 3D AOI description. Here is the previous extract where one layer is added to the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration: @@ -38,15 +38,19 @@ Here is the previous extract where one layer is added to the [ArUcoScene](../../ Now, let's understand the meaning of each JSON entry. -### "MyLayer" +### *layers* -The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically useful for visualisation purpose. +An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArLayers](../../argaze.md/#argaze.ArFeatures.ArLayer) stored by name. -### AOI Scene +### MyLayer + +The name of an [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically useful for visualisation purpose. + +### *aoi_scene* The set of 3D AOI into the layer as defined at [3D AOI description chapter](aoi_3d_description.md). -## Add ArLayer to ArUcoCamera to project 3D AOI +## Add ArLayer to ArUcoCamera to project 3D AOI into Here is the previous extract where one layer is added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) and displayed: @@ -91,9 +95,13 @@ Here is the previous extract where one layer is added to the [ArUcoCamera](../.. Now, let's understand the meaning of each JSON entry. -### "MyLayer" +### *layers* + +An [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) instance can contains multiples [ArLayers](../../argaze.md/#argaze.ArFeatures.ArLayer) stored by name. + +### MyLayer -The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically useful for visualisation purpose. +The name of an [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically useful for visualisation purpose. !!! warning "Layer name policy" @@ -103,11 +111,11 @@ The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers are projected into their dedicated [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layers when calling the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method. -## Add 2D AOI analysis +## Add AOI analysis features to ArUcoCamera -When a scene layer is projected into a camera layer, it means that the 3D [ArLayer.aoi_scene](../../argaze.md/#argaze.ArFeatures.ArLayer.aoi_scene) description of the scene becomes the 2D camera's [ArLayer.aoi_scene](../../argaze.md/#argaze.ArFeatures.ArLayer.aoi_scene) description of the camera. +When a scene layer is projected into a camera layer, it means that the 3D scene's AOI are transformed into 2D camera's AOI. -Therefore, it means that [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) benefits from all the services described in [2D AOI analysis pipeline section](../gaze_analysis_pipeline/aoi_analysis.md). +Therefore, it means that [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) benefits from all the services described in [AOI analysis pipeline section](../gaze_analysis_pipeline/aoi_analysis.md). Here is the previous extract where AOI matcher, AOI scan path and AOI scan path analyzers are added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer: @@ -156,4 +164,4 @@ Here is the previous extract where AOI matcher, AOI scan path and AOI scan path !!! warning - Adding scan path and scan path analyzers to an [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer doesn't make sense if the camera is moving. + Adding scan path and scan path analyzers to an [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer doesn't make sense as the space viewed thru camera frame doesn't necessary reflect the space the gaze is covering. diff --git a/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md index 6bf84a9..2205ed2 100644 --- a/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md +++ b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md @@ -62,15 +62,15 @@ aruco_camera = ArUcoCamera.ArUcoCamera.from_json('./configuration.json') Now, let's understand the meaning of each JSON entry. -### Name - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* +### *name - inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* The name of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. Basically useful for visualisation purpose. -### Size - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* +### *size - inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* The size of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame in pixels. Be aware that gaze positions have to be in the same range of value to be projected in. -### ArUco Detector +### *aruco_detector* The first [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) pipeline step is to detect ArUco markers inside input image and estimate their poses. @@ -81,21 +81,21 @@ The [ArUcoDetector](../../argaze.md/#argaze.ArUcoMarkers.ArUcoDetector) is in ch !!! warning "Mandatory" JSON *aruco_detector* entry is mandatory. -### Gaze Movement Identifier - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* +### *gaze_movement_identifier - inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* The first [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) pipeline step dedicated to identify fixations or saccades from consecutive timestamped gaze positions. ![Gaze movement identification](../../img/aruco_camera_gaze_movement_identification.png) -### Image parameters - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* +### *image_parameters - inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* The usual [ArFrame visualisation parameters](../gaze_analysis_pipeline/visualisation.md) plus one additional *draw_detected_markers* field. ## Pipeline execution -### Detect ArUco markers, estimate scene pose and project AOI +### Detect ArUco markers, estimate scene pose and project 3D AOI -Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole pipeline dedicated to ArUco markers detection, scene pose estimation and AOI projection. +Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole pipeline dedicated to ArUco markers detection, scene pose estimation and 3D AOI projection. ```python # Assuming that Full HD (1920x1080) video stream or file is opened @@ -107,10 +107,10 @@ Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures # Capture image from video stream of file image = video_capture.read() - # Detect ArUco markers, estimate scene pose then, project AOI into camera frame + # Detect ArUco markers, estimate scene pose then, project 3D AOI into camera frame aruco_camera.watch(image) - # Display ArUcoCamera frame image to display detected ArUco markers, scene pose, AOI projection and ArFrame visualisation. + # Display ArUcoCamera frame image to display detected ArUco markers, scene pose, 2D AOI projection and ArFrame visualisation. ... aruco_camera.image() ``` @@ -135,4 +135,4 @@ Particularly, timestamped gaze positions can be passed one by one to [ArUcoCamer At this point, the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method only detects ArUco markers and the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method only process gaze movement identification without any AOI support as no scene description is provided into the JSON configuration file. - Read the next chapters to learn [how to estimate scene pose](pose_estimation.md) and [how to project AOI](aoi_3d_projection.md). \ No newline at end of file + Read the next chapters to learn [how to estimate scene pose](pose_estimation.md) and [how to project 3D AOI](aoi_3d_projection.md). \ No newline at end of file diff --git a/docs/user_guide/aruco_markers_pipeline/pose_estimation.md b/docs/user_guide/aruco_markers_pipeline/pose_estimation.md index d7da336..5dcde6f 100644 --- a/docs/user_guide/aruco_markers_pipeline/pose_estimation.md +++ b/docs/user_guide/aruco_markers_pipeline/pose_estimation.md @@ -1,13 +1,13 @@ Estimate scene pose =================== -An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) class defines a space with [ArUco markers inside](aruco_markers_description.md) helping to estimate scene pose when they are watched by [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera). +Once [ArUco markers are placed into a scene](aruco_markers_description.md) and [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) is [configured](configuration_and_execution.md), scene pose can be estimated. ![Scene pose estimation](../../img/aruco_camera_pose_estimation.png) ## Add ArUcoScene to ArUcoCamera JSON configuration file -An [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) instance can contains multiples [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene). +An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) class defines a space with [ArUco markers inside](aruco_markers_description.md) helping to estimate scene pose when they are watched by [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera). Here is an extract from the JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration file with a sample where one scene is added and displayed: @@ -65,11 +65,15 @@ Here is an extract from the JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMark Now, let's understand the meaning of each JSON entry. -### "MyScene" +### *scenes* + +An [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) instance can contains multiples [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) stored by name. + +### MyScene -The name of the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene). Basically useful for visualisation purpose. +The name of an [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene). Basically useful for visualisation purpose. -### ArUco markers group +### *aruco_markers_group* The 3D places of ArUco markers into the scene as defined at [ArUco markers description chapter](aruco_markers_description.md). Thanks to this description, it is possible to estimate the pose of [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) in [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. @@ -77,6 +81,6 @@ The 3D places of ArUco markers into the scene as defined at [ArUco markers descr [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) pose estimation is done when calling the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method. -### Draw scenes +### *draw_scenes* The drawing parameters of each loaded [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) in [ArUcoCamera.image](../../argaze.md/#argaze.ArFeatures.ArFrame.image). -- cgit v1.1 From 611ab703f970d0403c73ea67a1b2e70fa4d6da27 Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Tue, 26 Sep 2023 16:46:47 +0200 Subject: Renaming a documentation chapter file. --- .../aruco_markers_pipeline/aoi_3d_frame.md | 102 +++++++++++++++++++++ .../user_guide/aruco_markers_pipeline/aoi_frame.md | 64 ------------- 2 files changed, 102 insertions(+), 64 deletions(-) create mode 100644 docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md delete mode 100644 docs/user_guide/aruco_markers_pipeline/aoi_frame.md (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md new file mode 100644 index 0000000..47dc73c --- /dev/null +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md @@ -0,0 +1,102 @@ +Define a 3D AOI as a frame +========================== + +When an 3D AOI of the scene contains others coplanar 3D AOI, like a screen with GUI elements displayed on, it is better to described them as 2D AOI inside 2D coordinates system related to the containing 3D AOI. + +## Add ArFrame to ArUcoScene + +The [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) class defines a rectangular area where timestamped gaze positions are projected in and inside which they need to be analyzed. + +Here is the previous extract where "MyScreen" AOI is defined as a frame into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration: + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + ... + "scenes": { + "MyScene" : { + "aruco_markers_group": { + ... + }, + "layers": { + "MyLayer": { + "aoi_scene": { + "MyScreen": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]] + } + } + }, + "frames": { + "MyScreen": { + "size": [350, 250], + "layers": { + "MyLayer": { + "aoi_scene": { + "BlueTriangle": [[100, 50], [250, 50], [175, 200]] + } + } + } + } + } + } + } + ... +} +``` +Now, let's understand the meaning of each JSON entry. + +### *frames* + +An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArFrames](../../argaze.md/#argaze.ArFeatures.ArFrame) stored by name. + +### MyScreen + +The name of a 3D AOI **and** of an [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame). Basically useful for visualisation purpose. + +!!! warning "AOI / Frame names policy" + + An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer 3D AOI is defined as an [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame, **provided they have the same name**. + +!!! warning "Layer name policy" + + An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame layer is projected into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer, **provided they have the same name**. + +!!! note + + [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame layers are projected into their dedicated [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers when the JSON configuration file is loaded. + +## Pipeline execution + +### Map ArUcoCamera image into ArUcoScene frames + +After camera image is passed to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method, it is possible to apply a perpective transformation in order to project watched image into each [ArUcoScenes](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) [frames background](../../argaze.md/#argaze.ArFeatures.ArFrame) image. + +```python +# Assuming that Full HD (1920x1080) video stream or file is opened +... + +# Assuming that the video reading is handled in a looping code block +...: + + # Capture image from video stream of file + image = video_capture.read() + + # Detect ArUco markers, estimate scene pose then, project 3D AOI into camera frame + aruco_camera.watch(image) + + # Map watched image into ArUcoScenes frames background + aruco_camera.map() +``` + +### Display each ArUcoScenes frames + +All [ArUcoScenes](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frames image can be displayed as any [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame). + +```python + ... + + # Display all ArUcoScenes frames + for frame in aruco_camera.scene_frames: + + ... frame.image() +``` \ No newline at end of file diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_frame.md b/docs/user_guide/aruco_markers_pipeline/aoi_frame.md deleted file mode 100644 index 6b87d52..0000000 --- a/docs/user_guide/aruco_markers_pipeline/aoi_frame.md +++ /dev/null @@ -1,64 +0,0 @@ -Define an AOI as a frame -======================== - - - - -## Add ArFrame to ArUcoScene - -The [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) class defines a rectangular area where timestamped gaze positions are projected in and inside which they need to be analyzed. An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArFrames](../../argaze.md/#argaze.ArFeatures.ArFrame). - -Here is the previous extract where the "GrayRectangle" AOI is defined as a frame into the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration: - -```json -{ - "name": "My FullHD camera", - "size": [1920, 1080], - ... - "scenes": { - "MyScene" : { - "aruco_markers_group": { - ... - }, - "layers": { - "MyLayer": { - "aoi_scene": { - "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], - "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]] - } - } - }, - "frames": { - "GrayRectangle": { - "size": [350, 250], - "layers": { - "MyLayer": { - "aoi_scene": { - "BlueTriangle": [[100, 50], [250, 50], [175, 200]] - } - } - } - } - } - } - } - ... -} -``` -Now, let's understand the meaning of each JSON entry. - -### "GrayRectangle" - -The name of the AOI and the [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame). Basically useful for visualisation purpose. - -!!! warning "Frame name policy" - - An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer AOI is defined as an [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame, **provided they have the same name**. - -!!! warning "Layer name policy" - - An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame layer is projected into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer, **provided they have the same name**. - -!!! note - - [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame layers are projected into their dedicated [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers when the JSON configuration file is loaded. \ No newline at end of file -- cgit v1.1 From 418952f8588457d3e0a07d9d2b9cf3f250470319 Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Tue, 26 Sep 2023 16:47:21 +0200 Subject: Updating documentation chapter changes. --- docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md | 4 ++-- docs/user_guide/aruco_markers_pipeline/introduction.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md index 7f98aa2..acbe31d 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md @@ -9,7 +9,7 @@ Once [ArUcoScene pose is estimated](pose_estimation.md) and [3D AOI are describe The [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer) class allows to load 3D AOI description. -Here is the previous extract where one layer is added to the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration: +Here is the previous extract where one layer is added to [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration: ```json { @@ -52,7 +52,7 @@ The set of 3D AOI into the layer as defined at [3D AOI description chapter](aoi_ ## Add ArLayer to ArUcoCamera to project 3D AOI into -Here is the previous extract where one layer is added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) and displayed: +Here is the previous extract where one layer is added to [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration and displayed: ```json { diff --git a/docs/user_guide/aruco_markers_pipeline/introduction.md b/docs/user_guide/aruco_markers_pipeline/introduction.md index dc3aa4a..ee424f0 100644 --- a/docs/user_guide/aruco_markers_pipeline/introduction.md +++ b/docs/user_guide/aruco_markers_pipeline/introduction.md @@ -19,8 +19,8 @@ To build your own ArUco markers pipeline, you need to know: * [How to describe scene's AOI](aoi_3d_description.md), * [How to load and execute ArUco markers pipeline](configuration_and_execution.md), * [How to estimate scene pose](pose_estimation.md), -* [How to project AOI into camera frame](aoi_3d_projection.md), -* [How to define an AOI as a frame](aoi_frame.md) +* [How to project 3D AOI into camera frame](aoi_3d_projection.md), +* [How to define a 3D AOI as a frame](aoi_3d_frame.md) More advanced features are also explained like: -- cgit v1.1 From 433e68397d3fcb7d3c9ab07e1fa0a8edc0c4e583 Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Tue, 26 Sep 2023 17:38:34 +0200 Subject: Documentating gaze analysis for ArUcoScenes frames. --- docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md index 47dc73c..8075426 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md @@ -67,7 +67,7 @@ The name of a 3D AOI **and** of an [ArFrame](../../argaze.md/#argaze.ArFeatures. ## Pipeline execution -### Map ArUcoCamera image into ArUcoScene frames +### Map ArUcoCamera image into ArUcoScenes frames After camera image is passed to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method, it is possible to apply a perpective transformation in order to project watched image into each [ArUcoScenes](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) [frames background](../../argaze.md/#argaze.ArFeatures.ArFrame) image. @@ -88,6 +88,14 @@ After camera image is passed to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFe aruco_camera.map() ``` +### Analyse timestamped gaze positions into ArUcoScenes frames + +[ArUcoScenes](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frames benefits from all the services described in [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md). + +!!! note + + Timestamped gaze positions passed to [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method are projected into [ArUcoScenes](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frames if applicable. + ### Display each ArUcoScenes frames All [ArUcoScenes](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frames image can be displayed as any [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame). -- cgit v1.1 From 6af30f98e4d8c89e4602c4048622ce5e66dbe774 Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Tue, 26 Sep 2023 17:39:10 +0200 Subject: Adding new aruco markers pipeline scripting chapter. --- .../advanced_topics/scripting.md | 132 +++++++++++++++++++++ .../aruco_markers_pipeline/introduction.md | 2 +- 2 files changed, 133 insertions(+), 1 deletion(-) create mode 100644 docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md b/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md new file mode 100644 index 0000000..c1db6d5 --- /dev/null +++ b/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md @@ -0,0 +1,132 @@ +Script the pipeline +=================== + +All aruco markers pipeline objects are accessible from Python script. +This could be particularly useful for realtime AR interaction applications. + +## Load ArUcoCamera configuration from dictionary + +First of all, [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration can be loaded from a python dictionary. + +```python +from argaze.ArUcoMarkers import ArUcoCamera + +# Edit a dict with ArUcoCamera configuration +configuration = { + "name": "My FullHD camera", + "size": (1920, 1080), + ... + "aruco_detector": { + ... + }, + "scenes": { + "MyScene" : { + "aruco_markers_group": { + ... + }, + "layers": { + "MyLayer": { + "aoi_scene": { + ... + } + }, + ... + } + }, + ... + } + "layers": { + "MyLayer": { + ... + }, + ... + }, + "image_parameters": { + ... + } +} + +# Load ArUcoCamera +aruco_camera = ArUcoCamera.ArUcoCamera.from_dict(configuration) + +# Do something with ArUcoCamera +... +``` + +## Access to ArUcoCamera and ArScenes attributes + +Then, once the configuration is loaded, it is possible to access to its attributes: [read ArUcoCamera code reference](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) to get a complete list of what is available. + +Thus, the [ArUcoCamera.scenes](../../../argaze.md/#argaze.ArFeatures.ArCamera) attribute allows to access each loaded aruco scene and so, access to their attributes: [read ArUcoScene code reference](../../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) to get a complete list of what is available. + +```python +from argaze import ArFeatures + +# Assuming the ArUcoCamera is loaded +... + +# Iterate over each ArUcoCamera scene +for name, aruco_scene in aruco_camera.scenes.items(): + ... +``` + +## Pipeline execution outputs + +[ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method returns many data about pipeline execution. + +```python +# Assuming that images are available +...: + + # Watch image with ArUco camera + detection_time, exception = aruco_camera.watch(image) + + # Do something with pipeline detection times + ... + + # Do something with pipeline exception + if exception: + ... +``` + +Let's understand the meaning of each returned data. + +### *detection_time* + +ArUco marker detection time in ms. + +### *exception* + +A [python Exception](https://docs.python.org/3/tutorial/errors.html#exceptions) object raised during pipeline execution. + +## Setup ArUcoCamera image parameters + +Specific [ArUcoCamera.image](../../../argaze.md/#argaze.ArFeatures.ArFrame.image) method parameters can be configured thanks to a python dictionary. + +```python +# Assuming ArUcoCamera is loaded +... + +# Edit a dict with ArUcoCamera image parameters +image_parameters = { + "draw_detected_markers": { + ... + }, + "draw_scenes": { + ... + }, + "draw_optic_parameters_grid": { + ... + }, + ... +} + +# Pass image parameters to ArUcoCamera +aruco_camera_image = aruco_camera.image(**image_parameters) + +# Do something with ArUcoCamera image +... +``` + +!!! note + [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) and so, benefits from all image parameters described in [gaze analysis pipeline visualisation section](../../gaze_analysis_pipeline/visualisation.md). \ No newline at end of file diff --git a/docs/user_guide/aruco_markers_pipeline/introduction.md b/docs/user_guide/aruco_markers_pipeline/introduction.md index ee424f0..f5bba18 100644 --- a/docs/user_guide/aruco_markers_pipeline/introduction.md +++ b/docs/user_guide/aruco_markers_pipeline/introduction.md @@ -24,6 +24,6 @@ To build your own ArUco markers pipeline, you need to know: More advanced features are also explained like: - +* [How to script ArUco markers pipeline](advanced_topics/scripting.md) * [How to calibrate optic parameters](advanced_topics/optic_parameters_calibration.md) -- cgit v1.1 From 06548cb7cb807f42fa42e4777288f67f259ae64a Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Tue, 26 Sep 2023 17:43:56 +0200 Subject: Fixing errors returned by mkdocs serve. --- .../advanced_topics/optic_parameters_calibration.md | 4 ++-- .../aruco_markers_pipeline/advanced_topics/scripting.md | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md b/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md index 0a7ec4d..3277216 100644 --- a/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md +++ b/docs/user_guide/aruco_markers_pipeline/advanced_topics/optic_parameters_calibration.md @@ -134,9 +134,9 @@ Below, an optic_parameters JSON file example: ## Load and display optic parameters -[ArUcoCamera.detector.optic_parameters](../../argaze.md/#argaze.ArUcoMarkers.ArUcoOpticCalibrator.OpticParameters) can be enabled thanks to a dedicated JSON entry. +[ArUcoCamera.detector.optic_parameters](../../../argaze.md/#argaze.ArUcoMarkers.ArUcoOpticCalibrator.OpticParameters) can be enabled thanks to a dedicated JSON entry. -Here is an extract from the JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration file where optic parameters are loaded and displayed: +Here is an extract from the JSON [ArUcoCamera](../../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration file where optic parameters are loaded and displayed: ```json { diff --git a/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md b/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md index c1db6d5..0b2ef52 100644 --- a/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md +++ b/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md @@ -6,7 +6,7 @@ This could be particularly useful for realtime AR interaction applications. ## Load ArUcoCamera configuration from dictionary -First of all, [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration can be loaded from a python dictionary. +First of all, [ArUcoCamera](../../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration can be loaded from a python dictionary. ```python from argaze.ArUcoMarkers import ArUcoCamera @@ -55,7 +55,7 @@ aruco_camera = ArUcoCamera.ArUcoCamera.from_dict(configuration) ## Access to ArUcoCamera and ArScenes attributes -Then, once the configuration is loaded, it is possible to access to its attributes: [read ArUcoCamera code reference](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) to get a complete list of what is available. +Then, once the configuration is loaded, it is possible to access to its attributes: [read ArUcoCamera code reference](../../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) to get a complete list of what is available. Thus, the [ArUcoCamera.scenes](../../../argaze.md/#argaze.ArFeatures.ArCamera) attribute allows to access each loaded aruco scene and so, access to their attributes: [read ArUcoScene code reference](../../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) to get a complete list of what is available. @@ -72,7 +72,7 @@ for name, aruco_scene in aruco_camera.scenes.items(): ## Pipeline execution outputs -[ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method returns many data about pipeline execution. +[ArUcoCamera.watch](../../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method returns many data about pipeline execution. ```python # Assuming that images are available @@ -129,4 +129,4 @@ aruco_camera_image = aruco_camera.image(**image_parameters) ``` !!! note - [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) and so, benefits from all image parameters described in [gaze analysis pipeline visualisation section](../../gaze_analysis_pipeline/visualisation.md). \ No newline at end of file + [ArUcoCamera](../../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../../argaze.md/#argaze.ArFeatures.ArFrame) and so, benefits from all image parameters described in [gaze analysis pipeline visualisation section](../../gaze_analysis_pipeline/visualisation.md). \ No newline at end of file -- cgit v1.1 From 1a0dc73d98fdbe0d45523ca3ac914928b0ae775a Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Wed, 27 Sep 2023 09:58:43 +0200 Subject: Adding new chapter to explain how to configure ArUco marker detection. --- .../aruco_detector_configuration.md | 39 ++++++++++++++++++++++ .../aruco_markers_pipeline/introduction.md | 2 +- 2 files changed, 40 insertions(+), 1 deletion(-) create mode 100644 docs/user_guide/aruco_markers_pipeline/advanced_topics/aruco_detector_configuration.md (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/advanced_topics/aruco_detector_configuration.md b/docs/user_guide/aruco_markers_pipeline/advanced_topics/aruco_detector_configuration.md new file mode 100644 index 0000000..98b0841 --- /dev/null +++ b/docs/user_guide/aruco_markers_pipeline/advanced_topics/aruco_detector_configuration.md @@ -0,0 +1,39 @@ +Improve ArUco markers detection +=============================== + +As explain in [OpenCV ArUco documentation](https://docs.opencv.org/4.x/d1/dcd/structcv_1_1aruco_1_1DetectorParameters.html), ArUco markers detection is highly configurable. + +## Load ArUcoDetector parameters + +[ArUcoCamera.detector.parameters](../../../argaze.md/#argaze.ArUcoMarkers.ArUcoDetector.Parameters) can be loaded thanks to a dedicated JSON entry. + +Here is an extract from the JSON [ArUcoCamera](../../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration file with ArUco detector parameters: + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + "aruco_detector": { + "dictionary": "DICT_APRILTAG_16h5", + "marker_size": 5, + "parameters": { + "cornerRefinementMethod": 3, + "aprilTagQuadSigma": 2, + "aprilTagDeglitch": 1 + } + }, + ... +``` + +## Print ArUcoDetector parameters + +```python +# Assuming ArUcoCamera is loaded +... + +# Print all ArUcoDetector parameters +print(aruco_camera.aruco_detector.parameters) + +# Print only modified ArUcoDetector parameters +print(f'{aruco_camera.aruco_detector.parameters:modified}') +``` diff --git a/docs/user_guide/aruco_markers_pipeline/introduction.md b/docs/user_guide/aruco_markers_pipeline/introduction.md index f5bba18..5a07b49 100644 --- a/docs/user_guide/aruco_markers_pipeline/introduction.md +++ b/docs/user_guide/aruco_markers_pipeline/introduction.md @@ -26,4 +26,4 @@ More advanced features are also explained like: * [How to script ArUco markers pipeline](advanced_topics/scripting.md) * [How to calibrate optic parameters](advanced_topics/optic_parameters_calibration.md) - +* [How to improve ArUco markers detection](advanced_topics/aruco_detector_configuration.md) -- cgit v1.1 From fbf4c80b9e7dabb6e2bbcb94df44e627de5646dc Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Wed, 27 Sep 2023 18:02:34 +0200 Subject: Updating illustrations. --- .../aruco_markers_pipeline/aoi_3d_description.md | 33 +++++++++------------- .../aruco_markers_pipeline/aoi_3d_frame.md | 30 ++++++++++++++++---- .../aruco_markers_pipeline/aoi_3d_projection.md | 5 ++-- .../aruco_markers_description.md | 30 ++++++++++---------- .../aruco_markers_pipeline/introduction.md | 6 ++-- .../aruco_markers_pipeline/pose_estimation.md | 8 +++--- 6 files changed, 61 insertions(+), 51 deletions(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md index 5a1a16e..502f905 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md @@ -3,7 +3,7 @@ Describe 3D AOI Once [ArUco markers are placed into a scene](aruco_markers_description.md), [areas of interest (AOI)](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AreaOfInterest) need to be described into the same 3D referential. -In the example scene, each screen is considered as an area of interest more the blue triangle area inside the top screen. +In the example scene, the screen and the sheet are considered as areas of interest. ![3D AOI description](../../img/aoi_3d_description.png) @@ -21,26 +21,20 @@ All AOI need to be described from same origin than markers in a [right-handed 3D OBJ file format could be exported from most 3D editors. ``` obj -o YellowSquare -v 6.200003 -7.275252 25.246159 -v 31.200003 -7.275252 25.246159 -v 6.200003 1.275252 1.753843 -v 31.200003 1.275252 1.753843 +o Sheet +v 14.200000 -3.000000 28.350000 +v 35.200000 -3.000000 28.350000 +v 14.200000 -3.000000 -1.35 +v 35.200000 -3.000000 -1.35 s off f 1 2 4 3 -o GrayRectangle -v 2.500000 2.500000 -0.500000 -v 37.500000 2.500000 -0.500000 -v 2.500000 27.500000 -0.500000 -v 37.500000 27.500000 -0.500000 +o Screen +v 2.750000 2.900000 -0.500000 +v 49.250000 2.900000 -0.500000 +v 2.750000 29.100000 -0.500000 +v 49.250000 29.100000 -0.500000 s off f 5 6 8 7 -o BlueTriangle -v 12.500002 7.500000 -0.500000 -v 27.500002 7.500000 -0.500000 -v 20.000002 22.500000 -0.500000 -s off -f 9 10 11 ``` Here are common OBJ file features needed to describe AOI: @@ -55,8 +49,7 @@ JSON file format allows to describe AOI vertices. ``` json { - "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], - "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]], - "BlueTriangle": [[12.5, 7.5, -0.5], [27.5, 7.5, -0.5], [20, 22.5, -0.5]] + "Sheet": [[14.2, -3, 28.35], [35.2, -3, 28.35], [14.2, -3, -1.35], [35.2, -3, -1.35]], + "Screen": [[2.75, 2.9, -0.5], [49.25, 2.9, -0.5], [2.75, 29.1, -0.5], [49.25, 29.1, -0.5]] } ``` diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md index 8075426..032e2b6 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md @@ -3,11 +3,13 @@ Define a 3D AOI as a frame When an 3D AOI of the scene contains others coplanar 3D AOI, like a screen with GUI elements displayed on, it is better to described them as 2D AOI inside 2D coordinates system related to the containing 3D AOI. +![3D AOI frame](../../img/aruco_camera_aoi_frame.png) + ## Add ArFrame to ArUcoScene The [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) class defines a rectangular area where timestamped gaze positions are projected in and inside which they need to be analyzed. -Here is the previous extract where "MyScreen" AOI is defined as a frame into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration: +Here is the previous extract where "Screen" AOI is defined as a frame into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration: ```json { @@ -22,18 +24,34 @@ Here is the previous extract where "MyScreen" AOI is defined as a frame into [Ar "layers": { "MyLayer": { "aoi_scene": { - "MyScreen": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]] + "Sheet": [[14.2, -3, 28.35], [35.2, -3, 28.35], [14.2, -3, -1.35], [35.2, -3, -1.35]], + "Screen": [[2.75, 2.9, -0.5], [49.25, 2.9, -0.5], [2.75, 29.1, -0.5], [49.25, 29.1, -0.5]] } } }, "frames": { - "MyScreen": { - "size": [350, 250], + "Screen": { + "size": [1920, 1080], "layers": { "MyLayer": { "aoi_scene": { - "BlueTriangle": [[100, 50], [250, 50], [175, 200]] - } + "GeoSector": [[860, 160], [1380, 100], [1660, 400], [1380, 740], [1440, 960], [920, 920], [680, 800], [640, 560]], + "LeftPanel": { + "Rectangle": { + "x": 0, + "y": 0, + "width": 350, + "height": 1080 + } + }, + "CircularWidget": { + "Circle": { + "cx": 1800, + "cy": 120, + "radius": 80 + } + } + } } } } diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md index acbe31d..0d58d9a 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md @@ -24,9 +24,8 @@ Here is the previous extract where one layer is added to [ArUcoScene](../../arga "layers": { "MyLayer": { "aoi_scene": { - "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], - "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]], - "BlueTriangle": [[12.5, 7.5, -0.5], [27.5, 7.5, -0.5], [20, 22.5, -0.5]] + "Sheet": [[14.2, -3, 28.35], [35.2, -3, 28.35], [14.2, -3, -1.35], [35.2, -3, -1.35]], + "Screen": [[2.75, 2.9, -0.5], [49.25, 2.9, -0.5], [2.75, 29.1, -0.5], [49.25, 29.1, -0.5]] } } } diff --git a/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md b/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md index b3ea2bb..3addcab 100644 --- a/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md +++ b/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md @@ -3,7 +3,7 @@ Set up ArUco markers First of all, ArUco markers needs to be printed and placed into the scene. -Here is an example scene where markers are surrounding a multi-screen workspace with a triangle area inside one of them. +Here is an example scene where markers are surrounding a workspace with a screen and a sheet on the table. ![Scene](../../img/scene.png) @@ -69,19 +69,19 @@ vn 0.0000 0.0000 1.0000 s off f 1//1 2//1 4//1 3//1 o DICT_APRILTAG_16h5#1_Marker -v -1.767767 23.000002 3.767767 -v 1.767767 23.000002 0.232233 -v -1.767767 28.000002 3.767767 -v 1.767767 28.000002 0.232233 -vn 0.7071 0.0000 0.7071 +v -0.855050 24.000002 4.349232 +v 0.855050 24.000002 -0.349231 +v -0.855050 29.000002 4.349232 +v 0.855050 29.000002 -0.349231 +vn 0.9397 0.0000 0.3420 s off f 5//2 6//2 8//2 7//2 o DICT_APRILTAG_16h5#2_Marker -v 33.000000 -1.767767 4.767767 -v 38.000000 -1.767767 4.767767 -v 33.000000 1.767767 1.232233 -v 38.000000 1.767767 1.232233 -vn 0.0000 0.7071 0.7071 +v 44.000000 0.000000 9.500000 +v 49.000000 0.000000 9.500000 +v 44.000000 -0.000000 4.500000 +v 49.000000 -0.000000 4.500000 +vn 0.0000 1.0000 -0.0000 s off f 9//3 10//3 12//3 11//3 ``` @@ -110,12 +110,12 @@ JSON file format allows to describe markers places using translation and euler a "rotation": [0, 0, 0] }, "1": { - "translation": [0, 25.5, 2], - "rotation": [0, 45, 0] + "translation": [0, 26.5, 2], + "rotation": [0, 70, 0] }, "2": { - "translation": [35.5, 0, 3], - "rotation": [-45, 0, 0] + "translation": [46.5, 0, 7], + "rotation": [-90, 0, 0] } } } diff --git a/docs/user_guide/aruco_markers_pipeline/introduction.md b/docs/user_guide/aruco_markers_pipeline/introduction.md index 5a07b49..26294f7 100644 --- a/docs/user_guide/aruco_markers_pipeline/introduction.md +++ b/docs/user_guide/aruco_markers_pipeline/introduction.md @@ -1,11 +1,11 @@ Overview ======== -This section explains how to build augmented reality pipelines based on ArUco Markers technology for various use cases. +This section explains how to build augmented reality pipelines based on [ArUco Markers technology](https://www.sciencedirect.com/science/article/abs/pii/S0031320314000235) for various use cases. -The OpenCV library provides a module to detect fiducial markers into a picture and estimate their poses (cf [OpenCV ArUco tutorial page](https://docs.opencv.org/4.x/d5/dae/tutorial_aruco_detection.html)). +The OpenCV library provides a module to detect fiducial markers into a picture and estimate their poses. -![OpenCV ArUco markers](https://pyimagesearch.com/wp-content/uploads/2020/12/aruco_generate_tags_header.png) +![OpenCV ArUco markers](../../img/opencv_aruco.png) The ArGaze [ArUcoMarkers submodule](../../argaze.md/#argaze.ArUcoMarkers) eases markers creation, markers detection and 3D scene pose estimation through a set of high level classes. diff --git a/docs/user_guide/aruco_markers_pipeline/pose_estimation.md b/docs/user_guide/aruco_markers_pipeline/pose_estimation.md index 5dcde6f..6027039 100644 --- a/docs/user_guide/aruco_markers_pipeline/pose_estimation.md +++ b/docs/user_guide/aruco_markers_pipeline/pose_estimation.md @@ -27,12 +27,12 @@ Here is an extract from the JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMark "rotation": [0, 0, 0] }, "1": { - "translation": [0, 25.5, 2], - "rotation": [0, 45, 0] + "translation": [0, 26.5, 2], + "rotation": [0, 70, 0] }, "2": { - "translation": [35.5, 0, 3], - "rotation": [-45, 0, 0] + "translation": [46.5, 0, 7], + "rotation": [-90, 0, 0] } } } -- cgit v1.1 From 0ab16a1a93ba62e791489f453d4ec20b7b5655bb Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Wed, 27 Sep 2023 23:22:47 +0200 Subject: Using Screen name instead of MyScreen --- docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md index 032e2b6..8affe91 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md @@ -67,7 +67,7 @@ Now, let's understand the meaning of each JSON entry. An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArFrames](../../argaze.md/#argaze.ArFeatures.ArFrame) stored by name. -### MyScreen +### Screen The name of a 3D AOI **and** of an [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame). Basically useful for visualisation purpose. -- cgit v1.1 From c12e429190b4f63064c81edfa08fb00b8ed8a28c Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Thu, 28 Sep 2023 23:31:33 +0200 Subject: Improving documentation details. --- .../aruco_markers_pipeline/advanced_topics/scripting.md | 4 ++-- .../aruco_markers_pipeline/aoi_3d_description.md | 4 +--- docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md | 2 +- .../user_guide/aruco_markers_pipeline/aoi_3d_projection.md | 4 ++-- .../aruco_markers_pipeline/aruco_markers_description.md | 5 +---- .../aruco_markers_pipeline/configuration_and_execution.md | 14 +++++++------- docs/user_guide/aruco_markers_pipeline/introduction.md | 4 ++-- 7 files changed, 16 insertions(+), 21 deletions(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md b/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md index 0b2ef52..529bff8 100644 --- a/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md +++ b/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md @@ -72,7 +72,7 @@ for name, aruco_scene in aruco_camera.scenes.items(): ## Pipeline execution outputs -[ArUcoCamera.watch](../../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method returns many data about pipeline execution. +[ArUcoCamera.watch](../../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method returns data about pipeline execution. ```python # Assuming that images are available @@ -81,7 +81,7 @@ for name, aruco_scene in aruco_camera.scenes.items(): # Watch image with ArUco camera detection_time, exception = aruco_camera.watch(image) - # Do something with pipeline detection times + # Do something with pipeline detection time ... # Do something with pipeline exception diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md index 502f905..b02bc9e 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md @@ -1,7 +1,7 @@ Describe 3D AOI =============== -Once [ArUco markers are placed into a scene](aruco_markers_description.md), [areas of interest (AOI)](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AreaOfInterest) need to be described into the same 3D referential. +Now [scene pose is estimated](aruco_markers_description.md) thanks to ArUco markers description, [areas of interest (AOI)](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AreaOfInterest) need to be described into the same 3D referential. In the example scene, the screen and the sheet are considered as areas of interest. @@ -26,14 +26,12 @@ v 14.200000 -3.000000 28.350000 v 35.200000 -3.000000 28.350000 v 14.200000 -3.000000 -1.35 v 35.200000 -3.000000 -1.35 -s off f 1 2 4 3 o Screen v 2.750000 2.900000 -0.500000 v 49.250000 2.900000 -0.500000 v 2.750000 29.100000 -0.500000 v 49.250000 29.100000 -0.500000 -s off f 5 6 8 7 ``` diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md index 8affe91..f1ae1f6 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_frame.md @@ -69,7 +69,7 @@ An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can co ### Screen -The name of a 3D AOI **and** of an [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame). Basically useful for visualisation purpose. +The name of a 3D AOI **and** an [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame). Basically useful for visualisation purpose. !!! warning "AOI / Frame names policy" diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md index 0d58d9a..8c7310b 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md @@ -96,7 +96,7 @@ Now, let's understand the meaning of each JSON entry. ### *layers* -An [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) instance can contains multiples [ArLayers](../../argaze.md/#argaze.ArFeatures.ArLayer) stored by name. +An [ArUcoCamera](../../argaze.md/#argaze.ArFeatures.ArFrame) instance can contains multiples [ArLayers](../../argaze.md/#argaze.ArFeatures.ArLayer) stored by name. ### MyLayer @@ -110,7 +110,7 @@ The name of an [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers are projected into their dedicated [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layers when calling the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method. -## Add AOI analysis features to ArUcoCamera +## Add AOI analysis features to ArUcoCamera layer When a scene layer is projected into a camera layer, it means that the 3D scene's AOI are transformed into 2D camera's AOI. diff --git a/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md b/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md index 3addcab..8104345 100644 --- a/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md +++ b/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md @@ -3,7 +3,7 @@ Set up ArUco markers First of all, ArUco markers needs to be printed and placed into the scene. -Here is an example scene where markers are surrounding a workspace with a screen and a sheet on the table. +Here is an example scene where markers are surrounding a workspace with a screen and a sheet on the table (considering the sheet stays static for the moment). ![Scene](../../img/scene.png) @@ -66,7 +66,6 @@ v 5.000000 0.000000 0.000000 v 0.000000 5.000000 0.000000 v 5.000000 5.000000 0.000000 vn 0.0000 0.0000 1.0000 -s off f 1//1 2//1 4//1 3//1 o DICT_APRILTAG_16h5#1_Marker v -0.855050 24.000002 4.349232 @@ -74,7 +73,6 @@ v 0.855050 24.000002 -0.349231 v -0.855050 29.000002 4.349232 v 0.855050 29.000002 -0.349231 vn 0.9397 0.0000 0.3420 -s off f 5//2 6//2 8//2 7//2 o DICT_APRILTAG_16h5#2_Marker v 44.000000 0.000000 9.500000 @@ -82,7 +80,6 @@ v 49.000000 0.000000 9.500000 v 44.000000 -0.000000 4.500000 v 49.000000 -0.000000 4.500000 vn 0.0000 1.0000 -0.0000 -s off f 9//3 10//3 12//3 11//3 ``` diff --git a/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md index 2205ed2..3bded3a 100644 --- a/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md +++ b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md @@ -29,6 +29,12 @@ Here is a simple JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCa }, "image_parameters": { "background_weight": 1, + "draw_detected_markers": { + "color": [0, 255, 0], + "draw_axes": { + "thickness": 3 + } + }, "draw_gaze_positions": { "color": [0, 255, 255], "size": 2 @@ -40,12 +46,6 @@ Here is a simple JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCa }, "draw_saccades": { "line_color": [255, 0, 255] - }, - "draw_detected_markers": { - "color": [0, 255, 0], - "draw_axes": { - "thickness": 3 - } } } } @@ -135,4 +135,4 @@ Particularly, timestamped gaze positions can be passed one by one to [ArUcoCamer At this point, the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method only detects ArUco markers and the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method only process gaze movement identification without any AOI support as no scene description is provided into the JSON configuration file. - Read the next chapters to learn [how to estimate scene pose](pose_estimation.md) and [how to project 3D AOI](aoi_3d_projection.md). \ No newline at end of file + Read the next chapters to learn [how to estimate scene pose](pose_estimation.md), [how to describe 3D scene's AOI](aoi_3d_description.md) and [how to project them into camera frame](aoi_3d_projection.md). \ No newline at end of file diff --git a/docs/user_guide/aruco_markers_pipeline/introduction.md b/docs/user_guide/aruco_markers_pipeline/introduction.md index 26294f7..a83da9a 100644 --- a/docs/user_guide/aruco_markers_pipeline/introduction.md +++ b/docs/user_guide/aruco_markers_pipeline/introduction.md @@ -9,16 +9,16 @@ The OpenCV library provides a module to detect fiducial markers into a picture a The ArGaze [ArUcoMarkers submodule](../../argaze.md/#argaze.ArUcoMarkers) eases markers creation, markers detection and 3D scene pose estimation through a set of high level classes. -First, let's look at the schema below: it gives an overview of the main notions involved in the following chapters. + To build your own ArUco markers pipeline, you need to know: * [How to setup ArUco markers into a scene](aruco_markers_description.md), -* [How to describe scene's AOI](aoi_3d_description.md), * [How to load and execute ArUco markers pipeline](configuration_and_execution.md), * [How to estimate scene pose](pose_estimation.md), +* [How to describe scene's AOI](aoi_3d_description.md), * [How to project 3D AOI into camera frame](aoi_3d_projection.md), * [How to define a 3D AOI as a frame](aoi_3d_frame.md) -- cgit v1.1 From 8abbb34fb00e63f29d00361b8cc3a65c12aea0cd Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Wed, 4 Oct 2023 13:38:28 +0200 Subject: Adding useAruco3Detection parameter in documentation. --- .../advanced_topics/aruco_detector_configuration.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/advanced_topics/aruco_detector_configuration.md b/docs/user_guide/aruco_markers_pipeline/advanced_topics/aruco_detector_configuration.md index 98b0841..f5b66c6 100644 --- a/docs/user_guide/aruco_markers_pipeline/advanced_topics/aruco_detector_configuration.md +++ b/docs/user_guide/aruco_markers_pipeline/advanced_topics/aruco_detector_configuration.md @@ -19,7 +19,8 @@ Here is an extract from the JSON [ArUcoCamera](../../../argaze.md/#argaze.ArUcoM "parameters": { "cornerRefinementMethod": 3, "aprilTagQuadSigma": 2, - "aprilTagDeglitch": 1 + "aprilTagDeglitch": 1, + "useAruco3Detection": 1 } }, ... -- cgit v1.1 From 8bb3cec466ace640c27b41106cac7f6a09dfcdbd Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Thu, 5 Oct 2023 22:07:12 +0200 Subject: Returning proejction time. --- .../aruco_markers_pipeline/advanced_topics/scripting.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md b/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md index 529bff8..892d6dd 100644 --- a/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md +++ b/docs/user_guide/aruco_markers_pipeline/advanced_topics/scripting.md @@ -79,9 +79,9 @@ for name, aruco_scene in aruco_camera.scenes.items(): ...: # Watch image with ArUco camera - detection_time, exception = aruco_camera.watch(image) + detection_time, projection_time, exception = aruco_camera.watch(image) - # Do something with pipeline detection time + # Do something with pipeline times ... # Do something with pipeline exception @@ -95,6 +95,10 @@ Let's understand the meaning of each returned data. ArUco marker detection time in ms. +### *projection_time* + +Scenes projection time in ms. + ### *exception* A [python Exception](https://docs.python.org/3/tutorial/errors.html#exceptions) object raised during pipeline execution. -- cgit v1.1 From cbf0dc4a328763970aa41e8d5081473cca7411a9 Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Wed, 11 Oct 2023 07:36:14 +0200 Subject: Improving documentation. --- docs/user_guide/aruco_markers_pipeline/introduction.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/introduction.md b/docs/user_guide/aruco_markers_pipeline/introduction.md index a83da9a..37ab055 100644 --- a/docs/user_guide/aruco_markers_pipeline/introduction.md +++ b/docs/user_guide/aruco_markers_pipeline/introduction.md @@ -20,10 +20,10 @@ To build your own ArUco markers pipeline, you need to know: * [How to estimate scene pose](pose_estimation.md), * [How to describe scene's AOI](aoi_3d_description.md), * [How to project 3D AOI into camera frame](aoi_3d_projection.md), -* [How to define a 3D AOI as a frame](aoi_3d_frame.md) +* [How to define a 3D AOI as a frame](aoi_3d_frame.md). More advanced features are also explained like: -* [How to script ArUco markers pipeline](advanced_topics/scripting.md) -* [How to calibrate optic parameters](advanced_topics/optic_parameters_calibration.md) -* [How to improve ArUco markers detection](advanced_topics/aruco_detector_configuration.md) +* [How to script ArUco markers pipeline](advanced_topics/scripting.md), +* [How to calibrate optic parameters](advanced_topics/optic_parameters_calibration.md), +* [How to improve ArUco markers detection](advanced_topics/aruco_detector_configuration.md). -- cgit v1.1 From d9dc8fc6f542c1ba46cba9d66a741890f946474a Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Mon, 23 Oct 2023 14:34:10 +0200 Subject: Removing draw_places_axes image parameter. Fixing aruco markers group drawing. --- docs/user_guide/aruco_markers_pipeline/pose_estimation.md | 4 ---- 1 file changed, 4 deletions(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/pose_estimation.md b/docs/user_guide/aruco_markers_pipeline/pose_estimation.md index 6027039..6b58b24 100644 --- a/docs/user_guide/aruco_markers_pipeline/pose_estimation.md +++ b/docs/user_guide/aruco_markers_pipeline/pose_estimation.md @@ -51,10 +51,6 @@ Here is an extract from the JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMark "draw_places": { "color": [0, 0, 0], "border_size": 1 - }, - "draw_places_axes": { - "thickness": 1, - "length": 2.5 } } } -- cgit v1.1 From ecfbc519e518948078b0831ccd0fa8bb18e439cb Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Mon, 23 Oct 2023 15:00:17 +0200 Subject: Removing parsing of normals for ArUcoMarkersGroup. --- docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md | 4 ---- 1 file changed, 4 deletions(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md b/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md index 8104345..6380f88 100644 --- a/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md +++ b/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md @@ -65,21 +65,18 @@ v 0.000000 0.000000 0.000000 v 5.000000 0.000000 0.000000 v 0.000000 5.000000 0.000000 v 5.000000 5.000000 0.000000 -vn 0.0000 0.0000 1.0000 f 1//1 2//1 4//1 3//1 o DICT_APRILTAG_16h5#1_Marker v -0.855050 24.000002 4.349232 v 0.855050 24.000002 -0.349231 v -0.855050 29.000002 4.349232 v 0.855050 29.000002 -0.349231 -vn 0.9397 0.0000 0.3420 f 5//2 6//2 8//2 7//2 o DICT_APRILTAG_16h5#2_Marker v 44.000000 0.000000 9.500000 v 49.000000 0.000000 9.500000 v 44.000000 -0.000000 4.500000 v 49.000000 -0.000000 4.500000 -vn 0.0000 1.0000 -0.0000 f 9//3 10//3 12//3 11//3 ``` @@ -87,7 +84,6 @@ Here are common OBJ file features needed to describe ArUco markers places: * Object lines (starting with *o* key) indicate markers dictionary and id by following this format: **DICTIONARY**#**ID**\_Marker. * Vertice lines (starting with *v* key) indicate markers corners. The marker size will be automatically deducted from the geometry. -* Plane normals (starting with *vn* key) need to be exported for further pose estimation. * Face (starting with *f* key) link vertices and normals indexes together. !!! warning -- cgit v1.1 From 0ad125b331799aab99c138bbe71e5576a2e7271c Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Mon, 23 Oct 2023 19:39:55 +0200 Subject: Removing a useless warning. --- docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md | 3 --- 1 file changed, 3 deletions(-) (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md index 3bded3a..60a1115 100644 --- a/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md +++ b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md @@ -114,9 +114,6 @@ Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures ... aruco_camera.image() ``` -!!! warning "Pose estimation error" - ArUco markers pose estimation algorithm can lead to errors due to geometric ambiguities as explain in [this article](https://ieeexplore.ieee.org/document/1717461). To discard such ambiguous cases, markers should **as less as possible be parallel to camera plan**. - ### Analyse timestamped gaze positions into camera frame As mentioned above, [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) and so, benefits from all the services described in [gaze analysis pipeline section](../gaze_analysis_pipeline/introduction.md). -- cgit v1.1