From 8f2b87bfec622dd32e90d9bfa17dfcda42add4fe Mon Sep 17 00:00:00 2001 From: Theo De La Hogue Date: Mon, 25 Sep 2023 14:45:52 +0200 Subject: Improving AOI description documentation. --- .../aruco_markers_pipeline/aoi_3d_description.md | 62 ++++++++ .../aruco_markers_pipeline/aoi_3d_projection.md | 159 +++++++++++++++++++++ .../aruco_markers_pipeline/aoi_description.md | 62 -------- .../user_guide/aruco_markers_pipeline/aoi_frame.md | 64 +++++++++ .../aruco_markers_pipeline/aoi_projection.md | 159 --------------------- .../configuration_and_execution.md | 2 +- .../aruco_markers_pipeline/introduction.md | 4 +- 7 files changed, 288 insertions(+), 224 deletions(-) create mode 100644 docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md create mode 100644 docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md delete mode 100644 docs/user_guide/aruco_markers_pipeline/aoi_description.md create mode 100644 docs/user_guide/aruco_markers_pipeline/aoi_frame.md delete mode 100644 docs/user_guide/aruco_markers_pipeline/aoi_projection.md (limited to 'docs/user_guide/aruco_markers_pipeline') diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md new file mode 100644 index 0000000..13f9c86 --- /dev/null +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_description.md @@ -0,0 +1,62 @@ +Describe 3D AOI +=============== + +Once [ArUco markers are placed into a scene](aruco_markers_description.md), areas of interest need to be described into the same 3D referential. + +In the example scene, each screen is considered as an area of interest more the blue triangle area inside the top screen. + +![3D AOI description](../../img/aoi_3d_description.png) + +All AOIs need to be described from same origin than markers in a [right-handed 3D axis](https://robotacademy.net.au/lesson/right-handed-3d-coordinate-frame/) where: + +* +X is pointing to the right, +* +Y is pointing to the top, +* +Z is pointing to the backward. + +!!! warning + All AOIs spatial values must be given in **centimeters**. + +### Edit OBJ file description + +OBJ file format could be exported from most 3D editors. + +``` obj +o YellowSquare +v 6.200003 -7.275252 25.246159 +v 31.200003 -7.275252 25.246159 +v 6.200003 1.275252 1.753843 +v 31.200003 1.275252 1.753843 +s off +f 1 2 4 3 +o GrayRectangle +v 2.500000 2.500000 -0.500000 +v 37.500000 2.500000 -0.500000 +v 2.500000 27.500000 -0.500000 +v 37.500000 27.500000 -0.500000 +s off +f 5 6 8 7 +o BlueTriangle +v 12.500002 7.500000 -0.500000 +v 27.500002 7.500000 -0.500000 +v 20.000002 22.500000 -0.500000 +s off +f 9 10 11 +``` + +Here are common OBJ file features needed to describe AOIs: + +* Object lines (starting with *o* key) indicate AOI name. +* Vertice lines (starting with *v* key) indicate AOI vertices. +* Face (starting with *f* key) link vertices together. + +### Edit JSON file description + +JSON file format allows to describe AOIs vertices. + +``` json +{ + "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], + "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]], + "BlueTriangle": [[12.5, 7.5, -0.5], [27.5, 7.5, -0.5], [20, 22.5, -0.5]] +} +``` diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md new file mode 100644 index 0000000..bdebd6c --- /dev/null +++ b/docs/user_guide/aruco_markers_pipeline/aoi_3d_projection.md @@ -0,0 +1,159 @@ +Project AOI into camera frame +============================= + +Once [ArUcoScene pose is estimated](pose_estimation.md) and [3D AOIs are described](aoi_3d_description.md), AOIs can be projected into [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. + +![3D AOI projection](../../img/aruco_camera_aoi_projection.png) + +## Add ArLayer to ArUcoScene to load AOI + +The [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer) class allows to load areas of interest description. An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArLayers](../../argaze.md/#argaze.ArFeatures.ArLayer). + +Here is the previous extract where one layer is added to the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration: + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + ... + "scenes": { + "MyScene" : { + "aruco_markers_group": { + ... + }, + "layers": { + "MyLayer": { + "aoi_scene": { + "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], + "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]], + "BlueTriangle": [[12.5, 7.5, -0.5], [27.5, 7.5, -0.5], [20, 22.5, -0.5]] + } + } + } + } + } + ... +} +``` + +Now, let's understand the meaning of each JSON entry. + +### "MyLayer" + +The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically useful for visualisation purpose. + +### AOI Scene + +The [AOIScene](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AOIScene) defines a set of 3D [AreaOfInterest](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AreaOfInterest) registered by name. + +## Add ArLayer to ArUcoCamera to project 3D AOIs + +Here is the previous extract where one layer is added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) and displayed: + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + ... + "scenes": { + "MyScene" : { + "aruco_markers_group": { + ... + }, + "layers": { + "MyLayer": { + "aoi_scene": { + ... + } + } + } + } + }, + "layers": { + "MyLayer": {} + } + ... + "image_parameters": { + ... + "draw_layers": { + "MyLayer": { + "draw_aoi_scene": { + "draw_aoi": { + "color": [255, 255, 255], + "border_size": 1 + } + } + } + } + } +} +``` + +Now, let's understand the meaning of each JSON entry. + +### "MyLayer" + +The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically useful for visualisation purpose. + +!!! warning "Layer name policy" + + An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer is projected into an [ ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer, **provided they have the same name**. + +!!! note + + [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers are projected into their dedicated [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layers when calling the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method. + +## Add 2D AOIs analysis + +When a scene layer is projected into a camera layer, it means that the 3D [ArLayer.aoi_scene](../../argaze.md/#argaze.ArFeatures.ArLayer.aoi_scene) description of the scene becomes the 2D camera's [ArLayer.aoi_scene](../../argaze.md/#argaze.ArFeatures.ArLayer.aoi_scene) description of the camera. + +Therefore, it means that [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) benefits from all the services described in [2D AOIs analysis pipeline section](../gaze_analysis_pipeline/aoi_2d_analysis.md). + +Here is the previous extract where AOI matcher, AOI scan path and AOI scan path analyzers are added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer: + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + ... + "scenes": { + "MyScene" : { + "aruco_markers_group": { + ... + }, + "layers": { + "MyLayer": { + "aoi_scene": { + ... + } + } + } + } + }, + "layers": { + "MyLayer": { + "aoi_matcher": { + "DeviationCircleCoverage": { + "coverage_threshold": 0.5 + } + }, + "aoi_scan_path": { + "duration_max": 30000 + }, + "aoi_scan_path_analyzers": { + "Basic": {}, + "TransitionMatrix": {}, + "NGram": { + "n_min": 3, + "n_max": 5 + } + } + } + } + ... +} +``` + +!!! warning + + Adding scan path and scan path analyzers to an [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer doesn't make sense if the camera is moving. diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_description.md b/docs/user_guide/aruco_markers_pipeline/aoi_description.md deleted file mode 100644 index 101ec9f..0000000 --- a/docs/user_guide/aruco_markers_pipeline/aoi_description.md +++ /dev/null @@ -1,62 +0,0 @@ -Describe AOI -============ - -Once [ArUco markers are placed into a scene](aruco_markers_description.md), areas of interest need to be described into the same 3D referential. - -In the example scene, each screen is considered as an area of interest more the blue triangle area inside the top screen. - -![AOI description](../../img/aoi_description.png) - -All AOIs need to be described from same origin than markers in a [right-handed 3D axis](https://robotacademy.net.au/lesson/right-handed-3d-coordinate-frame/) where: - -* +X is pointing to the right, -* +Y is pointing to the top, -* +Z is pointing to the backward. - -!!! warning - All AOIs spatial values must be given in **centimeters**. - -### Edit OBJ file description - -OBJ file format could be exported from most 3D editors. - -``` obj -o YellowSquare -v 6.200003 -7.275252 25.246159 -v 31.200003 -7.275252 25.246159 -v 6.200003 1.275252 1.753843 -v 31.200003 1.275252 1.753843 -s off -f 1 2 4 3 -o GrayRectangle -v 2.500000 2.500000 -0.500000 -v 37.500000 2.500000 -0.500000 -v 2.500000 27.500000 -0.500000 -v 37.500000 27.500000 -0.500000 -s off -f 5 6 8 7 -o BlueTriangle -v 12.500002 7.500000 -0.500000 -v 27.500002 7.500000 -0.500000 -v 20.000002 22.500000 -0.500000 -s off -f 9 10 11 -``` - -Here are common OBJ file features needed to describe AOIs: - -* Object lines (starting with *o* key) indicate AOI name. -* Vertice lines (starting with *v* key) indicate AOI vertices. -* Face (starting with *f* key) link vertices together. - -### Edit JSON file description - -JSON file format allows to describe AOIs vertices. - -``` json -{ - "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], - "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]], - "BlueTriangle": [[12.5, 7.5, -0.5], [27.5, 7.5, -0.5], [20, 22.5, -0.5]] -} -``` diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_frame.md b/docs/user_guide/aruco_markers_pipeline/aoi_frame.md new file mode 100644 index 0000000..6b87d52 --- /dev/null +++ b/docs/user_guide/aruco_markers_pipeline/aoi_frame.md @@ -0,0 +1,64 @@ +Define an AOI as a frame +======================== + + + + +## Add ArFrame to ArUcoScene + +The [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) class defines a rectangular area where timestamped gaze positions are projected in and inside which they need to be analyzed. An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArFrames](../../argaze.md/#argaze.ArFeatures.ArFrame). + +Here is the previous extract where the "GrayRectangle" AOI is defined as a frame into the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration: + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + ... + "scenes": { + "MyScene" : { + "aruco_markers_group": { + ... + }, + "layers": { + "MyLayer": { + "aoi_scene": { + "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], + "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]] + } + } + }, + "frames": { + "GrayRectangle": { + "size": [350, 250], + "layers": { + "MyLayer": { + "aoi_scene": { + "BlueTriangle": [[100, 50], [250, 50], [175, 200]] + } + } + } + } + } + } + } + ... +} +``` +Now, let's understand the meaning of each JSON entry. + +### "GrayRectangle" + +The name of the AOI and the [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame). Basically useful for visualisation purpose. + +!!! warning "Frame name policy" + + An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer AOI is defined as an [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame, **provided they have the same name**. + +!!! warning "Layer name policy" + + An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame layer is projected into [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer, **provided they have the same name**. + +!!! note + + [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) frame layers are projected into their dedicated [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers when the JSON configuration file is loaded. \ No newline at end of file diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_projection.md b/docs/user_guide/aruco_markers_pipeline/aoi_projection.md deleted file mode 100644 index 2f764f8..0000000 --- a/docs/user_guide/aruco_markers_pipeline/aoi_projection.md +++ /dev/null @@ -1,159 +0,0 @@ -Project AOI into camera frame -============================= - -Once [ArUcoScene pose is estimated](pose_estimation.md) and [AOI are described](aoi_description.md), AOI can be projected into [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. - -![AOI projection](../../img/aruco_camera_aoi_projection.png) - -## Add ArLayer to ArUcoScene to load AOI - -The [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer) class allows to load areas of interest description. An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArLayers](../../argaze.md/#argaze.ArFeatures.ArLayer). - -Here is the previous extract where one layer is added to the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) configuration: - -```json -{ - "name": "My FullHD camera", - "size": [1920, 1080], - ... - "scenes": { - "MyScene" : { - "aruco_markers_group": { - ... - }, - "layers": { - "MyLayer": { - "aoi_scene": { - "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], - "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]], - "BlueTriangle": [[12.5, 7.5, -0.5], [27.5, 7.5, -0.5], [20, 22.5, -0.5]] - } - } - } - } - } - ... -} -``` - -Now, let's understand the meaning of each JSON entry. - -### "MyLayer" - -The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically useful for visualisation purpose. - -### AOI Scene - -The [AOIScene](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AOIScene) defines a set of 3D [AreaOfInterest](../../argaze.md/#argaze.AreaOfInterest.AOIFeatures.AreaOfInterest) registered by name. - -## Add ArLayer to ArUcoCamera to project AOI - -Here is the previous extract where one layer is added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) and displayed: - -```json -{ - "name": "My FullHD camera", - "size": [1920, 1080], - ... - "scenes": { - "MyScene" : { - "aruco_markers_group": { - ... - }, - "layers": { - "MyLayer": { - "aoi_scene": { - ... - } - } - } - } - }, - "layers": { - "MyLayer": {} - } - ... - "image_parameters": { - ... - "draw_layers": { - "MyLayer": { - "draw_aoi_scene": { - "draw_aoi": { - "color": [255, 255, 255], - "border_size": 1 - } - } - } - } - } -} -``` - -Now, let's understand the meaning of each JSON entry. - -### "MyLayer" - -The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically useful for visualisation purpose. - -!!! warning "Layer name policy" - - An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layer is projected into an [ ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer, **provided they have the same name**. - -!!! note - - [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers are projected into their dedicated [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layers when calling the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method. - -## Add AOI analysis - -When a scene layer is projected into a camera layer, it means that the 3D [ArLayer.aoi_scene](../../argaze.md/#argaze.ArFeatures.ArLayer.aoi_scene) description of the scene becomes the 2D camera's [ArLayer.aoi_scene](../../argaze.md/#argaze.ArFeatures.ArLayer.aoi_scene) description of the camera. - -Therefore, it means that [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) benefits from all the services described in [AOI analysis pipeline section](../gaze_analysis_pipeline/aoi_analysis.md). - -Here is the previous extract where AOI matcher, AOI scan path and AOI scan path analyzers are added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer: - -```json -{ - "name": "My FullHD camera", - "size": [1920, 1080], - ... - "scenes": { - "MyScene" : { - "aruco_markers_group": { - ... - }, - "layers": { - "MyLayer": { - "aoi_scene": { - ... - } - } - } - } - }, - "layers": { - "MyLayer": { - "aoi_matcher": { - "DeviationCircleCoverage": { - "coverage_threshold": 0.5 - } - }, - "aoi_scan_path": { - "duration_max": 30000 - }, - "aoi_scan_path_analyzers": { - "Basic": {}, - "TransitionMatrix": {}, - "NGram": { - "n_min": 3, - "n_max": 5 - } - } - } - } - ... -} -``` - -!!! warning - - Adding scan path and scan path analyzers to an [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer doesn't make sense if the camera is moving. diff --git a/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md index 35b64f7..6bf84a9 100644 --- a/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md +++ b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md @@ -135,4 +135,4 @@ Particularly, timestamped gaze positions can be passed one by one to [ArUcoCamer At this point, the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method only detects ArUco markers and the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method only process gaze movement identification without any AOI support as no scene description is provided into the JSON configuration file. - Read the next chapters to learn [how to estimate scene pose](pose_estimation.md) and [how to project AOI](aoi_projection.md). \ No newline at end of file + Read the next chapters to learn [how to estimate scene pose](pose_estimation.md) and [how to project AOI](aoi_3d_projection.md). \ No newline at end of file diff --git a/docs/user_guide/aruco_markers_pipeline/introduction.md b/docs/user_guide/aruco_markers_pipeline/introduction.md index dd82900..917245d 100644 --- a/docs/user_guide/aruco_markers_pipeline/introduction.md +++ b/docs/user_guide/aruco_markers_pipeline/introduction.md @@ -16,10 +16,10 @@ First, let's look at the schema below: it gives an overview of the main notions To build your own ArUco markers pipeline, you need to know: * [How to setup ArUco markers into a scene](aruco_markers_description.md), -* [How to describe scene's AOI](aoi_description.md), +* [How to describe scene's AOI](aoi_3d_description.md), * [How to load and execute ArUco markers pipeline](configuration_and_execution.md), * [How to estimate scene pose](pose_estimation.md), -* [How to project AOI into camera frame](aoi_projection.md), +* [How to project AOIs into camera frame](aoi_3d_projection.md), * [How to define an AOI as a frame](aoi_frame.md) More advanced features are also explained like: -- cgit v1.1