From 935ca70dfa1cde21b6c3c43659db3dcac3223c44 Mon Sep 17 00:00:00 2001 From: Théo de la Hogue Date: Wed, 6 Sep 2023 10:01:12 +0200 Subject: Renaming aruco markers pipeline chapter. Working on aruco camera configuratin chapter. --- docs/img/aruco_camera_frame.png | Bin 0 -> 19406 bytes .../aruco_camera_configuration_and_execution.md | 86 ++++++-------- .../aruco_markers_description.md | 130 +++++++++++++++++++++ .../aruco_markers_pipeline/aruco_scene_creation.md | 130 --------------------- .../aruco_markers_pipeline/introduction.md | 3 +- .../optic_parameters_calibration.md | 2 +- mkdocs.yml | 2 +- 7 files changed, 170 insertions(+), 183 deletions(-) create mode 100644 docs/img/aruco_camera_frame.png create mode 100644 docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md delete mode 100644 docs/user_guide/aruco_markers_pipeline/aruco_scene_creation.md diff --git a/docs/img/aruco_camera_frame.png b/docs/img/aruco_camera_frame.png new file mode 100644 index 0000000..273675a Binary files /dev/null and b/docs/img/aruco_camera_frame.png differ diff --git a/docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md b/docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md index a0ec2bf..9b1db42 100644 --- a/docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md +++ b/docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md @@ -3,9 +3,7 @@ Configure and execute ArUcoCamera Once [ArUco markers are placed into a scene](aruco_scene_creation.md) and [the camera optic have been calibrated](optic_parameters_calibration.md), everything is ready to setup an ArUco marker pipeline thanks to [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class. -As it inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class benefits from all the services of a [gaze analysis pipeline](./user_guide/gaze_analysis_pipeline/introduction.md). - -Besides, the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class projects [ArUcoScenes](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene)'s layers into its own layers thanks to ArUco markers pose estimations made by its [ArUcoDetector](../../argaze.md/#argaze.ArUcoMarkers.ArUcoDetector). +As it inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class benefits from all the services of a [gaze analysis pipeline](./user_guide/gaze_analysis_pipeline/introduction.md). Besides, the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class projects [ArUcoScenes](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene)'s layers into its own layers thanks to ArUco markers pose estimations made by its [ArUcoDetector](../../argaze.md/#argaze.ArUcoMarkers.ArUcoDetector). ![ArUco camera frame](../../img/aruco_camera_frame.png) @@ -25,18 +23,21 @@ Here is a simple JSON ArUcoCamera configuration file example: }, "marker_size": 5, "optic_parameters": "optic_parameters.json", - "parameters": { - "cornerRefinementMethod": 1, - "aprilTagQuadSigma": 2, - "aprilTagDeglitch": 1 - } }, "scenes": { - - }, - "layers": { - "main_layer": {} + "main_scene" : { + "aruco_markers_group": "aruco_description.json" + } }, + "image_parameters": { + "background_weight": 1, + "draw_detected_markers": { + "color": [0, 255, 0], + "draw_axes": { + "thickness": 3 + } + } + } } ``` @@ -51,60 +52,45 @@ aruco_camera = ArUcoCamera.ArUcoCamera.from_json('./configuration.json') Now, let's understand the meaning of each JSON entry. -### Name - -The name of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera). Basically useful for visualisation purpose. +### Name (inherited from ArFrame) -### Size +The name of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. Basically useful for visualisation purpose. -The size of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) defines the dimension of the rectangular area where gaze positions are projected. Be aware that gaze positions have to be in the same range of value. +### Size (inherited from ArFrame) -!!! warning - **ArGaze doesn't impose any spatial unit.** Gaze positions can either be integer or float, pixels, millimeters or what ever you need. The only concern is that all spatial values used in further configurations have to be all the same unit. +The size of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame in pixels. Be aware that gaze positions have to be in the same range of value. ### ArUco detector -The first [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) pipeline step is to identify fixations or saccades from consecutive timestamped gaze positions. - -![Gaze Movement Identifier](../../img/ar_frame_gaze_movement_identifier.png) - -The identification algorithm can be selected by instantiating a particular [GazeMovementIdentifier](../../argaze.md/#argaze.GazeFeatures.GazeMovementIdentifier) from the [argaze.GazeAnalysis](../../argaze.md/#argaze.GazeAnalysis) submodule or [from another python package](advanced_topics/module_loading.md). - -In the example file, the choosen identification algorithm is the [Dispersion Threshold Identification (I-DT)](../../argaze.md/#argaze.GazeAnalysis.DispersionThresholdIdentification) which has two specific *deviation_max_threshold* and *duration_min_threshold* attributes. - -!!! note - In ArGaze, [Fixation](../../argaze.md/#argaze.GazeFeatures.Fixation) and [Saccade](../../argaze.md/#argaze.GazeFeatures.Saccade) are considered as particular [GazeMovements](../../argaze.md/#argaze.GazeFeatures.GazeMovement). - -!!! warning - JSON *gaze_movement_identifier* entry is mandatory. Otherwise, the ScanPath and ScanPathAnalyzers steps are disabled. - -### Scan Path - -The second [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) pipeline step aims to build a [ScanPath](../../argaze.md/#argaze.GazeFeatures.ScanPath) defined as a list of [ScanSteps](../../argaze.md/#argaze.GazeFeatures.ScanStep) made by a fixation and a consecutive saccade. - -![Scan Path](../../img/ar_frame_scan_path.png) - -Once fixations and saccades are identified, they are automatically appended to the ScanPath if required. - -The [ScanPath.duration_max](../../argaze.md/#argaze.GazeFeatures.ScanPath.duration_max) attribute is the duration from which older scan steps are removed each time new scan steps are added. +... -!!! note - JSON *scan_path* entry is not mandatory. If scan_path_analyzers entry is not empty, the ScanPath step is automatically enabled. +### Scenes -### Scan Path Analyzers +... -Finally, the last [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) pipeline step consists in passing the previously built [ScanPath](../../argaze.md/#argaze.GazeFeatures.ScanPath) to each loaded [ScanPathAnalyzer](../../argaze.md/#argaze.GazeFeatures.ScanPathAnalyzer). +### Image parameters (inherited from ArFrame) -Each analysis algorithm can be selected by instantiating a particular [ScanPathAnalyzer](../../argaze.md/#argaze.GazeFeatures.ScanPathAnalyzer) from the [argaze.GazeAnalysis](../../argaze.md/#argaze.GazeAnalysis) submodule or [from another python package](advanced_topics/module_loading.md). +... ## Pipeline execution Timestamped gaze positions have to be passed one by one to [ArFrame.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method to execute the whole intanciated pipeline. ```python -# Assuming that timestamped gaze positions are available +# Assuming that live Full HD (1920x1080) video stream is enabled +... + +# Assuming there is a way to escape the while loop ... - # Look ArFrame at a timestamped gaze position - ar_frame.look(timestamp, gaze_position) + while video_stream.is_alive(): + + # Capture image from video stream + image = video_stream.read() + + # Detect ArUco markers in image + aruco_camera.watch(image) + + # Do something with ArUcoCamera frame image + ... ``` diff --git a/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md b/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md new file mode 100644 index 0000000..061fb19 --- /dev/null +++ b/docs/user_guide/aruco_markers_pipeline/aruco_markers_description.md @@ -0,0 +1,130 @@ +Setup ArUco markers into a scene +================================ + +First of all, ArUco markers needs to be printed and placed into the scene. + +## Print ArUco markers from a ArUco dictionary + +ArUco markers always belongs to a set of markers called ArUco markers dictionary. + +![ArUco dictionaries](../../img/aruco_dictionaries.png) + +Many ArUco dictionaries exist with properties concerning the format, the number of markers or the difference between each markers to avoid error in tracking. + +Here is the documention [about ArUco markers dictionaries](https://docs.opencv.org/3.4/d9/d6a/group__aruco.html#gac84398a9ed9dd01306592dd616c2c975). + +The creation of [ArUcoMarkers](../../argaze.md/#argaze.ArUcoMarkers.ArUcoMarker) pictures from a dictionary is illustrated in the code below: + +``` python +from argaze.ArUcoMarkers import ArUcoMarkersDictionary + +# Create a dictionary of specific April tags +aruco_dictionary = ArUcoMarkersDictionary.ArUcoMarkersDictionary('DICT_APRILTAG_16h5') + +# Export marker n°5 as 3.5 cm picture with 300 dpi resolution +aruco_dictionary.create_marker(5, 3.5).save('./markers/', 300) + +# Export all dictionary markers as 3.5 cm pictures with 300 dpi resolution +aruco_dictionary.save('./markers/', 3.5, 300) +``` + +!!! note + There is **A4_DICT_APRILTAG_16h5_5cm_0-7.pdf** file located in *./src/argaze/ArUcoMarkers/utils/* folder ready to be printed on A4 paper sheet. + +Let's print some of them before to go further. + +!!! warning + Print markers with a blank zone around them to help in their detection. + +## Describe ArUco markers place + +Once [ArUcoMarkers](../../argaze.md/#argaze.ArUcoMarkers.ArUcoMarker) pictures are placed into a scene it is possible to describe their 3D places into a file. + +![ArUco scene](../../img/aruco_scene.png) + +Where ever the origin point is, all markers places need to be described in a [right-handed 3D axis](https://robotacademy.net.au/lesson/right-handed-3d-coordinate-frame/) where: + +* +X is pointing to the right, +* +Y is pointing to the top, +* +Z is pointing to the backward. + +!!! warning + All ArUco markers spatial values must be given in **centimeters**. + +### Edit OBJ file description + +OBJ file format could be exported from most 3D editors. + +``` obj +o DICT_APRILTAG_16h5#0_Marker +v -5.000000 14.960000 0.000000 +v 0.000000 14.960000 0.000000 +v -5.000000 19.959999 0.000000 +v 0.000000 19.959999 0.000000 +vn 0.0000 0.0000 1.0000 +s off +f 1//1 2//1 4//1 3//1 +o DICT_APRILTAG_16h5#1_Marker +v 25.000000 14.960000 0.000000 +v 30.000000 14.960000 0.000000 +v 25.000000 19.959999 0.000000 +v 30.000000 19.959999 0.000000 +vn 0.0000 0.0000 1.0000 +s off +f 5//2 6//2 8//2 7//2 +o DICT_APRILTAG_16h5#2_Marker +v -5.000000 -5.000000 0.000000 +v 0.000000 -5.000000 0.000000 +v -5.000000 0.000000 0.000000 +v 0.000000 0.000000 0.000000 +vn 0.0000 0.0000 1.0000 +s off +f 9//3 10//3 12//3 11//3 +o DICT_APRILTAG_16h5#3_Marker +v 25.000000 -5.000000 0.000000 +v 30.000000 -5.000000 0.000000 +v 25.000000 0.000000 0.000000 +v 30.000000 0.000000 0.000000 +vn 0.0000 0.0000 1.0000 +s off +f 13//4 14//4 16//4 15//4 +``` + +Here are common OBJ file features needed to describe ArUco markers places: + +* Object lines (starting with *o* key) indicate markers dictionary and id by following this format: **DICTIONARY**#**ID**\_Marker. +* Vertice lines (starting with *v* key) indicate markers corners. The marker size will be automatically deducted from the geometry. +* Plane normals (starting with *vn* key) need to be exported for further pose estimation. +* Face (starting with *f* key) link vertices and normals indexes together. + +!!! warning + All markers must have the same size and belong to the same dictionary. + +### Edit JSON file description + +JSON file format allows to describe markers places using translation and euler angle rotation vectors. + +``` json +{ + "dictionary": "DICT_APRILTAG_16h5", + "marker_size": 5, + "places": { + "0": { + "translation": [-2.5, 17.5, 0], + "rotation": [0.0, 0.0, 0.0] + }, + "1": { + "translation": [27.5, 17.5, 0], + "rotation": [0.0, 0.0, 0.0] + }, + "2": { + "translation": [-2.5, -2.5, 0], + "rotation": [0.0, 0.0, 0.0] + }, + "3": { + "translation": [27.5, -2.5, 0], + "rotation": [0.0, 0.0, 0.0] + } + } +} +``` diff --git a/docs/user_guide/aruco_markers_pipeline/aruco_scene_creation.md b/docs/user_guide/aruco_markers_pipeline/aruco_scene_creation.md deleted file mode 100644 index ee5cca7..0000000 --- a/docs/user_guide/aruco_markers_pipeline/aruco_scene_creation.md +++ /dev/null @@ -1,130 +0,0 @@ -Build ArUco markers scene -========================= - -First of all, ArUco markers needs to be printed and placed into the scene. - -## Print ArUco markers from a ArUco dictionary - -ArUco markers always belongs to a set of markers called ArUco markers dictionary. - -![ArUco dictionaries](../../img/aruco_dictionaries.png) - -Many ArUco dictionaries exist with properties concerning the format, the number of markers or the difference between each markers to avoid error in tracking. - -Here is the documention [about ArUco markers dictionaries](https://docs.opencv.org/3.4/d9/d6a/group__aruco.html#gac84398a9ed9dd01306592dd616c2c975). - -The creation of [ArUcoMarkers](../../argaze.md/#argaze.ArUcoMarkers.ArUcoMarker) pictures from a dictionary is illustrated in the code below: - -``` python -from argaze.ArUcoMarkers import ArUcoMarkersDictionary - -# Create a dictionary of specific April tags -aruco_dictionary = ArUcoMarkersDictionary.ArUcoMarkersDictionary('DICT_APRILTAG_16h5') - -# Export marker n°5 as 3.5 cm picture with 300 dpi resolution -aruco_dictionary.create_marker(5, 3.5).save('./markers/', 300) - -# Export all dictionary markers as 3.5 cm pictures with 300 dpi resolution -aruco_dictionary.save('./markers/', 3.5, 300) -``` - -!!! note - There is **A4_DICT_APRILTAG_16h5_5cm_0-7.pdf** file located in *./src/argaze/ArUcoMarkers/utils/* folder ready to be printed on A4 paper sheet. - -Let's print some of them before to go further. - -!!! warning - Print markers with a blank zone around them to help in their detection. - -## Describe ArUco markers place - -Once [ArUcoMarkers](../../argaze.md/#argaze.ArUcoMarkers.ArUcoMarker) pictures are placed into a scene it is possible to describe their 3D places into a file. - -![ArUco scene](../../img/aruco_scene.png) - -Where ever the origin point is, all markers places need to be described in a [right-handed 3D axis](https://robotacademy.net.au/lesson/right-handed-3d-coordinate-frame/) where: - -* +X is pointing to the right, -* +Y is pointing to the top, -* +Z is pointing to the backward. - -!!! warning - All ArUco markers spatial values must be given in **centimeters**. - -### Edit OBJ file description - -OBJ file format could be exported from most 3D editors. - -``` obj -o DICT_APRILTAG_16h5#0_Marker -v -5.000000 14.960000 0.000000 -v 0.000000 14.960000 0.000000 -v -5.000000 19.959999 0.000000 -v 0.000000 19.959999 0.000000 -vn 0.0000 0.0000 1.0000 -s off -f 1//1 2//1 4//1 3//1 -o DICT_APRILTAG_16h5#1_Marker -v 25.000000 14.960000 0.000000 -v 30.000000 14.960000 0.000000 -v 25.000000 19.959999 0.000000 -v 30.000000 19.959999 0.000000 -vn 0.0000 0.0000 1.0000 -s off -f 5//2 6//2 8//2 7//2 -o DICT_APRILTAG_16h5#2_Marker -v -5.000000 -5.000000 0.000000 -v 0.000000 -5.000000 0.000000 -v -5.000000 0.000000 0.000000 -v 0.000000 0.000000 0.000000 -vn 0.0000 0.0000 1.0000 -s off -f 9//3 10//3 12//3 11//3 -o DICT_APRILTAG_16h5#3_Marker -v 25.000000 -5.000000 0.000000 -v 30.000000 -5.000000 0.000000 -v 25.000000 0.000000 0.000000 -v 30.000000 0.000000 0.000000 -vn 0.0000 0.0000 1.0000 -s off -f 13//4 14//4 16//4 15//4 -``` - -Here are common OBJ file features needed to describe ArUco markers places: - -* Object lines (starting with *o* key) indicate markers dictionary and id by following this format: **DICTIONARY**#**ID**\_Marker. -* Vertice lines (starting with *v* key) indicate markers corners. The marker size will be automatically deducted from the geometry. -* Plane normals (starting with *vn* key) need to be exported for further pose estimation. -* Face (starting with *f* key) link vertices and normals indexes together. - -!!! warning - All markers must have the same size and belong to the same dictionary. - -### Edit JSON file description - -JSON file format allows to describe markers places using translation and euler angle rotation vectors. - -``` json -{ - "dictionary": "DICT_APRILTAG_16h5", - "marker_size": 5, - "places": { - "0": { - "translation": [-2.5, 17.5, 0], - "rotation": [0.0, 0.0, 0.0] - }, - "1": { - "translation": [27.5, 17.5, 0], - "rotation": [0.0, 0.0, 0.0] - }, - "2": { - "translation": [-2.5, -2.5, 0], - "rotation": [0.0, 0.0, 0.0] - }, - "3": { - "translation": [27.5, -2.5, 0], - "rotation": [0.0, 0.0, 0.0] - } - } -} -``` diff --git a/docs/user_guide/aruco_markers_pipeline/introduction.md b/docs/user_guide/aruco_markers_pipeline/introduction.md index d2b19eb..ae174f3 100644 --- a/docs/user_guide/aruco_markers_pipeline/introduction.md +++ b/docs/user_guide/aruco_markers_pipeline/introduction.md @@ -15,7 +15,7 @@ First, let's look at the schema below: it gives an overview of the main notions To build your own ArUco markers pipeline, you need to know: -* [How to build an ArUco markers scene](aruco_scene_creation.md), +* [How to setup ArUco markers into a scene](aruco_markers_description.md), * [How to calibrate optic parameters](optic_parameters_calibration.md), * [How to deal with an ArUcoCamera instance](aruco_camera_configuration_and_execution.md), * [How to add ArScene instance](ar_scene.md), @@ -24,3 +24,4 @@ To build your own ArUco markers pipeline, you need to know: More advanced features are also explained like: * [How to script ArUco markers pipeline](advanced_topics/scripting.md) +* [How to improve ArUco markers detection](advanced_topics/aruco_detector_configuration.md) diff --git a/docs/user_guide/aruco_markers_pipeline/optic_parameters_calibration.md b/docs/user_guide/aruco_markers_pipeline/optic_parameters_calibration.md index 0561112..455d95a 100644 --- a/docs/user_guide/aruco_markers_pipeline/optic_parameters_calibration.md +++ b/docs/user_guide/aruco_markers_pipeline/optic_parameters_calibration.md @@ -63,9 +63,9 @@ aruco_detector = ArUcoDetector.ArUcoDetector(dictionary=aruco_dictionary, marker # Assuming there is a way to escape the while loop ... - # Capture images from video stream while video_stream.is_alive(): + # Capture image from video stream image = video_stream.read() # Detect all board corners in image diff --git a/mkdocs.yml b/mkdocs.yml index 89f1ffc..0755961 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -23,7 +23,7 @@ nav: - user_guide/gaze_analysis_pipeline/advanced_topics/module_loading.md - ArUco markers pipeline: - user_guide/aruco_markers_pipeline/introduction.md - - user_guide/aruco_markers_pipeline/aruco_scene_creation.md + - user_guide/aruco_markers_pipeline/aruco_markers_description.md - user_guide/aruco_markers_pipeline/optic_parameters_calibration.md - user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md # - ArUco Markers: -- cgit v1.1