diff options
| -rw-r--r-- | docs/img/aruco_scene.png | bin | 11903 -> 13704 bytes | |||
| -rw-r--r-- | docs/user_guide/augmented_reality_pipeline/aruco_scene_creation.md | 131 | ||||
| -rw-r--r-- | docs/user_guide/augmented_reality_pipeline/introduction.md | 19 | ||||
| -rw-r--r-- | mkdocs.yml | 7 | ||||
| -rw-r--r-- | src/argaze/utils/demo_data/aruco_markers_group.json | 36 | ||||
| -rw-r--r-- | src/argaze/utils/demo_data/aruco_markers_group.obj | 2 | ||||
| -rw-r--r-- | src/argaze/utils/demo_data/demo_augmented_reality_setup.json | 2 |
7 files changed, 158 insertions, 39 deletions
diff --git a/docs/img/aruco_scene.png b/docs/img/aruco_scene.png Binary files differindex 611676e..eee548d 100644 --- a/docs/img/aruco_scene.png +++ b/docs/img/aruco_scene.png diff --git a/docs/user_guide/augmented_reality_pipeline/aruco_scene_creation.md b/docs/user_guide/augmented_reality_pipeline/aruco_scene_creation.md new file mode 100644 index 0000000..ce080ca --- /dev/null +++ b/docs/user_guide/augmented_reality_pipeline/aruco_scene_creation.md @@ -0,0 +1,131 @@ +Setup ArUco markers scene +========================= + +The OpenCV library provides a module to detect fiducial markers into a picture and estimate its pose (cf [OpenCV ArUco tutorial page](https://docs.opencv.org/4.x/d5/dae/tutorial_aruco_detection.html)). + + + +The ArGaze [ArUcoMarkers submodule](../../argaze.md/#argaze.ArUcoMarkers) eases markers creation and description of their expected place for further camera pose estimation. + +## Print ArUco markers from a ArUco dictionary + +ArUco markers always belongs to a set of markers called ArUco markers dictionary. + + + +Many ArUco dictionaries exist with properties concerning the format, the number of markers or the difference between each markers to avoid error in tracking. + +Here is the documention [about ArUco markers dictionaries](https://docs.opencv.org/3.4/d9/d6a/group__aruco.html#gac84398a9ed9dd01306592dd616c2c975). + +The creation of [ArUcoMarkers](../../argaze.md/#argaze.ArUcoMarkers.ArUcoMarker) pictures from a dictionary is illustrated in the code below: + +``` python +from argaze.ArUcoMarkers import ArUcoMarkersDictionary + +# Create a dictionary of specific April tags +aruco_dictionary = ArUcoMarkersDictionary.ArUcoMarkersDictionary('DICT_APRILTAG_16h5') + +# Export marker n°5 as 3.5 cm picture with 300 dpi resolution +aruco_dictionary.create_marker(5, 3.5).save('./markers/', 300) + +# Export all dictionary markers as 3.5 cm pictures with 300 dpi resolution +aruco_dictionary.save('./markers/', 3.5, 300) +``` + +Let's print some of them before to go further. + +!!! warning + Print markers with a blank zone around them to help in their detection. + +## Describe expected ArUco markers place + +Once [ArUcoMarkers](../../argaze.md/#argaze.ArUcoMarkers.ArUcoMarker) pictures are placed into a scene it is possible to describe their expected 3D place into a file. + + + +What ever the origin is, all expected markers places need to be described in an [right-handed 3D axis](https://robotacademy.net.au/lesson/right-handed-3d-coordinate-frame/) where, from a camera point of view: + +* +X is pointing to the right, +* +Y is pointing to the top, +* +Z is pointing to camera side. + +!!! warning + All ArUco markers spatial values must be given in **centimeters**. + +### Edit OBJ file description + +OBJ file format could be exported from most 3D editors. + +``` obj +o DICT_APRILTAG_16h5#0_Marker +v -5.000000 14.960000 0.000000 +v 0.000000 14.960000 0.000000 +v -5.000000 19.959999 0.000000 +v 0.000000 19.959999 0.000000 +vn 0.0000 0.0000 1.0000 +s off +f 1//1 2//1 4//1 3//1 +o DICT_APRILTAG_16h5#1_Marker +v 25.000000 14.960000 0.000000 +v 30.000000 14.960000 0.000000 +v 25.000000 19.959999 0.000000 +v 30.000000 19.959999 0.000000 +vn 0.0000 0.0000 1.0000 +s off +f 5//2 6//2 8//2 7//2 +o DICT_APRILTAG_16h5#2_Marker +v -5.000000 -5.000000 0.000000 +v 0.000000 -5.000000 0.000000 +v -5.000000 0.000000 0.000000 +v 0.000000 0.000000 0.000000 +vn 0.0000 0.0000 1.0000 +s off +f 9//3 10//3 12//3 11//3 +o DICT_APRILTAG_16h5#3_Marker +v 25.000000 -5.000000 0.000000 +v 30.000000 -5.000000 0.000000 +v 25.000000 0.000000 0.000000 +v 30.000000 0.000000 0.000000 +vn 0.0000 0.0000 1.0000 +s off +f 13//4 14//4 16//4 15//4 +``` + +Here are common OBJ file features needed to describe ArUco markers place: + +* Object lines (line starting with *o* key) indicate markers dictionary and id by following a format: **DICTIONARY**#**ID**\_Marker. +* Vertice lines (lines starting with *v* key) indicate markers corners. The marker size will be automatically deducted from the geometry. +* Plane normals (lines starting with *vn* key) need to be exported for further pose estimation. +* Face (lines starting with *f* key) link vertices and normals indexes together. + +!!! warning + All markers must have the same size and belong to the same dictionary. + +### Edit JSON file description + +JSON file format allows to describe markers places using translation and euler angle rotation vectors. + +``` json +{ + "dictionary": "DICT_APRILTAG_16h5", + "marker_size": 5, + "places": { + "0": { + "translation": [-2.5, 17.5, 0], + "rotation": [0.0, 0.0, 0.0] + }, + "1": { + "translation": [27.5, 17.5, 0], + "rotation": [0.0, 0.0, 0.0] + }, + "2": { + "translation": [-2.5, -2.5, 0], + "rotation": [0.0, 0.0, 0.0] + }, + "3": { + "translation": [27.5, -2.5, 0], + "rotation": [0.0, 0.0, 0.0] + } + } +} +``` diff --git a/docs/user_guide/augmented_reality_pipeline/introduction.md b/docs/user_guide/augmented_reality_pipeline/introduction.md new file mode 100644 index 0000000..a06b1e2 --- /dev/null +++ b/docs/user_guide/augmented_reality_pipeline/introduction.md @@ -0,0 +1,19 @@ +Overview +======== + +This section explains how to build augmented reality pipelines based on ArUco Markers technology for various use cases. + +First, let's look at the schema below: it gives an overview of the main notions involved in the following chapters. + + + +To build your own augmented reality pipeline, you need to know: + +* [How to setup an ArUco markers scene](aruco_scene_creation.md), +* [How to deal with an ArCamera instance](ar_camera_configuration_and_execution.md), +* [How to add ArScene instance](ar_scene.md), +* [How to visualize ArCamera and ArScenes](visualisation.md) + +More advanced features are also explained like: + +* [How to script augmented reality pipeline](advanced_topics/scripting.md) @@ -21,6 +21,9 @@ nav: - Advanced Topics: - user_guide/gaze_analysis_pipeline/advanced_topics/scripting.md - user_guide/gaze_analysis_pipeline/advanced_topics/module_loading.md + - Augmented Reality environment: + - user_guide/augmented_reality_pipeline/introduction.md + - user_guide/augmented_reality_pipeline/aruco_scene_creation.md # - ArUco Markers: # - user_guide/aruco_markers/introduction.md # - user_guide/aruco_markers/dictionary_selection.md @@ -36,10 +39,6 @@ nav: # - user_guide/areas_of_interest/vision_cone_filtering.md # - user_guide/areas_of_interest/aoi_matching.md # - user_guide/areas_of_interest/heatmap.md -# - Augmented Reality environment: -# - user_guide/ar_camera/introduction.md -# - user_guide/ar_camera/environment_setup.md -# - user_guide/ar_camera/environment_exploitation.md # - Gaze Analysis: # - user_guide/gaze_analysis/introduction.md # - user_guide/gaze_analysis/gaze_position.md diff --git a/src/argaze/utils/demo_data/aruco_markers_group.json b/src/argaze/utils/demo_data/aruco_markers_group.json index 11814d2..d824ee9 100644 --- a/src/argaze/utils/demo_data/aruco_markers_group.json +++ b/src/argaze/utils/demo_data/aruco_markers_group.json @@ -4,47 +4,19 @@ "places": { "0": { "translation": [-2.5, 17.5, 0], - "rotation": [0.0, 0.0, 0.0], - "marker": { - "dictionary": { - "name": "DICT_APRILTAG_16h5" - }, - "identifier": 0, - "size": 5 - } + "rotation": [0.0, 0.0, 0.0] }, "1": { "translation": [27.5, 17.5, 0], - "rotation": [0.0, 0.0, 0.0], - "marker": { - "dictionary": { - "name": "DICT_APRILTAG_16h5" - }, - "identifier": 1, - "size": 5 - } + "rotation": [0.0, 0.0, 0.0] }, "2": { "translation": [-2.5, -2.5, 0], - "rotation": [0.0, 0.0, 0.0], - "marker": { - "dictionary": { - "name": "DICT_APRILTAG_16h5" - }, - "identifier": 2, - "size": 5 - } + "rotation": [0.0, 0.0, 0.0] }, "3": { "translation": [27.5, -2.5, 0], - "rotation": [0.0, 0.0, 0.0], - "marker": { - "dictionary": { - "name": "DICT_APRILTAG_16h5" - }, - "identifier": 3, - "size": 5 - } + "rotation": [0.0, 0.0, 0.0] } } }
\ No newline at end of file diff --git a/src/argaze/utils/demo_data/aruco_markers_group.obj b/src/argaze/utils/demo_data/aruco_markers_group.obj index 1030d01..83935ef 100644 --- a/src/argaze/utils/demo_data/aruco_markers_group.obj +++ b/src/argaze/utils/demo_data/aruco_markers_group.obj @@ -1,5 +1,3 @@ -# Blender v3.0.1 OBJ File: 'ar_camera.blend' -# www.blender.org o DICT_APRILTAG_16h5#0_Marker v -5.000000 14.960000 0.000000 v 0.000000 14.960000 0.000000 diff --git a/src/argaze/utils/demo_data/demo_augmented_reality_setup.json b/src/argaze/utils/demo_data/demo_augmented_reality_setup.json index 5b25da5..3511b98 100644 --- a/src/argaze/utils/demo_data/demo_augmented_reality_setup.json +++ b/src/argaze/utils/demo_data/demo_augmented_reality_setup.json @@ -41,7 +41,7 @@ }, "scenes": { "ArScene Demo" : { - "aruco_markers_group": "aruco_markers_group.obj", + "aruco_markers_group": "aruco_markers_group.json", "layers": { "main_layer" : { "aoi_scene": "aoi_3d_scene.obj" |
