aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--docs/img/aruco_scene.pngbin11903 -> 13704 bytes
-rw-r--r--docs/user_guide/augmented_reality_pipeline/aruco_scene_creation.md131
-rw-r--r--docs/user_guide/augmented_reality_pipeline/introduction.md19
-rw-r--r--mkdocs.yml7
-rw-r--r--src/argaze/utils/demo_data/aruco_markers_group.json36
-rw-r--r--src/argaze/utils/demo_data/aruco_markers_group.obj2
-rw-r--r--src/argaze/utils/demo_data/demo_augmented_reality_setup.json2
7 files changed, 158 insertions, 39 deletions
diff --git a/docs/img/aruco_scene.png b/docs/img/aruco_scene.png
index 611676e..eee548d 100644
--- a/docs/img/aruco_scene.png
+++ b/docs/img/aruco_scene.png
Binary files differ
diff --git a/docs/user_guide/augmented_reality_pipeline/aruco_scene_creation.md b/docs/user_guide/augmented_reality_pipeline/aruco_scene_creation.md
new file mode 100644
index 0000000..ce080ca
--- /dev/null
+++ b/docs/user_guide/augmented_reality_pipeline/aruco_scene_creation.md
@@ -0,0 +1,131 @@
+Setup ArUco markers scene
+=========================
+
+The OpenCV library provides a module to detect fiducial markers into a picture and estimate its pose (cf [OpenCV ArUco tutorial page](https://docs.opencv.org/4.x/d5/dae/tutorial_aruco_detection.html)).
+
+![OpenCV ArUco markers](https://pyimagesearch.com/wp-content/uploads/2020/12/aruco_generate_tags_header.png)
+
+The ArGaze [ArUcoMarkers submodule](../../argaze.md/#argaze.ArUcoMarkers) eases markers creation and description of their expected place for further camera pose estimation.
+
+## Print ArUco markers from a ArUco dictionary
+
+ArUco markers always belongs to a set of markers called ArUco markers dictionary.
+
+![ArUco dictionaries](../../img/aruco_dictionaries.png)
+
+Many ArUco dictionaries exist with properties concerning the format, the number of markers or the difference between each markers to avoid error in tracking.
+
+Here is the documention [about ArUco markers dictionaries](https://docs.opencv.org/3.4/d9/d6a/group__aruco.html#gac84398a9ed9dd01306592dd616c2c975).
+
+The creation of [ArUcoMarkers](../../argaze.md/#argaze.ArUcoMarkers.ArUcoMarker) pictures from a dictionary is illustrated in the code below:
+
+``` python
+from argaze.ArUcoMarkers import ArUcoMarkersDictionary
+
+# Create a dictionary of specific April tags
+aruco_dictionary = ArUcoMarkersDictionary.ArUcoMarkersDictionary('DICT_APRILTAG_16h5')
+
+# Export marker n°5 as 3.5 cm picture with 300 dpi resolution
+aruco_dictionary.create_marker(5, 3.5).save('./markers/', 300)
+
+# Export all dictionary markers as 3.5 cm pictures with 300 dpi resolution
+aruco_dictionary.save('./markers/', 3.5, 300)
+```
+
+Let's print some of them before to go further.
+
+!!! warning
+ Print markers with a blank zone around them to help in their detection.
+
+## Describe expected ArUco markers place
+
+Once [ArUcoMarkers](../../argaze.md/#argaze.ArUcoMarkers.ArUcoMarker) pictures are placed into a scene it is possible to describe their expected 3D place into a file.
+
+![ArUco scene](../../img/aruco_scene.png)
+
+What ever the origin is, all expected markers places need to be described in an [right-handed 3D axis](https://robotacademy.net.au/lesson/right-handed-3d-coordinate-frame/) where, from a camera point of view:
+
+* +X is pointing to the right,
+* +Y is pointing to the top,
+* +Z is pointing to camera side.
+
+!!! warning
+ All ArUco markers spatial values must be given in **centimeters**.
+
+### Edit OBJ file description
+
+OBJ file format could be exported from most 3D editors.
+
+``` obj
+o DICT_APRILTAG_16h5#0_Marker
+v -5.000000 14.960000 0.000000
+v 0.000000 14.960000 0.000000
+v -5.000000 19.959999 0.000000
+v 0.000000 19.959999 0.000000
+vn 0.0000 0.0000 1.0000
+s off
+f 1//1 2//1 4//1 3//1
+o DICT_APRILTAG_16h5#1_Marker
+v 25.000000 14.960000 0.000000
+v 30.000000 14.960000 0.000000
+v 25.000000 19.959999 0.000000
+v 30.000000 19.959999 0.000000
+vn 0.0000 0.0000 1.0000
+s off
+f 5//2 6//2 8//2 7//2
+o DICT_APRILTAG_16h5#2_Marker
+v -5.000000 -5.000000 0.000000
+v 0.000000 -5.000000 0.000000
+v -5.000000 0.000000 0.000000
+v 0.000000 0.000000 0.000000
+vn 0.0000 0.0000 1.0000
+s off
+f 9//3 10//3 12//3 11//3
+o DICT_APRILTAG_16h5#3_Marker
+v 25.000000 -5.000000 0.000000
+v 30.000000 -5.000000 0.000000
+v 25.000000 0.000000 0.000000
+v 30.000000 0.000000 0.000000
+vn 0.0000 0.0000 1.0000
+s off
+f 13//4 14//4 16//4 15//4
+```
+
+Here are common OBJ file features needed to describe ArUco markers place:
+
+* Object lines (line starting with *o* key) indicate markers dictionary and id by following a format: **DICTIONARY**#**ID**\_Marker.
+* Vertice lines (lines starting with *v* key) indicate markers corners. The marker size will be automatically deducted from the geometry.
+* Plane normals (lines starting with *vn* key) need to be exported for further pose estimation.
+* Face (lines starting with *f* key) link vertices and normals indexes together.
+
+!!! warning
+ All markers must have the same size and belong to the same dictionary.
+
+### Edit JSON file description
+
+JSON file format allows to describe markers places using translation and euler angle rotation vectors.
+
+``` json
+{
+ "dictionary": "DICT_APRILTAG_16h5",
+ "marker_size": 5,
+ "places": {
+ "0": {
+ "translation": [-2.5, 17.5, 0],
+ "rotation": [0.0, 0.0, 0.0]
+ },
+ "1": {
+ "translation": [27.5, 17.5, 0],
+ "rotation": [0.0, 0.0, 0.0]
+ },
+ "2": {
+ "translation": [-2.5, -2.5, 0],
+ "rotation": [0.0, 0.0, 0.0]
+ },
+ "3": {
+ "translation": [27.5, -2.5, 0],
+ "rotation": [0.0, 0.0, 0.0]
+ }
+ }
+}
+```
diff --git a/docs/user_guide/augmented_reality_pipeline/introduction.md b/docs/user_guide/augmented_reality_pipeline/introduction.md
new file mode 100644
index 0000000..a06b1e2
--- /dev/null
+++ b/docs/user_guide/augmented_reality_pipeline/introduction.md
@@ -0,0 +1,19 @@
+Overview
+========
+
+This section explains how to build augmented reality pipelines based on ArUco Markers technology for various use cases.
+
+First, let's look at the schema below: it gives an overview of the main notions involved in the following chapters.
+
+![Augmented reality pipeline](../../img/augmented_reality_pipeline.png)
+
+To build your own augmented reality pipeline, you need to know:
+
+* [How to setup an ArUco markers scene](aruco_scene_creation.md),
+* [How to deal with an ArCamera instance](ar_camera_configuration_and_execution.md),
+* [How to add ArScene instance](ar_scene.md),
+* [How to visualize ArCamera and ArScenes](visualisation.md)
+
+More advanced features are also explained like:
+
+* [How to script augmented reality pipeline](advanced_topics/scripting.md)
diff --git a/mkdocs.yml b/mkdocs.yml
index 4681f20..c5427db 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -21,6 +21,9 @@ nav:
- Advanced Topics:
- user_guide/gaze_analysis_pipeline/advanced_topics/scripting.md
- user_guide/gaze_analysis_pipeline/advanced_topics/module_loading.md
+ - Augmented Reality environment:
+ - user_guide/augmented_reality_pipeline/introduction.md
+ - user_guide/augmented_reality_pipeline/aruco_scene_creation.md
# - ArUco Markers:
# - user_guide/aruco_markers/introduction.md
# - user_guide/aruco_markers/dictionary_selection.md
@@ -36,10 +39,6 @@ nav:
# - user_guide/areas_of_interest/vision_cone_filtering.md
# - user_guide/areas_of_interest/aoi_matching.md
# - user_guide/areas_of_interest/heatmap.md
-# - Augmented Reality environment:
-# - user_guide/ar_camera/introduction.md
-# - user_guide/ar_camera/environment_setup.md
-# - user_guide/ar_camera/environment_exploitation.md
# - Gaze Analysis:
# - user_guide/gaze_analysis/introduction.md
# - user_guide/gaze_analysis/gaze_position.md
diff --git a/src/argaze/utils/demo_data/aruco_markers_group.json b/src/argaze/utils/demo_data/aruco_markers_group.json
index 11814d2..d824ee9 100644
--- a/src/argaze/utils/demo_data/aruco_markers_group.json
+++ b/src/argaze/utils/demo_data/aruco_markers_group.json
@@ -4,47 +4,19 @@
"places": {
"0": {
"translation": [-2.5, 17.5, 0],
- "rotation": [0.0, 0.0, 0.0],
- "marker": {
- "dictionary": {
- "name": "DICT_APRILTAG_16h5"
- },
- "identifier": 0,
- "size": 5
- }
+ "rotation": [0.0, 0.0, 0.0]
},
"1": {
"translation": [27.5, 17.5, 0],
- "rotation": [0.0, 0.0, 0.0],
- "marker": {
- "dictionary": {
- "name": "DICT_APRILTAG_16h5"
- },
- "identifier": 1,
- "size": 5
- }
+ "rotation": [0.0, 0.0, 0.0]
},
"2": {
"translation": [-2.5, -2.5, 0],
- "rotation": [0.0, 0.0, 0.0],
- "marker": {
- "dictionary": {
- "name": "DICT_APRILTAG_16h5"
- },
- "identifier": 2,
- "size": 5
- }
+ "rotation": [0.0, 0.0, 0.0]
},
"3": {
"translation": [27.5, -2.5, 0],
- "rotation": [0.0, 0.0, 0.0],
- "marker": {
- "dictionary": {
- "name": "DICT_APRILTAG_16h5"
- },
- "identifier": 3,
- "size": 5
- }
+ "rotation": [0.0, 0.0, 0.0]
}
}
} \ No newline at end of file
diff --git a/src/argaze/utils/demo_data/aruco_markers_group.obj b/src/argaze/utils/demo_data/aruco_markers_group.obj
index 1030d01..83935ef 100644
--- a/src/argaze/utils/demo_data/aruco_markers_group.obj
+++ b/src/argaze/utils/demo_data/aruco_markers_group.obj
@@ -1,5 +1,3 @@
-# Blender v3.0.1 OBJ File: 'ar_camera.blend'
-# www.blender.org
o DICT_APRILTAG_16h5#0_Marker
v -5.000000 14.960000 0.000000
v 0.000000 14.960000 0.000000
diff --git a/src/argaze/utils/demo_data/demo_augmented_reality_setup.json b/src/argaze/utils/demo_data/demo_augmented_reality_setup.json
index 5b25da5..3511b98 100644
--- a/src/argaze/utils/demo_data/demo_augmented_reality_setup.json
+++ b/src/argaze/utils/demo_data/demo_augmented_reality_setup.json
@@ -41,7 +41,7 @@
},
"scenes": {
"ArScene Demo" : {
- "aruco_markers_group": "aruco_markers_group.obj",
+ "aruco_markers_group": "aruco_markers_group.json",
"layers": {
"main_layer" : {
"aoi_scene": "aoi_3d_scene.obj"