diff options
Diffstat (limited to 'docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md')
-rw-r--r-- | docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md | 138 |
1 files changed, 138 insertions, 0 deletions
diff --git a/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md new file mode 100644 index 0000000..81c577f --- /dev/null +++ b/docs/user_guide/aruco_markers_pipeline/configuration_and_execution.md @@ -0,0 +1,138 @@ +Load and execute pipeline +========================= + +Once [ArUco markers are placed into a scene](aruco_markers_description.md), they can be detected thanks to [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class. + +As [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class also benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md). + +![ArUco camera frame](../../img/aruco_camera_frame.png) + +## Load JSON configuration file + +The [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) internal pipeline loads from a JSON configuration file thanks to [ArUcoCamera.from_json](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera.from_json) class method. + +Here is a simple JSON [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) configuration file example: + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + "aruco_detector": { + "dictionary": "DICT_APRILTAG_16h5", + "marker_size": 5 + }, + "gaze_movement_identifier": { + "DispersionThresholdIdentification": { + "deviation_max_threshold": 25, + "duration_min_threshold": 150 + } + }, + "image_parameters": { + "background_weight": 1, + "draw_gaze_positions": { + "color": [0, 255, 255], + "size": 2 + }, + "draw_fixations": { + "deviation_circle_color": [255, 0, 255], + "duration_border_color": [127, 0, 127], + "duration_factor": 1e-2 + }, + "draw_saccades": { + "line_color": [255, 0, 255] + }, + "draw_detected_markers": { + "color": [0, 255, 0], + "draw_axes": { + "thickness": 3 + } + } + } +} +``` + +Then, here is how to load the JSON file: + +```python +from argaze.ArUcoMarkers import ArUcoCamera + +# Load ArUcoCamera +aruco_camera = ArUcoCamera.ArUcoCamera.from_json('./configuration.json') +``` + +Now, let's understand the meaning of each JSON entry. + +### Name - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* + +The name of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame. Basically useful for visualisation purpose. + +### Size - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* + +The size of the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) frame in pixels. Be aware that gaze positions have to be in the same range of value to be projected in. + +### ArUco Detector + +The first [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) pipeline step is to detect ArUco markers inside input image and estimate their poses. + +![ArUco markers detection](../../img/aruco_camera_markers_detection.png) + +The [ArUcoDetector](../../argaze.md/#argaze.ArUcoMarkers.ArUcoDetector) is in charge to detect all markers from a specific dictionary with a given size in centimeters. + +!!! warning "Mandatory" + JSON *aruco_detector* entry is mandatory. + +### Gaze Movement Identifier - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* + +The first [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) pipeline step dedicated to identify fixations or saccades from consecutive timestamped gaze positions. + +![Gaze movement identification](../../img/aruco_camera_gaze_movement_identification.png) + +### Image parameters - *inherited from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame)* + +The usual [ArFrame visualisation parameters](./user_guide/gaze_analysis_pipeline/visualisation.md) plus one additional *draw_detected_markers* field. + +## Pipeline execution + +### Detect ArUco markers, estimate scene pose and project AOI + +Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole pipeline dedicated to ArUco markers detection, scene pose estimation and AOI projection. + +```python +# Assuming that Full HD (1920x1080) video stream or file is opened +... + +# Assuming that the video reading is handled in a looping code block +...: + + # Capture image from video stream of file + image = video_capture.read() + + # Detect ArUco markers, estimate scene pose then, project AOI into camera frame + aruco_camera.watch(image) + + # Display ArUcoCamera frame image to display detected ArUco markers, scene pose, AOI projection and ArFrame visualisation. + ... aruco_camera.image() +``` + +!!! warning "Pose estimation error" + ArUco markers pose estimation algorithm can lead to errors due to geometric ambiguities as explain in [this article](https://ieeexplore.ieee.org/document/1717461). To discard such ambiguous cases, markers should **as less as possible be parallel to camera plan**. + +### Analyse timestamped gaze positions into camera frame + +As mentioned above, [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) and so, benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md). + +Particularly, timestamped gaze positions can be passed one by one to [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method to execute the whole pipeline dedicated to gaze analysis. + +```python +# Assuming that timestamped gaze positions are available +... + + # Look ArUcoCamera frame at a timestamped gaze position + aruco_camera.look(timestamp, gaze_position) +``` + +!!! note "" + + At this point, the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method only detects ArUco markers and the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method only process gaze movement identification without any AOI support as no scene description is provided into the JSON configuration file. + + Read the next chapters to learn [how to estimate scene pose](pose_estimation.md) and [how to project AOI](aoi_projection.md).
\ No newline at end of file |