diff options
3 files changed, 138 insertions, 15 deletions
diff --git a/docs/user_guide/aruco_markers_pipeline/aoi_description.md b/docs/user_guide/aruco_markers_pipeline/aoi_description.md index 8c57cd1..80ad858 100644 --- a/docs/user_guide/aruco_markers_pipeline/aoi_description.md +++ b/docs/user_guide/aruco_markers_pipeline/aoi_description.md @@ -55,8 +55,8 @@ JSON file format allows to describe AOIs vertices. ``` json { - "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [6.2, 1.275252, 1.753843], [31.2, 1.275252, 1.753843]], - "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [2.5, 27.5, -0.5], [37.5, 27.5, -0.5]], + "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], + "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]], "BlueTriangle": [[12.5, 7.5, -0.5], [27.5, 7.5, -0.5], [20, 22.5, -0.5]] } ``` diff --git a/docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md b/docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md index 824e466..ba19e45 100644 --- a/docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md +++ b/docs/user_guide/aruco_markers_pipeline/aruco_camera_configuration_and_execution.md @@ -3,9 +3,9 @@ Configure and execute ArUcoCamera Once [ArUco markers are placed into a scene](aruco_markers_description.md) and [areas of interest are described](aoi_description.md), everything is ready to setup an ArUco marker pipeline thanks to [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class. -As it inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md). +As [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame), the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) class benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md). - + ## Load JSON configuration file @@ -68,19 +68,21 @@ The usual [ArFrame visualisation parameters](./user_guide/gaze_analysis_pipeline ## Pipeline execution -Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole intanciated pipeline. +### Detect ArUco markers, estimate scene pose and project scene + +Pass each camera image to [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method to execute the whole pipeline dedicated to ArUco markers detection, scene pose estimation and projection. ```python # Assuming that Full HD (1920x1080) video stream or file is opened ... -# Assuming there is a way to escape the while loop -while ...: +# Assuming that the video reading is handled in a looping code block +...: # Capture image from video stream of file image = video_capture.read() - # Detect ArUco markers and more... + # Detect ArUco markers, estimate scene pose then, project scene into camera frame aruco_camera.watch(image) # Display ArUcoCamera frame image to check that ArUco markers are well detected and scene is well projected @@ -90,4 +92,24 @@ while ...: !!! warning ArUco markers pose estimation algorithm can lead to errors due to geometric ambiguities as explain in [this article](https://ieeexplore.ieee.org/document/1717461). To discard such ambiguous cases, markers should **not be parallel to camera plan**. -At this point, the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) only detects ArUco markers as no scene description is provided.
\ No newline at end of file + + +### Analyse timestamped gaze positions into camera frame + +As mentioned above, [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) inherits from [ArFrame](../../argaze.md/#argaze.ArFeatures.ArFrame) and so, benefits from all the services described in [gaze analysis pipeline section](./user_guide/gaze_analysis_pipeline/introduction.md). + +Particularly, timestamped gaze positions can be passed one by one to [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArFrame.look) method to execute the whole pipeline dedicated to gaze analysis. + +```python +# Assuming that timestamped gaze positions are available +... + + # Look ArUcoCamera frame at a timestamped gaze position + aruco_camera.look(timestamp, gaze_position) +``` + +!!! warning "" + + At this point, the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method only detects ArUco markers and the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method is not able to analyze gaze positions as no scene description is provided into the JSON configuration file. + + Read the next chapters to learn [how to enable scene pose estimation and its projection](aruco_scene.md).
\ No newline at end of file diff --git a/docs/user_guide/aruco_markers_pipeline/aruco_scene.md b/docs/user_guide/aruco_markers_pipeline/aruco_scene.md index b47fefb..91d2702 100644 --- a/docs/user_guide/aruco_markers_pipeline/aruco_scene.md +++ b/docs/user_guide/aruco_markers_pipeline/aruco_scene.md @@ -58,9 +58,7 @@ The 3D places of ArUco markers into the scene as defined at [ArUco markers descr ## Add ArLayer to ArUcoScene to load AOI -The [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer) class allows to load areas of interest description. - -An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArLayers](../../argaze.md/#argaze.ArFeatures.ArLayer). +The [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer) class allows to load areas of interest description. An [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) instance can contains multiples [ArLayers](../../argaze.md/#argaze.ArFeatures.ArLayer). Here is the previous extract where one layer is added to the [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene): @@ -77,8 +75,8 @@ Here is the previous extract where one layer is added to the [ArUcoScene](../../ "layers": { "MyLayer": { "aoi_scene": { - "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [6.2, 1.275252, 1.753843], [31.2, 1.275252, 1.753843]], - "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [2.5, 27.5, -0.5], [37.5, 27.5, -0.5]], + "YellowSquare": [[6.2, -7.275252, 25.246159], [31.2, -7.275252, 25.246159], [31.2, 1.275252, 1.753843], [6.2, 1.275252, 1.753843]], + "GrayRectangle": [[2.5, 2.5, -0.5], [37.5, 2.5, -0.5], [37.5, 27.5, -0.5], [2.5, 27.5, -0.5]], "BlueTriangle": [[12.5, 7.5, -0.5], [27.5, 7.5, -0.5], [20, 22.5, -0.5]] } } @@ -123,7 +121,17 @@ Here is the previous extract where one layer is added to the [ArUcoCamera](../.. } }, "layers": { - "MyLayer": {} + "MyLayer": { + "aoi_matcher": { + ... + }, + "aoi_scan_path": { + "duration_max": 30000 + }, + "aoi_scan_path_analyzers": { + ... + } + } } ... } @@ -142,3 +150,96 @@ The name of the [ArLayer](../../argaze.md/#argaze.ArFeatures.ArLayer). Basically !!! note [ArUcoScene](../../argaze.md/#argaze.ArUcoMarkers.ArUcoScene) layers are projected into their dedicated [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layers when calling the [ArUcoCamera.watch](../../argaze.md/#argaze.ArFeatures.ArCamera.watch) method. + +## Add GazeMovementIdentifier to ArUcocamera to enable gaze movement identification + +Here is the previous extract where gaze movement identifier is added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera): + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + ... + "scenes": { + "MyScene" : { + "aruco_markers_group": { + ... + }, + "layers": { + "MyLayer": { + "aoi_scene": { + ... + } + } + } + } + }, + "layers": { + "MyLayer": {} + }, + "gaze_movement_identifier": { + "DispersionThresholdIdentification": { + "deviation_max_threshold": 50, + "duration_min_threshold": 200 + } + } + ... +} +``` +!!! note + + Timestamped gaze positions are [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layers execution is done when calling the [ArUcoCamera.look](../../argaze.md/#argaze.ArFeatures.ArCamera.look) method. + +## Add AOIMatcher, AOIScanPath and AOIScanPathAnalyzers to ArUcoCamera layer to enable gaze analysis + +Here is the previous extract where AOI matcher, AOI scan path and AOI scan path analyzers are added to the [ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer: + +```json +{ + "name": "My FullHD camera", + "size": [1920, 1080], + ... + "scenes": { + "MyScene" : { + "aruco_markers_group": { + ... + }, + "layers": { + "MyLayer": { + "aoi_scene": { + ... + } + } + } + } + }, + "layers": { + "MyLayer": { + "aoi_matcher": { + "DeviationCircleCoverage": { + "coverage_threshold": 0.5 + } + }, + "aoi_scan_path": { + "duration_max": 30000 + }, + "aoi_scan_path_analyzers": { + "Basic": {}, + "TransitionMatrix": {}, + "NGram": { + "n_min": 3, + "n_max": 5 + } + } + } + }, + "gaze_movement_identifier": { + ... + } + ... +} +``` + +!!! warning + + Adding scan path and scan path analyzers to [an ArUcoCamera](../../argaze.md/#argaze.ArUcoMarkers.ArUcoCamera) layer doesn't make sense if the camera is moving.
\ No newline at end of file |