Mono-Hydra++: Real-Time Monocular Scene Graph Construction with Multi-Task Learning for 3D Indoor Mapping

Figure 2 : System overview of MONO-HYDRA++. Monocular RGB and IMU streams feed the M2H-MX perception model to produce depth and semantic predictions. SuperPoint tracks, IMU measurements, and sparse predicted-depth factors are used by the RVIO2-style robocentric VIO module to estimate metric odometry. The middle stage applies pose-aware temporal alignment and metric-semantic fusion, while the CPU runs the VIO pose-graph interface and the Mono-Hydra/Hydra scene-graph front-end/back-end. The rightmost stage performs hierarchical 3D scene graph construction, including room detection, loop-closure

Figure 3 : M2H-MX architecture overview. An input RGB image is first processed by the DINOv3 encoder, represented here by its internal Vision Transformer, ViT, block groups. Outputs from four selected ViT block ranges are used as multi-level token features for dense prediction. Low-Rank Adaptation, LoRA, is applied to the higher-level ViT blocks, blocks 13 to 24, to adapt the pretrained backbone efficiently while keeping most backbone parameters frozen. The Hierarchical Feature Adapter, HFA, converts the selected token features into spatial feature maps using Token Reassembly, TR, and then con

Figure 10 : Qualitative semantic mesh comparison on two representative ScanNet scenes. For each scene, the top row shows the predicted semantic mesh, the predicted labels transferred onto GT vertices, the GT semantic mesh, and the per-vertex mismatch map; the bottom row shows zoomed failure regions. Errors concentrate near object boundaries, thin structures, cluttered furniture regions, and, in scene0233, confusion involving the broad otherfurniture category.

Figure 1 : System outputs and data flow of Mono-Hydra++. Monocular RGB and IMU inputs are processed by the M2H-MX network to predict depth and semantics on the GPU, while the CPU-side RVIO2-style VIO module uses RGB, IMU, SuperPoint tracks, and sparse predicted-depth factors to estimate metric odometry. The odometry and temporally stabilized predictions are then fused in the Mono-Hydra/Hydra backend to produce a metric-semantic mesh and a hierarchical 3D scene graph with building, room, place, and object nodes.

Figure 12 : Trajectory and mesh overview for the uHumans2 ablation. The top row summarizes the depth-sparse-factor and semantic-masking comparison, while the bottom row summarizes the temporal pose-warp sweep. The figure provides qualitative context for the trajectory and mesh changes reported in Table 7 .

Figure 9 : Qualitative comparison on ScanNet scene0054. RGB-D results are on the top row, monocular results are on the bottom row, and ground truth is the rightmost panel.

Figure 11 : Qualitative radius-based object retrieval at r=0.5 m on the same two ScanNet scenes used in Fig. 10 . Each row shows ground-truth object centers, predicted object centers, and the matching overlay. Green connections indicate same-class matches within the radius threshold; missing GT and unmatched predictions highlight object-level failures in cluttered regions.

Figure 13 : Qualitative uHumans2 Office H12 ablation example. The top row shows the isometric reconstruction and the bottom row shows the corresponding top view for the baseline, sparse depth + semantic masking, and the sequence-specific ground-truth reference. The scene contains twelve moving humans and illustrates the dynamic complexity behind the aggregate Office results in Table 7 .

Figure 14 : Loop-closure candidate diagnostic on the highly dynamic uHumans2 Office H12 sequence. All panels use the same ground-truth mesh as spatial context. The comparison shows where the estimated trajectory remains close enough to revisited locations to support loop closure and where dynamic-scene drift leaves candidate revisits missed.