PROSE: Training-Free Egocentric Scene Registration with Vision-Language Models

Figure 1: Overview of PROSE. From two egocentric RGB sequences of the same scene captured at different times with partial overlap, PROSE prompts a pretrained VLM to (i) list the objects in each sequence, lifting them into an object-level 3D scene graph, and (ii) match instances across the two scans. The resulting correspondences drive a training-free pose estimation that registers the two scenes. No depth sensor, training, or annotated graph is required.

Figure 2: Overall pipeline of PROSE. From two egocentric RGB scans of the same scene at different times, PROSE parses each into a per-scan 3D scene graph, matches instances across scans, and estimates the rigid transform T\in SE(3) by generating one candidate per matched pair and selecting the highest-inlier-ratio hypothesis. The entire pipeline is training-free.

Figure A2: Instance-identity inconsistency across frames, and its resolution by fusion. Top row: the example of per-frame SAM 3 masks from the segmentation stage (Sec. 3.1 ); the same seat is assigned ID 1: sofa in frames 1-2 and 4-6 but a distinct ID 4: armchair in frame 3 (denoted by red box ), so its identity is not consistent over the sequence. Bottom row: fusion relabels every frame to one canonical instance, yielding the temporally consistent 3D instance used in the scene graph \mathcal{G} . Note that the fused 3D instance projected back into each RGB frame for better visual understandin

Figure A3: Height binning on the gravity axis. Each object instance is placed on the gravity ( h_{v} ) axis at its centroid height (colored markers, linked to the axis). The pooled heights are cut into K{=}5 equal-count quantile bins, with edges q_{0}{=}h_{\min},\dots,q_{5}{=}h_{\max} and native widths w_{1},\dots,w_{5} (Eq. 4 ). Each bin is then expanded across its edges by \alpha{=}0.2 of the smaller adjacent width into the overlapping bands B_{k} (Eq. 5 ), so an instance near a boundary still falls in a shared bin and stays matchable.

Figure A6: Qualitative registration results. Additional comparisons between registration baselines on the single clean ADT sequence. We display RGB reconstruction from VGGT- \Omega (top), colored subscans (middle), and a per point error heatmap (bottom). The error is the distance to the nearest GT point after registration, where green indicates low error and red indicates high error.

Figure A8: Qualitative result on scene graph generation. Additional scene graph visualization of PROSE on the path planning environment with nodes and edges overlayed on the VGGT- \Omega reconstruction. We furthest sample nodes to highlight their labels and display individual frames from each section of the room with every node in frame labeled and connected with edges.