Fourier Features Let Agents Learn High Precision Policies with Imitation Learning

Figure 1 : Method Overview. Adding a Fourier feature mapping from Cartesian coordinates into a higher-dimensional feature space improves performance for any point cloud encoder used for diffusion imitation learning. For high-precision policies, the network must learn to condition on fine details in the scene geometry e.g. to decide whether to insert the leg into the slot or reposition it, yet neural networks learn the high frequency components of the target function only slowly, if at all. While neighbouring points in the scene have very similar Cartesian features, the high-dimensional Fourier

Figure 2 : Overview of PointPatch encoder family. We group the input point cloud into neighborhoods \mathcal{N}(i) (patch centers indicated in blue on the left). We map point coordinates into Fourier feature space to amplify subtle geometric differences between similar observations. The tokenizer extracts and aggregates features for each neighborhood to produce a set of tokens which are then forwarded to a goal-conditioned diffusion policy D_{\theta} to denoise the next action chunk.

Figure 3 : Overview of all evalution tasks from RoboCasa, ManiSkill3, and Real World benchmarks. Left : 4 of 16 RoboCasa tasks used for evaluation. Middle : all evaluated ManiSkill3 tasks. Right : starting configurations for all real-world tasks.

Figure 5 : Mean success rate across all tasks of 3D encoders with and without Fourier features on RoboCasa (left), ManiSkill3 (middle), and the real world (right). Methods using Fourier Features are marked via hatched bars, and methods are displayed in the order of the legend at the top. Across diverse tasks and architectures, Fourier features provide a consistent and meaningful benefit to task performance.

Figure 4 : Left : Setup for the real-world drawer experiments. Right : RGB views from left, right and gripper cameras and depth view from the left camera.

Figure 6 : Success rates with and without Fourier features on point clouds of different sizes, achieved by voxel downsampling of the observations. Larger point clouds contain richer geometric detail, resulting in a larger benefit for Fourier features.

Figure 9 : Parameter study of different Fourier feature wavelength configurations. Performance is robust to different numbers L of log-spaced wavelengths ( left ), as well as to the minimum wavelength \lambda_{\text{min}} (right) around our default of \lambda_{\text{min}}{=}0.02 , L{=}16 .