Sim-to-real transfer

Sim-to-real transfer is the strategy of teaching a robot inside a simulator and then moving what it learned onto real hardware. Simulation is attractive because it is cheap, fast, safe, and endlessly repeatable. A policy can practice a task millions of times in simulation without wearing out a real robot or risking damage, and difficult or dangerous situations can be generated on demand. The promise of sim-to-real is to harness all of that virtual practice to produce a policy that then performs well in the physical world.

The obstacle is the reality gap. A simulator is always an approximation, and the ways it differs from reality, in physics like friction and contact, in how sensors behave, and in visual appearance like lighting and texture, can add up to a world that is subtly but importantly unlike the real one. A policy that has only ever seen the simulator can latch onto details that do not hold up, and then fail when confronted with the genuine article. Bridging this gap is the core of sim-to-real work, and it is really a specific, important instance of the broader problem of distribution shift between training and deployment.

The most common way to bridge the reality gap is domain randomization, which deliberately varies simulation parameters such as physics, textures, and lighting during training. By exposing the policy to a wide range of simulated conditions, the real world becomes just one more variation it has learned to handle, rather than a surprise. This has to be tuned carefully, since randomizing too little leaves the gap open while randomizing too much can make learning intractable. Higher simulation fidelity, in both dynamics and sensing, also narrows the gap directly. In practice, a frequently successful recipe combines aggressive domain randomization in simulation with a modest amount of fine-tuning on real-world data to close the remaining distance.

Sim-to-real transfer matters because it offers a path around the fundamental bottleneck of physical AI, the high cost and slowness of collecting real-world experience. For anyone building robots, it explains both the appeal of simulation and why simulated success does not guarantee real-world success, since the reality gap stands in between. Understanding sim-to-real, and the role of simulation and sensor fidelity in closing the gap, is key to reasoning about how much a robot's virtual training will actually carry over to the world.

The reality gap is the set of differences between a simulator and the real world, spanning physics such as friction and contact, sensor behavior, and visual appearance like lighting and texture. These differences can cause a policy that succeeds in simulation to fail on real hardware.

Domain randomization is a training technique that varies simulation parameters, such as physics, textures, and lighting, so a policy learns to handle a wide range of conditions. The aim is for the real world to look like just another variation the policy has already seen, which helps it transfer.

The reality gap is a specific case of distribution shift, where the deployment data, from the real world, differs from the training data, from simulation. The same underlying principles of broadening training conditions and adapting to the target domain apply.

Distribution shift / domain shift, Policy, World model, Imitation learning / behavior cloning, Physical AI.