Robot manipulation

Robot manipulation is a robot's ability to change its environment through physical contact, such as picking up, moving, pushing, inserting, or assembling objects. It combines perception, planning, and control, usually through an arm and an end-effector. Manipulation is the core skill behind most imitation learning datasets and vision-language-action models.

What is robot manipulation?

Manipulation covers anything a robot does to objects rather than to its own position. Grasping and pick-and-place are the classic cases, but manipulation also includes pushing, sliding, pouring, folding cloth, opening drawers, and fitting a peg into a hole. Researchers often split it into prehensile manipulation, where the robot holds the object, and non-prehensile manipulation, where it moves the object without grasping it, such as pushing a box across a table.
A manipulator is the arm that does the work. Its degrees of freedom set how many ways it can move, and its end-effector, a gripper, hand, or tool, makes contact with the world.
Manipulation covers moves that hold the object and moves that never pick it up.

Key takeaways

  • Robot manipulation is changing the world through contact, with or without holding the object.
  • It spans perception, planning, and control, and contact makes it hard to model by hand.
  • Learned manipulation policies depend on large, diverse sets of demonstrations.

How it works

A classical pipeline breaks manipulation into stages. Perception finds the object and estimates its pose. A planner chooses a grasp or contact point and a collision-free path, using motion planning and inverse kinematics. A controller executes the motion and reacts to contact forces.
Learning-based manipulation folds those stages into a policy that maps camera images and proprioception directly to actions. Policies are usually trained by imitation learning on demonstrations collected through teleoperation, and vision-language-action models add a language instruction so one model can handle many tasks.

See what a manipulation dataset actually covers before you train on it

Large manipulation datasets look diverse until you sort them. Watch for:
  • many episodes of the same task in nearly the same scene
  • successful episodes that needed several grasps to get there
  • tasks and objects that cluster in one lab
In Voxel51, DROID holds 1,000 real-world manipulation episodes as MCAP, with three synced cameras and fields for the task prompt, success, and number of grasps.

Robotics Droid

1,000 episodes

Real-world robot manipulation episodes from DROID, stored as MCAP with three synced cameras and fields for task prompt, success, and number of grasps.

Try this: Filter num_grasps to 2 or more, add the wrist camera from the Topics tab, and watch what happens between the first grasp and the second.

Explore DROID in Voxel51

Free with a work or .edu email.

Go deeper

Why it matters

Factories, warehouses, kitchens, and homes all need robots that handle objects, which makes manipulation one of the most valuable skills in physical AI. It is also where data is hardest. Contact is unforgiving, so small differences in object shape, friction, or lighting turn a success into a failed grasp, and a policy only generalizes as far as its demonstrations cover. Datasets such as DROID and Open X-Embodiment exist to widen that coverage, and checking which scenes, objects, and failure modes they actually contain is curation work.
Contact makes manipulation hard, and data makes it learnable.

Frequently asked questions

What is the difference between grasping and manipulation?

Grasping is one manipulation skill: securely picking up and holding an object. Manipulation is the broader category and includes pushing, pouring, inserting, folding, and other ways of changing the world through contact.

What is non-prehensile manipulation?

Moving an object without grasping it, such as pushing, sliding, flipping, or tipping it over. It lets a robot handle objects that are too large, flat, or awkward to pick up.

How do robots learn manipulation?

Most current systems learn from demonstrations collected by teleoperation, through imitation learning, sometimes combined with reinforcement learning in simulation. Vision-language-action models extend this to many tasks by conditioning on a language instruction.
black and white photo of Jesse Mostipak
Director of Growth
Jesse Mostipak is the Director of Growth at Voxel51, where the work is helping humans find and trust what the brand knows, and teaching the Google knowledge graph and the LLMs answering on their behalf to do the same. That question, how knowledge gets built inside a system, is one Jesse has been chasing for years. Earlier versions of it ran through a New York City high school science classroom, data science and machine learning, and developer relations at Kaggle, Posit (formerly RStudio), and Baseten. The answer doesn't change much depending on whether the learner is a teenager, a software engineer, or a knowledge graph. Jesse holds a Master's in Education from CUNY Hunter College. LinkedIn
See all articles by Jesse Mostipak
Last updated November 19, 2026

Building visual or physical AI?

Let's talk.