Robot Manipulation Problem

Manipulation involves robot to interact with the objects in the enviornment to arrange some specific arrangements of said objects. Example: picking and placing, stacking blocks, assembling a clock, or more abstractly making a coffee.

Interaction with the enviornment introduces alot more abstractions that the robot needs to recognize the achieve a given task. These abstraction include identifying what an object is, how will it behave upon interaction with the robot and with other objects, is it controllable by the robot. After having this apriori knowledge, it needs to come up with a plan which can change the current config/arrangment of the relevant objects in the enviornment to a desired config/arrangment. Depending on the task, the robot needs to make sure that these objects do not enter certain undesirable states, while exectuting the trajectory.

Identifying objects

Identifying interaction/behavior between objects and the robot

Planning from initial start to desired state

Executing the planned trajectory

When we think of robotic manipulation we think of the standard “pick and place” task where the robot is programmed to pick up objects from one location to a goal location. In this task, the robot has to use perception to identify the object and the enviornment, planning to plan the trajectory of the rigid robot arm in the enviornment while avoiding obstacles, and control to make sure the task is being executed in realtime while accounting for changing enviornments and disturbances. The tight interplay between perception, planning and control is required to succesfully execute a robot manipulation task.

Pick and place is just the tip of the iceberg when it comes to robot manipulation. It is the most synonymous with robot manipulation as it is the easiest to demonstrate and the most mature in terms of implementation. When we think broader, we can include any task where the robot “manipulates” or interacts with the objects around it as a robot manipulation task. This can include opening a door, solving a rubiks cube, tying shoelaces, folding laundry and the possibilities are endless.

The performance of these manipulation tasks are evaluated against their human counterparts. As humans, we are quite adept at manipulating the world around us to acheive our objective and our manipulation capbabilities are quite efficient in “learning” and very robust to variability in enviornment and disturbances. Hence, we want our robotic clones to have comparable if not better performances in such tasks.

But as we have found out, interacting with the enviornment is difficult for robots, partly because it is an endeavour to represent the enviornment in a formal way without making the problem intractable. With the rise of reinforcement learning and big data robotics, this problem is mostly solved with the cost (excuse the pun) of brittleness and lack of generalizability. If we can marry the learning based approaches with the traditional model-based approach which provides formal gurantees on robustness, perhaps maybe we can make robots our equals!