Cloth Funnels: Canonicalized-Alignment
for Multi-Purpose Garment Manipulation
ICRA 2023
Automating garment manipulation is challenging due to extremely high variability in object configurations. To reduce this intrinsic variation, we introduce the task of "canonicalized-alignment" that simplifies downstream applications by reducing the possible garment configurations. This task can be considered as "cloth state funnel" that manipulates arbitrarily configured clothing items into a predefined deformable configuration (i.e. canonicalization) at an appropriate rigid pose (i.e. alignment). In the end, the cloth items will result in a compact set of structured and highly visible configurations - which are desirable for downstream manipulation skills. To enable this task, we propose a novel canonicalized-alignment objective that effectively guides learning to avoid adverse local minima during learning. Using this objective, we learn a multi-arm, multi-primitive policy that strategically chooses between dynamic flings and quasi-static pick and place actions to achieve efficient canonicalized-alignment. We evaluate this approach on a real-world ironing and folding system that relies on this learned policy as the common first step. Empirically, we demonstrate that our task-agnostic canonicalized-alignment can enable even simple manually-designed policies to work well where they were previously inadequate, thus bridging the gap between automated non-deformable manufacturing and deformable manipulation.
Paper & Code
Latest Paper Version: ArXiv. Code can be found here.
Team
1 Columbia University 2 Toyota Research Institute
Technical Summary Video
Real World Results
Canonicalized Alignment
We compare our combined-primitive canonicalized alignment policy to three baselines. Although Flingbot (first column) can effectively increase coverage, it fails to account for the cloth's alignment. Aligned Fling (second column) is quite effective at unfolding and aligning, but it cannot canonicalize the arms. Aligned pick and place (third columns) cannot efficiently unfold the cloth. Our method (last column) is able to achieve high canonicalized-alignment performance by combining dynamic and quasi-static action primitives.
Cloth Instance #1
Cloth Instance #2
Cloth Instance #3
Folding
Within less than 1 hour, we can collect and label 200 simulated cloth configurations with coverage at least 60% and trained a DeepLabv3 detector for a given garment class, which achieves a high success rate on canonicalized cloth configurations. After a canonicalized-alignment episode rollout, we run our keypoint model, translate the pixel keypoints to real world 3D coordinates, and execute a fixed folding heuristic parametrized by the given keypoints. By representing cloths as a set keypoints, we sidestep their infinite DoF by using a few meaningful keypoints as the representation, which makes it simpler to define heuristics over them.
Ironing
For garment manipulation pipelines, specialized tools are placed at a fixed location in the workspace. For ironing, the extra tools involved are the ironing board and the arm holding the iron. Given a well canonicalized and aligned shirt, an open-loop ironing primitive where the end effector moves from one end of the ironing board and back without any perception can be suf- ficient. In our setup, we use two transforms such that the left and right side of the shirt is on the ironing board respectively.
Citation
@inproceedings{canberk2022clothfunnels,
title={Cloth Funnels: Canonicalized-Alignment for Multi-Purpose Garment Manipulation},
author={Canberk, Alper and Chi, Cheng and Ha, Huy and Burchfiel, Benjamin and Cousineau, Eric and Feng, Siyuan and Song, Shuran},
booktitle={International Conference of Robotics and Automation (ICRA)},
year={2022}
}
If you have any questions, please feel free to contact Alper
Canberk.