  Manipulating unseen articulated objects through visual feedback is a critical
but challenging task for real robots. Existing learning-based solutions mainly
focus on visual affordance learning or other pre-trained visual models to guide
manipulation policies, which face challenges for novel instances in real-world
scenarios. In this paper, we propose a novel part-guided 3D RL framework, which
can learn to manipulate articulated objects without demonstrations. We combine
the strengths of 2D segmentation and 3D RL to improve the efficiency of RL
policy training. To improve the stability of the policy on real robots, we
design a Frame-consistent Uncertainty-aware Sampling (FUS) strategy to get a
condensed and hierarchical 3D representation. In addition, a single versatile
RL policy can be trained on multiple articulated object manipulation tasks
simultaneously in simulation and shows great generalizability to novel
categories and instances. Experimental results demonstrate the effectiveness of
our framework in both simulation and real-world settings. Our code is available
at
https://github.com/THU-VCLab/Part-Guided-3D-RL-for-Sim2Real-Articulated-Object-Manipulation.
