  End-to-end robot learning, particularly for long-horizon tasks, often results
in unpredictable outcomes and poor generalization. To address these challenges,
we propose a novel Therblig-based Backbone Framework (TBBF) to enhance robot
task understanding and transferability. This framework uses therbligs (basic
action elements) as the backbone to decompose high-level robot tasks into
elemental robot configurations, which are then integrated with current
foundation models to improve task understanding. The approach consists of two
stages: offline training and online testing. During the offline training stage,
we developed the Meta-RGate SynerFusion (MGSF) network for accurate therblig
segmentation across various tasks. In the online testing stage, after a
one-shot demonstration of a new task is collected, our MGSF network extracts
high-level knowledge, which is then encoded into the image using Action
Registration (ActionREG). Additionally, the Large Language Model
(LLM)-Alignment Policy for Visual Correction (LAP-VC) is employed to ensure
precise action execution, facilitating trajectory transfer in novel robot
scenarios. Experimental results validate these methods, achieving 94.37% recall
in therblig segmentation and success rates of 94.4% and 80% in real-world
online robot testing for simple and complex scenarios, respectively.
Supplementary material is available at:
https://sites.google.com/view/therbligsbasedbackbone/home
