  3D scene graphs (3DSGs) are an emerging description; unifying symbolic,
topological, and metric scene representations. However, typical 3DSGs contain
hundreds of objects and symbols even for small environments; rendering task
planning on the full graph impractical. We construct TASKOGRAPHY, the first
large-scale robotic task planning benchmark over 3DSGs. While most benchmarking
efforts in this area focus on vision-based planning, we systematically study
symbolic planning, to decouple planning performance from visual representation
learning. We observe that, among existing methods, neither classical nor
learning-based planners are capable of real-time planning over full 3DSGs.
Enabling real-time planning demands progress on both (a) sparsifying 3DSGs for
tractable planning and (b) designing planners that better exploit 3DSG
hierarchies. Towards the former goal, we propose SCRUB, a task-conditioned 3DSG
sparsification method; enabling classical planners to match and in some cases
surpass state-of-the-art learning-based planners. Towards the latter goal, we
propose SEEK, a procedure enabling learning-based planners to exploit 3DSG
structure, reducing the number of replanning queries required by current best
approaches by an order of magnitude. We will open-source all code and baselines
to spur further research along the intersections of robot task planning,
learning and 3DSGs.
