  Embodied agents are expected to perform more complicated tasks in an
interactive environment, with the progress of Embodied AI in recent years.
Existing embodied tasks including Embodied Referring Expression (ERE) and other
QA-form tasks mainly focuses on interaction in term of linguistic instruction.
Therefore, enabling the agent to manipulate objects in the environment for
exploration actively has become a challenging problem for the community. To
solve this problem, We introduce a new embodied task: Remote Embodied
Manipulation Question Answering (REMQA) to combine ERE with manipulation tasks.
In the REMQA task, the agent needs to navigate to a remote position and perform
manipulation with the target object to answer the question. We build a
benchmark dataset for the REMQA task in the AI2-THOR simulator. To this end, a
framework with 3D semantic reconstruction and modular network paradigms is
proposed. The evaluation of the proposed framework on the REMQA dataset is
presented to validate its effectiveness.
