  This paper reviews the state-of-the-art of language models architectures and
strategies for "complex" question-answering (QA, CQA, CPS) with a focus on
hybridization. Large Language Models (LLM) are good at leveraging public data
on standard problems but once you want to tackle more specific complex
questions or problems (e.g. How does the concept of personal freedom vary
between different cultures ? What is the best mix of power generation methods
to reduce climate change ?) you may need specific architecture, knowledge,
skills, methods, sensitive data protection, explainability, human approval and
versatile feedback... Recent projects like ChatGPT and GALACTICA have allowed
non-specialists to grasp the great potential as well as the equally strong
limitations of LLM in complex QA. In this paper, we start by reviewing required
skills and evaluation techniques. We integrate findings from the robust
community edited research papers BIG, BLOOM and HELM which open source,
benchmark and analyze limits and challenges of LLM in terms of tasks complexity
and strict evaluation on accuracy (e.g. fairness, robustness, toxicity, ...) as
a baseline. We discuss some challenges associated with complex QA, including
domain adaptation, decomposition and efficient multi-step QA, long form and
non-factoid QA, safety and multi-sensitivity data protection, multimodal
search, hallucinations, explainability and truthfulness, temporal reasoning. We
analyze current solutions and promising research trends, using elements such
as: hybrid LLM architectural patterns, training and prompting strategies,
active human reinforcement learning supervised with AI, neuro-symbolic and
structured knowledge grounding, program synthesis, iterated decomposition and
others.
