  Sequence modeling is a crucial area across various domains, including Natural
Language Processing (NLP), speech recognition, time series forecasting, music
generation, and bioinformatics. Recurrent Neural Networks (RNNs) and Long Short
Term Memory Networks (LSTMs) have historically dominated sequence modeling
tasks like Machine Translation, Named Entity Recognition (NER), etc. However,
the advancement of transformers has led to a shift in this paradigm, given
their superior performance. Yet, transformers suffer from $O(N^2)$ attention
complexity and challenges in handling inductive bias. Several variations have
been proposed to address these issues which use spectral networks or
convolutions and have performed well on a range of tasks. However, they still
have difficulty in dealing with long sequences. State Space Models(SSMs) have
emerged as promising alternatives for sequence modeling paradigms in this
context, especially with the advent of S4 and its variants, such as S4nd,
Hippo, Hyena, Diagnol State Spaces (DSS), Gated State Spaces (GSS), Linear
Recurrent Unit (LRU), Liquid-S4, Mamba, etc. In this survey, we categorize the
foundational SSMs based on three paradigms namely, Gating architectures,
Structural architectures, and Recurrent architectures. This survey also
highlights diverse applications of SSMs across domains such as vision, video,
audio, speech, language (especially long sequence modeling), medical (including
genomics), chemical (like drug design), recommendation systems, and time series
analysis, including tabular data. Moreover, we consolidate the performance of
SSMs on benchmark datasets like Long Range Arena (LRA), WikiText, Glue, Pile,
ImageNet, Kinetics-400, sstv2, as well as video datasets such as Breakfast,
COIN, LVU, and various time series datasets. The project page for Mamba-360
work is available on this webpage.\url{https://github.com/badripatro/mamba360}.
