  Large language models are ubiquitous in natural language processing because
they can adapt to new tasks without retraining. However, their sheer scale and
complexity present unique challenges and opportunities, prompting researchers
and practitioners to explore novel model training, optimization, and deployment
methods. This literature review focuses on various techniques for reducing
resource requirements and compressing large language models, including
quantization, pruning, knowledge distillation, and architectural optimizations.
The primary objective is to explore each method in-depth and highlight its
unique challenges and practical applications. The discussed methods are
categorized into a taxonomy that presents an overview of the optimization
landscape and helps navigate it to understand the research trajectory better.
