  The rapid growth of large-scale AI models, particularly large language models
has brought significant challenges in data privacy, computational resources,
and accessibility. Traditional centralized architectures often struggle to meet
required data security and scalability needs which hinders the democratization
of AI systems. Nesa introduces a model-agnostic sharding framework designed for
decentralized AI inference. Our framework uses blockchain-based sequential deep
neural network sharding to distribute computational tasks across a diverse
network of nodes based on a personalised heuristic and routing mechanism. This
enables efficient distributed training and inference for recent large-scale
models even on consumer-grade hardware. We use compression techniques like
dynamic blockwise quantization and mixed matrix decomposition to reduce data
transfer and memory needs. We also integrate robust security measures,
including hardware-based trusted execution environments to ensure data
integrity and confidentiality. Evaluating our system across various natural
language processing and vision tasks shows that these compression strategies do
not compromise model accuracy. Our results highlight the potential to
democratize access to cutting-edge AI technologies by enabling secure and
efficient inference on a decentralized network.
