Dillinger

Problem statement

• Running larger LLM’s with parameters over 30B requires huge infra thereby restricting research efforts.
• ChatGPT and other closed source platforms do not make the cut.

Solution

• Decentralised Inference for LLMs!

• SwarmLM introduces a decentralised approach to LLMS computation.
• The Transformer can infer layers of the model in a distributed compute environment.
• Clients (orchestrators) can create chains of pipeline-parallel servers (Compute Nodes) for distributed inference.This eliminates the need for a single massive resource (HPC) to run LLM models.
• SwarmLM incentivises contributors to dedicate resources and earn tokens through Proof of Compute.

Stakeholders

Compute Node Providers:

• GPU providers, Incentivised by Proof of Compute/Inference i.e. generation of token(LLM) volume.

LLM Users:

• Research people/civilians who want to build and tune LLM’s on the fly without HPC infrastructure.
• Pledge token to Bond manager Contract, get API key to utilise the Decentralised inference Network.

Links

• https://swarm-lm.vercel.app/
• https://github.com/albyzyx/swarm-lm-node
• https://github.com/albyzyx/swarm-lm-interface
• https://github.com/albyzyx/swarm-lm-contracts
• https://github.com/yedhukrrish/waku_node.git

Since the entire decentralization part is built atop of python and flask we faced major issues with respect to creating an interoperable space.