walrus-s3-gateway

Walrus S3 Gateway is a high-performance, S3-compatible API gateway that provides seamless integration with the Walrus decentralized storage network. Built in Rust, it enables developers to use familiar S3 APIs while leveraging the robust, distributed storage capabilities of the Walrus blockchain-based storage system.

Key Features:

• S3 API Compatibility: Full support for standard S3 operations (GET, PUT, DELETE, LIST)
• Client-Side Signing: Secure authentication using SUI blockchain-based cryptographic signatures
• Decentralized Storage: Interfaces with Walrus network for distributed, fault-tolerant storage
• High Performance: Async request handling with configurable timeouts and concurrent operations
• Security-First: Fine-grained permission management and signature verification
• Production Ready: TLS support, CORS handling, and comprehensive error management

Architecture: The gateway acts as a bridge between traditional S3 client applications and the Walrus decentralized storage network. It implements a unique client-side signing workflow where clients maintain control of their private keys while the gateway validates and processes storage operations on the blockchain.

Use Cases: Web3 Applications: Decentralized apps requiring censorship-resistant storage Data Archival: Long-term storage with blockchain guarantees Multi-Cloud Strategy: Hybrid storage solutions combining traditional and decentralized storage Developer Tools: Easy migration from S3 to decentralized storage without code changes The project is part of the broader Walrus ecosystem, providing enterprise-grade decentralized storage solutions with the convenience of familiar S3 APIs.

Walrus S3 Gateway is built using a modern, high-performance technology stack centered around Rust for maximum performance and memory safety:

• Rust - Core language providing zero-cost abstractions and memory safety
• Tokio - Async runtime enabling concurrent request handling without blocking
• Hyper - High-performance HTTP server implementation
• Serde - Type-safe serialization/deserialization for configuration and data
• SUI SDK - Blockchain integration for cryptographic signature verification

The most hacky and notable aspect is our unique client-side signing workflow that bridges traditional S3 APIs with blockchain security.

How it works:

1. Client sends standard S3 PUT request
2. Gateway responds with HTTP 202 + unsigned transaction template (breaking S3 convention)
3. Client signs transaction locally with their SUI wallet
4. Client submits signed transaction via dedicated endpoint
5. Gateway validates and executes on Walrus network

This approach is particularly hacky because:

• We hijack the standard S3 PUT flow to return 202 instead of 200
• We embed blockchain transaction templates in HTTP responses
• We maintain S3 compatibility while adding blockchain security layer