Zubernetes | ZK8S

A distributed system for orchestrating secure containerized workloads with ZK-proof verification in Intel SGX enclaves (TEEs) using Gramine.

SYSTEM ARCHITECTURE

Scheduler

• The scheduler determines optimal placement of secure workloads across worker nodes through three critical phases:

Feasibility Analysis:

• The scheduler evaluates if a task can be scheduled on available workers by checking SGX enclave requirements, TEE capabilities, and ZK-proof verification readiness. It validates the worker's ability to handle proof verification workloads, considers memory requirements for execution traces, and verifies quote generation capacity for remote attestation. This ensures tasks are only assigned to properly equipped workers.

Worker Scoring:

• Candidate workers are scored based on multiple factors: available SGX enclave memory (EPC), ZK-proof verification performance metrics, historical success rates, current enclave utilization, network latency for proof distribution, and TEE attestation status. This comprehensive scoring ensures optimal task placement.

Optimal Selection:

• The final selection considers proof verification throughput requirements, enclave memory pressure, load balancing across TEE-enabled workers, and data locality for proof artifacts. This ensures efficient resource utilization while maintaining security guarantees.

Manager

• The Manager serves as the central orchestrator for secure workload execution and proof verification. It exposes an API that enables users to submit ZK-proof verification jobs, deploy secure containers in SGX enclaves, monitor verification status, query TEE capabilities, and control job lifecycle. The API also provides access to attestation reports and verification metrics.
• The Manager maintains a Job Storage system that tracks all proof verification jobs, enclave allocation status, proof execution traces, verification results, and attestation data. This enables informed scheduling decisions and provides comprehensive status information to users.
• The Manager collects critical metrics including SGX enclave utilization, proof verification throughput, memory usage per verification task, CPU load in secure/insecure worlds, TEE-specific performance metrics, and remote attestation statistics. These metrics guide scheduling decisions and system optimization.

Worker

• Workers are responsible for executing secure containers and performing proof verification in SGX enclaves. Each worker provides an API that enables task acceptance, secure container launch, enclave metrics reporting, attestation data provision, proof execution trace management, and container lifecycle control.

This architecture ensures secure proof verification in SGX enclaves, protected execution traces, verifiable computation results, scalable secure container deployment, and efficient resource utilization while maintaining strong security guarantees through hardware-based isolation and attestation.

Software

• Phala Dstack SDK to provide remote attestations for containers running in TEEs
• Next.js for frontend
• Solidity with Foundry for smart contract
• Backend using Python FastAPI and hosted on a cloud VM
• Gramine LibOS
• Intel SGX driver
• Docker runtime
• ZK-proof verifier in Rust and Golang
• Attestation service

Hardware:

• Intel SGX-enabled CPU
• Sufficient EPC memory allocation
• Secure storage for proofs
• High-bandwidth network for proof distribution
• TEE-enabled platform