Our application helps to curate high quality datasets by providing a framework to incentivize RLHF, with the LLM being fully verifiable and hosted on-chain.
The application allows the the user to submit instructions and queries to an AI chat interface, smart contract act as a middle-man between the user and hosted (centralized unsecured or Cartesi) LLM. The user is provided a chance to rate two responses to their prompt, creating an incentivized system to curate high quality RLHF data. User pays a query fee for the initial prompt and receives a rebate for providing feedback on the responses. All data is tracked on-chain, indexed by the graph, and with the ability to provide state checks of the model through Cartesi. As the model is tuned, datasets used for tuning can be uploaded to IPFS, verified by the graph, and checkpoints from Cartesi can be recorded in the smart contract, providing a fully transparent lineage of LLM evolution.
The application was developed using the Scaffold ETH and WAGMI libraries for blockchain functionality. Additionally, we employed the Graph and Catesi libraries. For the front-end technology, we utilized React JS along with Biconomy and Worldcoin, the application allows the user to recharge his account using fiat.