The front end works as follows:
- a user signs in with their account (a coinbase embedded wallet)
- a user is presented with a single button, where they can give instructions on what to do with assets in their wallet.
- in our example use case, a user can say 'send Max $10'
- we use the browsers native voice to text API, and send this to a agent running on the backend. The agent turns the natural language into an executable transaction (either EVM or Arb Stylus ERC20 transfer instructions) and sends this tx back to the user.
- A user can accept this transaction, or reject it. If they reject, back to starting screen.
- If the user accepts the transaction, they sign the message instructions with their embedded wallet, and send it to the agent to process.
- The agent returns the transaction hash, and the on chain transaction is complete
Once a user has signed up, they can send assets with nothing but 2 button presses and their voice, that's was our goal for this hackathon.
- This is a pretty simple NextJS app for the client, that makes use of the browsers native voice to text API.
- We wrote onchain contracts for the agent to interact with, both in Solidity and Rust (for the arbitrum stylus price track)
- For our accounts, we used the coinbase embedded wallets sdk.
- For our AI Agent, we used the coinbase CPD kit, with some custom language extensions for the stylus contracts interact
We didn't finish the job - a lot of the code is there, but we unfortunately did not have time to get a working demo.