VoiceSense

Driven by our belief in the web3 future for all, our project - VoiceSence - aims at making web3 inclusive for everyone by alleviating barriers preventing people to transact on blockchain. We make web3 transactions possible for anyone who finds voice input more convenient than manual text input. VoiceSence aims to:

• Enable users with visual impairment or reading difficulties to transact on web3 using their voice as the input mechanism.
• Enable users to record inputs in their native language without the need to write in an unfamiliar language.

Our project’s logic uses AI models running within an execution layer to transform voice input into text and map it to the fields necessary to create a transaction. We then create a wallet connection to create and sign transactions on-chain.

VoiceSense uses the following tech stack:

• We use Cartesi’s execution layer to bring the power of mature web2 solutions in Machine Learning voice recognition to web3.

• The language recognition models used are provided by the Hugging Face AI.

• The wallet provider is WalletConnect

• Languages: JS, Python, TypeScript

The project is divided in the following milestones:

1. Enable voice recognition and processing on the execution layer.
2. Map the transcribed voice into transaction fields, create a transaction and sign it with a wallet.
3. Generate a transaction request and send it to a recipient.

In this initial development stage of VoiceSense, we are running Cartesi Machine locally in the host mode. We achieved this by cloning the Cartesi Rollups-examples repository and using the Calculator DApp as a source. We took the Front-end echo as a source for our front-end application.

We integrate the Hugging Face AI model into the back-end in two phases:

• We initially used the first version of the Hugging Face model, which is trained with 960 hours and does not recognise blockchain terminology.
• We then used the send, larger, version of the Hugging Face model, which is trained with thousands of hours.