a(i)udit

MOTIVATION (Problem statement)

In 2022 alone, over $3.6 billion USD was lost due to exploits & hacks from 167 major security incidents - with a whopping 51.5% of these being from audited protocols! [1]. We do not dispute the value of audits, but they are centralized, expensive (both time & monetary), and are not enforced - the security of our web3 future cannot rely on audits alone. And so, how do we make it easier for developers be part of the solution & incorporate a security-first practices earlier in their developer loop?

SOLUTION

Our answer is A(i)udit: an all-in-one platform where engineers use natural language to author, audit, test, and deploy smart contracts to any chain & with any test.

• For developers, A(i)udit "shifts-left" security best practices by enabling security-specific testing and rapid prototyping to take place much earlier in the development lifecycle. Get ramped up & familiar with security and auditing tools in your dev loop before commissioning a (costly) audit.

• For junior auditors, A(i)udit lets you focus on finding vulnerabilities by removing the overhead of learning & managing multiple tools, programs, environments, audit report templates, and frameworks. Hit the ground running faster & more effectively with A(i)udit.

DESCRIPTION

Specifically, A(i)udit is a tool that developers can use to execute blockchain-specific workflows using natural language. We chose to focus specifically on security & auditing for the Hackathon. Below is a description of a sample workflow:

1. The user uploads their own smart contract to the platform. Alternatively, a developer can use A(i)udit to help them get started on a smart contract too. We currently only support EVM compatible smart contracts.

2. The user can then simply ask A(i)udit to write and execute simple unit and fuzz tests. We cannot gaurantee or vouch for the quality of the unit tests and currently only support Foundry Fuzz tests.

3. The user can ask A(i)udit to execute static analysis tests. Currently we support Static Analysis tests using Mythril & Slither.

4. The user can ask the model to store the test results to IPFS using web3.storage for streamlined review, triage, and collaboration to address identified vulnerabilities.

5. The user repeats steps 1-4 until no vulnerabilities are detected.

6. The user can then ask A(i)udit to deploy the smart contract to a testnet or mainnet. We currently only support Ethereum, Gnosis, Polygon, and a local Ethereum node as destinations.

[1] https://beosin.com/resources/Global_Web3_Security_Report_2022_.pdf

A(i)udit was made using the open-source LangChain framework, which gives LLMs like ChatGPT the ability to successfully execute tasks of varying complexity. We currently support ChatGPT 3.5 but we hope to enable an end-user to use any LLM they wish (ideally open-source ones).

CORE COMPONENTS:

• Large Language Model (LLM): OpenAI's ChatGPT 3.5
• AI Agent: LangChain Action Agents
• Auditing tools: Mythril, Foundry, and Slither
• Poetry: A simple python package and dependency management system

AGENT TOOLS:

• mythril_tool: Used for running static analyses on smart contracts. Returns the path to the results (a .txt file) that will be stored on IPFS.

• slither_test_analysis: Used for performing static analyses on EVM-compatible smart contracts. smart_contract_compiler: Useful for compiling EVM-compatible smart contracts. Returns the path to a file containing the bytecode.

• smart_contract_identifier: Used for identifying a contract on an EVM blockchain. Given an address, this tool returns whether it is a contract or not. If it is a proxy, returns its implementation address.

• smart_contract_tester: Used for writing EVM-compatible smart contracts test cases. Returns the path to the .js file containing the test.

• smart_contract_writer: Used for writing EVM-compatible smart contracts given a title and description. Returns the path to the .sol file.

• smart_contract_compiler: Useful for compiling EVM-compatible smart contracts a path to a .sol file. Returns the path to a file containing the bytecode.

• smart_contract_deployer: Useful for deploying EVM-compatible smart contracts given a path to a file containing the bytecode and the network to deploy to. Returns the address of the deployed contract.

• smart_contract_fuzzytester: Useful for writing EVM-compatible smart contracts fuzz testing use cases. Returns the path to the .sol file.

• view_test_tool_result: This stores the only files extension .txt to filecoin and returns an id that can be used to review the audit results. Useful for reviewing txt files generated by audit tools.

ROADMAP

• Additional tools for the AI agents, with a specific focus on expanding support for a wider range of tests and audit use cases.

• A frontend and IDE for a more interactive and tigher developer loop.

• Support for open-source models to make A(i)udit a true public good.

• Extending the product offering to support specific workflows unique to auditors, and perhaps even web2 use cases.