not so secret agent

Creating oracles

Users can create new oracles by providing a natural-language script describing how to get the information that they’re looking for. They can then subsidize that oracle with bounty funds, encouraging scraping nodes to scrape the data from the web using LLMs.

Preventing inaccurate data

Scraping nodes must stake funds that they risk having slashed if they provide incorrect information. Before they can stake, they must also authenticate via Worldcoin. This means that if they get slashed and banned for incorrect data, they can’t re-stake from a new account — their World ID is permanently blocked. The banning process happens via a voting mechanism among other stakers.

Scraping data using an LLM

The node software launches a virtual web browser using PupI built a browser-to-AI interface which is able to transform a page into an LLM-friendly providing it the ability to interact with elements by clicking and typing. This means that it can take any actions that a human would — even things like noticing that a cookie consent pop-up must be closed before interacting with the page.

Oracle contract

• Represents an individual oracle that stores and manages data scraped from the web.
• Integrates with Worldcoin for unique identity verification of stakers.
• Implements a staking and slashing mechanism to ensure accurate data provision and penalize malicious behavior.
• Allows stakers to update the oracle's value and receive bounties based on predefined conditions.
• Enables stakers to vote and ban other stakers who provide incorrect information.

Factory contract

• Responsible for creating and managing multiple Oracle instances.
• Maintains a registry of created oracles, allowing easy access and retrieval.
• Ensures uniqueness of oracle names and nodes during creation.
• Allows users to create new oracles by specifying data scraping scripts and parameters.

The smart contract leverages Worldcoin integration to prevent Sybil attacks and ensure the integrity of the staking system. It provides a flexible and scalable solution for creating and managing multiple oracles within the not-so-secret agent ecosystem.

Scraping node

The scraping node is responsible for executing the data scraping tasks defined by the oracles and updating the oracle's value based on the scraped data.

Key components of the scraping node:

1. Node Execution:
   • Connects to the Ethereum network using the provided credentials.
   • Instantiates the oracle contract using the provided oracle address and ABI.
   • Continuously checks the oracle's state (last updated timestamp, cooloff period, available bounties).
   • Triggers the scraping task when conditions are met (cooloff period elapsed, sufficient bounties).
2. Scraping Task:
   • Launches a headless browser and navigates to the specified web pages.
   • Interacts with an AI model to process the scraped content and determine the next actions.
   • Follows a conversational approach, where the AI model guides the scraping process based on the current page content.
   • Continues scraping until the AI model provides an answer or encounters an error.
   • Returns the scraped data as a number to be updated in the oracle contract.
3. Page Content Extraction:
   • Extracts the relevant content from the web page for the AI model to process.
   • Constructs a structured representation of the visible elements, filtering out unnecessary data.
   • Converts the structured content to a suitable format for the AI model to consume.

The scraping node leverages AI models to intelligently navigate and extract data from web pages. It follows a conversational approach, where the AI model guides the scraping process based on the current page content.

The scraped data is used to update the oracle's value on the smart contract, making it available for other applications. The scraping node continuously monitors the oracle's state and triggers the scraping task when conditions are met, ensuring the data remains up to date.