LST Optimizer

This project aims to create an automated system for managing a portfolio of Liquid Staked Tokens (LSTs) on the Starknet Sepolia testnet. The main objective is to optimize the portfolio composition using reinforcement learning and forecasting to achieve the best possible returns. The project includes various components, such as a Flask-based web application, a reinforcement learning environment, data fetching and processing scripts, and integration with blockchain smart contracts for rebalancing the portfolio.

Environment Setup:

The environment is set up to simulate the trading of Liquid Staked Tokens (LSTs) with historical price data. The StakedETHEnv class extends the gym.Env class from OpenAI's Gym to create a custom RL environment. This environment handles the portfolio's state, actions, rewards, and transition dynamics. Data Handling:

Historical price data is fetched using the YFinance API and processed using Pandas. The data is then used to calculate historical returns, which are necessary for training the RL agent. Additionally, the fetch_and_process_tbill_data function fetches risk-free rates from an external API, which are used to calculate metrics like the expected return. Reinforcement Learning:

The PPO (Proximal Policy Optimization) algorithm from Stable Baselines3 is used to train the RL agent. The agent interacts with the custom environment to learn an optimal policy for rebalancing the portfolio based on historical and forecasted data. During each episode, the environment simulates the portfolio's performance over a specified time period, considering both historical price changes and forecasted price movements. Forecasting:

The Prophet library is used to generate time series forecasts for each asset. These forecasts are used as part of the observation space for the RL agent, helping it make informed decisions about future portfolio compositions. Blockchain Integration:

The project integrates with the Starknet blockchain to manage real token balances. It uses the Starknet Python SDK to interact with smart contracts, including checking balances and executing token transfers for rebalancing. Two main smart contract functions are involved: transfer_tokens_to_fund and transfer_tokens_from_fund, which handle the actual movement of tokens based on the RL agent's decisions. Flask API:

The Flask application provides endpoints for triggering rebalances and fetching the latest data. It also includes background tasks managed by APScheduler to ensure periodic updates and rebalances. The Flask app interacts with the RL environment, fetches the necessary data, and uses the Starknet SDK to execute the rebalances. Visualization:

The results of the RL agent's actions, including portfolio compositions, rewards, and normalized comparisons, are visualized using Plotly. The visualizations are embedded in both the Flask app and the Streamlit interface, providing an interactive and user-friendly way to analyze the model's performance. Deployment and Testing:

The application is designed to be deployed locally using ngrok for exposing the Flask server. This setup allows for easy testing and development. The project also includes error handling and logging to ensure smooth operation and facilitate debugging.

Stable Baselines3 (PPO Algorithm):

Purpose: Used for implementing the reinforcement learning (RL) agent that optimizes the portfolio. Integration: The RL model interacts with the custom environment to learn and make decisions based on historical and forecasted data. Flask:

Purpose: Provides the web framework for the API endpoints and the main interface for interaction. Integration: Handles HTTP requests for initiating rebalances and fetching the latest data. It also serves the web pages for visualization. APScheduler:

Purpose: Schedules periodic tasks like fetching the latest data and triggering rebalances. Integration: Ensures that the system stays updated with the latest data and performs rebalances at defined intervals. Plotly:

Purpose: For data visualization. Integration: Generates interactive graphs to visualize portfolio compositions, rewards, and normalized comparisons. Pandas and NumPy:

Purpose: Data manipulation and analysis. Integration: Processes historical price data, calculates returns, and handles various data transformations required for the RL environment. Asyncio and Aiohttp:

Purpose: Handles asynchronous operations, especially for making non-blocking HTTP requests. Integration: Ensures smooth and efficient data fetching and interactions with external services without blocking the main application. Starknet Python SDK:

Purpose: Interacts with the Starknet blockchain. Integration: Manages smart contract interactions, including transferring tokens and checking balances. Pyngrok:

Purpose: Exposes the local Flask server to the internet. Integration: Allows external access to the locally hosted Flask application, which is useful for development and testing purposes. Streamlit:

Purpose: Provides a user interface for visualizing data and interacting with the model. Integration: Displays charts, tables, and other outputs in an easy-to-use web application format. YFinance and Flipside:

Purpose: Fetches historical and real-time financial data. Integration: Provides the necessary financial data inputs for model training and evaluation. Prophet:

Purpose: Time series forecasting. Integration: Generates forecasted prices for assets, which are used by the RL agent for decision-making. CVXPY:

Purpose: Optimization library. Integration: Used for calculating the optimal portfolio compositions based on various constraints and objectives.