Data Connectors

Data Connectors serve as the bridge between the AI agents and external market environments. They facilitate the ingestion of real-time prices, historical data, news sentiment, and technical indicators required for the decision-making process.

News & Sentiment Connectors

The system utilizes a hybrid approach to news gathering, prioritizing cost-effective RSS feeds while providing fallback support for professional APIs.

NewsAPIConnector

Located in data_connectors/newsapi_connector.py, this class manages the retrieval of cryptocurrency news. It is designed with a multi-layered fallback mechanism to ensure data availability even if specific APIs hit rate limits.

Public Interface:

• __init__(api_key: Optional[str] = None): Initializes the connector. It automatically attempts to load the NEWSAPI_KEY from environment variables if not provided.
• get_bitcoin_news(language: str = "en", limit: int = 20) -> List[Dict]: The primary method for fetching recent news.

Data Sourcing Logic:

1. RSS Feeds (Free): Attempts to fetch and parse feeds from high-authority sources like CoinTelegraph and CoinDesk.
2. Google News RSS (Free): If dedicated crypto feeds fail, it queries Google News via RSS.
3. NewsAPI (Paid/Tiered): If an API key is present and RSS feeds are insufficient, it utilizes the newsapi.org REST API.

Usage Example:

from data_connectors.newsapi_connector import NewsAPIConnector

connector = NewsAPIConnector()
news = connector.get_bitcoin_news(limit=5)

for article in news:
    print(f"Source: {article['source']} - Title: {article['title']}")

Fear & Greed Index

The system integrates market psychology via the Crypto Fear & Greed Index. This data is consumed primarily by the Sentiment Agent to populate the SentimentMetadata schema, allowing the model to interpret whether the market is in a state of "Extreme Greed" (potential reversal) or "Extreme Fear" (potential bottom).

Market Data Connectors

The system interfaces with standard financial APIs to retrieve price action and volume data.

CoinGecko Integration

Used primarily within agent tools to fetch current and historical asset prices.

Key Tools:

• get_price(crypto: str): Returns the current USD valuation.
• get_historical_price(crypto: str, date: str): Retrieves price data for a specific historical point (format: DD-MM-YYYY).

Usage Example:

# Typically invoked via Agent Tools
from pycoingecko import CoinGeckoAPI

cg = CoinGeckoAPI()
price = cg.get_price(ids='bitcoin', vs_currencies='usd')

Technical & On-Chain Data

While the system supports direct API connections, it often processes raw data through a RAG (Retrieval-Augmented Generation) pipeline for complex analysis.

Knowledge Indexer

The KnowledgeIndexer in agents/rag/indexer.py processes static data and research papers to provide agents with "expert" context that isn't available via standard price APIs.

• Supported Formats: .md, .txt, .pdf (via curated markdown).
• Domain Mapping: Data is partitioned into technical, sentiment, fundamental, and risk folders.

Configuration

Connectors rely on environment variables for authentication. These should be defined in a .env file at the project root.

| Variable | Description | Source | | :--- | :--- | :--- | | NEWSAPI_KEY | API key for newsapi.org (Optional fallback) | newsapi.org | | GROQ_API_KEY | Required for the Llama-3 based agents | groq.com | | OPENAI_API_KEY | Required if using OpenAI models for embedding | openai.com |

Error Handling

All connectors implement defensive programming patterns:

• Timeouts: Requests are capped at 10 seconds to prevent agent hang-ups.
• Graceful Degradation: If a data source (like a specific RSS feed) is down, the system logs a [WARN] and moves to the next available provider without crashing the orchestration loop.
• Type Safety: Responses are validated against Pydantic schemas (see agents/schemas.py) before being passed to the agents.