Case Study
Polymarket Sentiment Agent
Prediction-market agent where the LLM only labels headline sentiment and Python computes the Bayesian edge versus market price. Public track record joined to real resolutions, admin-gated controls, 77 tests. Paper trading; deploy pending.
Executive Summary
An autonomous agent that watches Polymarket markets and crypto news, uses an LLM purely as an NLP parser, then computes a deterministic Bayesian posterior in Python. Edge equals posterior minus market-implied prior. Every signal is logged and later joined to the market's real resolution, so the track record is verifiable rather than claimed.
Problem & Constraints
Most AI trading demos ask the model for price predictions, but LLMs are uncalibrated. This agent splits NLP parsing from the quantitative decision, runs in paper-trading mode only, and publishes its hit rate against actual Polymarket outcomes.
Architecture
News RSS ingestion → LLM sentiment labeling → Bayesian posterior update → edge = posterior - market price → paper trader → signal log → resolution join (real Polymarket outcomes) → public track-record page. FastAPI backend + React frontend, admin-gated controls.
Methodology
- Ingested real-time news via free RSS and Polymarket public APIs
- LLM labels headline sentiment; Python computes calibrated Bayesian updates
- Signals persisted and joined to real market resolutions for an auditable track record
- Admin endpoints authenticated; timezone-handling crash in the portfolio view found and fixed with a regression test
Results & Metrics
| Metric | Result |
|---|---|
| Mode | Paper trading only |
| Track record | Joined to real Polymarket resolutions |
| Tests | 77 (pytest) |
| NLP | LLM as parser only — no LLM price predictions |
Tech Stack
Python, FastAPI, React, Vite, Docker, Polymarket API
Future Work
Redeployment (previous Fly.io demo retired), multi-market expansion, backtesting framework.