choijaewook/polymarket-arb-bot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
49e033f373f12edf6860a9d6ff5b7073007f8668
polymarket-arb-bot/docs/gap-analysis.md
choijaewook 7f45211276 update 03-22 09:28
2026-03-22 09:28:14 +09:00

23 KiB
Raw Blame History

Design-Implementation Gap Analysis Report

Summary: Comprehensive comparison of polymarket-arb-bot-prompt.md specification against actual implementation

Author: gap-detector Created: 2026-03-18 Last Modified: 2026-03-18 Status: Review

Analysis Overview

  • Analysis Target: Polymarket Temporal Arbitrage Bot
  • Design Document: D:\PRJ\poly_company\dtr2_poly\polymarket-arb-bot-prompt.md
  • Implementation Path: D:\PRJ\poly_company\dtr2_poly\polymarket-arb-bot\
  • Analysis Date: 2026-03-18

Overall Scores

Category Score Status
Directory Structure 90% PARTIAL
Dependencies 100% IMPLEMENTED
API Integrations 88% PARTIAL
Core Strategy Logic 95% IMPLEMENTED
Risk Management 100% IMPLEMENTED
Implementation Phases 82% PARTIAL
Execution Commands 67% PARTIAL
Speed Requirements 70% PARTIAL
Asset-Specific Handling 100% IMPLEMENTED
Overall 87% PARTIAL

1. Directory Structure (Spec lines 75-123)

Status: PARTIAL (90%)

Specified File/Directory Status Notes
.env IMPLEMENTED .env.example exists (actual .env gitignored, correct)
config.toml IMPLEMENTED All sections present, values match spec
requirements.txt IMPLEMENTED All 14 dependencies present
src/__init__.py IMPLEMENTED Exists (empty)
src/main.py IMPLEMENTED Full ArbBot class with event loop
src/config.py IMPLEMENTED Nested dataclasses, TOML loader, env overlay
src/feeds/__init__.py IMPLEMENTED Exports BinanceFeed, PolymarketFeed
src/feeds/binance_ws.py IMPLEMENTED Full async WebSocket with reconnect
src/feeds/coinbase_ws.py MISSING Coinbase cross-validation feed not implemented
src/feeds/polymarket_ws.py IMPLEMENTED Full orderbook WebSocket with auto-reconnect
src/strategy/__init__.py IMPLEMENTED Exports all strategy classes
src/strategy/temporal_arb.py IMPLEMENTED Full strategy with probability model
src/strategy/sum_to_one.py IMPLEMENTED Full sum-to-one with signal generation
src/strategy/spread_capture.py IMPLEMENTED Full spread capture (disabled by default)
src/strategy/signal.py IMPLEMENTED SignalAggregator coordinating all strategies
src/execution/__init__.py IMPLEMENTED Exists
src/execution/clob_client.py IMPLEMENTED EIP-712 auth, order ops, async wrapper
src/execution/order_manager.py IMPLEMENTED Full lifecycle, expiry cancellation
src/execution/position_tracker.py IMPLEMENTED Mark-to-market, PnL tracking
src/market/__init__.py IMPLEMENTED Exports MarketDiscovery, WindowTracker
src/market/discovery.py IMPLEMENTED Gamma API with filtering
src/market/window_tracker.py IMPLEMENTED Clock-aligned windows for 3 assets x 2 TFs
src/market/oracle.py IMPLEMENTED Chainlink ABI, latency monitoring
src/risk/__init__.py IMPLEMENTED Exists
src/risk/position_sizer.py IMPLEMENTED Kelly Criterion with 4 safety caps
src/risk/risk_manager.py IMPLEMENTED Daily loss, exposure, position count checks
src/risk/fee_calculator.py IMPLEMENTED 5M vs 15M fees, breakeven calc, EV calc
src/data/__init__.py IMPLEMENTED Exports TradeDB
src/data/models.py IMPLEMENTED All models: Market, Position, Trade, etc.
src/data/db.py IMPLEMENTED SQLite with trades, windows, daily_summary, balance
src/utils/__init__.py IMPLEMENTED Exists
src/utils/logger.py IMPLEMENTED structlog with console/JSON renderers
src/utils/telegram.py IMPLEMENTED Trade, fill, daily, error, halt notifications
src/utils/metrics.py IMPLEMENTED Rolling windows, asset breakdown, hourly stats
paper_trade.py IMPLEMENTED Full paper bot with virtual execution engine
backtest.py IMPLEMENTED Synthetic + historical modes (not in spec but bonus)
dashboard/app.py MISSING Streamlit dashboard not implemented

2. Dependencies (Spec lines 126-143)

Status: IMPLEMENTED (100%)

Dependency Spec Version Impl Version Status
py-clob-client >=0.18.0 >=0.18.0 IMPLEMENTED
web3 >=6.0 >=6.0 IMPLEMENTED
websockets >=12.0 >=12.0 IMPLEMENTED
aiohttp >=3.9 >=3.9 IMPLEMENTED
python-dotenv >=1.0 >=1.0 IMPLEMENTED
toml >=0.10 >=0.10 IMPLEMENTED
eth-account >=0.11 >=0.11 IMPLEMENTED
structlog >=24.0 >=24.0 IMPLEMENTED
sqlite-utils >=3.36 >=3.36 IMPLEMENTED
ccxt >=4.2 >=4.2 IMPLEMENTED
numpy >=1.26 >=1.26 IMPLEMENTED
pandas >=2.2 >=2.2 IMPLEMENTED
streamlit >=1.32 >=1.32 IMPLEMENTED
python-telegram-bot >=21.0 >=21.0 IMPLEMENTED

Note: streamlit is listed in requirements.txt but dashboard/app.py is not implemented. python-telegram-bot is in requirements.txt but the actual Telegram implementation uses aiohttp directly instead.

3. API Integrations (Spec lines 147-240)

Status: PARTIAL (88%)

3.1 Polymarket Gamma API Market Discovery

Requirement Status Location
GET /events endpoint IMPLEMENTED src/market/discovery.py:16
Query params: tag=crypto, active=true, closed=false, limit=100 IMPLEMENTED src/market/discovery.py:17-22
Filter: "Up or Down" in title IMPLEMENTED src/market/discovery.py:179 (regex)
Filter: "5 Min" or "15 Min" IMPLEMENTED src/market/discovery.py:30-37
Filter: BTC / ETH / SOL IMPLEMENTED src/market/discovery.py:24-28
Filter: enableOrderBook == true IMPLEMENTED src/market/discovery.py:201
Filter: active == true IMPLEMENTED src/market/discovery.py:199
Continuous discovery loop IMPLEMENTED src/market/discovery.py:235
Rate limit handling (429) IMPLEMENTED src/market/discovery.py:143-147

3.2 CLOB API Authentication (EIP-712)

Requirement Status Location
py-clob-client ClobClient usage IMPLEMENTED src/execution/clob_client.py:53-61
host, key, chain_id, signature_type, funder params IMPLEMENTED src/execution/clob_client.py:56-61
API key creation (create_api_key) IMPLEMENTED src/execution/clob_client.py:68
Order placement (OrderArgs, GTC) IMPLEMENTED src/execution/clob_client.py:122-132
Market order (FOK) IMPLEMENTED src/execution/clob_client.py:155-167
Async wrapper (thread executor) IMPLEMENTED src/execution/clob_client.py:43-46

3.3 Binance WebSocket

Requirement Status Location
Combined stream URL format IMPLEMENTED src/feeds/binance_ws.py:31-34
btcusdt@trade, ethusdt@trade, solusdt@trade IMPLEMENTED src/feeds/binance_ws.py:21-25
Parse stream name -> symbol mapping IMPLEMENTED src/feeds/binance_ws.py:206-209
Extract price (data['p']), volume (data['q']) IMPLEMENTED src/feeds/binance_ws.py:212-213
Auto-reconnect with exponential backoff IMPLEMENTED src/feeds/binance_ws.py:96-131

3.4 Polymarket WebSocket (Orderbook)

Requirement Status Location
Subscribe message format IMPLEMENTED src/feeds/polymarket_ws.py:211-217
Channel: "book" (spec says "market") CHANGED Spec: "channel": "market", Impl: "channel": "book"
assets_ids field (spec: assets_id singular) CHANGED Spec: "assets_id", Impl: "assets_ids" (array)
Parse bid/ask updates IMPLEMENTED src/feeds/polymarket_ws.py:250-269
Maintain OrderBookSnapshot IMPLEMENTED src/feeds/polymarket_ws.py:271-278
Auto-reconnect IMPLEMENTED src/feeds/polymarket_ws.py:108-143

3.5 Coinbase WebSocket (Cross-validation)

Requirement Status Notes
coinbase_ws.py module MISSING File does not exist
Cross-validation feed MISSING No Coinbase integration at all

Note: The spec marks this as "auxiliary feed for cross-validation". The .env.example includes COINBASE_WS_URL but no implementation exists.

4. Core Strategy Logic (Spec lines 244-333)

Status: IMPLEMENTED (95%)

4.1 TemporalArbStrategy Class

Spec Requirement Status Implementation Detail
__init__ with config params IMPLEMENTED Takes TemporalArbConfig, RiskConfig, FeesConfig
min_price_move_pct = 0.15 CHANGED Config default 0.15, but config.toml has 0.03
max_poly_price = 0.65 IMPLEMENTED max_poly_entry_price = 0.65
min_edge = 0.20 CHANGED Config default 0.20, but config.toml has 0.05
exit_before_resolution_sec = 5 IMPLEMENTED Matches spec
max_position_per_market = 5000 IMPLEMENTED In RiskConfig
taker_fee_rate = 0.0156 IMPLEMENTED FeesConfig with timeframe-aware fee

4.2 Evaluate Function - 10 Steps

Step Spec Description Status Notes
1 Price direction calculation IMPLEMENTED (cex_price - window_start_price) / window_start_price * 100
2 Direction magnitude check (abs < min_price_move_pct) IMPLEMENTED temporal_arb.py:85-86
3 Direction determination (UP/DOWN) IMPLEMENTED temporal_arb.py:89
4 Probability estimation IMPLEMENTED Enhanced: multi-factor model (base + time decay + volatility)
5 Polymarket price selection IMPLEMENTED temporal_arb.py:98-99
6 Edge calculation (prob - price - fee) IMPLEMENTED temporal_arb.py:112-113
7 Edge threshold check IMPLEMENTED temporal_arb.py:115-116
8 Entry price ceiling check IMPLEMENTED temporal_arb.py:102-103
9 Time-to-resolution check IMPLEMENTED temporal_arb.py:93-95
10 Position sizing IMPLEMENTED temporal_arb.py:120-125

4.3 Kelly Criterion Position Sizing

Requirement Status Notes
Formula: f* = (bp - q) / b IMPLEMENTED temporal_arb.py:233
b = (1/price - 1) IMPLEMENTED temporal_arb.py:226
p = estimated_prob IMPLEMENTED temporal_arb.py:227
Max 25% Kelly cap IMPLEMENTED temporal_arb.py:234 via kelly_fraction_cap
Dollar size = balance * kelly_fraction IMPLEMENTED temporal_arb.py:236
max_position_per_market cap IMPLEMENTED temporal_arb.py:236
Return shares = dollar_size / price IMPLEMENTED temporal_arb.py:237

4.4 Signal Generation

Requirement Status Notes
Signal dataclass with all fields IMPLEMENTED models.py:72-81
Signal aggregation across strategies IMPLEMENTED signal.py SignalAggregator
Signal deduplication / cooldown IMPLEMENTED 30-second cooldown per direction/asset/timeframe

5. Risk Management (Spec lines 337-394)

Status: IMPLEMENTED (100%)

5.1 Config Parameters (config.toml)

Parameter Spec Value Impl Value Status
mode "paper" "paper" IMPLEMENTED
assets ["BTC","ETH","SOL"] ["BTC","ETH","SOL"] IMPLEMENTED
timeframes ["5M","15M"] ["5M","15M"] IMPLEMENTED
log_level "INFO" "INFO" IMPLEMENTED
temporal_arb.enabled true true IMPLEMENTED
temporal_arb.min_price_move_pct 0.15 0.03 CHANGED
temporal_arb.max_poly_entry_price 0.65 0.65 IMPLEMENTED
temporal_arb.min_edge 0.20 0.05 CHANGED
temporal_arb.exit_before_resolution_sec 5 5 IMPLEMENTED
sum_to_one.enabled true true IMPLEMENTED
sum_to_one.min_spread_after_fee 0.02 0.02 IMPLEMENTED
spread_capture.enabled false false IMPLEMENTED
spread_capture.spread_target 0.04 0.04 IMPLEMENTED
risk.max_position_per_market_usd 5000 5000 IMPLEMENTED
risk.max_total_exposure_usd 20000 20000 IMPLEMENTED
risk.max_daily_loss_usd 2000 2000 IMPLEMENTED
risk.kelly_fraction_cap 0.25 0.25 IMPLEMENTED
risk.max_concurrent_positions 6 6 IMPLEMENTED
fees.taker_fee_5m 0.0156 0.0156 IMPLEMENTED
fees.taker_fee_15m 0.03 0.03 IMPLEMENTED
exchange.binance.ws_url spec URL matches IMPLEMENTED
exchange.binance.symbols 3 pairs matches IMPLEMENTED
exchange.polymarket.* all URLs matches IMPLEMENTED
notifications.* all fields matches IMPLEMENTED

Note on CHANGED values: min_price_move_pct (0.15->0.03) and min_edge (0.20->0.05) were intentionally lowered in config.toml (comments explain: "lower for evaluation"). The default dataclass values still match the spec. This appears to be parameter tuning, not a design deviation.

5.2 Risk Manager Features

Feature Status Location
Maximum exposure check IMPLEMENTED risk_manager.py:110-117
Daily loss limit with auto-halt IMPLEMENTED risk_manager.py:93-108
Position count limit IMPLEMENTED risk_manager.py:119-125
Trading halt/resume IMPLEMENTED risk_manager.py:131-141
Risk summary for dashboard IMPLEMENTED risk_manager.py:147-161

5.3 Fee Calculator

Feature Status Notes
5M taker fee (1.56%) IMPLEMENTED fee_calculator.py:39
15M taker fee (3%) IMPLEMENTED fee_calculator.py:39
Fee applied to profit, not cost IMPLEMENTED fee_calculator.py:40-43
Breakeven probability calc IMPLEMENTED fee_calculator.py:45-57
Expected value calculation IMPLEMENTED fee_calculator.py:80-90

6. Implementation Phases (Spec lines 398-430)

Status: PARTIAL (82%)

Phase 1: Infrastructure & Data Pipeline

Item Status Notes
Project scaffolding IMPLEMENTED All directories, deps, config
Binance WebSocket BTC/ETH/SOL IMPLEMENTED Full async with reconnect
Polymarket Gamma API discovery IMPLEMENTED With filtering and continuous loop
Window tracker (start price / end time) IMPLEMENTED Clock-aligned for 3x2 windows
Logging & SQLite trade log IMPLEMENTED structlog + SQLite with 4 tables

Phase 2: Strategy Engine

Item Status Notes
Temporal arb signal generator IMPLEMENTED Enhanced probability model
Sum-to-one arb monitor IMPLEMENTED Full with signal generation
Probability estimation model IMPLEMENTED 3-factor model (base + time + volatility)
Kelly Criterion sizing IMPLEMENTED With 4 safety caps in PositionSizer

Phase 3: Execution Engine

Item Status Notes
CLOB auth (EIP-712 + API Key) IMPLEMENTED py-clob-client wrapper
Order manager (create/modify/cancel) IMPLEMENTED Full lifecycle management
Position tracker (real-time PnL) IMPLEMENTED Mark-to-market updates
Fee calculator (5M vs 15M) IMPLEMENTED With breakeven and EV calcs

Phase 4: Risk & Monitoring

Item Status Notes
Risk manager (daily loss, max exposure) IMPLEMENTED Auto-halt on breach
Telegram notification bot IMPLEMENTED Trade, fill, daily, error, halt alerts
Streamlit real-time dashboard MISSING dashboard/app.py does not exist
Paper trading mode IMPLEMENTED Full virtual execution engine

Phase 5: Optimization & Live

Item Status Notes
Paper trading validation IMPLEMENTED paper_trade.py with simulated orderbooks
Small-amount live trading IMPLEMENTED src/main.py with live mode support
Parameter tuning PARTIAL Config supports it, no auto-tuning
Performance analysis IMPLEMENTED backtest.py (bonus: not in original spec)

7. Execution Commands (Spec lines 482-496)

Status: PARTIAL (67%)

Command Status Notes
python paper_trade.py IMPLEMENTED Full paper trading bot at project root
python src/main.py IMPLEMENTED Full live/paper ArbBot with all components
streamlit run dashboard/app.py MISSING Dashboard directory does not exist

8. Speed Requirements (Spec lines 463-467)

Status: PARTIAL (70%)

Requirement Status Implementation
Opportunity window: 2-5 seconds IMPLEMENTED Strategy evaluates on every tick
Order detect -> fill < 500ms PARTIAL Async architecture supports it; actual latency depends on network
Binance tick -> signal: < 50ms PARTIAL Callback-based, should be <50ms; no explicit measurement
Signal -> CLOB order: < 200ms PARTIAL Uses thread executor for sync SDK; no latency metrics
Server near Polygon RPC N/A Deployment concern, not code

Notes:

  • src/utils/logger.py provides log_timing context manager for measurement, but it is not used in the hot path (strategy eval -> order submission).
  • No explicit latency benchmarking or monitoring exists in the strategy pipeline.
  • The signal.py aggregator has rate limiting at 0.5s intervals per market, which could theoretically delay signals.

9. Asset-Specific Handling (BTC, ETH, SOL Simultaneous)

Status: IMPLEMENTED (100%)

Requirement Status Notes
BTC support IMPLEMENTED Asset.BTC enum, Binance stream, Gamma filter
ETH support IMPLEMENTED Asset.ETH enum, Binance stream, Gamma filter
SOL support IMPLEMENTED Asset.SOL enum, Binance stream, Gamma filter
Simultaneous operation IMPLEMENTED WindowTracker manages 6 windows (3 assets x 2 TFs)
Per-asset price tracking IMPLEMENTED BinanceFeed._latest_prices keyed by symbol
Oracle monitoring per asset IMPLEMENTED OracleMonitor.FEEDS has all 3 Chainlink addresses

Differences Found

MISSING Features (Design O, Implementation X)

Item Spec Location Description Impact
src/feeds/coinbase_ws.py Spec line 87 Coinbase WebSocket for cross-validation Low
dashboard/app.py Spec line 122 Streamlit real-time dashboard Medium
Explicit latency measurement Spec lines 463-467 No tick-to-order latency tracking in hot path Medium

ADDED Features (Design X, Implementation O)

Item Implementation Location Description
backtest.py Project root Synthetic + historical backtester (not in spec)
Enhanced probability model temporal_arb.py:169-205 Multi-factor model (base + time decay + volatility boost) vs spec's simple linear model
PositionSizer class src/risk/position_sizer.py Standalone Kelly sizer with 4 caps (half-Kelly, exposure cap) beyond spec
OracleMonitor class src/market/oracle.py Chainlink oracle latency tracking (spec mentions oracle but no dedicated monitor)
Signal deduplication signal.py:78-84 30-second cooldown per market direction
Simulated orderbooks in paper mode paper_trade.py:461-529 Generates fake Polymarket prices for paper trading when no real market exists
Balance history table src/data/db.py:102-113 Tracks balance over time (not in spec)
Early exit logic temporal_arb.py:245-284 should_exit_early() method for reversal/take-profit detection

CHANGED Features (Design != Implementation)

Item Spec Implementation Impact
WebSocket subscribe channel "channel": "market" "channel": "book" Low (likely API correction)
WebSocket subscribe field "assets_id": TOKEN_ID "assets_ids": [token_id] (array) Low (likely API correction)
config.toml min_price_move_pct 0.15 0.03 Low (tuning, default still 0.15)
config.toml min_edge 0.20 0.05 Low (tuning, default still 0.20)
Probability estimation min(0.95, 0.55 + abs(pct) * 100) Multi-factor: base + time_decay + vol_boost Low (improvement)
Telegram implementation python-telegram-bot SDK Direct aiohttp HTTP calls Low (simpler, fewer deps)
Fee calculation scope taker_fee_rate = 0.0156 (fixed) Timeframe-aware via FeesConfig.fee_for_timeframe() Low (improvement)

Summary by Component

Component Files Completeness Quality
Config & Setup 4 100% High - frozen dataclasses, env overlay
Data Feeds 2/3 67% High - auto-reconnect, heartbeat
Market Discovery 3 100% High - rate limiting, continuous loop
Strategy Engine 4 100% High - 3 strategies, aggregator, dedup
Execution Engine 3 100% High - async SDK wrapper, lifecycle
Risk Management 3 100% High - auto-halt, multi-check
Data Layer 2 100% High - 4 tables, aggregation queries
Utilities 3 100% High - structured logging, metrics
Entry Points 2/3 67% High - paper + live, missing dashboard
Bonus 1 N/A backtest.py not in spec

Recommended Actions

Immediate Actions (Priority: High)

  1. Implement dashboard/app.py -- The Streamlit dashboard is specified in the design and streamlit is already in requirements.txt. This provides visibility into bot performance and is explicitly called out as an execution command (streamlit run dashboard/app.py). Should display: live positions, PnL chart, trade log, risk status, asset breakdown using data from MetricsCollector and TradeDB.

Medium Priority

  1. Add latency instrumentation -- The log_timing utility exists in src/utils/logger.py but is never used in the critical path. Wrap the strategy evaluation and order submission flows to verify the spec's <50ms and <200ms targets.

  2. Implement src/feeds/coinbase_ws.py -- Coinbase cross-validation was specified as auxiliary. Consider adding as a secondary price source to detect exchange-specific anomalies or confirm Binance prices.

Low Priority / Documentation

  1. Document config.toml parameter changes -- The deployed config.toml has min_price_move_pct=0.03 and min_edge=0.05 vs spec defaults of 0.15 and 0.20. These are clearly intentional tuning choices (inline comments explain), but should be documented in a tuning log.

  2. WebSocket field differences -- The Polymarket WebSocket uses "channel": "book" and "assets_ids" (array) instead of spec's "channel": "market" and "assets_id" (singular). These appear to be corrections reflecting the actual API behavior. Update the spec to match.

Match Rate: 87%

The implementation covers the vast majority of the specification with high code quality. The two primary gaps (Streamlit dashboard and Coinbase feed) are well-contained and do not affect core trading functionality. Several additions (backtester, oracle monitor, enhanced probability model, early exit logic) represent improvements beyond the original spec.

Verdict

Match Rate >= 70% && < 90%: There are some differences. Document update and focused implementation recommended.

Specific focus areas:

  • Build the Streamlit dashboard to complete Phase 4
  • Add latency metrics to validate speed requirements
  • Update spec to reflect API corrections and intentional improvements
Reference in New Issue View Git Blame Copy Permalink