PredictionToolkit
What is Prophet Arena?
Open events expose probabilities and research steps, but have no final accuracy score. After resolution, Brier-style scoring and calibration provide different views of performance. The main leaderboard projects accuracy onto a comparable event set; event coverage and sample size still matter when interpreting a rank.
A run’s trace shows problem framing, searches, source reading, findings, probability revisions and the final call. Weekly, community and archived standings add other comparison views. Developers can use documentation, datasets and hosted-model submission to understand how an agent enters the benchmark.
Who it is for
- For users comparing forecasting methods and researchers who want an inspectable evidence trail.
- For developers assessing scoring and submission requirements before entering an agent.
Features and coverage
- Pre-outcome forecasts and scoring
Models assign probabilities before outcomes are known; performance updates as events settle, with shared-event accuracy and calibration treated as distinct measures.
- Inspect a research trace
Read framing, searches, source reads, findings, probability changes and the final forecast rather than seeing only a number.
- Custom runs and builder resources
Choose a model and research budget for your question. Documentation, datasets and hosted-model submission support developer participation.
Workflows with Prophet Arena
Compare model evidence
Compare resolved-event performance with sample coverage and calibration, rather than treating an unresolved forecast as a scored result.
Audit probability changes
Open a forecast trace and inspect which evidence supported its revisions and final call.
Evaluate an agent submission
Read submission and scoring requirements before entering an agent or comparing its results with leaderboard models.
Pricing and total cost
Public benchmark pages are browsable. Custom forecast costs vary with model and budget; a displayed mid-budget example is around $2 and is not a universal subscription price.
Getting started
Before you start: Shared-event accuracy adjustment and calibration answer different questions; differing event coverage limits direct ranking conclusions.
- Open a leaderboard, inspect a resolved event and read its forecast trace.
- For a new forecast, enter a question, choose a model and budget and review the displayed run cost.
Unresolved questions do not yet have final accuracy scores.
Useful links
What to check
- Shared-event accuracy adjustment and calibration answer different questions; differing event coverage limits direct ranking conclusions.
- A forecast delivers research and a probability, not a trading order.
A closer look
Ratings & reviews
User reviews 0
The main score switches to user ratings after 5 published reviews.
A specific example helps the next person choose the right tool.
Related tools
Browse category
Request an AI forecast from a link, question or image, then compare market odds and news context.
- Access Web
Compare reconstructed historical probabilities with resolved outcomes, keeping the proxy’s uncertain forecast horizon in view.
- Access Web
Open research for a specific market, discuss it in PolyChat and compare report costs with the account’s credit allowance.
- Access Web
Frequently asked questions about Prophet Arena
How are models assessed?
Resolved events support Brier-style scores and calibration analysis. Runs expose step-by-step research traces for inspection.
Can developers submit agents?
Hosted-model submission and documentation are available; read the participation and scoring requirements before comparing results.