Executive forecast
Base horizon median
83.493
percent resolved
Base 80% interval
43.591–86.089
Model, residual and scenario uncertainty
Recent annualized pace
27.7%
Log-interpolated six-month window
Recalculate
2026-08-27
Earlier if a trigger fires
Decision implication. Use the case to test bounded forecasting and saturation; do not treat system leaderboard gains as isolated model capability or verified organizational value.
Compact Canonical Model
Ys(t) = Bs(t) min { Q0 exp[∫0t gQ,s(u)du], Ms(t)Ds(t)Us(t) }Realized output is limited by technical capacity and absorptive demand, with branch-specific and transfer constraints applied exactly once.
Generalized realization family
Y = BX[ α(BQQ)−ρ + (1−α)(BZZ)−ρ ]−1/ρAs ρ → ∞, the generalized family converges to the compact hard minimum.
Scenario forecast
| Scenario | Probability | Horizon median | 80% interval | Assumptions |
|---|---|---|---|---|
| Base | 58.0% | 83.493 | 43.591 – 86.089 | Model-averaged historical dynamics continue within the benchmark definition. |
| Accelerated | 20.0% | 86.772 | 45.303 – 89.470 | Recent acceleration persists. |
| Saturation / measurement stress | 17.0% | 75.797 | 39.573 – 78.153 | Benchmark saturation or measurement limitations dominate. |
| Discontinuous | 5.0% | 83.493 | 43.591 – 86.184 | A structural improvement loop materially changes the trajectory. |
Current pace
Current
79.200
6 months ago
70.079
12 months ago
54.874
Log-growth acceleration
-0.494
per year²
Rates use log interpolation at six- and twelve-month anchors.
Model competition
| Model | Converged | AICc | Log RMSE | Horizon prediction |
|---|---|---|---|---|
| logistic | Yes | -14.545 | 0.012 | 84.009 |
| accelerating | Yes | -11.497 | 0.014 | 42.479 |
| linear | Yes | -6.008 | 0.058 | 123.439 |
| exponential | Yes | 0.545 | 0.093 | 159.322 |
| decaying_acceleration | Yes | 27.115 | 0.011 | 67.547 |
| change_point | Yes | 35.667 | 0.021 | 117.027 |
Milestones
| Milestone | Scenario | Probability | Median crossing | Early 10% | Late 90% |
|---|---|---|---|---|---|
| 80% resolved | base | 75.0% | 2026-07-28 | 2026-07-28 | 2026-07-28 |
| 80% resolved | accelerated | 75.0% | 2026-07-28 | 2026-07-28 | 2026-07-28 |
| 80% resolved | downside | 74.0% | 2026-07-28 | 2026-07-28 | 2026-07-28 |
| 80% resolved | discontinuous | 75.0% | 2026-07-28 | 2026-07-28 | 2026-07-28 |
| 90% resolved | base | 1.0% | 2026-07-28 | 2026-07-28 | 2026-07-28 |
| 90% resolved | accelerated | 8.0% | 2026-12-31 | 2026-12-01 | 2026-12-31 |
| 90% resolved | downside | 1.0% | 2026-07-28 | 2026-07-28 | 2026-07-28 |
| 90% resolved | discontinuous | 3.0% | 2026-09-01 | 2026-07-28 | 2026-11-01 |
Bottlenecks
| Constraint | Severity | Mechanism | Release condition | Mitigation |
|---|---|---|---|---|
| Benchmark saturation and configuration drift | high | As scores rise, remaining tasks and changes in scaffolds, tools, attempts, and model versions limit comparability. | A harder fresh benchmark with controlled configuration is available. | Track system metadata and use a bounded model with explicit version breaks. |
Trigger dashboard
| Trigger | Direction | Threshold | Forecast effect | Action |
|---|---|---|---|---|
| Official leaderboard or benchmark revision | event | New verified record or benchmark version | May change both the level and the measurement regime. | Freeze the upstream file and rerun the bounded hindcast. |
Optimal plan for today’s regime
no regret actions
Freeze source snapshots and record benchmark/configuration changes.; Use rolling-origin scores rather than in-sample fit alone.
option preserving actions
Maintain multiple model classes and trigger-based recalibration.
next experiment
Add the complete upstream dataset and rerun the preregistered hindcast.
delay pending evidence
Do not translate benchmark movement directly into revenue or deployment.
stop expand pivot
Expand confidence only after independent replication and prospective calibration.
Evidence ledger
| ID | Evidence | Grade | Measurement date | Publisher |
|---|---|---|---|---|
| swe-bench-official-leaderboard | SWE-bench official leaderboard data — Verified track | A | 2025-12-05 | SWE-bench |
| swe-bench-frontier-excerpt | FWAA selected SWE-bench Verified frontier excerpt | B | 2025-12-05 | MONTREAL.AI |
Confidence and limitations
Evidence quality score
0.900
Model disagreement
0.437
Base p80 relative width
50.9%
Boundary. Research and decision-support output only. No warranty, assurance, legal, investment, tax, medical, or regulatory advice is provided.
Reproducibility
Input SHA-256
6ac8b2c7b01fd1825561c5dfc5eae5a03467b31fd6c2e2f41d28fff50263fc3a
Seed
20260729
Bootstrap samples
100
Open complete machine-readable result
{
"protocol_version": "2.0.0",
"engine_version": "2.0.0",
"forecast_id": "swe-bench-verified-frontier-demonstration-2026-07-28",
"generated_at": "2026-07-28T16:00:00+00:00",
"research_cutoff": "2026-07-28T18:00:00+00:00",
"target": {
"question": "What bounded trajectory is implied by the selected public SWE-bench Verified frontier records through December 2026?",
"metric": "SWE-bench Verified frontier resolved rate",
"unit": "percent resolved",
"scope": "Selected official leaderboard frontier excerpt",
"horizon_end": "2026-12-31",
"target_kind": "probability",
"success_definition": "Percent of the 500 human-filtered SWE-bench Verified instances resolved under the recorded system configuration.",
"direction": "higher_is_better",
"lower_bound": 0,
"upper_bound": 100
},
"executive_forecast": {
"base_horizon_median": 83.49338009688697,
"base_horizon_p80": {
"level": 0.8,
"lower": 43.59097247237167,
"upper": 86.0893224190951
},
"decision_implication": "Use the case to test bounded forecasting and saturation; do not treat system leaderboard gains as isolated model capability or verified organizational value.",
"validation_status": "Probabilistic reference forecast; not a guarantee or assurance engagement."
},
"observations": [
{
"date": "2024-06-20",
"value": 33.6,
"evidence_ids": [
"swe-bench-official-leaderboard",
"swe-bench-frontier-excerpt"
],
"comparable": true,
"note": "Official leaderboard record; system-level result"
},
{
"date": "2024-10-22",
"value": 49.0,
"evidence_ids": [
"swe-bench-official-leaderboard",
"swe-bench-frontier-excerpt"
],
"comparable": true,
"note": "Official leaderboard record; checked flag false"
},
{
"date": "2025-01-10",
"value": 60.2,
"evidence_ids": [
"swe-bench-official-leaderboard",
"swe-bench-frontier-excerpt"
],
"comparable": true,
"note": "Official leaderboard record; multiple attempts"
},
{
"date": "2025-02-03",
"value": 60.8,
"evidence_ids": [
"swe-bench-official-leaderboard",
"swe-bench-frontier-excerpt"
],
"comparable": true,
"note": "Official leaderboard record; multiple attempts"
},
{
"date": "2025-07-31",
"value": 74.8,
"evidence_ids": [
"swe-bench-official-leaderboard",
"swe-bench-frontier-excerpt"
],
"comparable": true,
"note": "Official leaderboard record"
},
{
"date": "2025-09-28",
"value": 78.8,
"evidence_ids": [
"swe-bench-official-leaderboard",
"swe-bench-frontier-excerpt"
],
"comparable": true,
"note": "Official leaderboard record; multiple attempts"
},
{
"date": "2025-12-05",
"value": 79.2,
"evidence_ids": [
"swe-bench-official-leaderboard",
"swe-bench-frontier-excerpt"
],
"comparable": true,
"note": "Official leaderboard record"
}
],
"evidence": [
{
"id": "swe-bench-official-leaderboard",
"title": "SWE-bench official leaderboard data — Verified track",
"publisher": "SWE-bench",
"source_url": "https://github.com/SWE-bench/swe-bench.github.io/blob/master/data/leaderboards.json",
"quality_grade": "A",
"measurement_date": "2025-12-05",
"publication_date": "2025-12-05",
"accessed_at": "2026-07-28T14:00:00-04:00",
"primary": true,
"claim": "Official leaderboard records for system-level percent resolved on SWE-bench Verified.",
"license": null,
"sha256": null,
"notes": "The hash is the observed Git blob identifier, not a SHA-256 digest."
},
{
"id": "swe-bench-frontier-excerpt",
"title": "FWAA selected SWE-bench Verified frontier excerpt",
"publisher": "MONTREAL.AI",
"source_url": "https://github.com/MontrealAI/forecasting-a-world-that-accelerates",
"quality_grade": "B",
"measurement_date": "2025-12-05",
"publication_date": "2026-07-28",
"accessed_at": "2026-07-28T14:00:00-04:00",
"primary": false,
"claim": "Selected source-linked public frontier records for bounded-model demonstration.",
"license": null,
"sha256": null,
"notes": "Not a complete leaderboard mirror."
}
],
"current_pace": {
"anchor_dates": {
"current": "2025-12-05",
"six_months_ago": "2025-06-05",
"twelve_months_ago": "2024-12-05"
},
"anchor_values": {
"current": 79.19999999999999,
"six_months_ago": 70.07895457571365,
"twelve_months_ago": 54.873960486796626
},
"recent_continuous_growth": 0.24420107539965977,
"prior_continuous_growth": 0.4908369554499001,
"annualized_recent_change": 0.2766009977396815,
"annualized_prior_change": 0.6336829677853701,
"log_growth_acceleration": -0.49359948174931456,
"recent_doubling_time_years": 2.8384280430605795,
"note": "Rates use log interpolation at six- and twelve-month anchors."
},
"compact_canonical_model": {
"equation": "Y_s(t)=B_s(t) min{Q_0 exp[int_0^t g_Q,s(u) du], M_s(t)D_s(t)U_s(t)}",
"technical_growth_equation": "g_Q,s=beta_theta theta_dot-beta_c c_dot/c-beta_tau tau_dot/tau+beta_A A_dot/(A+epsilon)+beta_P P_dot/P+beta_R R_dot/R",
"generalized_family": "Y=B_X[alpha(B_Q Q)^(-rho)+(1-alpha)(B_Z Z)^(-rho)]^(-1/rho); rho->infinity yields the hard minimum.",
"configuration": {
"mode": "reduced_form",
"input_series_semantics": "realized_outcome"
},
"diagnostics": {
"base": {},
"accelerated": {},
"downside": {},
"discontinuous": {}
},
"plain_language": "Realized output is limited by technical capacity and absorptive demand, with branch-specific and transfer constraints applied exactly once."
},
"model_fit": {
"candidate_models": [
"linear",
"exponential",
"accelerating",
"decaying_acceleration",
"logistic",
"change_point"
],
"fits": [
{
"model": "linear",
"converged": true,
"parameters": {
"intercept": 35.98861631630591,
"slope": 34.567795267697214,
"time_origin": 0.0
},
"n_observations": 7,
"n_parameters": 3,
"log_likelihood": 10.004135337966835,
"aicc": -6.0082706759336695,
"bic": -14.170540228767731,
"rmse_log": 0.05795428758087129,
"mae_log": 0.048903546173661806,
"horizon_prediction": 123.43912612103078,
"message": "Gaussian log-residual maximum likelihood: `ftol` termination condition is satisfied."
},
{
"model": "exponential",
"converged": true,
"parameters": {
"log_x0": 3.670748124111733,
"growth": 0.5534688840685614,
"time_origin": 0.0
},
"n_observations": 7,
"n_parameters": 3,
"log_likelihood": 6.727404524603309,
"aicc": 0.5451909507933816,
"bic": -7.617078602040679,
"rmse_log": 0.09255091787185245,
"mae_log": 0.0774699008147346,
"horizon_prediction": 159.32213173430515,
"message": "Gaussian log-residual maximum likelihood (closed form)"
},
{
"model": "accelerating",
"converged": true,
"parameters": {
"log_x0": 3.520819344453681,
"growth": 1.2409099945969424,
"acceleration": -0.9097134828941319,
"time_origin": 0.0
},
"n_observations": 7,
"n_parameters": 4,
"log_likelihood": 19.748339827575897,
"aicc": -11.496679655151794,
"bic": -31.71303905893054,
"rmse_log": 0.014405719883396279,
"mae_log": 0.01150079132064422,
"horizon_prediction": 42.47915892634429,
"message": "Gaussian log-residual maximum likelihood (closed form quadratic)"
},
{
"model": "decaying_acceleration",
"converged": true,
"parameters": {
"log_x0": 3.513353386117663,
"growth": 1.3783511937241322,
"initial_acceleration": -1.5278558037525773,
"kappa": 0.7833671923915402,
"time_origin": 0.0
},
"n_observations": 7,
"n_parameters": 5,
"log_likelihood": 21.442515914627194,
"aicc": 27.114968170745612,
"bic": -33.15548108397782,
"rmse_log": 0.011309014901379467,
"mae_log": 0.009469617754784134,
"horizon_prediction": 67.54739482740639,
"message": "`ftol` termination condition is satisfied."
},
{
"model": "logistic",
"converged": true,
"parameters": {
"x0": 33.52666681117868,
"growth": 2.2549044664844544,
"capacity": 84.43367981674535,
"time_origin": 0.0
},
"n_observations": 7,
"n_parameters": 4,
"log_likelihood": 21.27263908237333,
"aicc": -14.545278164746662,
"bic": -34.76163756852541,
"rmse_log": 0.011586820697469314,
"mae_log": 0.010198676667102788,
"horizon_prediction": 84.00879411435186,
"message": "`ftol` termination condition is satisfied."
},
{
"model": "change_point",
"converged": true,
"parameters": {
"log_x0": 3.520842336117868,
"growth_pre": 1.0452753625003322,
"growth_delta": -0.7116139023613083,
"change_time": 0.5585330294256555,
"time_origin": 0.0
},
"n_observations": 7,
"n_parameters": 5,
"log_likelihood": 17.16642562252155,
"aicc": 35.6671487549569,
"bic": -24.603300499766537,
"rmse_log": 0.02083158257773717,
"mae_log": 0.018727086095665868,
"horizon_prediction": 117.02734982710624,
"message": "Gaussian log-residual maximum likelihood with grid-searched continuous change point"
}
],
"base_model_weights": {
"linear": 0.011363030820848436,
"exponential": 0.00042897113871288417,
"accelerating": 0.17672054886064706,
"decaying_acceleration": 7.292463393037677e-10,
"logistic": 0.8114874484404111,
"change_point": 1.0134255276305417e-11
},
"horizon_log_disagreement": 0.43692887800358887,
"rolling_origin_hindcast": [
{
"model": "linear",
"n": 3,
"rmse_log": 0.11181174468498256,
"mae_log": 0.11031844086187832,
"smape": 0.11019782614569994
},
{
"model": "exponential",
"n": 3,
"rmse_log": 0.2233591733505312,
"mae_log": 0.21337885643797203,
"smape": 0.212331458257889
},
{
"model": "accelerating",
"n": 3,
"rmse_log": 0.034347027226883355,
"mae_log": 0.0334210156892271,
"smape": 0.03341735769559921
},
{
"model": "decaying_acceleration",
"n": 3,
"rmse_log": 0.03139306027517948,
"mae_log": 0.02997117995756149,
"smape": 0.029968281608200493
},
{
"model": "logistic",
"n": 3,
"rmse_log": 0.018631243138609702,
"mae_log": 0.018545081087221266,
"smape": 0.01854453441807657
},
{
"model": "change_point",
"n": 0,
"rmse_log": null,
"mae_log": null,
"smape": null
},
{
"model": "model_average",
"n": 2,
"rmse_log": 0.11099865620439549,
"mae_log": 0.10896302918084722,
"smape": 0.10884319945417648
}
]
},
"forecast_dates": [
"2026-07-28",
"2026-08-01",
"2026-09-01",
"2026-10-01",
"2026-11-01",
"2026-12-01",
"2026-12-31"
],
"scenarios": [
{
"id": "base",
"label": "Base",
"kind": "base",
"probability": 0.58,
"assumptions": [
"Model-averaged historical dynamics continue within the benchmark definition."
],
"parameters": {},
"forecast_points": [
{
"date": "2026-07-28",
"median": 82.64115607779445,
"mean": 78.20630446434788,
"intervals": {
"p50": {
"level": 0.5,
"lower": 76.9880448528882,
"upper": 84.1029701454302
},
"p80": {
"level": 0.8,
"lower": 62.07686442641476,
"upper": 85.03852898186966
},
"p95": {
"level": 0.95,
"lower": 59.3358765446527,
"upper": 86.42801327848296
}
}
},
{
"date": "2026-08-01",
"median": 82.69399122775738,
"mean": 78.04937700977305,
"intervals": {
"p50": {
"level": 0.5,
"lower": 76.90227542505598,
"upper": 84.02688081435187
},
"p80": {
"level": 0.8,
"lower": 61.60174667991402,
"upper": 85.0776776556434
},
"p95": {
"level": 0.95,
"lower": 57.64805384772033,
"upper": 86.29523797864468
}
}
},
{
"date": "2026-09-01",
"median": 82.83158599716074,
"mean": 77.42011856357485,
"intervals": {
"p50": {
"level": 0.5,
"lower": 76.72743265987316,
"upper": 84.40303466615377
},
"p80": {
"level": 0.8,
"lower": 58.14241826806194,
"upper": 85.77401457061272
},
"p95": {
"level": 0.95,
"lower": 53.23279643842664,
"upper": 87.03700447973952
}
}
},
{
"date": "2026-10-01",
"median": 83.10425116317252,
"mean": 76.60300735766042,
"intervals": {
"p50": {
"level": 0.5,
"lower": 75.33771814191589,
"upper": 84.4902443480685
},
"p80": {
"level": 0.8,
"lower": 54.23186745356451,
"upper": 85.51034799368462
},
"p95": {
"level": 0.95,
"lower": 49.873242511014425,
"upper": 86.98695453161807
}
}
},
{
"date": "2026-11-01",
"median": 83.27456107025844,
"mean": 75.73532174660518,
"intervals": {
"p50": {
"level": 0.5,
"lower": 74.65680731584172,
"upper": 84.3879289524519
},
"p80": {
"level": 0.8,
"lower": 50.26942413115372,
"upper": 85.62157427449092
},
"p95": {
"level": 0.95,
"lower": 46.44754059868354,
"upper": 86.70434168694935
}
}
},
{
"date": "2026-12-01",
"median": 83.4490351117366,
"mean": 74.814371372104,
"intervals": {
"p50": {
"level": 0.5,
"lower": 73.89330309387692,
"upper": 84.47302065263888
},
"p80": {
"level": 0.8,
"lower": 46.684420549779574,
"upper": 85.36986543817882
},
"p95": {
"level": 0.95,
"lower": 42.72111605479985,
"upper": 87.25408800647614
}
}
},
{
"date": "2026-12-31",
"median": 83.49338009688697,
"mean": 74.17875169067196,
"intervals": {
"p50": {
"level": 0.5,
"lower": 73.02811182541507,
"upper": 85.04026274136041
},
"p80": {
"level": 0.8,
"lower": 43.59097247237167,
"upper": 86.0893224190951
},
"p95": {
"level": 0.95,
"lower": 38.74834906028312,
"upper": 87.25200891687139
}
}
}
],
"horizon_summary": {
"date": "2026-12-31",
"median": 83.49338009688697,
"mean": 74.17875169067196,
"intervals": {
"p50": {
"level": 0.5,
"lower": 73.02811182541507,
"upper": 85.04026274136041
},
"p80": {
"level": 0.8,
"lower": 43.59097247237167,
"upper": 86.0893224190951
},
"p95": {
"level": 0.95,
"lower": 38.74834906028312,
"upper": 87.25200891687139
}
}
},
"invalidation_conditions": [
"A benchmark version change breaks comparability."
]
},
{
"id": "accelerated",
"label": "Accelerated",
"kind": "accelerated",
"probability": 0.2,
"assumptions": [
"Recent acceleration persists."
],
"parameters": {
"annual_growth_shift": 0.08,
"annual_acceleration_shift": 0.05
},
"forecast_points": [
{
"date": "2026-07-28",
"median": 82.64115607779445,
"mean": 78.20630446434788,
"intervals": {
"p50": {
"level": 0.5,
"lower": 76.9880448528882,
"upper": 84.1029701454302
},
"p80": {
"level": 0.8,
"lower": 62.07686442641476,
"upper": 85.03852898186966
},
"p95": {
"level": 0.95,
"lower": 59.3358765446527,
"upper": 86.42801327848296
}
}
},
{
"date": "2026-08-01",
"median": 82.76672185355056,
"mean": 78.1180226265634,
"intervals": {
"p50": {
"level": 0.5,
"lower": 76.96991214851728,
"upper": 84.10078373685374
},
"p80": {
"level": 0.8,
"lower": 61.655926355117295,
"upper": 85.15250476998453
},
"p95": {
"level": 0.95,
"lower": 57.69875618981136,
"upper": 86.37113595584923
}
}
},
{
"date": "2026-09-01",
"median": 83.48818883648211,
"mean": 78.02589794382042,
"intervals": {
"p50": {
"level": 0.5,
"lower": 77.33564810729968,
"upper": 85.07209431944877
},
"p80": {
"level": 0.8,
"lower": 58.6033110115756,
"upper": 86.45394192959128
},
"p95": {
"level": 0.95,
"lower": 53.65477079598252,
"upper": 87.72694351180613
}
}
},
{
"date": "2026-10-01",
"median": 84.3626487610814,
"mean": 77.74781814076663,
"intervals": {
"p50": {
"level": 0.5,
"lower": 76.47851181028764,
"upper": 85.76962920559623
},
"p80": {
"level": 0.8,
"lower": 55.05306794305188,
"upper": 86.80517966601778
},
"p95": {
"level": 0.95,
"lower": 50.628442969442965,
"upper": 88.30414556697278
}
}
},
{
"date": "2026-11-01",
"median": 85.19113238975245,
"mean": 77.45536172486281,
"intervals": {
"p50": {
"level": 0.5,
"lower": 76.37504027759596,
"upper": 86.33012453130716
},
"p80": {
"level": 0.8,
"lower": 51.42637933210595,
"upper": 87.59216230852411
},
"p95": {
"level": 0.95,
"lower": 47.51653481526506,
"upper": 88.69984970785163
}
}
},
{
"date": "2026-12-01",
"median": 86.03975063011009,
"mean": 77.10597411066938,
"intervals": {
"p50": {
"level": 0.5,
"lower": 76.18735630579067,
"upper": 87.09552629570169
},
"p80": {
"level": 0.8,
"lower": 48.13376088813872,
"upper": 88.02021405989753
},
"p95": {
"level": 0.95,
"lower": 44.04741369475709,
"upper": 89.9629332260431
}
}
},
{
"date": "2026-12-31",
"median": 86.77243958013216,
"mean": 77.052721811715,
"intervals": {
"p50": {
"level": 0.5,
"lower": 75.89616582378885,
"upper": 88.38007339073329
},
"p80": {
"level": 0.8,
"lower": 45.302933247028825,
"upper": 89.47033309032331
},
"p95": {
"level": 0.95,
"lower": 40.2701240956066,
"upper": 90.67868210867488
}
}
}
],
"horizon_summary": {
"date": "2026-12-31",
"median": 86.77243958013216,
"mean": 77.052721811715,
"intervals": {
"p50": {
"level": 0.5,
"lower": 75.89616582378885,
"upper": 88.38007339073329
},
"p80": {
"level": 0.8,
"lower": 45.302933247028825,
"upper": 89.47033309032331
},
"p95": {
"level": 0.95,
"lower": 40.2701240956066,
"upper": 90.67868210867488
}
}
},
"invalidation_conditions": [
"Recent-window growth falls materially below the prior window."
]
},
{
"id": "downside",
"label": "Saturation / measurement stress",
"kind": "downside",
"probability": 0.17,
"assumptions": [
"Benchmark saturation or measurement limitations dominate."
],
"parameters": {
"annual_drag": 0.18,
"immediate_multiplier": 0.98,
"floor_multiplier": 0.6
},
"forecast_points": [
{
"date": "2026-07-28",
"median": 80.98833295623857,
"mean": 76.66217837506093,
"intervals": {
"p50": {
"level": 0.5,
"lower": 75.44828395583043,
"upper": 82.4209107425216
},
"p80": {
"level": 0.8,
"lower": 60.83532713788646,
"upper": 83.33775840223227
},
"p95": {
"level": 0.95,
"lower": 58.14915901375964,
"upper": 84.6994530129133
}
}
},
{
"date": "2026-08-01",
"median": 80.88051495259364,
"mean": 76.33775695858093,
"intervals": {
"p50": {
"level": 0.5,
"lower": 75.21581127066123,
"upper": 82.18417431814287
},
"p80": {
"level": 0.8,
"lower": 60.25082257461854,
"upper": 83.21192722222195
},
"p95": {
"level": 0.95,
"lower": 56.38383408507437,
"upper": 84.40278649081093
}
}
},
{
"date": "2026-09-01",
"median": 79.78678909659598,
"mean": 74.61100006919828,
"intervals": {
"p50": {
"level": 0.5,
"lower": 73.90701764139118,
"upper": 81.30047306171251
},
"p80": {
"level": 0.8,
"lower": 56.005167691452144,
"upper": 82.62105727093538
},
"p95": {
"level": 0.95,
"lower": 51.27601809532365,
"upper": 83.83762107684976
}
}
},
{
"date": "2026-10-01",
"median": 78.87463067581383,
"mean": 72.75516534777302,
"intervals": {
"p50": {
"level": 0.5,
"lower": 71.50337812123219,
"upper": 80.1900832434956
},
"p80": {
"level": 0.8,
"lower": 51.47171722732715,
"upper": 81.15826835042883
},
"p95": {
"level": 0.95,
"lower": 47.334926054959595,
"upper": 82.55972247224402
}
}
},
{
"date": "2026-11-01",
"median": 77.83797112243036,
"mean": 70.85621708687009,
"intervals": {
"p50": {
"level": 0.5,
"lower": 69.78282848036272,
"upper": 78.8786526456833
},
"p80": {
"level": 0.8,
"lower": 46.987578602313775,
"upper": 80.03175928134817
},
"p95": {
"level": 0.95,
"lower": 43.41520720569102,
"upper": 81.04383809029129
}
}
},
{
"date": "2026-12-01",
"median": 76.85631574245386,
"mean": 68.98281879363856,
"intervals": {
"p50": {
"level": 0.5,
"lower": 68.0555146770907,
"upper": 77.79940341197472
},
"p80": {
"level": 0.8,
"lower": 42.99621393133077,
"upper": 78.62539481998967
},
"p95": {
"level": 0.95,
"lower": 39.34602215569444,
"upper": 80.3607582600128
}
}
},
{
"date": "2026-12-31",
"median": 75.79662979173041,
"mean": 67.43284466483873,
"intervals": {
"p50": {
"level": 0.5,
"lower": 66.2960913787039,
"upper": 77.20091467034415
},
"p80": {
"level": 0.8,
"lower": 39.572584064937836,
"upper": 78.15326787404028
},
"p95": {
"level": 0.95,
"lower": 35.17637284961865,
"upper": 79.20877332768855
}
}
}
],
"horizon_summary": {
"date": "2026-12-31",
"median": 75.79662979173041,
"mean": 67.43284466483871,
"intervals": {
"p50": {
"level": 0.5,
"lower": 66.2960913787039,
"upper": 77.20091467034415
},
"p80": {
"level": 0.8,
"lower": 39.572584064937836,
"upper": 78.15326787404028
},
"p95": {
"level": 0.95,
"lower": 35.17637284961865,
"upper": 79.20877332768855
}
}
},
"invalidation_conditions": [
"Fresh harder tasks preserve comparable headroom."
]
},
{
"id": "discontinuous",
"label": "Discontinuous",
"kind": "discontinuous",
"probability": 0.05,
"assumptions": [
"A structural improvement loop materially changes the trajectory."
],
"parameters": {
"annual_transition_hazard": 0.15,
"jump_multiplier": 1.15,
"growth_rate_compounding": 0.25,
"initial_growth_bonus": 0.12
},
"forecast_points": [
{
"date": "2026-07-28",
"median": 82.64115607779445,
"mean": 78.20630446434788,
"intervals": {
"p50": {
"level": 0.5,
"lower": 76.9880448528882,
"upper": 84.1029701454302
},
"p80": {
"level": 0.8,
"lower": 62.07686442641476,
"upper": 85.03852898186966
},
"p95": {
"level": 0.95,
"lower": 59.3358765446527,
"upper": 86.42801327848296
}
}
},
{
"date": "2026-08-01",
"median": 82.69399122775738,
"mean": 78.04937700977305,
"intervals": {
"p50": {
"level": 0.5,
"lower": 76.90227542505598,
"upper": 84.02688081435187
},
"p80": {
"level": 0.8,
"lower": 61.60174667991402,
"upper": 85.0776776556434
},
"p95": {
"level": 0.95,
"lower": 57.64805384772033,
"upper": 86.29523797864468
}
}
},
{
"date": "2026-09-01",
"median": 82.85864250473102,
"mean": 77.54519416987817,
"intervals": {
"p50": {
"level": 0.5,
"lower": 76.72743265987316,
"upper": 84.5885315614538
},
"p80": {
"level": 0.8,
"lower": 58.14241826806194,
"upper": 85.84142955269596
},
"p95": {
"level": 0.95,
"lower": 53.23279643842664,
"upper": 87.16795573493489
}
}
},
{
"date": "2026-10-01",
"median": 83.13352665232067,
"mean": 76.73780813042201,
"intervals": {
"p50": {
"level": 0.5,
"lower": 75.33771814191589,
"upper": 84.51176831402344
},
"p80": {
"level": 0.8,
"lower": 54.23186745356451,
"upper": 85.57073239493846
},
"p95": {
"level": 0.95,
"lower": 49.873242511014425,
"upper": 87.22371948287972
}
}
},
{
"date": "2026-11-01",
"median": 83.27456107025844,
"mean": 76.09538009188326,
"intervals": {
"p50": {
"level": 0.5,
"lower": 76.0421751272369,
"upper": 84.48751052423293
},
"p80": {
"level": 0.8,
"lower": 50.26942413115372,
"upper": 85.7199982646618
},
"p95": {
"level": 0.95,
"lower": 46.44754059868354,
"upper": 93.04762360816935
}
}
},
{
"date": "2026-12-01",
"median": 83.49102105301245,
"mean": 75.19102059499183,
"intervals": {
"p50": {
"level": 0.5,
"lower": 75.19145867833988,
"upper": 84.5346145197651
},
"p80": {
"level": 0.8,
"lower": 46.684420549779574,
"upper": 85.82011773002365
},
"p95": {
"level": 0.95,
"lower": 42.72111605479985,
"upper": 93.44026993503242
}
}
},
{
"date": "2026-12-31",
"median": 83.49338009688697,
"mean": 74.56431168111884,
"intervals": {
"p50": {
"level": 0.5,
"lower": 74.08655362280804,
"upper": 85.20402608063064
},
"p80": {
"level": 0.8,
"lower": 43.59097247237167,
"upper": 86.18395420467277
},
"p95": {
"level": 0.95,
"lower": 38.74834906028312,
"upper": 94.12810676470423
}
}
}
],
"horizon_summary": {
"date": "2026-12-31",
"median": 83.49338009688697,
"mean": 74.56431168111882,
"intervals": {
"p50": {
"level": 0.5,
"lower": 74.08655362280804,
"upper": 85.20402608063064
},
"p80": {
"level": 0.8,
"lower": 43.59097247237167,
"upper": 86.18395420467277
},
"p95": {
"level": 0.95,
"lower": 38.74834906028312,
"upper": 94.12810676470423
}
}
},
"invalidation_conditions": [
"No verified regime-change signal appears."
]
}
],
"milestones": [
{
"id": "p80",
"label": "80% resolved",
"threshold": 80,
"direction": "at_least",
"prerequisites": [
"Benchmark definition and system configuration remain comparable."
],
"scenario_results": {
"base": {
"probability_by_horizon": 0.75,
"median_crossing_date": "2026-07-28",
"p10_crossing_date": "2026-07-28",
"p90_crossing_date": "2026-07-28"
},
"accelerated": {
"probability_by_horizon": 0.75,
"median_crossing_date": "2026-07-28",
"p10_crossing_date": "2026-07-28",
"p90_crossing_date": "2026-07-28"
},
"downside": {
"probability_by_horizon": 0.74,
"median_crossing_date": "2026-07-28",
"p10_crossing_date": "2026-07-28",
"p90_crossing_date": "2026-07-28"
},
"discontinuous": {
"probability_by_horizon": 0.75,
"median_crossing_date": "2026-07-28",
"p10_crossing_date": "2026-07-28",
"p90_crossing_date": "2026-07-28"
}
}
},
{
"id": "p90",
"label": "90% resolved",
"threshold": 90,
"direction": "at_least",
"prerequisites": [
"Fresh unsaturated tasks preserve meaning."
],
"scenario_results": {
"base": {
"probability_by_horizon": 0.01,
"median_crossing_date": "2026-07-28",
"p10_crossing_date": "2026-07-28",
"p90_crossing_date": "2026-07-28"
},
"accelerated": {
"probability_by_horizon": 0.08,
"median_crossing_date": "2026-12-31",
"p10_crossing_date": "2026-12-01",
"p90_crossing_date": "2026-12-31"
},
"downside": {
"probability_by_horizon": 0.01,
"median_crossing_date": "2026-07-28",
"p10_crossing_date": "2026-07-28",
"p90_crossing_date": "2026-07-28"
},
"discontinuous": {
"probability_by_horizon": 0.03,
"median_crossing_date": "2026-09-01",
"p10_crossing_date": "2026-07-28",
"p90_crossing_date": "2026-11-01"
}
}
}
],
"bottlenecks": [
{
"name": "Benchmark saturation and configuration drift",
"severity": "high",
"mechanism": "As scores rise, remaining tasks and changes in scaffolds, tools, attempts, and model versions limit comparability.",
"release_condition": "A harder fresh benchmark with controlled configuration is available.",
"mitigation": "Track system metadata and use a bounded model with explicit version breaks.",
"evidence_ids": [
"swe-bench-official-leaderboard"
]
}
],
"triggers": [
{
"name": "Official leaderboard or benchmark revision",
"direction": "event",
"threshold": "New verified record or benchmark version",
"forecast_effect": "May change both the level and the measurement regime.",
"action": "Freeze the upstream file and rerun the bounded hindcast.",
"evidence_ids": [
"swe-bench-official-leaderboard"
]
}
],
"optimal_plan": {
"no_regret_actions": [
"Freeze source snapshots and record benchmark/configuration changes.",
"Use rolling-origin scores rather than in-sample fit alone."
],
"option_preserving_actions": [
"Maintain multiple model classes and trigger-based recalibration."
],
"next_experiment": [
"Add the complete upstream dataset and rerun the preregistered hindcast."
],
"delay_pending_evidence": [
"Do not translate benchmark movement directly into revenue or deployment."
],
"stop_expand_pivot": [
"Expand confidence only after independent replication and prospective calibration."
]
},
"confidence": {
"evidence_quality_score": 0.9,
"horizon_log_model_disagreement": 0.43692887800358887,
"base_horizon_p80_relative_width": 0.5090026286803542,
"limitations": [
"Only seven selected records.",
"Records combine model and scaffold effects and may differ in attempts.",
"Checked status varies across entries.",
"Submission date is not model release date."
],
"assumptions": [
"The selected records approximate a cumulative public system frontier."
]
},
"recalculation": {
"scheduled_date": "2026-08-27",
"early_triggers": [
"New official leaderboard record",
"Benchmark version change"
],
"required_evidence": [
"Complete leaderboard snapshot",
"Configuration metadata",
"Checked or independently reproduced status"
]
},
"reproducibility": {
"input_sha256": "6ac8b2c7b01fd1825561c5dfc5eae5a03467b31fd6c2e2f41d28fff50263fc3a",
"random_seed": 20260729,
"bootstrap_samples_requested": 100,
"bootstrap_samples_successful": 100,
"models": [
"linear",
"exponential",
"accelerating",
"decaying_acceleration",
"logistic",
"change_point"
],
"code_version": "2.0.0"
},
"legal_notice": "Research and decision-support output only. No warranty, assurance, legal, investment, tax, medical, or regulatory advice is provided."
}