Files
iddaai-be/ai-engine/scripts/test_score_coherence.py
T
fahricansecer 988ee2f50d Add backtest pipeline, betting_brain filters, score coherence + social v3
betting_brain.py:
- HARD_MIN_SAMPLES=50 floor for calibrator bypass
- ev_edge < 0 + >= 0.20 hard vetoes
- BTTS muted (grid search found no profitable config)
- Per-market optimal envelopes (MS, OU25)
- Score coherence filter: main_pick must agree with score prediction
- HTFT reversal cross-check for MS picks

feature_builder.py / data_loader.py:
- Real home/away_position from data (was hardcoded 10)
- Cup detection wired into UpsetEngine
- _estimate_league_position with 300-day season filter

New scripts:
- diagnostic_backtest.py: per-bet diagnostic backtest with loss patterns
- optimize_filters.py: grid search per-market optimal thresholds
- analyze_backtest_csv.py: root-cause hypothesis testing on CSV
- compare_backtests.py: side-by-side validation with verdict
- test_score_coherence.py: smoke test for coherence filter (20/20 pass)

Reports:
- diagnostic_backtest_20260525_024437 (50-match smoke)
- diagnostic_backtest_20260525_035649 (1000-match in-sample)
- filter_optimization_patch.json (grid search winners per market)

Social poster v3:
- satori + resvg HTML/CSS rendering pipeline
- Twemoji football/basketball + flag SVGs
- caption SEO: 12 curated hashtags per post
- image SEO: descriptive filenames + .json metadata sidecar
- /health, /preview-png, /run-now endpoints

Docs:
- mds/SESSION_HANDOFF.md: full session state for cross-machine continuity
- mds/SOCIAL_POSTER_SETUP.md: API keys + test commands

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 20:43:28 +03:00

55 lines
2.0 KiB
Python

"""Smoke test for the score-coherence filter using the LAFC vs Sounders
1-0 scenario from production. Verifies that markets that contradict the
predicted score are correctly excluded from the coherent set, and that
the markets the model got right are all included.
"""
import os, sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from services.betting_brain import BettingBrain
brain = BettingBrain()
pkg = {
"score_prediction": {"ft": "1-0", "ht": "0-0"},
}
coh = brain._score_consistent_markets(pkg)
print(f"Predicted: 1-0 (HT 0-0)")
print(f"Coherent set size: {len(coh)}")
print()
# Each pick the system actually offered for the LAFC match, with whether
# it was the *actual* winning pick.
test_picks = [
("MS", "1", True, "correct"),
("MS", "2", False, "wrong"),
("MS", "X", False, "wrong"),
("DC", "1X", True, "correct"),
("DC", "12", True, "correct"),
("DC", "X2", False, "wrong"),
("OU25", "Üst", False, "WRONG — system featured this"),
("OU25", "Alt", True, "correct"),
("OU35", "Alt", True, "correct"),
("OU35", "Üst", False, "wrong"),
("BTTS", "Var", False, "wrong"),
("BTTS", "Yok", True, "correct"),
("HT", "X", True, "correct"),
("HT", "1", False, "wrong"),
("HTFT", "X/1", True, "correct"),
("HTFT", "1/1", False, "wrong (HT was 0-0)"),
("HT_OU05", "Üst", False, "wrong"),
("HT_OU05", "Alt", True, "correct"),
("OE", "Çift", False, "wrong (1 is odd)"),
("OE", "Tek", True, "correct"),
]
print(f"{'market':<10}{'pick':<10}{'real-win?':<12}{'in-coherent?':<14}{'match?'}")
print("-" * 60)
ok = 0
for market, pick, would_win, note in test_picks:
in_coh = (market, pick) in coh
match = "" if in_coh == would_win else "✗ MISMATCH"
if in_coh == would_win: ok += 1
print(f"{market:<10}{pick:<10}{str(would_win):<12}{str(in_coh):<14}{match} {note}")
print()
print(f"Result: {ok}/{len(test_picks)} picks correctly classified")