30 Commits

Author SHA1 Message Date
fahricansecer f3362f266c gg 2026-05-10 22:52:05 +03:00
fahricansecer c525b12dfd gg 2026-05-10 10:37:45 +03:00
fahricansecer 4f7090e2d9 feat(ai-engine): value sniper thresholds and logic relaxed 2026-05-06 17:44:45 +03:00
fahricansecer 5b5f83c8cf fix(ai-engine): remove target leakage from training data extraction
Deploy Iddaai Backend / build-and-deploy (push) Successful in 6s
- goals_form now uses avg of last 5 historical matches instead of current match goals
- squad_quality removes current match goals/assists, uses only pre-match known data
- adds temporal filtering via match_id -> mst_utc mapping
2026-05-05 22:35:04 +03:00
fahricansecer bfddcaca7d gg
Deploy Iddaai Backend / build-and-deploy (push) Successful in 6s
2026-05-05 21:27:06 +03:00
fahricansecer 56d560af08 Update single_match_orchestrator.py
Deploy Iddaai Backend / build-and-deploy (push) Successful in 8s
2026-05-05 20:59:59 +03:00
fahricansecer 4bc51cfa99 fix(ai-engine): hoist ms_edge before score prediction branch to prevent UnboundLocalError
Deploy Iddaai Backend / build-and-deploy (push) Successful in 5s
2026-05-05 20:34:14 +03:00
fahricansecer fdb8a5d0f0 fix(ai-engine): sync FEATURE_COLS with trained models (82→102 features)
Deploy Iddaai Backend / build-and-deploy (push) Successful in 6s
- Load feature columns dynamically from feature_cols.json
- Add 20 missing odds_*_present boolean flags to fallback list
- Fixes LightGBM 'features in data (82) != training data (102)' crash
2026-05-05 20:29:55 +03:00
fahricansecer 22596e69f2 fix(predictions): circuit breaker resilience + graceful degradation
Deploy Iddaai Backend / build-and-deploy (push) Successful in 27s
- Reset consecutiveFailures on cooldown expiry (half-open state)
  so a single retry failure doesn't immediately re-open the circuit
- Exclude AI Engine app-level 500s from circuit breaker count
  (only network/infra errors: timeout, 502, 503, 504, 429)
- Return null gracefully instead of throwing 503 when no cache exists
- Add DB fallback for non-cooldown AI Engine failures
- Remove blocking wait-and-retry that held requests for up to 20s
2026-05-05 20:19:25 +03:00
fahricansecer f32badbd8f fix(predictions): cooldown fallback cascade + circuit breaker tuning
Deploy Iddaai Backend / build-and-deploy (push) Successful in 27s
- Add 4-level fallback when AI circuit breaker fires cooldown:
  1) In-memory cache (10min TTL)
  2) DB stored prediction (no TTL filter)
  3) DB cached prediction (with model version check)
  4) Wait out cooldown + retry once (max 20s wait)
- Raise circuit breaker threshold from 3 to 5 consecutive failures
- Reduce cooldown duration from 30s to 15s for faster recovery
- Add extractCooldownMs helper to parse remaining ms from error detail
2026-05-05 20:11:19 +03:00
fahricansecer 5645b38f20 main
Deploy Iddaai Backend / build-and-deploy (push) Successful in 32s
2026-05-05 17:09:11 +03:00
fahricansecer 244d8f5366 feat(ai): expand training to 68K+ matches, add score model, backfill implied odds
Deploy Iddaai Backend / build-and-deploy (push) Successful in 6s
- extract_training_data.py: switch from top_leagues.json (23) to qualified_leagues.json (265)
- update_implied_odds.py: new script to backfill implied odds from real market data
- train_score_model.py: rewrite with v25 102-feature set + temporal split
- single_match_orchestrator.py: integrate ML score model with heuristic fallback
2026-05-05 16:04:00 +03:00
fahricansecer 9bb8f39bca gg
Deploy Iddaai Backend / build-and-deploy (push) Successful in 2m45s
2026-05-05 14:06:20 +03:00
fahricansecer 7a1cf14e2f Update matches.service.ts
Deploy Iddaai Backend / build-and-deploy (push) Successful in 28s
2026-05-05 10:47:00 +03:00
fahricansecer 62c797d299 Update matches.service.ts
Deploy Iddaai Backend / build-and-deploy (push) Successful in 29s
2026-05-05 10:13:23 +03:00
fahricansecer 34cc4a6cbb Update matches.service.ts
Deploy Iddaai Backend / build-and-deploy (push) Successful in 30s
2026-05-05 01:04:56 +03:00
fahricansecer 27e96da31d main
Deploy Iddaai Backend / build-and-deploy (push) Successful in 29s
2026-05-04 18:00:40 +03:00
fahricansecer 145a8b336b fix(feeder): preserve pre-match odds when match goes live
Deploy Iddaai Backend / build-and-deploy (push) Successful in 29s
Live odds have missing selections (e.g. '1' key removed from Maç Sonucu
after kickoff), causing the AI model to produce wildly incorrect predictions
(e.g. 3.5% home win for Bristol City). Two guards added:

1. fetchOddsForMatches: Exclude live/finished matches from odds fetch query
2. processMatchOdds: Skip odds/lineups/sidelined overwrite if match already
   has pre-match odds and is live/finished
2026-05-02 16:32:42 +03:00
fahricansecer 7a8960edb8 chore: remove debug checkpoint logs and temp SQL files
Deploy Iddaai Backend / build-and-deploy (push) Successful in 37s
2026-04-26 17:09:22 +03:00
fahricansecer 691c52f610 perf: replace Prisma relation queries with raw SQL for getExistingMatchIds and getMissingScopes - fixes Pi hang
Deploy Iddaai Backend / build-and-deploy (push) Successful in 39s
2026-04-26 17:07:19 +03:00
fahricansecer bc461429f6 debug: add checkpoint timestamps to processDate for hang diagnosis
Deploy Iddaai Backend / build-and-deploy (push) Successful in 46s
2026-04-26 17:04:46 +03:00
fahricansecer a338d02244 main
Deploy Iddaai Backend / build-and-deploy (push) Successful in 2m42s
2026-04-26 03:07:18 +03:00
fahricansecer 1623432039 fix: watchdog force-kill with SIGKILL fallback when process.exit is blocked 2026-04-26 02:27:51 +03:00
fahricansecer 4c7930e9d2 feat: add watchdog timer to detect and recover from hung API requests
Deploy Iddaai Backend / build-and-deploy (push) Successful in 27s
2026-04-25 11:20:30 +03:00
fahricansecer ec463cb927 fix: make canvas import optional for ARM64 compatibility 2026-04-25 02:41:53 +03:00
fahricansecer eab95c4e5c Update feeder.service.ts
Deploy Iddaai Backend / build-and-deploy (push) Successful in 30s
2026-04-25 02:23:38 +03:00
fahricansecer 9027cc9900 v28
Deploy Iddaai Backend / build-and-deploy (push) Successful in 3m21s
2026-04-24 23:46:28 +03:00
fahricansecer 3875f2a512 Create v28-pro-max-architecture.md
Deploy Iddaai Backend / build-and-deploy (push) Successful in 27s
2026-04-24 02:30:26 +03:00
fahricansecer 300dceeb4b Merge branch 'main' of https://gitea.bilgich.com/fahricansecer/iddaai-be
Deploy Iddaai Backend / build-and-deploy (push) Successful in 27s
2026-04-24 02:10:48 +03:00
fahricansecer ad01976fb9 fix: lineup data normalization + tomorrow match sync + player field mapping 2026-04-24 02:09:58 +03:00
87 changed files with 13768 additions and 1712 deletions
+6 -6
View File
@@ -25,11 +25,11 @@ jobs:
--network iddaai_iddaai-network \
-p 127.0.0.1:1810:3005 \
-e NODE_ENV=production \
-e DATABASE_URL='postgresql://iddaai_user:IddaA1_S4crET!@iddaai-postgres:5432/iddaai_db?schema=public' \
-e REDIS_HOST='iddaai-redis' \
-e REDIS_PORT='6379' \
-e REDIS_PASSWORD='IddaA1_Redis_Pass!' \
-e AI_ENGINE_URL='http://iddaai-ai-engine:8000' \
-e JWT_SECRET='b7V8jM2wP1L5mQxs2RdfFkAsLpI2oG!w' \
-e DATABASE_URL='${{ secrets.DATABASE_URL }}' \
-e REDIS_HOST='${{ secrets.REDIS_HOST }}' \
-e REDIS_PORT='${{ secrets.REDIS_PORT }}' \
-e REDIS_PASSWORD='${{ secrets.REDIS_PASSWORD }}' \
-e AI_ENGINE_URL='${{ secrets.AI_ENGINE_URL }}' \
-e JWT_SECRET='${{ secrets.JWT_SECRET }}' \
-e JWT_ACCESS_EXPIRATION='1d' \
iddaai-be:latest /bin/sh -c "npx prisma migrate deploy && node dist/src/main.js"
+4 -2
View File
@@ -42,7 +42,9 @@ uploads/
public/uploads/
# Large Datasets and ML Models
ai-engine/models/
models/
ai-engine/models/*
!ai-engine/models/*.py
models/*
!models/*.py
colab_export/
+1 -1
View File
@@ -47,7 +47,7 @@ COPY --from=builder /app/dist ./dist
COPY --from=builder /app/src/i18n ./dist/i18n
# Copy league filter config files (critical: without these, feeder stores ALL matches)
COPY top_leagues.json basketball_top_leagues.json ./
COPY qualified_leagues.json top_leagues.json basketball_top_leagues.json ./
# Set environment
ENV NODE_ENV=production
+69
View File
@@ -1,7 +1,9 @@
import os
import json
import yaml
from typing import Dict, Any, Optional
class EnsembleConfig:
_instance: Optional['EnsembleConfig'] = None
_config: Dict[str, Any] = {}
@@ -35,12 +37,79 @@ class EnsembleConfig:
except (KeyError, TypeError):
return default
# Singleton accessor
def get_config() -> EnsembleConfig:
return EnsembleConfig()
# ── Market Thresholds Loader ────────────────────────────────────────────
_market_thresholds_cache: Optional[Dict[str, Any]] = None
def load_market_thresholds() -> Dict[str, Any]:
"""
Load market thresholds from JSON config file.
Returns the full config dict with 'markets' and 'defaults' keys.
Caches after first load for performance.
"""
global _market_thresholds_cache
if _market_thresholds_cache is not None:
return _market_thresholds_cache
config_path = os.path.join(os.path.dirname(__file__), 'market_thresholds.json')
try:
with open(config_path, 'r', encoding='utf-8') as f:
data = json.load(f)
_market_thresholds_cache = data
print(f"✅ Market thresholds loaded: {len(data.get('markets', {}))} markets (v={data.get('_meta', {}).get('version', '?')})")
return data
except Exception as e:
print(f"❌ Failed to load market thresholds: {e} — using built-in defaults")
_market_thresholds_cache = {"markets": {}, "defaults": {
"calibration": 0.55,
"min_conf": 55.0,
"min_play_score": 68.0,
"min_edge": 0.02,
"odds_band_min_sample": 0.0,
"odds_band_min_edge": 0.0,
}}
return _market_thresholds_cache
def build_threshold_dict(field: str) -> Dict[str, float]:
"""
Build a flat {market: value} dict for a specific threshold field.
Usage:
calibration_map = build_threshold_dict("calibration")
# → {"MS": 0.62, "DC": 0.82, ...}
"""
data = load_market_thresholds()
markets = data.get("markets", {})
result: Dict[str, float] = {}
for market, cfg in markets.items():
if field in cfg:
result[market] = float(cfg[field])
return result
def get_threshold_default(field: str) -> float:
"""Get the default fallback value for a threshold field."""
data = load_market_thresholds()
defaults = data.get("defaults", {})
return float(defaults.get(field, 0.0))
if __name__ == "__main__":
# Test
cfg = get_config()
print(f"Weights: {cfg.get('engine_weights')}")
print(f"Team Weight: {cfg.get('engine_weights.team')}")
print()
print("--- Market Thresholds ---")
for field in ["calibration", "min_conf", "min_play_score", "min_edge"]:
d = build_threshold_dict(field)
print(f"{field}: {d}")
print(f"Default calibration: {get_threshold_default('calibration')}")
+115
View File
@@ -0,0 +1,115 @@
{
"_meta": {
"version": "v34",
"description": "Market-specific thresholds for the betting engine pipeline — V34 odds-aware gate fix",
"rule": "max_reachable (100 × calibration) MUST be > min_conf + 8",
"updated_at": "2026-05-10",
"changelog": "V34: Reduced min_edge to realistic levels for odds-aware V25 model. Model output ≈ market-implied, so large EV edges are mathematically impossible."
},
"markets": {
"MS": {
"calibration": 0.62,
"min_conf": 20.0,
"min_play_score": 28.0,
"min_edge": 0.005,
"odds_band_min_sample": 8.0,
"odds_band_min_edge": 0.005
},
"DC": {
"calibration": 0.82,
"min_conf": 40.0,
"min_play_score": 50.0,
"min_edge": 0.003,
"odds_band_min_sample": 8.0,
"odds_band_min_edge": 0.005
},
"OU15": {
"calibration": 0.84,
"min_conf": 45.0,
"min_play_score": 50.0,
"min_edge": 0.003,
"odds_band_min_sample": 8.0,
"odds_band_min_edge": 0.005
},
"OU25": {
"calibration": 0.68,
"min_conf": 30.0,
"min_play_score": 40.0,
"min_edge": 0.005,
"odds_band_min_sample": 8.0,
"odds_band_min_edge": 0.005
},
"OU35": {
"calibration": 0.60,
"min_conf": 20.0,
"min_play_score": 30.0,
"min_edge": 0.008,
"odds_band_min_sample": 8.0,
"odds_band_min_edge": 0.008
},
"BTTS": {
"calibration": 0.65,
"min_conf": 30.0,
"min_play_score": 40.0,
"min_edge": 0.005,
"odds_band_min_sample": 8.0,
"odds_band_min_edge": 0.005
},
"HT": {
"calibration": 0.58,
"min_conf": 20.0,
"min_play_score": 28.0,
"min_edge": 0.01,
"odds_band_min_sample": 8.0,
"odds_band_min_edge": 0.008
},
"HT_OU05": {
"calibration": 0.68,
"min_conf": 35.0,
"min_play_score": 42.0,
"min_edge": 0.005,
"odds_band_min_sample": 8.0,
"odds_band_min_edge": 0.005
},
"HT_OU15": {
"calibration": 0.60,
"min_conf": 25.0,
"min_play_score": 32.0,
"min_edge": 0.008,
"odds_band_min_sample": 8.0,
"odds_band_min_edge": 0.008
},
"OE": {
"calibration": 0.62,
"min_conf": 35.0,
"min_play_score": 32.0,
"min_edge": 0.005
},
"CARDS": {
"calibration": 0.58,
"min_conf": 30.0,
"min_play_score": 35.0,
"min_edge": 0.008
},
"HCAP": {
"calibration": 0.56,
"min_conf": 25.0,
"min_play_score": 30.0,
"min_edge": 0.015
},
"HTFT": {
"calibration": 0.45,
"min_conf": 10.0,
"min_play_score": 18.0,
"min_edge": 0.02
}
},
"defaults": {
"calibration": 0.55,
"min_conf": 55.0,
"min_play_score": 60.0,
"min_edge": 0.008,
"odds_band_min_sample": 0.0,
"odds_band_min_edge": 0.0
}
}
+38 -11
View File
@@ -18,10 +18,15 @@ from features.sidelined_analyzer import get_sidelined_analyzer
@dataclass
class PlayerPrediction:
"""Player engine prediction output."""
home_squad_quality: float = 50.0 # 0-100
away_squad_quality: float = 50.0
squad_diff: float = 0.0 # -100 to +100
"""Player engine prediction output.
IMPORTANT: squad_quality uses the SAME composite formula as
extract_training_data.py so that inference values match the
distribution the model was trained on (~3-36 range).
"""
home_squad_quality: float = 12.0 # training-scale composite (~3-36)
away_squad_quality: float = 12.0
squad_diff: float = 0.0 # home - away (training scale)
home_key_players: int = 0
away_key_players: int = 0
home_missing_impact: float = 0.0 # 0-1, how much weaker due to missing players
@@ -100,10 +105,12 @@ class PlayerPredictorEngine:
"home_goals_last_5": home_analysis.total_goals_last_5,
"home_assists_last_5": home_analysis.total_assists_last_5,
"home_key_players": home_analysis.key_players_count,
"home_forwards": home_analysis.forward_count or 2,
"away_starting_11": away_analysis.starting_count or 11,
"away_goals_last_5": away_analysis.total_goals_last_5,
"away_assists_last_5": away_analysis.total_assists_last_5,
"away_key_players": away_analysis.key_players_count,
"away_forwards": away_analysis.forward_count or 2,
}
elif match_id:
# Try to get from database
@@ -131,13 +138,31 @@ class PlayerPredictorEngine:
away_goals = features.get("away_goals_last_5", 0)
home_key = features.get("home_key_players", 0)
away_key = features.get("away_key_players", 0)
home_assists = features.get("home_assists_last_5", 0)
away_assists = features.get("away_assists_last_5", 0)
home_starting = features.get("home_starting_11", 11)
away_starting = features.get("away_starting_11", 11)
home_fwd = features.get("home_forwards", 2)
away_fwd = features.get("away_forwards", 2)
# Calculate squad quality (0-100)
# Based on: goals scored, key players, assists
home_quality = min(100, 50 + (home_goals * 3) + (home_key * 5) +
features.get("home_assists_last_5", 0) * 2)
away_quality = min(100, 50 + (away_goals * 3) + (away_key * 5) +
features.get("away_assists_last_5", 0) * 2)
# Calculate squad quality — MUST match extract_training_data.py formula
# Formula: starting_count * 0.3 + goals * 2.0 + assists * 1.0
# + key_players * 3.0 + fwd_count * 1.5
# Typical range: ~3 36 (model trained on this distribution)
home_quality = (
home_starting * 0.3 +
home_goals * 2.0 +
home_assists * 1.0 +
home_key * 3.0 +
home_fwd * 1.5
)
away_quality = (
away_starting * 0.3 +
away_goals * 2.0 +
away_assists * 1.0 +
away_key * 3.0 +
away_fwd * 1.5
)
# Squad difference
squad_diff = home_quality - away_quality
@@ -186,8 +211,10 @@ class PlayerPredictorEngine:
Calculate 1X2 probability modifiers based on squad analysis.
Returns modifiers to apply to base probabilities.
squad_diff is in training scale (~-33 to +33), normalize to -1..+1.
"""
diff = prediction.squad_diff / 100 # -1 to +1
diff = prediction.squad_diff / 33.0 # training-scale normalisation
diff = max(-1.0, min(1.0, diff)) # clamp
return {
"home_modifier": 1.0 + (diff * 0.3), # Up to +/-30%
+16 -16
View File
@@ -323,8 +323,8 @@ class OddsBandAnalyzer:
m.home_team_id,
m.away_team_id,
CASE
WHEN m.home_team_id = %(team_id)s THEN os_sel.odd_value
ELSE os_sel2.odd_value
WHEN m.home_team_id = %(team_id)s THEN os_sel.odd_value::numeric
ELSE os_sel2.odd_value::numeric
END AS team_odds
FROM matches m
JOIN odd_categories oc
@@ -344,7 +344,7 @@ class OddsBandAnalyzer:
AND m.score_home IS NOT NULL
AND m.score_away IS NOT NULL
AND m.mst_utc < %(before_ts)s
AND COALESCE(os_sel.odd_value, os_sel2.odd_value)
AND COALESCE(os_sel.odd_value::numeric, os_sel2.odd_value::numeric)
BETWEEN %(band_low)s AND %(band_high)s
ORDER BY m.mst_utc DESC
LIMIT %(max_lookback)s
@@ -432,7 +432,7 @@ class OddsBandAnalyzer:
AND m.score_home IS NOT NULL
AND m.score_away IS NOT NULL
AND m.mst_utc < %(before_ts)s
AND os_h.odd_value BETWEEN %(band_low)s AND %(band_high)s
AND os_h.odd_value::numeric BETWEEN %(band_low)s AND %(band_high)s
ORDER BY m.mst_utc DESC
LIMIT %(max_lookback)s
)
@@ -508,7 +508,7 @@ class OddsBandAnalyzer:
f"İlk Yarı {line_str} Alt/Üst",
f"Ilk Yari {line_str} Alt/Ust",
]
score_expr = "COALESCE(m.score_ht_home, 0) + COALESCE(m.score_ht_away, 0)"
score_expr = "COALESCE(m.ht_score_home, 0) + COALESCE(m.ht_score_away, 0)"
else:
cat_names = [
f"{line_str} Alt/Üst",
@@ -535,7 +535,7 @@ class OddsBandAnalyzer:
AND m.status = 'FT'
AND m.score_home IS NOT NULL
AND m.mst_utc < %(before_ts)s
AND os_over.odd_value BETWEEN %(band_low)s AND %(band_high)s
AND os_over.odd_value::numeric BETWEEN %(band_low)s AND %(band_high)s
ORDER BY m.mst_utc DESC
LIMIT %(max_lookback)s
)
@@ -620,7 +620,7 @@ class OddsBandAnalyzer:
AND m.status = 'FT'
AND m.score_home IS NOT NULL
AND m.mst_utc < %(before_ts)s
AND os_yes.odd_value BETWEEN %(band_low)s AND %(band_high)s
AND os_yes.odd_value::numeric BETWEEN %(band_low)s AND %(band_high)s
ORDER BY m.mst_utc DESC
LIMIT %(max_lookback)s
)
@@ -696,7 +696,7 @@ class OddsBandAnalyzer:
AND m.sport = 'football' AND m.status = 'FT'
AND m.score_home IS NOT NULL
AND m.mst_utc < %(before_ts)s
AND os_sel.odd_value BETWEEN %(bl)s AND %(bh)s
AND os_sel.odd_value::numeric BETWEEN %(bl)s AND %(bh)s
ORDER BY m.mst_utc DESC LIMIT %(ml)s
)
SELECT COUNT(*) AS ss,
@@ -748,7 +748,7 @@ class OddsBandAnalyzer:
try:
cur.execute("""
WITH ht_matches AS (
SELECT m.score_ht_home, m.score_ht_away,
SELECT m.ht_score_home, m.ht_score_away,
m.home_team_id, m.away_team_id
FROM matches m
JOIN odd_categories oc ON oc.match_id = m.id
@@ -761,18 +761,18 @@ class OddsBandAnalyzer:
AND os2.name = '2' AND m.away_team_id = %(tid)s
WHERE (m.home_team_id = %(tid)s OR m.away_team_id = %(tid)s)
AND m.sport = 'football' AND m.status = 'FT'
AND m.score_ht_home IS NOT NULL
AND m.ht_score_home IS NOT NULL
AND m.mst_utc < %(before_ts)s
AND COALESCE(os1.odd_value, os2.odd_value)
AND COALESCE(os1.odd_value::numeric, os2.odd_value::numeric)
BETWEEN %(bl)s AND %(bh)s
ORDER BY m.mst_utc DESC LIMIT %(ml)s
)
SELECT COUNT(*) AS ss,
COALESCE(AVG(CASE
WHEN (home_team_id = %(tid)s AND score_ht_home > score_ht_away)
OR (away_team_id = %(tid)s AND score_ht_away > score_ht_home)
WHEN (home_team_id = %(tid)s AND ht_score_home > ht_score_away)
OR (away_team_id = %(tid)s AND ht_score_away > ht_score_home)
THEN 1.0 ELSE 0.0 END), 0.33) AS win_rate,
COALESCE(AVG(CASE WHEN score_ht_home = score_ht_away
COALESCE(AVG(CASE WHEN ht_score_home = ht_score_away
THEN 1.0 ELSE 0.0 END), 0.40) AS draw_rate
FROM ht_matches
""", {
@@ -824,7 +824,7 @@ class OddsBandAnalyzer:
AND m.sport = 'football' AND m.status = 'FT'
AND m.score_home IS NOT NULL
AND m.mst_utc < %(before_ts)s
AND os_odd.odd_value BETWEEN %(bl)s AND %(bh)s
AND os_odd.odd_value::numeric BETWEEN %(bl)s AND %(bh)s
ORDER BY m.mst_utc DESC LIMIT %(ml)s
)
SELECT COUNT(*) AS ss,
@@ -1185,7 +1185,7 @@ class OddsBandAnalyzer:
'IY/MS'
)
JOIN odd_selections os ON os.odd_category_db_id = oc.db_id
AND os.odd_value BETWEEN %(bl)s AND %(bh)s
AND os.odd_value::numeric BETWEEN %(bl)s AND %(bh)s
WHERE m.sport = 'football'
AND m.status = 'FT'
AND m.score_home IS NOT NULL
+12 -6
View File
@@ -14,10 +14,13 @@ from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse
from pydantic import BaseModel
from models.basketball_v25 import get_basketball_v25_predictor
try:
from models.basketball_v25 import get_basketball_v25_predictor
HAS_BASKETBALL = True
except ImportError:
HAS_BASKETBALL = False
from services.single_match_orchestrator import get_single_match_orchestrator
from services.v26_shadow_engine import get_v26_shadow_engine
from data.database import dispose_engine
load_dotenv()
@@ -49,9 +52,6 @@ async def lifespan(_: FastAPI):
yield
# Cleanup async DB connections on shutdown
await dispose_engine()
app = FastAPI(
title="Suggest-Bet AI Engine",
@@ -123,9 +123,15 @@ def health_check() -> dict[str, Any]:
try:
orchestrator = get_single_match_orchestrator()
shadow_engine = get_v26_shadow_engine()
if HAS_BASKETBALL:
basketball_predictor = get_basketball_v25_predictor()
basketball_readiness = basketball_predictor.readiness_summary()
ready = bool(basketball_readiness["fully_loaded"])
ready = bool(basketball_readiness.get("fully_loaded", True))
else:
basketball_readiness = {"fully_loaded": False, "error": "Basketball module not found"}
ready = True
return {
"status": "healthy" if ready else "degraded",
"engine": "v28.main",
+413
View File
@@ -0,0 +1,413 @@
"""
Calibration Module for XGBoost Models
=====================================
Calibrates raw probabilities from XGBoost models using Isotonic Regression.
Ensures that a predicted probability of 70% actually corresponds to a 70% win rate.
Usage:
from ai_engine.models.calibration import Calibrator
calibrator = Calibrator()
calibrated_prob = calibrator.calibrate("ms", raw_prob)
# Training new calibration models:
calibrator.train_calibration(valid_df, market="ms")
"""
import os
import pickle
import json
import numpy as np
import pandas as pd
from datetime import datetime
from typing import Dict, List, Optional, Tuple, Any
from sklearn.isotonic import IsotonicRegression
from sklearn.calibration import calibration_curve
from sklearn.metrics import brier_score_loss
AI_ENGINE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CALIBRATION_DIR = os.path.join(AI_ENGINE_DIR, "models", "calibration")
os.makedirs(CALIBRATION_DIR, exist_ok=True)
# Supported markets for calibration
SUPPORTED_MARKETS = [
"ms", # Match Result (1X2) - multi-class, calibrated per class
"ms_home", # Standard Home win probability
"ms_home_heavy_fav", # Context: home odds <= 1.40
"ms_home_fav", # Context: 1.40 < home odds <= 1.80
"ms_home_balanced", # Context: 1.80 < home odds <= 2.50
"ms_home_underdog", # Context: home odds > 2.50
"ms_draw", # Draw probability
"ms_away", # Away win probability
"ou15", # Over/Under 1.5
"ou25", # Over/Under 2.5
"ou35", # Over/Under 3.5
"btts", # Both Teams to Score
"ht_ft", # Half-Time/Full-Time
"dc", # Double Chance
"ht", # Half-Time Result
]
class CalibrationMetrics:
"""Stores calibration quality metrics for a market."""
def __init__(self):
self.brier_score: float = 0.0
self.calibration_error: float = 0.0
self.sample_count: int = 0
self.last_trained: str = ""
self.mean_predicted: float = 0.0
self.mean_actual: float = 0.0
def to_dict(self) -> Dict:
return {
"brier_score": round(self.brier_score, 4),
"calibration_error": round(self.calibration_error, 4),
"sample_count": self.sample_count,
"last_trained": self.last_trained,
"mean_predicted": round(self.mean_predicted, 4),
"mean_actual": round(self.mean_actual, 4),
}
class Calibrator:
"""
Probability calibration using Isotonic Regression.
Isotonic Regression is a non-parametric method that fits a piecewise
constant function that is monotonically increasing. It's ideal for
calibrating probabilities because:
1. It preserves ranking (if P(A) > P(B) before, P(A) > P(B) after)
2. It doesn't assume a specific distribution shape
3. It can correct systematic over/under-confidence
Example:
# Before calibration: model predicts 70% but actual win rate is 60%
# After calibration: model predicts 70% → calibrated to 60%
"""
def __init__(self):
self.calibrators: Dict[str, IsotonicRegression] = {}
self.metrics: Dict[str, CalibrationMetrics] = {}
self.heuristic_fallback: Dict[str, float] = {
"ms": 0.90,
"ms_home": 0.90,
"ms_home_heavy_fav": 0.95,
"ms_home_fav": 0.90,
"ms_home_balanced": 0.85,
"ms_home_underdog": 0.80,
"ms_draw": 0.90,
"ms_away": 0.90,
"ou15": 0.90,
"ou25": 0.90,
"ou35": 0.90,
"btts": 0.90,
"ht_ft": 0.85,
"dc": 0.93,
"ht": 0.85,
}
self._load_calibrators()
def _load_calibrators(self):
"""Load trained calibrators for each market from disk."""
for market in SUPPORTED_MARKETS:
model_path = os.path.join(CALIBRATION_DIR, f"{market}_calibrator.pkl")
metrics_path = os.path.join(CALIBRATION_DIR, f"{market}_metrics.json")
if os.path.exists(model_path):
try:
with open(model_path, "rb") as f:
self.calibrators[market] = pickle.load(f)
print(f"[Calibrator] Loaded calibration model for {market}")
except Exception as e:
print(f"[Calibrator] Warning: Failed to load {market}: {e}")
if os.path.exists(metrics_path):
try:
with open(metrics_path, "r") as f:
data = json.load(f)
metrics = CalibrationMetrics()
metrics.brier_score = data.get("brier_score", 0.0)
metrics.calibration_error = data.get("calibration_error", 0.0)
metrics.sample_count = data.get("sample_count", 0)
metrics.last_trained = data.get("last_trained", "")
metrics.mean_predicted = data.get("mean_predicted", 0.0)
metrics.mean_actual = data.get("mean_actual", 0.0)
self.metrics[market] = metrics
except Exception as e:
print(f"[Calibrator] Warning: Failed to load metrics for {market}: {e}")
def calibrate(self, market_type: str, raw_prob: float, odds_val: Optional[float] = None) -> float:
"""
Calibrate a raw probability using Isotonic Regression.
Args:
market_type (str): 'ms_home', 'ou25', 'btts', 'ht_ft', etc.
raw_prob (float): The raw probability from XGBoost (0.0 - 1.0)
odds_val (float, optional): The pre-match odds, used for context-aware bucket mapping
Returns:
float: Calibrated probability (0.0 - 1.0)
"""
# Normalize market type
market_key = market_type.lower().replace("-", "_")
# Route to bucket if ms_home and odds provided
if market_key == "ms_home" and odds_val is not None and odds_val > 1.0:
if odds_val <= 1.40:
bucket_key = "ms_home_heavy_fav"
elif odds_val <= 1.80:
bucket_key = "ms_home_fav"
elif odds_val <= 2.50:
bucket_key = "ms_home_balanced"
else:
bucket_key = "ms_home_underdog"
if bucket_key in self.calibrators:
market_key = bucket_key
# If we have a trained Isotonic Regression model, use it
if market_key in self.calibrators:
try:
calibrated = self.calibrators[market_key].predict([raw_prob])[0]
# Ensure output is valid probability
return float(np.clip(calibrated, 0.01, 0.99))
except Exception as e:
print(f"[Calibrator] Warning: Isotonic failed for {market_key}: {e}")
# Fall through to heuristic
# Fallback to heuristic calibration
return self._heuristic_calibrate(market_key, raw_prob)
def _heuristic_calibrate(self, market_type: str, raw_prob: float) -> float:
"""
Heuristic calibration fallback when no trained model exists.
This applies a conservative shrinkage towards the mean:
- Binary markets (OU, BTTS): shrink towards 0.5
- Multi-class (MS): shrink towards 0.33
- HT/FT: stronger shrinkage due to higher variance
"""
# Get shrinkage factor for this market
shrinkage = self.heuristic_fallback.get(market_type, 0.90)
if market_type in ["ms", "ms_home", "ms_home_heavy_fav", "ms_home_fav", "ms_home_balanced", "ms_home_underdog", "ms_draw", "ms_away"]:
# Pull towards 0.33 (uniform for 3-class)
return (raw_prob * shrinkage) + (0.33 * (1.0 - shrinkage))
elif market_type in ["ou15", "ou25", "ou35", "btts"]:
# Pull towards 0.5 (uniform for binary)
return (raw_prob * shrinkage) + (0.5 * (1.0 - shrinkage))
elif market_type in ["ht_ft", "ht"]:
# Stronger shrinkage for high-variance markets
return raw_prob * shrinkage
elif market_type == "dc":
# Double chance is more reliable
return (raw_prob * shrinkage) + (0.66 * (1.0 - shrinkage))
return raw_prob
def train_calibration(
self,
df: pd.DataFrame,
market: str,
prob_col: str,
actual_col: str,
min_samples: int = 100,
save: bool = True,
) -> CalibrationMetrics:
"""
Train an Isotonic Regression calibration model for a specific market.
Args:
df: DataFrame with predictions and actual outcomes
market: Market identifier (e.g., 'ms_home', 'ou25', 'btts')
prob_col: Column name for raw probabilities
actual_col: Column name for actual outcomes (0 or 1)
min_samples: Minimum samples required to train
save: Whether to save the model to disk
Returns:
CalibrationMetrics with quality metrics
"""
# Filter valid data
valid_df = df[[prob_col, actual_col]].dropna()
n_samples = len(valid_df)
if n_samples < min_samples:
print(f"[Calibrator] Warning: Only {n_samples} samples for {market}, "
f"need at least {min_samples}")
metrics = CalibrationMetrics()
metrics.sample_count = n_samples
return metrics
# Extract arrays
raw_probs = valid_df[prob_col].values
actuals = valid_df[actual_col].values
# Train Isotonic Regression
iso = IsotonicRegression(out_of_bounds="clip", increasing=True)
iso.fit(raw_probs, actuals)
# Calculate calibrated probabilities
calibrated_probs = iso.predict(raw_probs)
# Calculate metrics
metrics = CalibrationMetrics()
metrics.sample_count = n_samples
metrics.last_trained = datetime.utcnow().isoformat()
metrics.brier_score = brier_score_loss(actuals, calibrated_probs)
metrics.mean_predicted = np.mean(raw_probs)
metrics.mean_actual = np.mean(actuals)
# Calculate Expected Calibration Error (ECE)
metrics.calibration_error = self._calculate_ece(
calibrated_probs, actuals, n_bins=10
)
# Store in memory
self.calibrators[market] = iso
self.metrics[market] = metrics
# Save to disk
if save:
self._save_calibration(market, iso, metrics)
print(f"[Calibrator] Trained {market}: "
f"Brier={metrics.brier_score:.4f}, "
f"ECE={metrics.calibration_error:.4f}, "
f"n={n_samples}")
return metrics
def train_all_markets(
self,
df: pd.DataFrame,
market_config: Dict[str, Tuple[str, str]],
min_samples: int = 100,
) -> Dict[str, CalibrationMetrics]:
"""
Train calibration models for multiple markets at once.
Args:
df: DataFrame with all predictions and outcomes
market_config: Dict mapping market -> (prob_col, actual_col)
e.g., {'ou25': ('ou25_over_prob', 'ou25_over_actual')}
min_samples: Minimum samples per market
Returns:
Dict of market -> CalibrationMetrics
"""
results = {}
for market, (prob_col, actual_col) in market_config.items():
print(f"\n[Calibrator] Training {market}...")
try:
metrics = self.train_calibration(
df=df,
market=market,
prob_col=prob_col,
actual_col=actual_col,
min_samples=min_samples,
save=True,
)
results[market] = metrics
except Exception as e:
print(f"[Calibrator] Failed to train {market}: {e}")
return results
def _calculate_ece(
self,
probs: np.ndarray,
actuals: np.ndarray,
n_bins: int = 10
) -> float:
"""
Calculate Expected Calibration Error (ECE).
ECE = sum(|bin_accuracy - bin_confidence| * bin_weight)
Lower is better. Perfect calibration = 0.
"""
bin_boundaries = np.linspace(0, 1, n_bins + 1)
ece = 0.0
for i in range(n_bins):
in_bin = (probs >= bin_boundaries[i]) & (probs < bin_boundaries[i + 1])
prop_in_bin = np.mean(in_bin)
if prop_in_bin > 0:
accuracy_in_bin = np.mean(actuals[in_bin])
avg_confidence_in_bin = np.mean(probs[in_bin])
ece += np.abs(accuracy_in_bin - avg_confidence_in_bin) * prop_in_bin
return ece
def _save_calibration(
self,
market: str,
calibrator: IsotonicRegression,
metrics: CalibrationMetrics
):
"""Save calibration model and metrics to disk."""
# Save model
model_path = os.path.join(CALIBRATION_DIR, f"{market}_calibrator.pkl")
with open(model_path, "wb") as f:
pickle.dump(calibrator, f)
# Save metrics
metrics_path = os.path.join(CALIBRATION_DIR, f"{market}_metrics.json")
with open(metrics_path, "w") as f:
json.dump(metrics.to_dict(), f, indent=2)
print(f"[Calibrator] Saved {market} to {CALIBRATION_DIR}")
def get_calibration_report(self) -> Dict[str, Any]:
"""Generate a summary report of all calibration models."""
report = {
"trained_markets": list(self.calibrators.keys()),
"metrics": {},
"heuristic_only": [],
}
for market in SUPPORTED_MARKETS:
if market in self.metrics:
report["metrics"][market] = self.metrics[market].to_dict()
elif market not in self.calibrators:
report["heuristic_only"].append(market)
return report
def get_calibrated_probabilities(
self,
market: str,
raw_probs: np.ndarray
) -> np.ndarray:
"""
Batch calibration for array of probabilities.
Args:
market: Market type
raw_probs: Array of raw probabilities
Returns:
Array of calibrated probabilities
"""
return np.array([self.calibrate(market, p) for p in raw_probs])
# Singleton instance
_calibrator_instance: Optional[Calibrator] = None
def get_calibrator() -> Calibrator:
"""Get or create the global Calibrator instance."""
global _calibrator_instance
if _calibrator_instance is None:
_calibrator_instance = Calibrator()
return _calibrator_instance
File diff suppressed because it is too large Load Diff
+676
View File
@@ -0,0 +1,676 @@
"""
V25 Ensemble Predictor - NO TARGET LEAKAGE
===========================================
Multi-model ensemble for match prediction using XGBoost and LightGBM.
Features:
- 73 engineered features (NO target leakage)
- Market-specific models (MS, OU25, BTTS)
- Weighted ensemble predictions
- Value bet detection
"""
import os
import json
import numpy as np
import pandas as pd
from typing import Dict, List, Optional, Any
from dataclasses import dataclass, field
import xgboost as xgb
import lightgbm as lgb
# CatBoost is optional
try:
from catboost import CatBoostClassifier
CATBOOST_AVAILABLE = True
except ImportError:
CatBoostClassifier = None
CATBOOST_AVAILABLE = False
# Paths
MODELS_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'v25')
@dataclass
class MarketPrediction:
"""Prediction for a single betting market."""
market_type: str
pick: str
probability: float
confidence: float
odds: float = 0.0
is_value_bet: bool = False
edge: float = 0.0
def to_dict(self) -> dict:
return {
'market_type': self.market_type,
'pick': self.pick,
'probability': round(self.probability * 100, 1),
'confidence': round(self.confidence, 1),
'odds': self.odds,
'is_value_bet': self.is_value_bet,
'edge': round(self.edge * 100, 1),
}
@dataclass
class ValueBet:
"""Detected value bet opportunity."""
market_type: str
pick: str
probability: float
odds: float
edge: float
confidence: float
def to_dict(self) -> dict:
return {
'market_type': self.market_type,
'pick': self.pick,
'probability': round(self.probability * 100, 1),
'odds': self.odds,
'edge': round(self.edge * 100, 1),
'confidence': round(self.confidence, 1),
}
@dataclass
class MatchPrediction:
"""Complete match prediction with all markets."""
match_id: str
home_team: str
away_team: str
# MS predictions
home_prob: float = 0.0
draw_prob: float = 0.0
away_prob: float = 0.0
ms_pick: str = ''
ms_confidence: float = 0.0
# OU25 predictions
over_prob: float = 0.0
under_prob: float = 0.0
ou25_pick: str = ''
ou25_confidence: float = 0.0
# BTTS predictions
btts_yes_prob: float = 0.0
btts_no_prob: float = 0.0
btts_pick: str = ''
btts_confidence: float = 0.0
# Value bets
value_bets: List[ValueBet] = field(default_factory=list)
def to_dict(self) -> dict:
return {
'match_id': self.match_id,
'home_team': self.home_team,
'away_team': self.away_team,
'ms': {
'home_prob': round(self.home_prob * 100, 1),
'draw_prob': round(self.draw_prob * 100, 1),
'away_prob': round(self.away_prob * 100, 1),
'pick': self.ms_pick,
'confidence': round(self.ms_confidence, 1),
},
'ou25': {
'over_prob': round(self.over_prob * 100, 1),
'under_prob': round(self.under_prob * 100, 1),
'pick': self.ou25_pick,
'confidence': round(self.ou25_confidence, 1),
},
'btts': {
'yes_prob': round(self.btts_yes_prob * 100, 1),
'no_prob': round(self.btts_no_prob * 100, 1),
'pick': self.btts_pick,
'confidence': round(self.btts_confidence, 1),
},
'value_bets': [vb.to_dict() for vb in self.value_bets],
}
class V25Predictor:
"""
V25 Ensemble Predictor - NO TARGET LEAKAGE
Uses market-specific XGBoost and LightGBM models.
Each market (MS, OU25, BTTS) has its own trained models.
"""
# Feature columns — loaded dynamically from feature_cols.json to stay
# in sync with the trained models. The hardcoded list below is only a
# fallback in case the JSON file is missing.
_FALLBACK_FEATURE_COLS = [
# ELO Features (8)
'home_overall_elo', 'away_overall_elo', 'elo_diff',
'home_home_elo', 'away_away_elo',
'home_form_elo', 'away_form_elo', 'form_elo_diff',
# Form Features (12)
'home_goals_avg', 'home_conceded_avg',
'away_goals_avg', 'away_conceded_avg',
'home_clean_sheet_rate', 'away_clean_sheet_rate',
'home_scoring_rate', 'away_scoring_rate',
'home_winning_streak', 'away_winning_streak',
'home_unbeaten_streak', 'away_unbeaten_streak',
# H2H Features (6)
'h2h_total_matches', 'h2h_home_win_rate', 'h2h_draw_rate',
'h2h_avg_goals', 'h2h_btts_rate', 'h2h_over25_rate',
# Team Stats Features (8)
'home_avg_possession', 'away_avg_possession',
'home_avg_shots_on_target', 'away_avg_shots_on_target',
'home_shot_conversion', 'away_shot_conversion',
'home_avg_corners', 'away_avg_corners',
# Odds Features (24)
'odds_ms_h', 'odds_ms_d', 'odds_ms_a',
'implied_home', 'implied_draw', 'implied_away',
'odds_ht_ms_h', 'odds_ht_ms_d', 'odds_ht_ms_a',
'odds_ou05_o', 'odds_ou05_u',
'odds_ou15_o', 'odds_ou15_u',
'odds_ou25_o', 'odds_ou25_u',
'odds_ou35_o', 'odds_ou35_u',
'odds_ht_ou05_o', 'odds_ht_ou05_u',
'odds_ht_ou15_o', 'odds_ht_ou15_u',
'odds_btts_y', 'odds_btts_n',
# Odds Presence Flags (20)
'odds_ms_h_present', 'odds_ms_d_present', 'odds_ms_a_present',
'odds_ht_ms_h_present', 'odds_ht_ms_d_present', 'odds_ht_ms_a_present',
'odds_ou05_o_present', 'odds_ou05_u_present',
'odds_ou15_o_present', 'odds_ou15_u_present',
'odds_ou25_o_present', 'odds_ou25_u_present',
'odds_ou35_o_present', 'odds_ou35_u_present',
'odds_ht_ou05_o_present', 'odds_ht_ou05_u_present',
'odds_ht_ou15_o_present', 'odds_ht_ou15_u_present',
'odds_btts_y_present', 'odds_btts_n_present',
# League Features (4)
'home_xga', 'away_xga',
'league_avg_goals', 'league_zero_goal_rate',
# Upset Engine (4)
'upset_atmosphere', 'upset_motivation', 'upset_fatigue', 'upset_potential',
# Referee Engine (5)
'referee_home_bias', 'referee_avg_goals', 'referee_cards_total',
'referee_avg_yellow', 'referee_experience',
# Momentum Engine (3)
'home_momentum_score', 'away_momentum_score', 'momentum_diff',
# Squad Features (9)
'home_squad_quality', 'away_squad_quality', 'squad_diff',
'home_key_players', 'away_key_players',
'home_missing_impact', 'away_missing_impact',
'home_goals_form', 'away_goals_form',
]
@staticmethod
def _load_feature_cols() -> list:
"""Load feature columns from feature_cols.json, falling back to hardcoded list."""
feature_json = os.path.join(MODELS_DIR, 'feature_cols.json')
try:
if os.path.exists(feature_json):
with open(feature_json, 'r', encoding='utf-8') as f:
cols = json.load(f)
if isinstance(cols, list) and len(cols) > 0:
print(f"[V25] Loaded {len(cols)} feature columns from feature_cols.json")
return cols
except Exception as e:
print(f"[V25] Warning: could not load feature_cols.json: {e}")
print(f"[V25] Using fallback feature columns ({len(V25Predictor._FALLBACK_FEATURE_COLS)} features)")
return V25Predictor._FALLBACK_FEATURE_COLS
FEATURE_COLS = _load_feature_cols.__func__()
# Model weights for ensemble
DEFAULT_WEIGHTS = {
'xgb': 0.50,
'lgb': 0.50,
}
def __init__(self, models_dir: str = None):
"""
Initialize V25 Predictor.
Args:
models_dir: Directory containing model files. Defaults to v25/ directory.
"""
self.models_dir = models_dir or MODELS_DIR
self.models = {} # market -> {'xgb': model, 'lgb': model}
self._loaded = False
# All trained market models available in V25
ALL_MARKETS = [
'ms', 'ou25', 'btts', # Core markets
'ou15', 'ou35', # Additional OU lines
'ht_result', 'ht_ou05', 'ht_ou15', # HT markets
'htft', # HT/FT combo
'cards_ou45', # Cards market
'handicap_ms', # Handicap
'odd_even', # Odd/Even goals
]
# Multi-class markets (output > 2 classes)
MULTICLASS_MARKETS = {'ms', 'ht_result', 'htft', 'handicap_ms'}
def load_models(self) -> bool:
"""Load all market-specific models from disk."""
try:
loaded_count = 0
for market in self.ALL_MARKETS:
self.models[market] = {}
# Load XGBoost (read content in Python to avoid non-ASCII path issues)
xgb_path = os.path.join(self.models_dir, f'xgb_v25_{market}.json')
if os.path.exists(xgb_path) and os.path.getsize(xgb_path) > 0:
with open(xgb_path, 'r', encoding='utf-8') as f:
xgb_content = f.read()
booster = xgb.Booster()
booster.load_model(bytearray(xgb_content, 'utf-8'))
self.models[market]['xgb'] = booster
loaded_count += 1
# Load LightGBM (read content in Python to avoid non-ASCII path issues)
lgb_path = os.path.join(self.models_dir, f'lgb_v25_{market}.txt')
if os.path.exists(lgb_path) and os.path.getsize(lgb_path) > 0:
with open(lgb_path, 'r', encoding='utf-8') as f:
model_str = f.read()
self.models[market]['lgb'] = lgb.Booster(model_str=model_str)
loaded_count += 1
# Remove empty entries
if not self.models[market]:
del self.models[market]
print(f"[V25] Loaded {loaded_count} model files across {len(self.models)} markets: {list(self.models.keys())}")
self._loaded = loaded_count > 0
return self._loaded
except Exception as e:
print(f"[ERROR] Error loading models: {e}")
import traceback
traceback.print_exc()
return False
def _ensure_loaded(self):
"""Ensure models are loaded before prediction."""
if not self._loaded:
if not self.load_models():
raise RuntimeError("Failed to load V25 models")
def _prepare_features(self, features: Dict[str, float]) -> pd.DataFrame:
"""Prepare feature vector for prediction."""
X = pd.DataFrame([{col: features.get(col, 0.0) for col in self.FEATURE_COLS}])
return X
def predict_ms(self, features: Dict[str, float]) -> tuple:
"""
Predict match result (1X2).
Returns:
(home_prob, draw_prob, away_prob)
"""
self._ensure_loaded()
X = self._prepare_features(features)
probs = []
# XGBoost
if 'xgb' in self.models.get('ms', {}):
dmat = xgb.DMatrix(X)
xgb_proba = self.models['ms']['xgb'].predict(dmat)
if len(xgb_proba.shape) == 1:
xgb_proba = np.array([xgb_proba])
probs.append(xgb_proba[0] * self.DEFAULT_WEIGHTS['xgb'])
# LightGBM
if 'lgb' in self.models.get('ms', {}):
lgb_proba = self.models['ms']['lgb'].predict(X)
if len(lgb_proba.shape) == 2:
probs.append(lgb_proba[0] * self.DEFAULT_WEIGHTS['lgb'])
if not probs:
return 0.33, 0.33, 0.33
ensemble_proba = np.sum(probs, axis=0)
ensemble_proba = ensemble_proba / ensemble_proba.sum()
return float(ensemble_proba[0]), float(ensemble_proba[1]), float(ensemble_proba[2])
def predict_ou25(self, features: Dict[str, float]) -> tuple:
"""
Predict Over/Under 2.5 goals.
Returns:
(over_prob, under_prob)
"""
self._ensure_loaded()
X = self._prepare_features(features)
probs = []
# XGBoost
if 'xgb' in self.models.get('ou25', {}):
dmat = xgb.DMatrix(X)
xgb_proba = self.models['ou25']['xgb'].predict(dmat)
if isinstance(xgb_proba, np.ndarray) and len(xgb_proba.shape) == 1:
probs.append(xgb_proba[0])
# LightGBM
if 'lgb' in self.models.get('ou25', {}):
lgb_proba = self.models['ou25']['lgb'].predict(X)
if isinstance(lgb_proba, np.ndarray):
probs.append(lgb_proba[0])
if not probs:
return 0.5, 0.5
# Average probability
avg_prob = np.mean(probs)
return float(avg_prob), float(1 - avg_prob)
def predict_btts(self, features: Dict[str, float]) -> tuple:
"""
Predict Both Teams To Score.
Returns:
(yes_prob, no_prob)
"""
self._ensure_loaded()
X = self._prepare_features(features)
probs = []
# XGBoost
if 'xgb' in self.models.get('btts', {}):
dmat = xgb.DMatrix(X)
xgb_proba = self.models['btts']['xgb'].predict(dmat)
if isinstance(xgb_proba, np.ndarray) and len(xgb_proba.shape) == 1:
probs.append(xgb_proba[0])
# LightGBM
if 'lgb' in self.models.get('btts', {}):
lgb_proba = self.models['btts']['lgb'].predict(X)
if isinstance(lgb_proba, np.ndarray):
probs.append(lgb_proba[0])
if not probs:
return 0.5, 0.5
# Average probability
avg_prob = np.mean(probs)
return float(avg_prob), float(1 - avg_prob)
def predict_market(self, market: str, features: Dict[str, float]) -> np.ndarray:
"""
Generic prediction for any loaded market.
Args:
market: Market key (e.g. 'ht_result', 'htft', 'cards_ou45')
features: Feature dictionary.
Returns:
numpy array of probabilities.
For binary markets: [positive_prob]
For multi-class markets: [class0_prob, class1_prob, ...]
"""
self._ensure_loaded()
if market not in self.models:
return None
X = self._prepare_features(features)
probs = []
weights = []
is_multiclass = market in self.MULTICLASS_MARKETS
# XGBoost
if 'xgb' in self.models[market]:
dmat = xgb.DMatrix(X)
xgb_proba = self.models[market]['xgb'].predict(dmat)
if isinstance(xgb_proba, np.ndarray):
if is_multiclass and len(xgb_proba.shape) == 2:
probs.append(xgb_proba[0])
elif is_multiclass and len(xgb_proba.shape) == 1:
probs.append(xgb_proba)
else:
probs.append(np.array([xgb_proba[0]]))
weights.append(self.DEFAULT_WEIGHTS['xgb'])
# LightGBM
if 'lgb' in self.models[market]:
lgb_proba = self.models[market]['lgb'].predict(X)
if isinstance(lgb_proba, np.ndarray):
if is_multiclass and len(lgb_proba.shape) == 2:
probs.append(lgb_proba[0])
elif is_multiclass and len(lgb_proba.shape) == 1:
probs.append(lgb_proba)
else:
probs.append(np.array([lgb_proba[0]]))
weights.append(self.DEFAULT_WEIGHTS['lgb'])
if not probs:
return None
# Weighted average
if len(probs) == 1:
return probs[0]
total_w = sum(weights[:len(probs)])
result = np.zeros_like(probs[0])
for p, w in zip(probs, weights):
result += p * (w / total_w)
# Normalize multi-class
if is_multiclass and result.sum() > 0:
result = result / result.sum()
return result
def has_market(self, market: str) -> bool:
"""Check if a specific market model is loaded."""
return market in self.models
def predict_match(
self,
match_id: str,
home_team: str,
away_team: str,
features: Dict[str, float],
odds: Optional[Dict[str, float]] = None,
) -> MatchPrediction:
"""
Predict all markets for a match.
Args:
match_id: Match identifier.
home_team: Home team name.
away_team: Away team name.
features: Feature dictionary.
odds: Optional odds dictionary for value bet detection.
Returns:
MatchPrediction object.
"""
# Get predictions for each market
home_prob, draw_prob, away_prob = self.predict_ms(features)
over_prob, under_prob = self.predict_ou25(features)
btts_yes_prob, btts_no_prob = self.predict_btts(features)
# Determine picks
ms_probs = {'1': home_prob, 'X': draw_prob, '2': away_prob}
ms_pick = max(ms_probs, key=ms_probs.get)
ms_confidence = ms_probs[ms_pick] * 100
ou25_probs = {'Over': over_prob, 'Under': under_prob}
ou25_pick = max(ou25_probs, key=ou25_probs.get)
ou25_confidence = ou25_probs[ou25_pick] * 100
btts_probs = {'Yes': btts_yes_prob, 'No': btts_no_prob}
btts_pick = max(btts_probs, key=btts_probs.get)
btts_confidence = btts_probs[btts_pick] * 100
# Create prediction
prediction = MatchPrediction(
match_id=match_id,
home_team=home_team,
away_team=away_team,
home_prob=home_prob,
draw_prob=draw_prob,
away_prob=away_prob,
ms_pick=ms_pick,
ms_confidence=ms_confidence,
over_prob=over_prob,
under_prob=under_prob,
ou25_pick=ou25_pick,
ou25_confidence=ou25_confidence,
btts_yes_prob=btts_yes_prob,
btts_no_prob=btts_no_prob,
btts_pick=btts_pick,
btts_confidence=btts_confidence,
)
# Detect value bets
if odds:
prediction.value_bets = self._detect_value_bets(
prediction, odds, home_prob, draw_prob, away_prob,
over_prob, under_prob, btts_yes_prob, btts_no_prob
)
return prediction
def _detect_value_bets(
self,
prediction: MatchPrediction,
odds: Dict[str, float],
home_prob: float,
draw_prob: float,
away_prob: float,
over_prob: float,
under_prob: float,
btts_yes_prob: float,
btts_no_prob: float,
) -> List[ValueBet]:
"""Detect value bets based on model vs market odds."""
value_bets = []
min_edge = 0.05 # 5% minimum edge
# MS value bets
if 'ms_h' in odds and odds['ms_h'] > 0:
implied = 1 / odds['ms_h']
edge = home_prob - implied
if edge > min_edge:
value_bets.append(ValueBet(
market_type='MS',
pick='1',
probability=home_prob,
odds=odds['ms_h'],
edge=edge,
confidence=home_prob * 100,
))
if 'ms_d' in odds and odds['ms_d'] > 0:
implied = 1 / odds['ms_d']
edge = draw_prob - implied
if edge > min_edge:
value_bets.append(ValueBet(
market_type='MS',
pick='X',
probability=draw_prob,
odds=odds['ms_d'],
edge=edge,
confidence=draw_prob * 100,
))
if 'ms_a' in odds and odds['ms_a'] > 0:
implied = 1 / odds['ms_a']
edge = away_prob - implied
if edge > min_edge:
value_bets.append(ValueBet(
market_type='MS',
pick='2',
probability=away_prob,
odds=odds['ms_a'],
edge=edge,
confidence=away_prob * 100,
))
# OU25 value bets
if 'ou25_o' in odds and odds['ou25_o'] > 0:
implied = 1 / odds['ou25_o']
edge = over_prob - implied
if edge > min_edge:
value_bets.append(ValueBet(
market_type='OU25',
pick='Over',
probability=over_prob,
odds=odds['ou25_o'],
edge=edge,
confidence=over_prob * 100,
))
if 'ou25_u' in odds and odds['ou25_u'] > 0:
implied = 1 / odds['ou25_u']
edge = under_prob - implied
if edge > min_edge:
value_bets.append(ValueBet(
market_type='OU25',
pick='Under',
probability=under_prob,
odds=odds['ou25_u'],
edge=edge,
confidence=under_prob * 100,
))
# BTTS value bets
if 'btts_y' in odds and odds['btts_y'] > 0:
implied = 1 / odds['btts_y']
edge = btts_yes_prob - implied
if edge > min_edge:
value_bets.append(ValueBet(
market_type='BTTS',
pick='Yes',
probability=btts_yes_prob,
odds=odds['btts_y'],
edge=edge,
confidence=btts_yes_prob * 100,
))
if 'btts_n' in odds and odds['btts_n'] > 0:
implied = 1 / odds['btts_n']
edge = btts_no_prob - implied
if edge > min_edge:
value_bets.append(ValueBet(
market_type='BTTS',
pick='No',
probability=btts_no_prob,
odds=odds['btts_n'],
edge=edge,
confidence=btts_no_prob * 100,
))
return value_bets
# Singleton instance
_v25_predictor: Optional[V25Predictor] = None
def get_v25_predictor() -> V25Predictor:
"""Get or create V25 predictor instance."""
global _v25_predictor
if _v25_predictor is None:
_v25_predictor = V25Predictor()
_v25_predictor.load_models()
return _v25_predictor
+343
View File
@@ -0,0 +1,343 @@
"""
V27 Pro Predictor Odds-Free Fundamentals + Value Edge Detection
This module loads V27 ensemble models (XGBoost, LightGBM, CatBoost)
and produces market-independent probability estimates.
The key insight: V27 is trained WITHOUT odds features, so it produces
"true" probabilities unbiased by market pricing. The divergence between
V25 (odds-aware) and V27 (odds-free) predictions signals market mispricing.
"""
import json
import logging
import os
import pickle
from pathlib import Path
from typing import Dict, List, Optional, Tuple
import numpy as np
logger = logging.getLogger(__name__)
V27_DIR = Path(__file__).parent / "v27"
class V27Predictor:
"""
Loads V27 ensemble models and provides predictions using the
82-feature odds-free vector.
"""
MARKETS = ['ms', 'ou25', 'btts']
def __init__(self):
self.models: Dict[str, Dict[str, object]] = {}
self.feature_cols: List[str] = []
self._loaded = False
def load_models(self) -> bool:
"""Load all V27 ensemble models and feature column spec."""
if self._loaded:
return True
# Feature columns
cols_path = V27_DIR / "v27_feature_cols.json"
if not cols_path.exists():
logger.error("[V27] Feature columns file not found: %s", cols_path)
return False
try:
with open(cols_path, "r", encoding="utf-8") as f:
self.feature_cols = json.load(f)
logger.info("[V27] Loaded %d feature columns", len(self.feature_cols))
except Exception as e:
logger.error("[V27] Failed to load feature columns: %s", e)
return False
# Load models per market
model_types = {"xgb": "xgb", "lgb": "lgb"}
for market in self.MARKETS:
self.models[market] = {}
for short, label in model_types.items():
# Try market-specific file first: v27_ms_xgb.pkl
path = V27_DIR / f"v27_{market}_{short}.pkl"
if not path.exists():
# Fallback to generic: v27_xgboost.pkl (for MS only)
generic_names = {"xgb": "v27_xgboost.pkl", "lgb": "v27_lightgbm.pkl", "cb": "v27_catboost.pkl"}
path = V27_DIR / generic_names.get(short, "")
if not path.exists():
logger.warning("[V27] Model file not found for %s/%s", market, short)
continue
try:
with open(path, "rb") as f:
model = pickle.load(f)
self.models[market][label] = model
logger.info("[V27] ✓ Loaded %s/%s from %s", market, label, path.name)
except Exception as e:
logger.error("[V27] ✗ Failed to load %s/%s: %s", market, label, e)
loaded_count = sum(len(v) for v in self.models.values())
if loaded_count == 0:
logger.error("[V27] No models loaded!")
return False
self._loaded = True
logger.info("[V27] Total models loaded: %d across %d markets", loaded_count, len(self.models))
return True
def _build_feature_array(self, features: Dict[str, float]) -> np.ndarray:
"""
Build ordered feature array from the full feature dict.
V27 uses only its 82 features (odds-free subset).
"""
row = []
for col in self.feature_cols:
row.append(float(features.get(col, 0.0)))
return np.array([row])
def _predict_with_model(self, model, X: np.ndarray, label: str, expected_classes: int) -> Optional[np.ndarray]:
"""
Predict probabilities from a model, handling both sklearn wrappers
(predict_proba) and raw Booster objects (predict).
For raw XGBoost Boosters, DMatrix is created WITH feature_names
to match the training schema.
"""
import xgboost as xgb
import lightgbm as lgbm
import pandas as pd
# 1. Try sklearn-style predict_proba first
if hasattr(model, 'predict_proba'):
try:
proba = model.predict_proba(X)[0]
if len(proba) == expected_classes:
return proba
logger.warning("[V27] %s predict_proba returned %d classes, expected %d", label, len(proba), expected_classes)
except Exception:
pass # Fall through to raw predict
# 2. Raw xgboost.Booster — MUST pass feature_names
if isinstance(model, xgb.Booster):
try:
feature_names = self.feature_cols if self.feature_cols else None
dmat = xgb.DMatrix(X, feature_names=feature_names)
raw = model.predict(dmat)
if isinstance(raw, np.ndarray):
if raw.ndim == 2 and raw.shape[1] == expected_classes:
return raw[0]
elif raw.ndim == 1 and expected_classes == 2:
p = float(raw[0])
return np.array([1.0 - p, p])
elif raw.ndim == 1 and len(raw) == expected_classes:
return raw
except Exception as e:
logger.warning("[V27] %s xgb.Booster predict failed: %s", label, e)
return None
# 3. Raw lightgbm.Booster — pass as DataFrame with column names
if isinstance(model, lgbm.Booster):
try:
if self.feature_cols:
X_named = pd.DataFrame(X, columns=self.feature_cols)
raw = model.predict(X_named)
else:
raw = model.predict(X)
if isinstance(raw, np.ndarray):
if raw.ndim == 2 and raw.shape[1] == expected_classes:
return raw[0]
elif raw.ndim == 1 and expected_classes == 2:
p = float(raw[0])
return np.array([1.0 - p, p])
elif raw.ndim == 1 and len(raw) == expected_classes:
return raw
except Exception as e:
logger.warning("[V27] %s lgb.Booster predict failed: %s", label, e)
return None
# 4. Generic fallback (CatBoost, etc.)
try:
if hasattr(model, 'predict'):
raw = model.predict(X)
if isinstance(raw, np.ndarray):
if raw.ndim == 2 and raw.shape[1] == expected_classes:
return raw[0]
elif raw.ndim == 1 and expected_classes == 2:
p = float(raw[0])
return np.array([1.0 - p, p])
elif raw.ndim == 1 and len(raw) == expected_classes:
return raw
except Exception as e:
logger.warning("[V27] %s generic predict failed: %s", label, e)
return None
def predict_ms(self, features: Dict[str, float]) -> Optional[Dict[str, float]]:
"""
Predict Match Score probabilities (Home/Draw/Away).
Returns dict with keys: home, draw, away.
"""
if not self._loaded or "ms" not in self.models or not self.models["ms"]:
return None
X = self._build_feature_array(features)
probs_list = []
for label, model in self.models["ms"].items():
proba = self._predict_with_model(model, X, f"MS/{label}", expected_classes=3)
if proba is not None and len(proba) == 3:
probs_list.append(proba)
if not probs_list:
return None
# Ensemble average
avg = np.mean(probs_list, axis=0)
return {
"home": float(avg[0]),
"draw": float(avg[1]),
"away": float(avg[2]),
}
def predict_ou25(self, features: Dict[str, float]) -> Optional[Dict[str, float]]:
"""
Predict Over/Under 2.5 probabilities.
Returns dict with keys: under, over.
"""
if not self._loaded or "ou25" not in self.models or not self.models["ou25"]:
return None
X = self._build_feature_array(features)
probs_list = []
for label, model in self.models["ou25"].items():
proba = self._predict_with_model(model, X, f"OU25/{label}", expected_classes=2)
if proba is not None and len(proba) == 2:
probs_list.append(proba)
if not probs_list:
return None
avg = np.mean(probs_list, axis=0)
return {
"under": float(avg[0]),
"over": float(avg[1]),
}
def predict_btts(self, features: Dict[str, float]) -> Optional[Dict[str, float]]:
"""
Predict Both Teams To Score probabilities.
Returns dict with keys: no, yes.
"""
if not self._loaded or 'btts' not in self.models or not self.models['btts']:
return None
X = self._build_feature_array(features)
probs_list = []
for label, model in self.models['btts'].items():
proba = self._predict_with_model(model, X, f'BTTS/{label}', expected_classes=2)
if proba is not None and len(proba) == 2:
probs_list.append(proba)
if not probs_list:
return None
avg = np.mean(probs_list, axis=0)
return {
'no': float(avg[0]),
'yes': float(avg[1]),
}
def predict_dc(self, features: Dict[str, float]) -> Optional[Dict[str, float]]:
"""
Predict Double Chance probabilities.
DC is algebraically derived from MS predictions:
1X = home + draw
X2 = draw + away
12 = home + away
This gives an odds-free DC estimate for divergence detection.
"""
ms_probs = self.predict_ms(features)
if not ms_probs:
return None
home = ms_probs['home']
draw = ms_probs['draw']
away = ms_probs['away']
return {
'1x': round(home + draw, 4),
'x2': round(draw + away, 4),
'12': round(home + away, 4),
}
def predict_all(self, features: Dict[str, float]) -> Dict[str, Optional[Dict[str, float]]]:
"""Run predictions for all supported markets."""
return {
'ms': self.predict_ms(features),
'ou25': self.predict_ou25(features),
'btts': self.predict_btts(features),
'dc': self.predict_dc(features),
}
def compute_divergence(
v25_probs: Dict[str, float],
v27_probs: Dict[str, float],
) -> Dict[str, float]:
"""
Compute the divergence signal between V25 (odds-aware) and V27 (odds-free).
Positive divergence = V27 thinks it's MORE likely than the market → VALUE BET
Negative divergence = V27 thinks it's LESS likely than the market → PASS
Returns per-outcome divergence values.
"""
divergence = {}
for key in v27_probs:
v25_val = v25_probs.get(key, 0.33)
v27_val = v27_probs.get(key, 0.33)
divergence[key] = round(v27_val - v25_val, 4)
return divergence
def compute_value_edge(
v25_probs: Dict[str, float],
v27_probs: Dict[str, float],
odds: Dict[str, float],
) -> Dict[str, Dict]:
"""
Detect value bets by combining V25/V27 divergence with odds.
A value bet exists when:
1. V27 (odds-free) probability > implied odds probability (model says it's underpriced)
2. V27 and V25 divergence is positive (V27 sees more signal than the market)
Returns per-outcome: { probability, implied_prob, edge, is_value }
"""
results = {}
for key in v27_probs:
v27_p = v27_probs[key]
v25_p = v25_probs.get(key, 0.33)
odds_val = odds.get(key, 0.0)
implied_p = (1.0 / odds_val) if odds_val > 1.01 else 0.0
divergence = v27_p - v25_p
edge = v27_p - implied_p if implied_p > 0 else 0.0
results[key] = {
"v27_prob": round(v27_p, 4),
"v25_prob": round(v25_p, 4),
"implied_prob": round(implied_p, 4),
"divergence": round(divergence, 4),
"edge": round(edge, 4),
"is_value": edge > 0.05 and divergence > 0.02, # 5% edge + 2% divergence
}
return results
@@ -1,8 +1,8 @@
{
"trained_at": "2026-04-14 17:20:03",
"trained_at": "2026-05-06 15:53:36",
"market_results": {
"MS": {
"samples": 9791,
"samples": 106428,
"features_used": [
"home_overall_elo",
"away_overall_elo",
@@ -107,19 +107,19 @@
"home_goals_form",
"away_goals_form"
],
"train_samples": 6853,
"val_samples": 1469,
"test_samples": 1469,
"xgb_accuracy": 0.8938,
"xgb_logloss": 0.2263,
"lgb_accuracy": 0.8938,
"lgb_logloss": 0.2214,
"ensemble_accuracy": 0.8945,
"ensemble_logloss": 0.2226,
"train_samples": 74499,
"val_samples": 15964,
"test_samples": 15965,
"xgb_accuracy": 0.5437,
"xgb_logloss": 0.9429,
"lgb_accuracy": 0.5436,
"lgb_logloss": 0.9423,
"ensemble_accuracy": 0.5442,
"ensemble_logloss": 0.9418,
"class_count": 3
},
"OU15": {
"samples": 9791,
"samples": 106428,
"features_used": [
"home_overall_elo",
"away_overall_elo",
@@ -224,19 +224,19 @@
"home_goals_form",
"away_goals_form"
],
"train_samples": 6853,
"val_samples": 1469,
"test_samples": 1469,
"xgb_accuracy": 0.9088,
"xgb_logloss": 0.1758,
"lgb_accuracy": 0.9067,
"lgb_logloss": 0.1783,
"ensemble_accuracy": 0.9108,
"ensemble_logloss": 0.1753,
"train_samples": 74499,
"val_samples": 15964,
"test_samples": 15965,
"xgb_accuracy": 0.753,
"xgb_logloss": 0.5256,
"lgb_accuracy": 0.7523,
"lgb_logloss": 0.5262,
"ensemble_accuracy": 0.7533,
"ensemble_logloss": 0.5254,
"class_count": 2
},
"OU25": {
"samples": 9791,
"samples": 106428,
"features_used": [
"home_overall_elo",
"away_overall_elo",
@@ -341,19 +341,19 @@
"home_goals_form",
"away_goals_form"
],
"train_samples": 6853,
"val_samples": 1469,
"test_samples": 1469,
"xgb_accuracy": 0.9204,
"xgb_logloss": 0.1535,
"lgb_accuracy": 0.9224,
"lgb_logloss": 0.1523,
"ensemble_accuracy": 0.9217,
"ensemble_logloss": 0.1518,
"train_samples": 74499,
"val_samples": 15964,
"test_samples": 15965,
"xgb_accuracy": 0.6253,
"xgb_logloss": 0.635,
"lgb_accuracy": 0.6246,
"lgb_logloss": 0.6347,
"ensemble_accuracy": 0.6262,
"ensemble_logloss": 0.6343,
"class_count": 2
},
"OU35": {
"samples": 9791,
"samples": 106428,
"features_used": [
"home_overall_elo",
"away_overall_elo",
@@ -458,19 +458,19 @@
"home_goals_form",
"away_goals_form"
],
"train_samples": 6853,
"val_samples": 1469,
"test_samples": 1469,
"xgb_accuracy": 0.9578,
"xgb_logloss": 0.1171,
"lgb_accuracy": 0.9564,
"lgb_logloss": 0.1144,
"ensemble_accuracy": 0.9571,
"ensemble_logloss": 0.1149,
"train_samples": 74499,
"val_samples": 15964,
"test_samples": 15965,
"xgb_accuracy": 0.7283,
"xgb_logloss": 0.5463,
"lgb_accuracy": 0.7304,
"lgb_logloss": 0.546,
"ensemble_accuracy": 0.7297,
"ensemble_logloss": 0.5456,
"class_count": 2
},
"BTTS": {
"samples": 9791,
"samples": 106428,
"features_used": [
"home_overall_elo",
"away_overall_elo",
@@ -575,19 +575,19 @@
"home_goals_form",
"away_goals_form"
],
"train_samples": 6853,
"val_samples": 1469,
"test_samples": 1469,
"xgb_accuracy": 0.9238,
"xgb_logloss": 0.1439,
"lgb_accuracy": 0.9265,
"lgb_logloss": 0.143,
"ensemble_accuracy": 0.9265,
"ensemble_logloss": 0.1424,
"train_samples": 74499,
"val_samples": 15964,
"test_samples": 15965,
"xgb_accuracy": 0.5894,
"xgb_logloss": 0.6636,
"lgb_accuracy": 0.5928,
"lgb_logloss": 0.6633,
"ensemble_accuracy": 0.5897,
"ensemble_logloss": 0.6628,
"class_count": 2
},
"HT_RESULT": {
"samples": 9786,
"samples": 103208,
"features_used": [
"home_overall_elo",
"away_overall_elo",
@@ -692,19 +692,19 @@
"home_goals_form",
"away_goals_form"
],
"train_samples": 6850,
"val_samples": 1468,
"test_samples": 1468,
"xgb_accuracy": 0.5627,
"xgb_logloss": 0.8712,
"lgb_accuracy": 0.5715,
"lgb_logloss": 0.8649,
"ensemble_accuracy": 0.5811,
"ensemble_logloss": 0.8649,
"train_samples": 72245,
"val_samples": 15481,
"test_samples": 15482,
"xgb_accuracy": 0.4695,
"xgb_logloss": 1.0174,
"lgb_accuracy": 0.4677,
"lgb_logloss": 1.0166,
"ensemble_accuracy": 0.4688,
"ensemble_logloss": 1.0164,
"class_count": 3
},
"HT_OU05": {
"samples": 9786,
"samples": 103208,
"features_used": [
"home_overall_elo",
"away_overall_elo",
@@ -809,19 +809,19 @@
"home_goals_form",
"away_goals_form"
],
"train_samples": 6850,
"val_samples": 1468,
"test_samples": 1468,
"xgb_accuracy": 0.7221,
"xgb_logloss": 0.5122,
"lgb_accuracy": 0.7268,
"lgb_logloss": 0.5092,
"ensemble_accuracy": 0.7275,
"ensemble_logloss": 0.5084,
"train_samples": 72245,
"val_samples": 15481,
"test_samples": 15482,
"xgb_accuracy": 0.7011,
"xgb_logloss": 0.5939,
"lgb_accuracy": 0.7002,
"lgb_logloss": 0.5936,
"ensemble_accuracy": 0.7009,
"ensemble_logloss": 0.5932,
"class_count": 2
},
"HT_OU15": {
"samples": 9786,
"samples": 103208,
"features_used": [
"home_overall_elo",
"away_overall_elo",
@@ -926,19 +926,19 @@
"home_goals_form",
"away_goals_form"
],
"train_samples": 6850,
"val_samples": 1468,
"test_samples": 1468,
"xgb_accuracy": 0.752,
"xgb_logloss": 0.5252,
"lgb_accuracy": 0.7595,
"lgb_logloss": 0.5213,
"ensemble_accuracy": 0.7595,
"ensemble_logloss": 0.5192,
"train_samples": 72245,
"val_samples": 15481,
"test_samples": 15482,
"xgb_accuracy": 0.6723,
"xgb_logloss": 0.6126,
"lgb_accuracy": 0.6736,
"lgb_logloss": 0.6118,
"ensemble_accuracy": 0.6734,
"ensemble_logloss": 0.6117,
"class_count": 2
},
"HTFT": {
"samples": 9786,
"samples": 103208,
"features_used": [
"home_overall_elo",
"away_overall_elo",
@@ -1043,19 +1043,19 @@
"home_goals_form",
"away_goals_form"
],
"train_samples": 6850,
"val_samples": 1468,
"test_samples": 1468,
"xgb_accuracy": 0.5136,
"xgb_logloss": 1.1384,
"lgb_accuracy": 0.5184,
"lgb_logloss": 1.1469,
"ensemble_accuracy": 0.5143,
"ensemble_logloss": 1.1339,
"train_samples": 72245,
"val_samples": 15481,
"test_samples": 15482,
"xgb_accuracy": 0.3337,
"xgb_logloss": 1.8208,
"lgb_accuracy": 0.3332,
"lgb_logloss": 1.8203,
"ensemble_accuracy": 0.3358,
"ensemble_logloss": 1.8186,
"class_count": 9
},
"ODD_EVEN": {
"samples": 9791,
"samples": 106428,
"features_used": [
"home_overall_elo",
"away_overall_elo",
@@ -1160,19 +1160,19 @@
"home_goals_form",
"away_goals_form"
],
"train_samples": 6853,
"val_samples": 1469,
"test_samples": 1469,
"xgb_accuracy": 0.8863,
"xgb_logloss": 0.3565,
"lgb_accuracy": 0.8802,
"lgb_logloss": 0.3338,
"ensemble_accuracy": 0.8863,
"ensemble_logloss": 0.3423,
"train_samples": 74499,
"val_samples": 15964,
"test_samples": 15965,
"xgb_accuracy": 0.5296,
"xgb_logloss": 0.6841,
"lgb_accuracy": 0.5359,
"lgb_logloss": 0.6822,
"ensemble_accuracy": 0.531,
"ensemble_logloss": 0.6826,
"class_count": 2
},
"CARDS_OU45": {
"samples": 9791,
"samples": 106428,
"features_used": [
"home_overall_elo",
"away_overall_elo",
@@ -1277,19 +1277,19 @@
"home_goals_form",
"away_goals_form"
],
"train_samples": 6853,
"val_samples": 1469,
"test_samples": 1469,
"xgb_accuracy": 0.6283,
"xgb_logloss": 0.6174,
"lgb_accuracy": 0.6413,
"lgb_logloss": 0.615,
"ensemble_accuracy": 0.6372,
"ensemble_logloss": 0.6142,
"train_samples": 74499,
"val_samples": 15964,
"test_samples": 15965,
"xgb_accuracy": 0.6009,
"xgb_logloss": 0.6489,
"lgb_accuracy": 0.5988,
"lgb_logloss": 0.6487,
"ensemble_accuracy": 0.6024,
"ensemble_logloss": 0.6479,
"class_count": 2
},
"HANDICAP_MS": {
"samples": 9791,
"samples": 106428,
"features_used": [
"home_overall_elo",
"away_overall_elo",
@@ -1394,15 +1394,15 @@
"home_goals_form",
"away_goals_form"
],
"train_samples": 6853,
"val_samples": 1469,
"test_samples": 1469,
"xgb_accuracy": 0.936,
"xgb_logloss": 0.1903,
"lgb_accuracy": 0.9346,
"lgb_logloss": 0.1843,
"ensemble_accuracy": 0.936,
"ensemble_logloss": 0.1861,
"train_samples": 74499,
"val_samples": 15964,
"test_samples": 15965,
"xgb_accuracy": 0.6058,
"xgb_logloss": 0.8691,
"lgb_accuracy": 0.608,
"lgb_logloss": 0.8677,
"ensemble_accuracy": 0.6068,
"ensemble_logloss": 0.8677,
"class_count": 3
}
}
@@ -0,0 +1,692 @@
{
"trained_at": "2026-05-10 19:48:06",
"trainer": "v25_pro",
"optuna_trials": 50,
"total_features": 114,
"markets": {
"MS": {
"market": "MS",
"samples": 106861,
"train": 64116,
"val": 16029,
"cal": 10686,
"test": 16030,
"features_used": 114,
"xgb_best_params": {
"max_depth": 4,
"eta": 0.022329400652878233,
"subsample": 0.6690795757813364,
"colsample_bytree": 0.5042256538541441,
"min_child_weight": 6,
"gamma": 9.960129417155444e-05,
"reg_lambda": 0.5132295377582388,
"reg_alpha": 6.804503659726287e-08
},
"lgb_best_params": {
"max_depth": 4,
"learning_rate": 0.023142410802706542,
"feature_fraction": 0.5728681432360808,
"bagging_fraction": 0.6781774410065095,
"bagging_freq": 2,
"min_child_samples": 26,
"lambda_l1": 3.25216937188593e-05,
"lambda_l2": 4.8081236902660474e-08
},
"xgb_best_iteration": 643,
"lgb_best_iteration": 441,
"xgb_optuna_best_logloss": 0.9155,
"lgb_optuna_best_logloss": 0.9146,
"test_xgb_raw": {
"accuracy": 0.5442,
"logloss": 0.943
},
"test_xgb_calibrated": {
"accuracy": 0.5404,
"logloss": 0.9438
},
"test_lgb_raw": {
"accuracy": 0.5427,
"logloss": 0.943
},
"test_lgb_calibrated": {
"accuracy": 0.5417,
"logloss": 0.9447
},
"test_ensemble_raw": {
"accuracy": 0.5437,
"logloss": 0.9426
},
"test_ensemble_calibrated": {
"accuracy": 0.5418,
"logloss": 0.9435
}
},
"OU15": {
"market": "OU15",
"samples": 106861,
"train": 64116,
"val": 16029,
"cal": 10686,
"test": 16030,
"features_used": 114,
"xgb_best_params": {
"max_depth": 5,
"eta": 0.020779487257177966,
"subsample": 0.8109935286948485,
"colsample_bytree": 0.9525413847213635,
"min_child_weight": 6,
"gamma": 0.35330347775044696,
"reg_lambda": 5.373541021746059e-07,
"reg_alpha": 0.2959430087754284
},
"lgb_best_params": {
"max_depth": 6,
"learning_rate": 0.013402310027682367,
"feature_fraction": 0.7404728146233901,
"bagging_fraction": 0.9712026511549247,
"bagging_freq": 6,
"min_child_samples": 39,
"lambda_l1": 0.39893027986899576,
"lambda_l2": 0.0626443611997599
},
"xgb_best_iteration": 353,
"lgb_best_iteration": 370,
"xgb_optuna_best_logloss": 0.499,
"lgb_optuna_best_logloss": 0.4989,
"test_xgb_raw": {
"accuracy": 0.7521,
"logloss": 0.5267
},
"test_xgb_calibrated": {
"accuracy": 0.7521,
"logloss": 0.5344
},
"test_lgb_raw": {
"accuracy": 0.7528,
"logloss": 0.5261
},
"test_lgb_calibrated": {
"accuracy": 0.7505,
"logloss": 0.5362
},
"test_ensemble_raw": {
"accuracy": 0.7518,
"logloss": 0.5261
},
"test_ensemble_calibrated": {
"accuracy": 0.7522,
"logloss": 0.5364
}
},
"OU25": {
"market": "OU25",
"samples": 106861,
"train": 64116,
"val": 16029,
"cal": 10686,
"test": 16030,
"features_used": 114,
"xgb_best_params": {
"max_depth": 5,
"eta": 0.01274409160014454,
"subsample": 0.8300258899365814,
"colsample_bytree": 0.7336425662264429,
"min_child_weight": 9,
"gamma": 2.5382243933649716e-06,
"reg_lambda": 5.096723080351853e-05,
"reg_alpha": 0.00040919711449493223
},
"lgb_best_params": {
"max_depth": 6,
"learning_rate": 0.02301514680733822,
"feature_fraction": 0.9569492061944688,
"bagging_fraction": 0.7249143523144639,
"bagging_freq": 1,
"min_child_samples": 40,
"lambda_l1": 9.954995248644963e-08,
"lambda_l2": 3.82413187126927e-06
},
"xgb_best_iteration": 475,
"lgb_best_iteration": 235,
"xgb_optuna_best_logloss": 0.6202,
"lgb_optuna_best_logloss": 0.62,
"test_xgb_raw": {
"accuracy": 0.6221,
"logloss": 0.6352
},
"test_xgb_calibrated": {
"accuracy": 0.6226,
"logloss": 0.6344
},
"test_lgb_raw": {
"accuracy": 0.6236,
"logloss": 0.6348
},
"test_lgb_calibrated": {
"accuracy": 0.6231,
"logloss": 0.6343
},
"test_ensemble_raw": {
"accuracy": 0.6239,
"logloss": 0.6349
},
"test_ensemble_calibrated": {
"accuracy": 0.6236,
"logloss": 0.6338
}
},
"OU35": {
"market": "OU35",
"samples": 106861,
"train": 64116,
"val": 16029,
"cal": 10686,
"test": 16030,
"features_used": 114,
"xgb_best_params": {
"max_depth": 4,
"eta": 0.012538827444713596,
"subsample": 0.7947923612828379,
"colsample_bytree": 0.9717654601553765,
"min_child_weight": 6,
"gamma": 0.011265216242399128,
"reg_lambda": 0.12152579364613436,
"reg_alpha": 0.013995120492957489
},
"lgb_best_params": {
"max_depth": 6,
"learning_rate": 0.013456307557939324,
"feature_fraction": 0.8208768633332759,
"bagging_fraction": 0.929472334516626,
"bagging_freq": 6,
"min_child_samples": 35,
"lambda_l1": 0.05522724221034949,
"lambda_l2": 0.21689047644122147
},
"xgb_best_iteration": 696,
"lgb_best_iteration": 412,
"xgb_optuna_best_logloss": 0.552,
"lgb_optuna_best_logloss": 0.5515,
"test_xgb_raw": {
"accuracy": 0.7314,
"logloss": 0.5466
},
"test_xgb_calibrated": {
"accuracy": 0.7293,
"logloss": 0.5482
},
"test_lgb_raw": {
"accuracy": 0.73,
"logloss": 0.5462
},
"test_lgb_calibrated": {
"accuracy": 0.7298,
"logloss": 0.5485
},
"test_ensemble_raw": {
"accuracy": 0.7312,
"logloss": 0.5462
},
"test_ensemble_calibrated": {
"accuracy": 0.7301,
"logloss": 0.5478
}
},
"BTTS": {
"market": "BTTS",
"samples": 106861,
"train": 64116,
"val": 16029,
"cal": 10686,
"test": 16030,
"features_used": 114,
"xgb_best_params": {
"max_depth": 4,
"eta": 0.023533647209064805,
"subsample": 0.7469060816054074,
"colsample_bytree": 0.8445418254808608,
"min_child_weight": 8,
"gamma": 1.0503733400514561e-08,
"reg_lambda": 2.0919595769527735e-06,
"reg_alpha": 0.027277017326535417
},
"lgb_best_params": {
"max_depth": 4,
"learning_rate": 0.03900730648793646,
"feature_fraction": 0.6968255358438369,
"bagging_fraction": 0.7078349435778689,
"bagging_freq": 1,
"min_child_samples": 46,
"lambda_l1": 1.1796591413903922e-05,
"lambda_l2": 1.574367227995052e-08
},
"xgb_best_iteration": 462,
"lgb_best_iteration": 339,
"xgb_optuna_best_logloss": 0.6557,
"lgb_optuna_best_logloss": 0.6554,
"test_xgb_raw": {
"accuracy": 0.5908,
"logloss": 0.6637
},
"test_xgb_calibrated": {
"accuracy": 0.5885,
"logloss": 0.6647
},
"test_lgb_raw": {
"accuracy": 0.5891,
"logloss": 0.6638
},
"test_lgb_calibrated": {
"accuracy": 0.5891,
"logloss": 0.6702
},
"test_ensemble_raw": {
"accuracy": 0.5892,
"logloss": 0.6635
},
"test_ensemble_calibrated": {
"accuracy": 0.5885,
"logloss": 0.6655
}
},
"HT_RESULT": {
"market": "HT_RESULT",
"samples": 103641,
"train": 62184,
"val": 15546,
"cal": 10364,
"test": 15547,
"features_used": 114,
"xgb_best_params": {
"max_depth": 4,
"eta": 0.01736265891311687,
"subsample": 0.8370935625192159,
"colsample_bytree": 0.8091927356001175,
"min_child_weight": 9,
"gamma": 0.0006570311316367184,
"reg_lambda": 0.5206211670360164,
"reg_alpha": 0.0004530536252850605
},
"lgb_best_params": {
"max_depth": 4,
"learning_rate": 0.04842652289664568,
"feature_fraction": 0.6277272818879166,
"bagging_fraction": 0.9526964840164693,
"bagging_freq": 3,
"min_child_samples": 23,
"lambda_l1": 0.09429192580834124,
"lambda_l2": 5.5433175427148124e-08
},
"xgb_best_iteration": 516,
"lgb_best_iteration": 136,
"xgb_optuna_best_logloss": 1.0128,
"lgb_optuna_best_logloss": 1.0126,
"test_xgb_raw": {
"accuracy": 0.4689,
"logloss": 1.0174
},
"test_xgb_calibrated": {
"accuracy": 0.4685,
"logloss": 1.0193
},
"test_lgb_raw": {
"accuracy": 0.4696,
"logloss": 1.018
},
"test_lgb_calibrated": {
"accuracy": 0.4685,
"logloss": 1.0248
},
"test_ensemble_raw": {
"accuracy": 0.4699,
"logloss": 1.0172
},
"test_ensemble_calibrated": {
"accuracy": 0.4693,
"logloss": 1.0195
}
},
"HT_OU05": {
"market": "HT_OU05",
"samples": 103641,
"train": 62184,
"val": 15546,
"cal": 10364,
"test": 15547,
"features_used": 114,
"xgb_best_params": {
"max_depth": 4,
"eta": 0.02440515089624656,
"subsample": 0.7173767988211683,
"colsample_bytree": 0.5705266148307722,
"min_child_weight": 10,
"gamma": 0.00010295747493868653,
"reg_lambda": 0.00048367003442154754,
"reg_alpha": 0.00018303274057896783
},
"lgb_best_params": {
"max_depth": 4,
"learning_rate": 0.043477055106943,
"feature_fraction": 0.5704621124873813,
"bagging_fraction": 0.9208787923016158,
"bagging_freq": 1,
"min_child_samples": 50,
"lambda_l1": 0.015064619068942013,
"lambda_l2": 6.143857495033091e-07
},
"xgb_best_iteration": 315,
"lgb_best_iteration": 133,
"xgb_optuna_best_logloss": 0.5756,
"lgb_optuna_best_logloss": 0.5757,
"test_xgb_raw": {
"accuracy": 0.7021,
"logloss": 0.5949
},
"test_xgb_calibrated": {
"accuracy": 0.7011,
"logloss": 0.5976
},
"test_lgb_raw": {
"accuracy": 0.7009,
"logloss": 0.5954
},
"test_lgb_calibrated": {
"accuracy": 0.7019,
"logloss": 0.6002
},
"test_ensemble_raw": {
"accuracy": 0.7012,
"logloss": 0.5947
},
"test_ensemble_calibrated": {
"accuracy": 0.7016,
"logloss": 0.5994
}
},
"HT_OU15": {
"market": "HT_OU15",
"samples": 103641,
"train": 62184,
"val": 15546,
"cal": 10364,
"test": 15547,
"features_used": 114,
"xgb_best_params": {
"max_depth": 4,
"eta": 0.032235943414662994,
"subsample": 0.9298749893021518,
"colsample_bytree": 0.8077813949235508,
"min_child_weight": 8,
"gamma": 0.00020929324388600622,
"reg_lambda": 3.2154973975232725e-05,
"reg_alpha": 1.5945155621686738e-08
},
"lgb_best_params": {
"max_depth": 5,
"learning_rate": 0.013909897616748226,
"feature_fraction": 0.5585477334219859,
"bagging_fraction": 0.9398770580467641,
"bagging_freq": 2,
"min_child_samples": 22,
"lambda_l1": 0.001865897980802303,
"lambda_l2": 2.6934572591055333e-06
},
"xgb_best_iteration": 188,
"lgb_best_iteration": 387,
"xgb_optuna_best_logloss": 0.616,
"lgb_optuna_best_logloss": 0.6159,
"test_xgb_raw": {
"accuracy": 0.6749,
"logloss": 0.6109
},
"test_xgb_calibrated": {
"accuracy": 0.6747,
"logloss": 0.6137
},
"test_lgb_raw": {
"accuracy": 0.6745,
"logloss": 0.6112
},
"test_lgb_calibrated": {
"accuracy": 0.6745,
"logloss": 0.6201
},
"test_ensemble_raw": {
"accuracy": 0.674,
"logloss": 0.6109
},
"test_ensemble_calibrated": {
"accuracy": 0.6744,
"logloss": 0.6174
}
},
"HTFT": {
"market": "HTFT",
"samples": 103641,
"train": 62184,
"val": 15546,
"cal": 10364,
"test": 15547,
"features_used": 114,
"xgb_best_params": {
"max_depth": 4,
"eta": 0.015239309183459821,
"subsample": 0.7923828997985648,
"colsample_bytree": 0.686316507387916,
"min_child_weight": 6,
"gamma": 0.005249577944740401,
"reg_lambda": 2.1813455810361064e-08,
"reg_alpha": 3.454483107951557e-06
},
"lgb_best_params": {
"max_depth": 4,
"learning_rate": 0.010347899501864056,
"feature_fraction": 0.9585697341293057,
"bagging_fraction": 0.9413628962257758,
"bagging_freq": 2,
"min_child_samples": 36,
"lambda_l1": 0.0015332771659626943,
"lambda_l2": 7.3640280079715765
},
"xgb_best_iteration": 714,
"lgb_best_iteration": 602,
"xgb_optuna_best_logloss": 1.7863,
"lgb_optuna_best_logloss": 1.7862,
"test_xgb_raw": {
"accuracy": 0.3349,
"logloss": 1.8179
},
"test_xgb_calibrated": {
"accuracy": 0.3332,
"logloss": 1.824
},
"test_lgb_raw": {
"accuracy": 0.3367,
"logloss": 1.8187
},
"test_lgb_calibrated": {
"accuracy": 0.335,
"logloss": 1.8338
},
"test_ensemble_raw": {
"accuracy": 0.3363,
"logloss": 1.8176
},
"test_ensemble_calibrated": {
"accuracy": 0.3338,
"logloss": 1.828
}
},
"ODD_EVEN": {
"market": "ODD_EVEN",
"samples": 106861,
"train": 64116,
"val": 16029,
"cal": 10686,
"test": 16030,
"features_used": 114,
"xgb_best_params": {
"max_depth": 8,
"eta": 0.01010929937405026,
"subsample": 0.9492996501687384,
"colsample_bytree": 0.9061960005014683,
"min_child_weight": 7,
"gamma": 2.664416507237002e-08,
"reg_lambda": 0.0003748192960525308,
"reg_alpha": 0.005287068300306146
},
"lgb_best_params": {
"max_depth": 8,
"learning_rate": 0.0634879805509945,
"feature_fraction": 0.9993568368122896,
"bagging_fraction": 0.9246236397710591,
"bagging_freq": 3,
"min_child_samples": 16,
"lambda_l1": 0.0016414429853061781,
"lambda_l2": 6.112007631403553e-05
},
"xgb_best_iteration": 322,
"lgb_best_iteration": 55,
"xgb_optuna_best_logloss": 0.6777,
"lgb_optuna_best_logloss": 0.6762,
"test_xgb_raw": {
"accuracy": 0.5216,
"logloss": 0.684
},
"test_xgb_calibrated": {
"accuracy": 0.5236,
"logloss": 0.6834
},
"test_lgb_raw": {
"accuracy": 0.5279,
"logloss": 0.6826
},
"test_lgb_calibrated": {
"accuracy": 0.5274,
"logloss": 0.6861
},
"test_ensemble_raw": {
"accuracy": 0.5239,
"logloss": 0.6828
},
"test_ensemble_calibrated": {
"accuracy": 0.5236,
"logloss": 0.6861
}
},
"CARDS_OU45": {
"market": "CARDS_OU45",
"samples": 106861,
"train": 64116,
"val": 16029,
"cal": 10686,
"test": 16030,
"features_used": 114,
"xgb_best_params": {
"max_depth": 8,
"eta": 0.010098671964329344,
"subsample": 0.9969616653360747,
"colsample_bytree": 0.5085930751344795,
"min_child_weight": 10,
"gamma": 0.8600893137103568,
"reg_lambda": 7.556243125116086,
"reg_alpha": 0.5596869360839299
},
"lgb_best_params": {
"max_depth": 8,
"learning_rate": 0.0183440412249233,
"feature_fraction": 0.5416111323291537,
"bagging_fraction": 0.9754210612419695,
"bagging_freq": 2,
"min_child_samples": 5,
"lambda_l1": 0.09157782079463243,
"lambda_l2": 2.559000594641019
},
"xgb_best_iteration": 973,
"lgb_best_iteration": 503,
"xgb_optuna_best_logloss": 0.6408,
"lgb_optuna_best_logloss": 0.6407,
"test_xgb_raw": {
"accuracy": 0.597,
"logloss": 0.6501
},
"test_xgb_calibrated": {
"accuracy": 0.6019,
"logloss": 0.6471
},
"test_lgb_raw": {
"accuracy": 0.5977,
"logloss": 0.6486
},
"test_lgb_calibrated": {
"accuracy": 0.6019,
"logloss": 0.6498
},
"test_ensemble_raw": {
"accuracy": 0.5964,
"logloss": 0.6487
},
"test_ensemble_calibrated": {
"accuracy": 0.6034,
"logloss": 0.6467
}
},
"HANDICAP_MS": {
"market": "HANDICAP_MS",
"samples": 106861,
"train": 64116,
"val": 16029,
"cal": 10686,
"test": 16030,
"features_used": 114,
"xgb_best_params": {
"max_depth": 4,
"eta": 0.01475719431584365,
"subsample": 0.867899230696633,
"colsample_bytree": 0.6518567347674479,
"min_child_weight": 9,
"gamma": 0.34932767754310273,
"reg_lambda": 3.3257801082201637e-07,
"reg_alpha": 4.6977721450875555e-06
},
"lgb_best_params": {
"max_depth": 7,
"learning_rate": 0.019649745228555244,
"feature_fraction": 0.7903699430858344,
"bagging_fraction": 0.7932436899357213,
"bagging_freq": 3,
"min_child_samples": 30,
"lambda_l1": 9.496143774926949e-08,
"lambda_l2": 0.0049885051588706136
},
"xgb_best_iteration": 1016,
"lgb_best_iteration": 364,
"xgb_optuna_best_logloss": 0.8328,
"lgb_optuna_best_logloss": 0.8322,
"test_xgb_raw": {
"accuracy": 0.6062,
"logloss": 0.871
},
"test_xgb_calibrated": {
"accuracy": 0.6039,
"logloss": 0.8729
},
"test_lgb_raw": {
"accuracy": 0.6079,
"logloss": 0.8713
},
"test_lgb_calibrated": {
"accuracy": 0.6067,
"logloss": 0.8736
},
"test_ensemble_raw": {
"accuracy": 0.6072,
"logloss": 0.8707
},
"test_ensemble_calibrated": {
"accuracy": 0.6066,
"logloss": 0.8728
}
}
}
}
+146
View File
@@ -0,0 +1,146 @@
import os
import sys
import psycopg2
from psycopg2.extras import RealDictCursor
# Path ayarları
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from services.single_match_orchestrator import SingleMatchOrchestrator
from services.feature_enrichment import FeatureEnrichmentService
DSN = "postgresql://suggestbet:SuGGesT2026SecuRe@localhost:15432/boilerplate_db"
def run_backtest(target_date="2026-05-03"):
conn = psycopg2.connect(DSN)
cur = conn.cursor(cursor_factory=RealDictCursor)
# 1. Hedef tarihteki bitmiş maçları ve takım isimlerini getir
cur.execute("""
SELECT m.id, m.score_home, m.score_away, m.mst_utc,
t1.name as home_name, t2.name as away_name
FROM matches m
LEFT JOIN teams t1 ON m.home_team_id = t1.id
LEFT JOIN teams t2 ON m.away_team_id = t2.id
WHERE m.status IN ('FT', 'AET', 'PEN')
AND to_timestamp(m.mst_utc / 1000.0)::date = %s::date
AND m.score_home IS NOT NULL
ORDER BY m.mst_utc ASC
""", (target_date,))
matches = cur.fetchall()
if not matches:
print(f"{target_date} tarihinde bitmiş maç bulunamadı.")
return
print(f"🚀 {target_date} için Orkestratör Backtesti Başlatılıyor... ({len(matches)} maç bulundu)")
print("-" * 60)
orchestrator = SingleMatchOrchestrator()
bets_placed = 0
won = 0
lost = 0
total_odds_won = 0.0
for match in matches:
# 3. Üst Akıl (Orkestratör) analizi yapar
try:
package = orchestrator.analyze_match(match['id'])
except Exception as e:
print(f"Hata ({match['id']}): {e}")
continue
if not package:
continue
package_data = package
# 4. Üst akıl bu maça bahis yapmaya karar verdi mi?
bet_advice = package_data.get("bet_advice", {})
if bet_advice.get("playable") == True:
bets_placed += 1
main_pick = package_data.get("main_pick", {})
market = main_pick.get("market")
pick = main_pick.get("pick")
odds = float(main_pick.get("odds", 0.0) or 0.0)
# Skora göre kazanıp kazanmadığını kontrol et
is_won = False
h = match['score_home']
a = match['score_away']
if market == "MS":
if pick == "1" and h > a: is_won = True
elif pick in ("X", "0") and h == a: is_won = True
elif pick == "2" and a > h: is_won = True
elif market == "OU25":
if pick == "Üst" and (h+a) > 2.5: is_won = True
elif pick == "Alt" and (h+a) < 2.5: is_won = True
elif market == "OU15":
if pick == "Üst" and (h+a) > 1.5: is_won = True
elif pick == "Alt" and (h+a) < 1.5: is_won = True
elif market == "BTTS":
if pick == "KG Var" and h > 0 and a > 0: is_won = True
elif pick == "KG Yok" and (h == 0 or a == 0): is_won = True
elif market == "DC":
if pick == "1X" and h >= a: is_won = True
elif pick == "12" and h != a: is_won = True
elif pick == "X2" and h <= a: is_won = True
if is_won:
won += 1
total_odds_won += odds
res = "✅ KAZANDI"
else:
lost += 1
res = "❌ KAYBETTİ"
print(f"[{res}] {match['home_name']} {h}-{a} {match['away_name']} | Tahmin: {market} {pick} (Oran: {odds})")
else:
main_pick = package_data.get("main_pick", {})
reasons = main_pick.get("reasons", ["Bilinmeyen Neden"]) if main_pick else ["No main pick"]
reason = " | ".join(reasons) if isinstance(reasons, list) else str(reasons)
market_board = package_data.get("market_board", {})
main_pick_market = main_pick.get('market', 'N/A') if main_pick else 'N/A'
main_pick_pick = main_pick.get('pick', 'N/A') if main_pick else 'N/A'
print(f"[PAS] {match['home_name']} {match['score_home']}-{match['score_away']} {match['away_name']} | Reddedilen: {main_pick_market} {main_pick_pick} -> Neden: {reason}")
if "market_passed_all_gates" in reason:
print(f" DEBUG: bet_advice = {bet_advice}")
v25_ms = market_board.get("MS", {}).get("probs", {})
v27_ms = {} # V27 is merged into V25 probabilities in market_board, or we don't have separate V27 access here
# Skora göre ms kontrolü
h = match['score_home']
a = match['score_away']
actual_ms = "1" if h > a else ("X" if h == a else "2")
v25_top = max(v25_ms, key=v25_ms.get) if v25_ms else "N/A"
v27_top = "N/A"
rejected_market = main_pick.get("market", "N/A") if main_pick else "N/A"
rejected_pick = main_pick.get("pick", "N/A") if main_pick else "N/A"
print(f"[PAS] {match['home_name']} {h}-{a} {match['away_name']} | Reddedilen: {rejected_market} {rejected_pick} -> Neden: {reason}")
print(f" [V25 MS Raw: {v25_top}] [Gerçek MS: {actual_ms}]")
# Sonuç Raporu
print("\n" + "=" * 60)
print(f"📊 BACKTEST SONUÇLARI ({target_date})")
print("=" * 60)
print(f"Toplam Maç Sayısı : {len(matches)}")
print(f"Oynanan Bahis Sayısı: {bets_placed} (Oynama Oranı: %{bets_placed/len(matches)*100:.1f})")
print(f"Riskli Bulunup Pas Geçilen: {len(matches) - bets_placed}")
if bets_placed > 0:
win_rate = won / bets_placed * 100
roi = ((total_odds_won - bets_placed) / bets_placed) * 100
print(f"Kazanılan : {won}")
print(f"Kaybedilen : {lost}")
print(f"İsabet Oranı : %{win_rate:.1f}")
print(f"Net Kar (ROI) : %{roi:.1f} {'📈' if roi > 0 else '📉'}")
if __name__ == "__main__":
run_backtest("2026-05-03")
+23 -7
View File
@@ -59,7 +59,7 @@ def fetch_matches(conn, sport: str):
def flush_features_batch(conn, rows, dry_run: bool, sport: str = 'football'):
"""Bulk upsert a batch of (match_id, home_elo, away_elo) into sport-partitioned ai_features table."""
"""Bulk upsert ELO features into sport-partitioned ai_features table."""
if not rows or dry_run:
return
@@ -70,19 +70,27 @@ def flush_features_batch(conn, rows, dry_run: bool, sport: str = 'football'):
f"""
INSERT INTO {table_name}
(match_id, home_elo, away_elo,
home_home_elo, away_away_elo,
home_form_elo, away_form_elo,
elo_diff,
home_form_score, away_form_score,
missing_players_impact, calculator_ver, updated_at)
VALUES %s
ON CONFLICT (match_id) DO UPDATE SET
home_elo = EXCLUDED.home_elo,
away_elo = EXCLUDED.away_elo,
home_home_elo = EXCLUDED.home_home_elo,
away_away_elo = EXCLUDED.away_away_elo,
home_form_elo = EXCLUDED.home_form_elo,
away_form_elo = EXCLUDED.away_form_elo,
elo_diff = EXCLUDED.elo_diff,
home_form_score = EXCLUDED.home_form_score,
away_form_score = EXCLUDED.away_form_score,
calculator_ver = EXCLUDED.calculator_ver,
updated_at = EXCLUDED.updated_at
""",
rows,
template="(%s, %s, %s, %s, %s, 0.0, %s, NOW())",
template="(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, 0.0, %s, NOW())",
page_size=500,
)
conn.commit()
@@ -136,16 +144,24 @@ def backfill(sport: str, batch_size: int, dry_run: bool):
if not home_id or not away_id:
continue
# Snapshot PRE-match ELO
# Snapshot PRE-match ELO (all dimensions)
home_rating = elo.get_or_create_rating(home_id, h_name or "")
away_rating = elo.get_or_create_rating(away_id, a_name or "")
h_overall = round(home_rating.overall_elo, 2)
a_overall = round(away_rating.overall_elo, 2)
feature_buf.append((
match_id,
round(home_rating.overall_elo, 2),
round(away_rating.overall_elo, 2),
round(form_to_score(home_rating.recent_form), 2),
round(form_to_score(away_rating.recent_form), 2),
h_overall, # home_elo
a_overall, # away_elo
round(home_rating.home_elo, 2), # home_home_elo
round(away_rating.away_elo, 2), # away_away_elo
round(home_rating.form_elo, 2), # home_form_elo
round(away_rating.form_elo, 2), # away_form_elo
round(h_overall - a_overall, 2), # elo_diff
round(form_to_score(home_rating.recent_form), 2), # home_form_score
round(form_to_score(away_rating.recent_form), 2), # away_form_score
CALCULATOR_VER,
))
+459
View File
@@ -0,0 +1,459 @@
#!/usr/bin/env python3
"""
AI Features Full Enrichment Script
====================================
Fills empty/default columns in football_ai_features that were not populated
by the original elo_backfill_v1 script.
Enriches: H2H, referee, team_stats, league_averages, form_streaks,
rolling_goals, implied_odds, and clean_sheet/scoring rates.
Usage:
python scripts/enrich_ai_features.py # enrich all
python scripts/enrich_ai_features.py --batch-size 500 # smaller batches
python scripts/enrich_ai_features.py --dry-run # preview only
python scripts/enrich_ai_features.py --force # re-enrich all rows
python scripts/enrich_ai_features.py --limit 1000 # process N rows max
Designed to be idempotent: uses ON CONFLICT upserts, skips already-enriched rows.
"""
from __future__ import annotations
import os
import sys
import time
import argparse
from typing import Any, Dict, List, Optional, Tuple
# Add ai-engine root to path
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import psycopg2
from psycopg2.extras import RealDictCursor, execute_values
from data.db import get_clean_dsn
from services.feature_enrichment import FeatureEnrichmentService
# ────────────────────────── constants ──────────────────────────
CALCULATOR_VER = 'enrichment_v2.0'
DEFAULT_BATCH_SIZE = 200
# ────────────────────────── helpers ────────────────────────────
def fetch_unenriched_matches(
conn: psycopg2.extensions.connection,
force: bool = False,
limit: Optional[int] = None,
) -> List[Dict[str, Any]]:
"""
Fetch matches from football_ai_features that still have default values
in the enrichment columns (h2h_total=0 AND referee_avg_cards=0).
If force=True, fetches ALL rows regardless of current state.
"""
with conn.cursor(cursor_factory=RealDictCursor) as cur:
where_clause = "WHERE 1=1" if force else (
"WHERE (faf.h2h_total = 0 AND faf.referee_avg_cards = 0)"
)
limit_clause = f"LIMIT {limit}" if limit else ""
cur.execute(f"""
SELECT
faf.match_id,
m.home_team_id,
m.away_team_id,
m.mst_utc,
m.league_id,
m.score_home,
m.score_away
FROM football_ai_features faf
JOIN matches m ON m.id = faf.match_id
WHERE m.status = 'FT'
AND m.score_home IS NOT NULL
AND m.sport = 'football'
AND ({where_clause.replace('WHERE ', '')})
ORDER BY m.mst_utc ASC
{limit_clause}
""")
return cur.fetchall()
def fetch_referee_for_match(
cur: RealDictCursor,
match_id: str,
) -> Optional[str]:
"""Get the head referee name for a match from match_officials."""
try:
cur.execute("""
SELECT mo.name
FROM match_officials mo
WHERE mo.match_id = %s
AND mo.role_id = 1
LIMIT 1
""", (match_id,))
row = cur.fetchone()
return row['name'] if row else None
except Exception:
return None
def fetch_implied_odds(
cur: RealDictCursor,
match_id: str,
) -> Dict[str, float]:
"""Get implied probabilities from odd_categories + odd_selections."""
defaults = {
'implied_home': 0.33,
'implied_draw': 0.33,
'implied_away': 0.33,
'implied_over25': 0.50,
'implied_btts_yes': 0.50,
'odds_overround': 0.0,
}
try:
cur.execute("""
SELECT oc.name AS cat_name, os.name AS sel_name, os.odd_value
FROM odd_selections os
JOIN odd_categories oc ON os.odd_category_db_id = oc.db_id
WHERE oc.match_id = %s
""", (match_id,))
rows = cur.fetchall()
except Exception:
return defaults
odds: Dict[str, float] = {}
for row in rows:
try:
cat = (row.get('cat_name') or '').lower().strip()
sel = (row.get('sel_name') or '').strip()
val = float(row.get('odd_value', 0))
if val <= 0:
continue
if cat == 'maç sonucu':
if sel == '1':
odds['ms_h'] = val
elif sel in ('0', 'X'):
odds['ms_d'] = val
elif sel == '2':
odds['ms_a'] = val
elif cat == '2,5 alt/üst':
if 'üst' in sel.lower():
odds['ou25_o'] = val
elif 'alt' in sel.lower():
odds['ou25_u'] = val
elif cat == 'karşılıklı gol':
if 'var' in sel.lower():
odds['btts_y'] = val
elif 'yok' in sel.lower():
odds['btts_n'] = val
except (ValueError, TypeError):
continue
# Compute implied probabilities
ms_h = odds.get('ms_h', 0)
ms_d = odds.get('ms_d', 0)
ms_a = odds.get('ms_a', 0)
if ms_h > 1.0 and ms_d > 1.0 and ms_a > 1.0:
raw_sum = 1 / ms_h + 1 / ms_d + 1 / ms_a
overround = raw_sum - 1.0
defaults['implied_home'] = round((1 / ms_h) / raw_sum, 4)
defaults['implied_draw'] = round((1 / ms_d) / raw_sum, 4)
defaults['implied_away'] = round((1 / ms_a) / raw_sum, 4)
defaults['odds_overround'] = round(overround, 4)
ou25_o = odds.get('ou25_o', 0)
ou25_u = odds.get('ou25_u', 0)
if ou25_o > 1.0 and ou25_u > 1.0:
raw_sum = 1 / ou25_o + 1 / ou25_u
defaults['implied_over25'] = round((1 / ou25_o) / raw_sum, 4)
btts_y = odds.get('btts_y', 0)
btts_n = odds.get('btts_n', 0)
if btts_y > 1.0 and btts_n > 1.0:
raw_sum = 1 / btts_y + 1 / btts_n
defaults['implied_btts_yes'] = round((1 / btts_y) / raw_sum, 4)
return defaults
def enrich_single_match(
enrichment: FeatureEnrichmentService,
cur: RealDictCursor,
match: Dict[str, Any],
) -> Dict[str, Any]:
"""
Compute all enrichment features for a single match and return
a dict ready for DB upsert.
"""
match_id = match['match_id']
home_id = str(match['home_team_id'])
away_id = str(match['away_team_id'])
mst_utc = int(match['mst_utc']) if match['mst_utc'] else 0
league_id = str(match['league_id']) if match['league_id'] else None
# 1. Team stats
home_stats = enrichment.compute_team_stats(cur, home_id, mst_utc)
away_stats = enrichment.compute_team_stats(cur, away_id, mst_utc)
# 2. H2H
h2h = enrichment.compute_h2h(cur, home_id, away_id, mst_utc)
# 3. Form & streaks
home_form = enrichment.compute_form_streaks(cur, home_id, mst_utc)
away_form = enrichment.compute_form_streaks(cur, away_id, mst_utc)
# 4. Referee
referee_name = fetch_referee_for_match(cur, match_id)
referee = enrichment.compute_referee_stats(cur, referee_name, mst_utc)
# 5. League averages
league = enrichment.compute_league_averages(cur, league_id, mst_utc)
# 6. Rolling stats (for goals avg)
home_rolling = enrichment.compute_rolling_stats(cur, home_id, mst_utc)
away_rolling = enrichment.compute_rolling_stats(cur, away_id, mst_utc)
# 7. Implied odds
implied = fetch_implied_odds(cur, match_id)
return {
'match_id': match_id,
# Team stats
'home_avg_possession': round(home_stats['avg_possession'], 2),
'away_avg_possession': round(away_stats['avg_possession'], 2),
'home_avg_shots_on_target': round(home_stats['avg_shots_on_target'], 2),
'away_avg_shots_on_target': round(away_stats['avg_shots_on_target'], 2),
'home_shot_conversion': round(home_stats['shot_conversion'], 4),
'away_shot_conversion': round(away_stats['shot_conversion'], 4),
'home_avg_corners': round(home_stats['avg_corners'], 2),
'away_avg_corners': round(away_stats['avg_corners'], 2),
# H2H
'h2h_total': h2h['total_matches'],
'h2h_home_win_rate': round(h2h['home_win_rate'], 4),
'h2h_avg_goals': round(h2h['avg_goals'], 2),
'h2h_over25_rate': round(h2h['over25_rate'], 4),
'h2h_btts_rate': round(h2h['btts_rate'], 4),
# Form
'home_clean_sheet_rate': round(home_form['clean_sheet_rate'], 4),
'away_clean_sheet_rate': round(away_form['clean_sheet_rate'], 4),
'home_scoring_rate': round(home_form['scoring_rate'], 4),
'away_scoring_rate': round(away_form['scoring_rate'], 4),
'home_win_streak': home_form['winning_streak'],
'away_win_streak': away_form['winning_streak'],
# Rolling goals
'home_goals_avg_5': round(home_rolling['rolling5_goals'], 2),
'away_goals_avg_5': round(away_rolling['rolling5_goals'], 2),
'home_conceded_avg_5': round(home_rolling['rolling5_conceded'], 2),
'away_conceded_avg_5': round(away_rolling['rolling5_conceded'], 2),
# Referee
'referee_avg_cards': round(referee['cards_total'], 2),
'referee_home_bias': round(referee['home_bias'], 4),
'referee_avg_goals': round(referee['avg_goals'], 2),
# League
'league_avg_goals': round(league['avg_goals'], 2),
'league_home_win_pct': round(league['home_win_rate'], 4),
'league_over25_pct': round(league['ou25_rate'], 4),
# Implied odds
'implied_home': implied['implied_home'],
'implied_draw': implied['implied_draw'],
'implied_away': implied['implied_away'],
'implied_over25': implied['implied_over25'],
'implied_btts_yes': implied['implied_btts_yes'],
'odds_overround': implied['odds_overround'],
# Missing players impact — default (no lineup data for historical)
'missing_players_impact': 0.0,
# Version
'calculator_ver': CALCULATOR_VER,
}
def flush_enrichment_batch(
conn: psycopg2.extensions.connection,
rows: List[Dict[str, Any]],
dry_run: bool,
) -> int:
"""Bulk upsert enriched features into football_ai_features."""
if not rows or dry_run:
return 0
columns = [
'match_id',
'home_avg_possession', 'away_avg_possession',
'home_avg_shots_on_target', 'away_avg_shots_on_target',
'home_shot_conversion', 'away_shot_conversion',
'home_avg_corners', 'away_avg_corners',
'h2h_total', 'h2h_home_win_rate', 'h2h_avg_goals',
'h2h_over25_rate', 'h2h_btts_rate',
'home_clean_sheet_rate', 'away_clean_sheet_rate',
'home_scoring_rate', 'away_scoring_rate',
'home_win_streak', 'away_win_streak',
'home_goals_avg_5', 'away_goals_avg_5',
'home_conceded_avg_5', 'away_conceded_avg_5',
'referee_avg_cards', 'referee_home_bias', 'referee_avg_goals',
'league_avg_goals', 'league_home_win_pct', 'league_over25_pct',
'implied_home', 'implied_draw', 'implied_away',
'implied_over25', 'implied_btts_yes', 'odds_overround',
'missing_players_impact', 'calculator_ver',
]
# Build update SET clause (skip match_id)
update_cols = [c for c in columns if c != 'match_id']
set_clause = ', '.join(f'{c} = EXCLUDED.{c}' for c in update_cols)
placeholders = ', '.join(['%s'] * len(columns))
values = [
tuple(row[c] for c in columns)
for row in rows
]
with conn.cursor() as cur:
execute_values(
cur,
f"""
INSERT INTO football_ai_features ({', '.join(columns)})
VALUES %s
ON CONFLICT (match_id) DO UPDATE SET
{set_clause},
updated_at = NOW()
""",
values,
template=f"({placeholders})",
page_size=200,
)
conn.commit()
return len(rows)
# ────────────────────────── main ───────────────────────────────
def run_enrichment(
batch_size: int,
dry_run: bool,
force: bool,
limit: Optional[int],
) -> None:
"""Core enrichment loop."""
dsn = get_clean_dsn()
conn = psycopg2.connect(dsn)
print(f"\n{'=' * 60}")
print(f"🧠 AI Features Full Enrichment — {CALCULATOR_VER}")
print(f" batch_size={batch_size} dry_run={dry_run} force={force}")
print(f"{'=' * 60}")
# 1. Fetch unenriched matches
t0 = time.time()
matches = fetch_unenriched_matches(conn, force=force, limit=limit)
print(f"\n📊 {len(matches):,} matches to enrich ({time.time() - t0:.1f}s)")
if not matches:
print("✅ Nothing to enrich — all rows already populated.")
conn.close()
return
# 2. Initialize enrichment service
enrichment = FeatureEnrichmentService()
# 3. Process in batches
total = len(matches)
processed = 0
written = 0
errors = 0
batch_buf: List[Dict[str, Any]] = []
t_start = time.time()
# Use a dedicated cursor with RealDictCursor for all enrichment queries
enrich_cur = conn.cursor(cursor_factory=RealDictCursor)
for idx, match in enumerate(matches):
try:
enriched = enrich_single_match(enrichment, enrich_cur, match)
batch_buf.append(enriched)
except Exception as e:
errors += 1
if errors <= 10:
print(f" ⚠️ Error enriching {match.get('match_id', '?')}: {e}")
processed += 1
# Flush batch
if len(batch_buf) >= batch_size:
flushed = flush_enrichment_batch(conn, batch_buf, dry_run)
written += flushed
batch_buf.clear()
# Progress reporting
if processed % 500 == 0:
elapsed = time.time() - t_start
rate = processed / elapsed if elapsed > 0 else 0
remaining = (total - processed) / rate if rate > 0 else 0
pct = processed / total * 100
print(
f" [{processed:>8,} / {total:,}] "
f"({pct:.1f}%) | {rate:.0f} matches/s | "
f"ETA: {remaining / 60:.1f} min | "
f"errors: {errors}"
)
# Flush remaining
if batch_buf:
flushed = flush_enrichment_batch(conn, batch_buf, dry_run)
written += flushed
enrich_cur.close()
elapsed = time.time() - t_start
print(f"\n{'=' * 60}")
print(f"✅ Enrichment complete:")
print(f" Processed: {processed:,} matches in {elapsed:.1f}s")
print(f" Written: {written:,} rows")
print(f" Errors: {errors:,}")
print(f" Rate: {processed / elapsed:.0f} matches/s")
print(f"{'=' * 60}")
conn.close()
def main() -> None:
parser = argparse.ArgumentParser(
description="Enrich football_ai_features with H2H, referee, stats, and odds data"
)
parser.add_argument(
'--batch-size',
type=int,
default=DEFAULT_BATCH_SIZE,
help=f'DB insert batch size (default: {DEFAULT_BATCH_SIZE})',
)
parser.add_argument(
'--dry-run',
action='store_true',
help='Compute features but do not write to DB',
)
parser.add_argument(
'--force',
action='store_true',
help='Re-enrich ALL rows, not just empty ones',
)
parser.add_argument(
'--limit',
type=int,
default=None,
help='Max number of matches to process',
)
args = parser.parse_args()
run_enrichment(
batch_size=args.batch_size,
dry_run=args.dry_run,
force=args.force,
limit=args.limit,
)
if __name__ == '__main__':
main()
+254 -24
View File
@@ -14,6 +14,7 @@ import json
import csv
import math
import time
import bisect
from datetime import datetime
from collections import defaultdict
@@ -33,7 +34,7 @@ from features.upset_engine import get_upset_engine
from features.referee_engine import get_referee_engine
from features.momentum_engine import get_momentum_engine
TOP_LEAGUES_PATH = os.path.join(AI_ENGINE_DIR, "..", "top_leagues.json")
TOP_LEAGUES_PATH = os.path.join(AI_ENGINE_DIR, "..", "qualified_leagues.json")
OUTPUT_CSV = os.path.join(AI_ENGINE_DIR, "data", "training_data.csv")
# Ensure output dir exists
@@ -120,6 +121,14 @@ FEATURE_COLS = [
"home_missing_impact", "away_missing_impact",
"home_goals_form", "away_goals_form",
# Player-Level Features (12)
"home_lineup_goals_per90", "away_lineup_goals_per90",
"home_lineup_assists_per90", "away_lineup_assists_per90",
"home_squad_continuity", "away_squad_continuity",
"home_top_scorer_form", "away_top_scorer_form",
"home_avg_player_exp", "away_avg_player_exp",
"home_goals_diversity", "away_goals_diversity",
# Labels
"score_home", "score_away", "total_goals",
"ht_score_home", "ht_score_away", "ht_total_goals",
@@ -336,7 +345,7 @@ class BatchDataLoader:
self.team_stats[tid].append((mst, poss, sot, tshots, corn, team_goals))
def _load_squad_data(self):
"""Bulk load squad participation + player events for squad features."""
"""Bulk load squad participation + player events + player career for squad features."""
ph = ",".join(["%s"] * len(self.top_league_ids))
# 1) Participation: starting XI count + position distribution per (match, team)
@@ -424,12 +433,99 @@ class BatchDataLoader:
for mid, tid, pid in self.cur.fetchall():
starting_players[(mid, tid)].append(pid)
# 5) Build combined cache
# 5) Build match_id → mst_utc mapping for temporal filtering
match_mst = {}
for m in self.matches:
match_mst[m[0]] = m[7] # m[0]=id, m[7]=mst_utc
# ─── NEW: Player Career Stats (prefix-sum for O(1) temporal lookup) ───
# 6a) Goals per player per match date
self.cur.execute(f"""
SELECT mpe.player_id, m.mst_utc,
SUM(CASE WHEN mpe.event_type = 'goal'
AND COALESCE(mpe.event_subtype, '') NOT ILIKE '%%penaltı kaçırma%%'
THEN 1 ELSE 0 END) AS goals
FROM match_player_events mpe
JOIN matches m ON mpe.match_id = m.id
WHERE m.status = 'FT' AND m.sport = 'football' AND m.league_id IN ({ph})
GROUP BY mpe.player_id, m.mst_utc
""", self.top_league_ids)
player_goals_raw = defaultdict(dict)
for pid, mst, goals in self.cur.fetchall():
player_goals_raw[pid][mst] = (player_goals_raw[pid].get(mst, 0)) + (goals or 0)
# 6b) Assists per player per match date
self.cur.execute(f"""
SELECT mpe.assist_player_id, m.mst_utc, COUNT(*) AS assists
FROM match_player_events mpe
JOIN matches m ON mpe.match_id = m.id
WHERE m.status = 'FT' AND m.sport = 'football' AND m.league_id IN ({ph})
AND mpe.event_type = 'goal' AND mpe.assist_player_id IS NOT NULL
GROUP BY mpe.assist_player_id, m.mst_utc
""", self.top_league_ids)
player_assists_raw = defaultdict(dict)
for pid, mst, assists in self.cur.fetchall():
player_assists_raw[pid][mst] = (player_assists_raw[pid].get(mst, 0)) + (assists or 0)
# 6c) Player participation dates (starts only)
self.cur.execute(f"""
SELECT mpp.player_id, m.mst_utc
FROM match_player_participation mpp
JOIN matches m ON mpp.match_id = m.id
WHERE mpp.is_starting = true
AND m.status = 'FT' AND m.sport = 'football' AND m.league_id IN ({ph})
ORDER BY mpp.player_id, m.mst_utc
""", self.top_league_ids)
player_starts_raw = defaultdict(list)
for pid, mst in self.cur.fetchall():
player_starts_raw[pid].append(mst)
# 6d) Build prefix sums per player (goals_prefix[i] = total goals up to start i)
player_career = {}
all_pids = set(player_starts_raw.keys()) | set(player_goals_raw.keys()) | set(player_assists_raw.keys())
for pid in all_pids:
starts = sorted(set(player_starts_raw.get(pid, [])))
if not starts:
continue
g_map = player_goals_raw.get(pid, {})
a_map = player_assists_raw.get(pid, {})
cum_g, cum_a = 0, 0
goals_pf, assists_pf = [], []
for mst in starts:
cum_g += g_map.get(mst, 0)
cum_a += a_map.get(mst, 0)
goals_pf.append(cum_g)
assists_pf.append(cum_a)
player_career[pid] = {'msts': starts, 'gp': goals_pf, 'ap': assists_pf}
# Free raw dicts
del player_goals_raw, player_assists_raw, player_starts_raw
print(f" 📊 Player careers built: {len(player_career)} players", flush=True)
# ─── NEW: Team Lineup History (for squad continuity) ───
# 7) Per-team sorted lineups: [(mst, frozenset(player_ids))]
team_lineup_map = defaultdict(list)
for (mid, tid), pids in starting_players.items():
mst = match_mst.get(mid, 0)
if mst > 0 and pids:
team_lineup_map[tid].append((mst, frozenset(pids)))
team_lineup_history = {}
team_lineup_msts = {}
for tid, ll in team_lineup_map.items():
ll.sort(key=lambda x: x[0])
team_lineup_history[tid] = ll
team_lineup_msts[tid] = [x[0] for x in ll]
del team_lineup_map
# ─── 8) Build combined cache — NO DATA LEAKAGE ───
all_keys = set(participation.keys()) | set(events.keys())
for key in all_keys:
mid, tid = key
part = participation.get(key, {'starting_count': 0, 'total_squad': 0, 'fwd_count': 0})
evt = events.get(key, {'goals': 0, 'assists': 0, 'unique_scorers': 0})
# Count key players in starting XI
starters = starting_players.get(key, [])
@@ -437,22 +533,78 @@ class BatchDataLoader:
kp_total = len(key_players_by_team.get(tid, set()))
kp_missing = max(0, kp_total - kp_in_starting)
# Squad quality: composite score
# Squad quality: composite score — ONLY pre-match info
squad_quality = (
part['starting_count'] * 0.3 +
evt['goals'] * 2.0 +
evt['assists'] * 1.0 +
kp_in_starting * 3.0 +
part['fwd_count'] * 1.5
)
# Missing impact: how many key players are missing
missing_impact = min(kp_missing / max(kp_total, 1), 1.0)
# goals_form: avg goals from last 5 matches BEFORE this match
current_mst = match_mst.get(mid, 0)
team_history = self.team_matches.get(tid, [])
recent_goals = [
tm[2] for tm in team_history if tm[0] < current_mst
][-5:]
goals_form = sum(recent_goals) / len(recent_goals) if recent_goals else 1.3
# ─── NEW: Player-level aggregation for starting XI ───
lineup_g90, lineup_a90, total_exp = 0.0, 0.0, 0
best_scorer_total, best_scorer_id = 0, None
scorers_in_lineup = 0
for pid in starters:
pc = player_career.get(pid)
if not pc:
continue
idx = bisect.bisect_left(pc['msts'], current_mst)
if idx == 0:
continue # no prior matches for this player
prior_starts = idx
prior_goals = pc['gp'][idx - 1]
prior_assists = pc['ap'][idx - 1]
lineup_g90 += prior_goals / prior_starts
lineup_a90 += prior_assists / prior_starts
total_exp += prior_starts
if prior_goals > 0:
scorers_in_lineup += 1
if prior_goals > best_scorer_total:
best_scorer_total = prior_goals
best_scorer_id = pid
n_st = len(starters) or 1
# Top scorer recent form (goals in last 5 starts)
top_scorer_form = 0
if best_scorer_id:
pc = player_career.get(best_scorer_id)
if pc:
idx = bisect.bisect_left(pc['msts'], current_mst)
if idx > 0:
s5 = max(0, idx - 5)
top_scorer_form = pc['gp'][idx - 1] - (pc['gp'][s5 - 1] if s5 > 0 else 0)
# Squad continuity (overlap with previous match lineup)
squad_continuity = 0.5
msts_list = team_lineup_msts.get(tid)
if msts_list:
li = bisect.bisect_left(msts_list, current_mst)
if li > 0:
prev_lineup = team_lineup_history[tid][li - 1][1]
squad_continuity = len(frozenset(starters) & prev_lineup) / n_st
self.squad_cache[key] = {
'squad_quality': squad_quality,
'key_players': kp_in_starting,
'missing_impact': missing_impact,
'goals_form': evt['goals'],
'goals_form': round(goals_form, 2),
'lineup_goals_per90': round(lineup_g90, 3),
'lineup_assists_per90': round(lineup_a90, 3),
'squad_continuity': round(squad_continuity, 3),
'top_scorer_form': top_scorer_form,
'avg_player_exp': round(total_exp / n_st, 1),
'goals_diversity': round(scorers_in_lineup / n_st, 3),
}
def _load_cards_data(self):
@@ -496,12 +648,20 @@ class FeatureExtractor:
self.referee_engine = get_referee_engine()
self.momentum_engine = get_momentum_engine()
# ── Data Quality Thresholds ──
# Matches below these thresholds produce default-only features that
# teach the model noise rather than signal.
DQ_MIN_FORM_MATCHES = 3 # team must have ≥3 prior matches
DQ_MIN_FEATURE_COVERAGE = 0.30 # ≥30% of key features must be non-default
def extract_all(self) -> list:
"""Extract features for all matches, yield row dicts."""
"""Extract features for all matches with data quality validation."""
matches = self.loader.matches
total = len(matches)
rows = []
skipped = 0
dq_rejected = 0
dq_reasons: dict = defaultdict(int)
t_start = time.time()
print(f"\n🔄 Extracting features for {total} matches...", flush=True)
@@ -528,32 +688,37 @@ class FeatureExtractor:
rate = i / elapsed # matches per second
remaining = (total - i) / rate if rate > 0 else 0
pct = i / total * 100
print(f" [{i}/{total}] ({pct:.0f}%) | {rate:.1f} maç/s | ETA: {remaining/60:.1f} dk | skipped: {skipped}", flush=True)
print(
f" [{i}/{total}] ({pct:.0f}%) | {rate:.1f} maç/s | "
f"ETA: {remaining/60:.1f} dk | skipped: {skipped} | "
f"dq_rejected: {dq_rejected}",
flush=True,
)
row = self._extract_one(
mid,
hid,
aid,
sh,
sa,
hth,
hta,
mst,
lid,
home_name,
away_name,
league_name,
mid, hid, aid, sh, sa, hth, hta, mst, lid,
home_name, away_name, league_name,
)
if row:
# ── Data Quality Gate ──
dq_pass, reason = self._validate_row_quality(row, hid, aid, mst)
if dq_pass:
rows.append(row)
else:
dq_rejected += 1
dq_reasons[reason] += 1
else:
skipped += 1
# Update ELO after processing (so ELO is calculated BEFORE the match)
self._update_elo(hid, aid, sh, sa)
print(f" ✅ Extracted {len(rows)} rows, skipped {skipped}", flush=True)
print(f" ✅ Extracted {len(rows)} rows, skipped {skipped}, DQ rejected {dq_rejected}", flush=True)
if dq_reasons:
print(f" 📊 DQ Rejection reasons:")
for reason, count in sorted(dq_reasons.items(), key=lambda x: -x[1]):
print(f" {reason}: {count}")
return rows
def _extract_one(
@@ -829,6 +994,20 @@ class FeatureExtractor:
"home_goals_form": home_goals_form,
"away_goals_form": away_goals_form,
# Player-Level Features
"home_lineup_goals_per90": home_sq.get('lineup_goals_per90', 0.0),
"away_lineup_goals_per90": away_sq.get('lineup_goals_per90', 0.0),
"home_lineup_assists_per90": home_sq.get('lineup_assists_per90', 0.0),
"away_lineup_assists_per90": away_sq.get('lineup_assists_per90', 0.0),
"home_squad_continuity": home_sq.get('squad_continuity', 0.5),
"away_squad_continuity": away_sq.get('squad_continuity', 0.5),
"home_top_scorer_form": home_sq.get('top_scorer_form', 0),
"away_top_scorer_form": away_sq.get('top_scorer_form', 0),
"home_avg_player_exp": home_sq.get('avg_player_exp', 0.0),
"away_avg_player_exp": away_sq.get('avg_player_exp', 0.0),
"home_goals_diversity": home_sq.get('goals_diversity', 0.0),
"away_goals_diversity": away_sq.get('goals_diversity', 0.0),
# Labels
"score_home": sh,
"score_away": sa,
@@ -854,6 +1033,57 @@ class FeatureExtractor:
return row
def _validate_row_quality(
self,
row: dict,
home_id: str,
away_id: str,
before_date: int,
) -> tuple:
"""
Data quality gate for training rows.
Ensures the feature vector has enough real signal to be useful for
training. Rejects rows where critical features are all at their
default/fallback values these teach the model noise, not patterns.
Returns (pass: bool, reason: str | None).
"""
# 1. Minimum form history: both teams must have enough prior matches
home_history = self.loader.team_matches.get(home_id, [])
away_history = self.loader.team_matches.get(away_id, [])
home_prior = sum(1 for m in home_history if m[0] < before_date)
away_prior = sum(1 for m in away_history if m[0] < before_date)
if home_prior < self.DQ_MIN_FORM_MATCHES:
return False, 'home_insufficient_history'
if away_prior < self.DQ_MIN_FORM_MATCHES:
return False, 'away_insufficient_history'
# 2. Feature coverage check: count how many key features are non-default
key_features = [
('home_goals_avg', 1.3),
('away_goals_avg', 1.3),
('home_clean_sheet_rate', 0.25),
('away_clean_sheet_rate', 0.25),
('home_avg_possession', 0.50),
('away_avg_possession', 0.50),
('home_avg_shots_on_target', 3.5),
('away_avg_shots_on_target', 3.5),
('h2h_total_matches', 0),
('odds_ms_h', 0.0),
]
non_default = sum(
1 for feat_name, default_val in key_features
if abs(float(row.get(feat_name, default_val)) - default_val) > 0.01
)
coverage = non_default / len(key_features)
if coverage < self.DQ_MIN_FEATURE_COVERAGE:
return False, f'low_feature_coverage_{coverage:.0%}'
return True, None
# -------------------------------------------------------------------------
# ELO (simplified inline version — doesn't need DB, grows incrementally)
# -------------------------------------------------------------------------
+225 -137
View File
@@ -1,183 +1,271 @@
"""
V25-Compatible Score Prediction Model Trainer
===============================================
Trains 4 independent XGBoost regression models for:
- FT Home Goals
- FT Away Goals
- HT Home Goals
- HT Away Goals
Uses the same 102-feature set as v25_ensemble for full compatibility.
Temporal train/test split (80/20) to avoid future leakage.
Usage:
python3 scripts/train_score_model.py
"""
import os
import sys
import pickle
import numpy as np
import pandas as pd
import xgboost as xgb
import pickle
import os
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error, r2_score
from datetime import datetime
from sklearn.metrics import mean_absolute_error, r2_score, mean_squared_error
# Paths
DATA_PATH = os.path.join(os.path.dirname(__file__), "../data/training_data.csv")
MODEL_PATH = os.path.join(os.path.dirname(__file__), "../models/xgb_score.pkl")
# Add parent directory to path
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
# Import unified 56-feature array from markets trainer
from train_xgboost_markets import FEATURES
# Config
AI_ENGINE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA_PATH = os.path.join(AI_ENGINE_DIR, "data", "training_data.csv")
MODEL_PATH = os.path.join(AI_ENGINE_DIR, "models", "xgb_score.pkl")
# Import the EXACT same feature set as v25 market models
from train_v25_clean import FEATURES
TARGETS = ["score_home", "score_away", "ht_score_home", "ht_score_away"]
def train():
print("🚀 Training Score Prediction Model (XGBoost) - Full Time & Half Time")
print("=" * 60)
# Model hyperparameters (tuned for goal count regression)
XGB_PARAMS = {
"objective": "reg:squarederror",
"n_estimators": 1200,
"learning_rate": 0.02,
"max_depth": 6,
"subsample": 0.8,
"colsample_bytree": 0.7,
"min_child_weight": 5,
"reg_alpha": 0.1,
"reg_lambda": 1.0,
"n_jobs": -1,
"random_state": 42,
}
def load_data() -> pd.DataFrame:
"""Load and validate training data."""
if not os.path.exists(DATA_PATH):
print(f"❌ Data file not found: {DATA_PATH}")
return
print(" Run extract_training_data.py first")
sys.exit(1)
print(f"📦 Loading data from {DATA_PATH}...")
df = pd.read_csv(DATA_PATH)
# Preprocessing
# Drop rows where target is missing (should verify)
# Fill feature NaNs with 0 (same as v25 training)
for col in FEATURES:
if col in df.columns:
df[col] = df[col].fillna(0)
# Backward-compatible: add odds presence flags if missing
odds_base_columns = [
"odds_ms_h", "odds_ms_d", "odds_ms_a",
"odds_ht_ms_h", "odds_ht_ms_d", "odds_ht_ms_a",
"odds_ou05_o", "odds_ou05_u",
"odds_ou15_o", "odds_ou15_u",
"odds_ou25_o", "odds_ou25_u",
"odds_ou35_o", "odds_ou35_u",
"odds_ht_ou05_o", "odds_ht_ou05_u",
"odds_ht_ou15_o", "odds_ht_ou15_u",
"odds_btts_y", "odds_btts_n",
]
for base_col in odds_base_columns:
pres_col = f"{base_col}_present"
if pres_col not in df.columns and base_col in df.columns:
df[pres_col] = (df[base_col] > 1.0).astype(int)
# Drop rows where any target is missing
df = df.dropna(subset=TARGETS)
# Fill feature NaNs with median/mean or 0
print(f" Original rows: {len(df)}")
# Filter valid odds (at least ms_h > 1.0)
# Filter: at least MS odds must be present
df = df[df["odds_ms_h"] > 1.0].copy()
print(f" Rows with valid odds: {len(df)}")
X = df[FEATURES]
y_home = df["score_home"]
y_away = df["score_away"]
y_ht_home = df["ht_score_home"]
y_ht_away = df["ht_score_away"]
# Ensure all features exist
missing = [f for f in FEATURES if f not in df.columns]
if missing:
print(f"⚠️ Missing {len(missing)} features, filling with 0: {missing[:5]}...")
for f in missing:
df[f] = 0
# Train/Test Split
X_train, X_test, y_h_train, y_h_test, y_a_train, y_a_test, y_ht_h_train, y_ht_h_test, y_ht_a_train, y_ht_a_test = train_test_split(
X, y_home, y_away, y_ht_home, y_ht_away, test_size=0.2, random_state=42
return df
def temporal_split(df: pd.DataFrame, train_ratio: float = 0.80):
"""
Temporal train/test split by match date.
Ensures no future information leaks into training.
"""
if "match_date" in df.columns:
df = df.sort_values("match_date").reset_index(drop=True)
elif "round" in df.columns:
df = df.sort_values("round").reset_index(drop=True)
split_idx = int(len(df) * train_ratio)
return df.iloc[:split_idx].copy(), df.iloc[split_idx:].copy()
def train_single_model(X_train, y_train, X_test, y_test, name: str):
"""Train a single XGBoost regression model with early stopping."""
print(f"\n🏗️ Training {name} model...")
model = xgb.XGBRegressor(**XGB_PARAMS)
model.fit(
X_train, y_train,
eval_set=[(X_test, y_test)],
verbose=False,
)
print(f" Training set: {len(X_train)} matches")
print(f" Test set: {len(X_test)} matches")
preds = model.predict(X_test)
# --- HOME GOALS MODEL ---
print("\n🏠 Training Home Goals Model...")
xgb_home = xgb.XGBRegressor(
objective='reg:squarederror',
n_estimators=1000,
learning_rate=0.01,
max_depth=5,
subsample=0.7,
colsample_bytree=0.7,
n_jobs=-1,
random_state=42,
early_stopping_rounds=50 # Configure here for newer XGBoost or remove if not supported in constructor (depends on version)
)
# Actually, to be safe across versions, let's remove early stopping for now or use validation set properly
# Using 'eval_set' without early_stopping_rounds just prints metrics
xgb_home = xgb.XGBRegressor(
objective='reg:squarederror',
n_estimators=1000,
learning_rate=0.01,
max_depth=5,
subsample=0.7,
colsample_bytree=0.7,
n_jobs=-1,
random_state=42
)
xgb_home.fit(X_train, y_h_train, eval_set=[(X_test, y_h_test)], verbose=False)
mae = mean_absolute_error(y_test, preds)
rmse = np.sqrt(mean_squared_error(y_test, preds))
r2 = r2_score(y_test, preds)
home_preds = xgb_home.predict(X_test)
mae_home = mean_absolute_error(y_h_test, home_preds)
r2_home = r2_score(y_h_test, home_preds)
print(f" ✅ FT Home MAE: {mae_home:.4f} goals")
print(f" ✅ FT Home R2: {r2_home:.4f}")
print(f" MAE: {mae:.4f} goals")
print(f" RMSE: {rmse:.4f}")
print(f" R²: {r2:.4f}")
# --- AWAY GOALS MODEL ---
print("\n✈️ Training FT Away Goals Model...")
xgb_away = xgb.XGBRegressor(
objective='reg:squarederror',
n_estimators=1000,
learning_rate=0.01,
max_depth=5,
subsample=0.7,
colsample_bytree=0.7,
n_jobs=-1,
random_state=42
)
xgb_away.fit(X_train, y_a_train, eval_set=[(X_test, y_a_test)], verbose=False)
return model, {"mae": mae, "rmse": rmse, "r2": r2}
away_preds = xgb_away.predict(X_test)
mae_away = mean_absolute_error(y_a_test, away_preds)
r2_away = r2_score(y_a_test, away_preds)
print(f" ✅ FT Away MAE: {mae_away:.4f} goals")
print(f" ✅ FT Away R2: {r2_away:.4f}")
# --- HT HOME GOALS MODEL ---
print("\n🏠 Training HT Home Goals Model...")
xgb_ht_home = xgb.XGBRegressor(
objective='reg:squarederror',
n_estimators=1000,
learning_rate=0.01,
max_depth=5,
subsample=0.7,
colsample_bytree=0.7,
n_jobs=-1,
random_state=42
)
xgb_ht_home.fit(X_train, y_ht_h_train, eval_set=[(X_test, y_ht_h_test)], verbose=False)
def evaluate_combined(models: dict, X_test, y_test_dict: dict):
"""Evaluate combined score accuracy (FT and HT)."""
print("\n🎯 Combined Score Evaluation (Test Set):")
ht_home_preds = xgb_ht_home.predict(X_test)
mae_ht_home = mean_absolute_error(y_ht_h_test, ht_home_preds)
print(f" ✅ HT Home MAE: {mae_ht_home:.4f} goals")
# FT Score
ft_h_preds = models["ft_home"].predict(X_test)
ft_a_preds = models["ft_away"].predict(X_test)
# --- HT AWAY GOALS MODEL ---
print("\n✈️ Training HT Away Goals Model...")
xgb_ht_away = xgb.XGBRegressor(
objective='reg:squarederror',
n_estimators=1000,
learning_rate=0.01,
max_depth=5,
subsample=0.7,
colsample_bytree=0.7,
n_jobs=-1,
random_state=42
)
xgb_ht_away.fit(X_train, y_ht_a_train, eval_set=[(X_test, y_ht_a_test)], verbose=False)
y_ft_h = y_test_dict["score_home"].values
y_ft_a = y_test_dict["score_away"].values
ht_away_preds = xgb_ht_away.predict(X_test)
mae_ht_away = mean_absolute_error(y_ht_a_test, ht_away_preds)
print(f" ✅ HT Away MAE: {mae_ht_away:.4f} goals")
exact = 0
close = 0
result_correct = 0
total = len(X_test)
# --- EVALUATE EXACT SCORE ACCURACY (ROUNDED) ---
print("\n🎯 Exact FT Score Accuracy (Test Set):")
correct = 0
close = 0 # Within 1 goal diff for both
for h_true, a_true, h_pred, a_pred in zip(y_ft_h, y_ft_a, ft_h_preds, ft_a_preds):
hp = max(0, round(h_pred))
ap = max(0, round(a_pred))
for h_true, a_true, h_pred, a_pred in zip(y_h_test, y_a_test, home_preds, away_preds):
h_p = round(h_pred)
a_p = round(a_pred)
if h_p == h_true and a_p == a_true:
correct += 1
if abs(h_p - h_true) <= 1 and abs(a_p - a_true) <= 1:
# Exact score
if hp == h_true and ap == a_true:
exact += 1
# Close (±1 each)
if abs(hp - h_true) <= 1 and abs(ap - a_true) <= 1:
close += 1
acc = correct / len(X_test) * 100
close_acc = close / len(X_test) * 100
print(f" Exact Match: {acc:.2f}%")
print(f" Close Match (+/- 1 goal): {close_acc:.2f}%")
# Result direction (1X2)
true_result = 1 if h_true > a_true else (0 if h_true == a_true else -1)
pred_result = 1 if hp > ap else (0 if hp == ap else -1)
if true_result == pred_result:
result_correct += 1
print(f" FT Exact Score: {exact / total * 100:.2f}% ({exact}/{total})")
print(f" FT Close (±1): {close / total * 100:.2f}% ({close}/{total})")
print(f" FT Result (1X2): {result_correct / total * 100:.2f}% ({result_correct}/{total})")
# HT Score
ht_h_preds = models["ht_home"].predict(X_test)
ht_a_preds = models["ht_away"].predict(X_test)
y_ht_h = y_test_dict["ht_score_home"].values
y_ht_a = y_test_dict["ht_score_away"].values
ht_exact = 0
ht_total = len(X_test)
for h_true, a_true, h_pred, a_pred in zip(y_ht_h, y_ht_a, ht_h_preds, ht_a_preds):
hp = max(0, round(h_pred))
ap = max(0, round(a_pred))
if hp == h_true and ap == a_true:
ht_exact += 1
print(f" HT Exact Score: {ht_exact / ht_total * 100:.2f}% ({ht_exact}/{ht_total})")
return {
"ft_exact_pct": exact / total * 100,
"ft_close_pct": close / total * 100,
"ft_result_pct": result_correct / total * 100,
"ht_exact_pct": ht_exact / ht_total * 100,
}
def train():
"""Main training pipeline."""
print("🚀 Score Prediction Model Trainer (V25-Compatible)")
print(f" Feature count: {len(FEATURES)}")
print("=" * 60)
# Load data
df = load_data()
print(f" Total valid rows: {len(df)}")
# Temporal split
train_df, test_df = temporal_split(df)
print(f" Training set: {len(train_df)} matches")
print(f" Test set: {len(test_df)} matches (temporally after training)")
X_train = train_df[FEATURES]
X_test = test_df[FEATURES]
# Train 4 models
models = {}
metrics = {}
for target_name, model_key in [
("score_home", "ft_home"),
("score_away", "ft_away"),
("ht_score_home", "ht_home"),
("ht_score_away", "ht_away"),
]:
model, metric = train_single_model(
X_train, train_df[target_name],
X_test, test_df[target_name],
model_key,
)
models[model_key] = model
metrics[model_key] = metric
# Combined evaluation
y_test_dict = {t: test_df[t] for t in TARGETS}
combined = evaluate_combined(models, X_test, y_test_dict)
# Save
print(f"\n💾 Saving models to {MODEL_PATH}...")
print(f"\n💾 Saving to {MODEL_PATH}...")
model_data = {
"home_model": xgb_home,
"away_model": xgb_away,
"ht_home_model": xgb_ht_home,
"ht_away_model": xgb_ht_away,
"home_model": models["ft_home"],
"away_model": models["ft_away"],
"ht_home_model": models["ht_home"],
"ht_away_model": models["ht_away"],
"features": FEATURES,
"meta": {
"mae_home": mae_home,
"mae_away": mae_away,
"mae_ht_home": mae_ht_home,
"mae_ht_away": mae_ht_away,
"acc": acc
}
**{f"{k}_{mk}": mv for k, m in metrics.items() for mk, mv in m.items()},
**combined,
"trained_at": datetime.now().isoformat(),
"feature_count": len(FEATURES),
"train_size": len(train_df),
"test_size": len(test_df),
},
}
with open(MODEL_PATH, "wb") as f:
pickle.dump(model_data, f)
print("✅ Done.")
print("\n✅ Score model training complete!")
print(f" Saved: {MODEL_PATH}")
if __name__ == "__main__":
train()
+553
View File
@@ -0,0 +1,553 @@
"""
V25 Pro Model Trainer Optuna + Isotonic Calibration
=====================================================
Combines V25's 83 features + 12 markets + temporal split
with Optuna hyperparameter tuning and Isotonic Regression calibration.
Usage:
python scripts/train_v25_pro.py
python scripts/train_v25_pro.py --markets MS,OU25,BTTS # specific markets
python scripts/train_v25_pro.py --trials 30 # fewer trials
"""
import os
import sys
import json
import pickle
import argparse
import numpy as np
import pandas as pd
import xgboost as xgb
import lightgbm as lgb
import optuna
from optuna.samplers import TPESampler
from datetime import datetime
from sklearn.metrics import accuracy_score, log_loss, classification_report
from sklearn.isotonic import IsotonicRegression
from sklearn.base import BaseEstimator, ClassifierMixin
optuna.logging.set_verbosity(optuna.logging.WARNING)
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
AI_ENGINE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA_PATH = os.path.join(AI_ENGINE_DIR, "data", "training_data.csv")
MODELS_DIR = os.path.join(AI_ENGINE_DIR, "models", "v25")
REPORTS_DIR = os.path.join(AI_ENGINE_DIR, "reports", "training_v25")
os.makedirs(MODELS_DIR, exist_ok=True)
os.makedirs(REPORTS_DIR, exist_ok=True)
# ─── Feature Columns (95 features, NO target leakage) ───────────────
FEATURES = [
# ELO (8)
"home_overall_elo", "away_overall_elo", "elo_diff",
"home_home_elo", "away_away_elo",
"home_form_elo", "away_form_elo", "form_elo_diff",
# Form (12)
"home_goals_avg", "home_conceded_avg",
"away_goals_avg", "away_conceded_avg",
"home_clean_sheet_rate", "away_clean_sheet_rate",
"home_scoring_rate", "away_scoring_rate",
"home_winning_streak", "away_winning_streak",
"home_unbeaten_streak", "away_unbeaten_streak",
# H2H (6)
"h2h_total_matches", "h2h_home_win_rate", "h2h_draw_rate",
"h2h_avg_goals", "h2h_btts_rate", "h2h_over25_rate",
# Team Stats (8)
"home_avg_possession", "away_avg_possession",
"home_avg_shots_on_target", "away_avg_shots_on_target",
"home_shot_conversion", "away_shot_conversion",
"home_avg_corners", "away_avg_corners",
# Odds (24 + 20 presence flags)
"odds_ms_h", "odds_ms_d", "odds_ms_a",
"implied_home", "implied_draw", "implied_away",
"odds_ht_ms_h", "odds_ht_ms_d", "odds_ht_ms_a",
"odds_ou05_o", "odds_ou05_u",
"odds_ou15_o", "odds_ou15_u",
"odds_ou25_o", "odds_ou25_u",
"odds_ou35_o", "odds_ou35_u",
"odds_ht_ou05_o", "odds_ht_ou05_u",
"odds_ht_ou15_o", "odds_ht_ou15_u",
"odds_btts_y", "odds_btts_n",
"odds_ms_h_present", "odds_ms_d_present", "odds_ms_a_present",
"odds_ht_ms_h_present", "odds_ht_ms_d_present", "odds_ht_ms_a_present",
"odds_ou05_o_present", "odds_ou05_u_present",
"odds_ou15_o_present", "odds_ou15_u_present",
"odds_ou25_o_present", "odds_ou25_u_present",
"odds_ou35_o_present", "odds_ou35_u_present",
"odds_ht_ou05_o_present", "odds_ht_ou05_u_present",
"odds_ht_ou15_o_present", "odds_ht_ou15_u_present",
"odds_btts_y_present", "odds_btts_n_present",
# League (4)
"home_xga", "away_xga",
"league_avg_goals", "league_zero_goal_rate",
# Upset Engine (4)
"upset_atmosphere", "upset_motivation", "upset_fatigue", "upset_potential",
# Referee Engine (5)
"referee_home_bias", "referee_avg_goals", "referee_cards_total",
"referee_avg_yellow", "referee_experience",
# Momentum (3)
"home_momentum_score", "away_momentum_score", "momentum_diff",
# Squad (9)
"home_squad_quality", "away_squad_quality", "squad_diff",
"home_key_players", "away_key_players",
"home_missing_impact", "away_missing_impact",
"home_goals_form", "away_goals_form",
# Player-Level Features (12)
"home_lineup_goals_per90", "away_lineup_goals_per90",
"home_lineup_assists_per90", "away_lineup_assists_per90",
"home_squad_continuity", "away_squad_continuity",
"home_top_scorer_form", "away_top_scorer_form",
"home_avg_player_exp", "away_avg_player_exp",
"home_goals_diversity", "away_goals_diversity",
]
MARKET_CONFIGS = [
{"target": "label_ms", "name": "MS", "num_class": 3},
{"target": "label_ou15", "name": "OU15", "num_class": 2},
{"target": "label_ou25", "name": "OU25", "num_class": 2},
{"target": "label_ou35", "name": "OU35", "num_class": 2},
{"target": "label_btts", "name": "BTTS", "num_class": 2},
{"target": "label_ht_result", "name": "HT_RESULT", "num_class": 3},
{"target": "label_ht_ou05", "name": "HT_OU05", "num_class": 2},
{"target": "label_ht_ou15", "name": "HT_OU15", "num_class": 2},
{"target": "label_ht_ft", "name": "HTFT", "num_class": 9},
{"target": "label_odd_even", "name": "ODD_EVEN", "num_class": 2},
{"target": "label_cards_ou45", "name": "CARDS_OU45", "num_class": 2},
{"target": "label_handicap_ms", "name": "HANDICAP_MS", "num_class": 3},
]
def load_data():
"""Load and prepare training data."""
if not os.path.exists(DATA_PATH):
print(f"[ERROR] Data not found: {DATA_PATH}")
sys.exit(1)
print(f"[INFO] Loading {DATA_PATH}...")
df = pd.read_csv(DATA_PATH)
for col in FEATURES:
if col in df.columns:
df[col] = df[col].fillna(0)
# Derive odds presence flags for older CSVs
odds_flag_sources = {
"odds_ms_h_present": "odds_ms_h", "odds_ms_d_present": "odds_ms_d",
"odds_ms_a_present": "odds_ms_a", "odds_ht_ms_h_present": "odds_ht_ms_h",
"odds_ht_ms_d_present": "odds_ht_ms_d", "odds_ht_ms_a_present": "odds_ht_ms_a",
"odds_ou05_o_present": "odds_ou05_o", "odds_ou05_u_present": "odds_ou05_u",
"odds_ou15_o_present": "odds_ou15_o", "odds_ou15_u_present": "odds_ou15_u",
"odds_ou25_o_present": "odds_ou25_o", "odds_ou25_u_present": "odds_ou25_u",
"odds_ou35_o_present": "odds_ou35_o", "odds_ou35_u_present": "odds_ou35_u",
"odds_ht_ou05_o_present": "odds_ht_ou05_o", "odds_ht_ou05_u_present": "odds_ht_ou05_u",
"odds_ht_ou15_o_present": "odds_ht_ou15_o", "odds_ht_ou15_u_present": "odds_ht_ou15_u",
"odds_btts_y_present": "odds_btts_y", "odds_btts_n_present": "odds_btts_n",
}
for flag_col, odds_col in odds_flag_sources.items():
if flag_col not in df.columns:
df[flag_col] = (
pd.to_numeric(df.get(odds_col, 0), errors="coerce").fillna(0) > 1.01
).astype(float)
print(f"[INFO] Shape: {df.shape}, Features: {len(FEATURES)}")
return df
def temporal_split_4way(valid_df: pd.DataFrame):
"""Chronological 60/15/10/15 split: train/val/cal/test."""
ordered = valid_df.sort_values("mst_utc").reset_index(drop=True)
n = len(ordered)
i1 = int(n * 0.60)
i2 = int(n * 0.75)
i3 = int(n * 0.85)
train = ordered.iloc[:i1].copy()
val = ordered.iloc[i1:i2].copy()
cal = ordered.iloc[i2:i3].copy()
test = ordered.iloc[i3:].copy()
return train, val, cal, test
# ─── XGBoost Wrapper for sklearn CalibratedClassifierCV ─────────────
class XGBWrapper(BaseEstimator, ClassifierMixin):
"""Thin sklearn-compatible wrapper around xgb.train for Isotonic calibration."""
def __init__(self, params, num_boost_round=500):
self.params = params
self.num_boost_round = num_boost_round
self.model_ = None
self.classes_ = None
def fit(self, X, y, **kwargs):
self.classes_ = np.unique(y)
dtrain = xgb.DMatrix(X, label=y)
self.model_ = xgb.train(self.params, dtrain, num_boost_round=self.num_boost_round)
return self
def predict_proba(self, X):
dm = xgb.DMatrix(X)
probs = self.model_.predict(dm)
if len(probs.shape) == 1:
probs = np.column_stack([1 - probs, probs])
return probs
def predict(self, X):
return np.argmax(self.predict_proba(X), axis=1)
# ─── Optuna Objectives ──────────────────────────────────────────────
def xgb_objective(trial, X_train, y_train, X_val, y_val, num_class):
params = {
"objective": "multi:softprob" if num_class > 2 else "binary:logistic",
"eval_metric": "mlogloss" if num_class > 2 else "logloss",
"max_depth": trial.suggest_int("max_depth", 3, 8),
"eta": trial.suggest_float("eta", 0.01, 0.15, log=True),
"subsample": trial.suggest_float("subsample", 0.6, 1.0),
"colsample_bytree": trial.suggest_float("colsample_bytree", 0.5, 1.0),
"min_child_weight": trial.suggest_int("min_child_weight", 1, 10),
"gamma": trial.suggest_float("gamma", 1e-8, 1.0, log=True),
"reg_lambda": trial.suggest_float("reg_lambda", 1e-8, 10.0, log=True),
"reg_alpha": trial.suggest_float("reg_alpha", 1e-8, 1.0, log=True),
"n_jobs": 4,
"random_state": 42,
}
if num_class > 2:
params["num_class"] = num_class
dtrain = xgb.DMatrix(X_train, label=y_train)
dval = xgb.DMatrix(X_val, label=y_val)
model = xgb.train(
params, dtrain, num_boost_round=1000,
evals=[(dval, "val")], early_stopping_rounds=50, verbose_eval=False,
)
preds = model.predict(dval)
if len(preds.shape) == 1:
preds = np.column_stack([1 - preds, preds])
return log_loss(y_val, preds)
def lgb_objective(trial, X_train, y_train, X_val, y_val, num_class):
params = {
"objective": "multiclass" if num_class > 2 else "binary",
"metric": "multi_logloss" if num_class > 2 else "binary_logloss",
"max_depth": trial.suggest_int("max_depth", 3, 8),
"learning_rate": trial.suggest_float("learning_rate", 0.01, 0.15, log=True),
"feature_fraction": trial.suggest_float("feature_fraction", 0.5, 1.0),
"bagging_fraction": trial.suggest_float("bagging_fraction", 0.6, 1.0),
"bagging_freq": trial.suggest_int("bagging_freq", 1, 7),
"min_child_samples": trial.suggest_int("min_child_samples", 5, 50),
"lambda_l1": trial.suggest_float("lambda_l1", 1e-8, 1.0, log=True),
"lambda_l2": trial.suggest_float("lambda_l2", 1e-8, 10.0, log=True),
"n_jobs": 4, "random_state": 42, "verbose": -1,
}
if num_class > 2:
params["num_class"] = num_class
train_data = lgb.Dataset(X_train, label=y_train)
val_data = lgb.Dataset(X_val, label=y_val, reference=train_data)
model = lgb.train(
params, train_data, num_boost_round=1000,
valid_sets=[val_data], valid_names=["val"],
callbacks=[lgb.early_stopping(50), lgb.log_evaluation(0)],
)
preds = model.predict(X_val, num_iteration=model.best_iteration)
if len(preds.shape) == 1:
preds = np.column_stack([1 - preds, preds])
return log_loss(y_val, preds)
# ─── Main Training Pipeline ─────────────────────────────────────────
def train_market(df, target_col, market_name, num_class, n_trials):
"""Full pipeline for one market: Optuna → Train → Calibrate → Evaluate."""
print(f"\n{'='*60}")
print(f"[MARKET] {market_name} (classes={num_class})")
print(f"{'='*60}")
valid_df = df[df[target_col].notna()].copy()
valid_df = valid_df[valid_df[target_col].astype(str) != ""].copy()
print(f"[INFO] Valid samples: {len(valid_df)}")
if len(valid_df) < 500:
print(f"[SKIP] Not enough data for {market_name}")
return None
available_features = [f for f in FEATURES if f in valid_df.columns]
print(f"[INFO] Features: {len(available_features)}/{len(FEATURES)}")
train_df, val_df, cal_df, test_df = temporal_split_4way(valid_df)
X_train = train_df[available_features].values
X_val = val_df[available_features].values
X_cal = cal_df[available_features].values
X_test = test_df[available_features].values
y_train = train_df[target_col].astype(int).values
y_val = val_df[target_col].astype(int).values
y_cal = cal_df[target_col].astype(int).values
y_test = test_df[target_col].astype(int).values
print(f"[INFO] Split: train={len(X_train)} val={len(X_val)} cal={len(X_cal)} test={len(X_test)}")
# ── Phase 1: Optuna XGBoost ──────────────────────────────────
print(f"\n[OPTUNA] XGBoost tuning ({n_trials} trials)...")
xgb_study = optuna.create_study(direction="minimize", sampler=TPESampler(seed=42))
xgb_study.optimize(
lambda trial: xgb_objective(trial, X_train, y_train, X_val, y_val, num_class),
n_trials=n_trials,
)
xgb_best = xgb_study.best_params
print(f"[OK] XGB best logloss: {xgb_study.best_value:.4f}")
# ── Phase 2: Optuna LightGBM ─────────────────────────────────
print(f"[OPTUNA] LightGBM tuning ({n_trials} trials)...")
lgb_study = optuna.create_study(direction="minimize", sampler=TPESampler(seed=42))
lgb_study.optimize(
lambda trial: lgb_objective(trial, X_train, y_train, X_val, y_val, num_class),
n_trials=n_trials,
)
lgb_best = lgb_study.best_params
print(f"[OK] LGB best logloss: {lgb_study.best_value:.4f}")
# ── Phase 3: Train final models with best params ─────────────
# XGBoost final
xgb_params = {
"objective": "multi:softprob" if num_class > 2 else "binary:logistic",
"eval_metric": "mlogloss" if num_class > 2 else "logloss",
"n_jobs": 4, "random_state": 42,
**{k: v for k, v in xgb_best.items()},
}
if num_class > 2:
xgb_params["num_class"] = num_class
dtrain = xgb.DMatrix(X_train, label=y_train)
dval = xgb.DMatrix(X_val, label=y_val)
xgb_model = xgb.train(
xgb_params, dtrain, num_boost_round=1500,
evals=[(dtrain, "train"), (dval, "val")],
early_stopping_rounds=80, verbose_eval=200,
)
print(f"[OK] XGB final: iter={xgb_model.best_iteration}, score={xgb_model.best_score:.4f}")
# LightGBM final
lgb_params = {
"objective": "multiclass" if num_class > 2 else "binary",
"metric": "multi_logloss" if num_class > 2 else "binary_logloss",
"n_jobs": 4, "random_state": 42, "verbose": -1,
**{k: v for k, v in lgb_best.items()},
}
if num_class > 2:
lgb_params["num_class"] = num_class
lgb_train_data = lgb.Dataset(X_train, label=y_train)
lgb_val_data = lgb.Dataset(X_val, label=y_val, reference=lgb_train_data)
lgb_model = lgb.train(
lgb_params, lgb_train_data, num_boost_round=1500,
valid_sets=[lgb_train_data, lgb_val_data],
valid_names=["train", "val"],
callbacks=[lgb.early_stopping(80), lgb.log_evaluation(200)],
)
print(f"[OK] LGB final: iter={lgb_model.best_iteration}")
# ── Phase 4: Isotonic Calibration on cal set ─────────────────
print("[CAL] Fitting Isotonic Regression (per-class)...")
# XGB calibration — manual IsotonicRegression per class
dcal = xgb.DMatrix(X_cal)
xgb_cal_raw = xgb_model.predict(dcal)
if len(xgb_cal_raw.shape) == 1:
xgb_cal_raw = np.column_stack([1 - xgb_cal_raw, xgb_cal_raw])
xgb_iso_calibrators = []
for cls_idx in range(num_class):
ir = IsotonicRegression(out_of_bounds="clip")
y_binary = (y_cal == cls_idx).astype(float)
ir.fit(xgb_cal_raw[:, cls_idx], y_binary)
xgb_iso_calibrators.append(ir)
print(f"[OK] XGB Isotonic calibrators fitted: {num_class} classes")
# LGB calibration — manual IsotonicRegression per class
lgb_cal_raw = lgb_model.predict(X_cal, num_iteration=lgb_model.best_iteration)
if len(lgb_cal_raw.shape) == 1:
lgb_cal_raw = np.column_stack([1 - lgb_cal_raw, lgb_cal_raw])
lgb_iso_calibrators = []
for cls_idx in range(num_class):
ir = IsotonicRegression(out_of_bounds="clip")
y_binary = (y_cal == cls_idx).astype(float)
ir.fit(lgb_cal_raw[:, cls_idx], y_binary)
lgb_iso_calibrators.append(ir)
print(f"[OK] LGB Isotonic calibrators fitted: {num_class} classes")
# ── Phase 5: Evaluate on test set ────────────────────────────
print("\n[EVAL] Test set evaluation...")
dtest = xgb.DMatrix(X_test)
# Raw XGB
xgb_raw_probs = xgb_model.predict(dtest)
if len(xgb_raw_probs.shape) == 1:
xgb_raw_probs = np.column_stack([1 - xgb_raw_probs, xgb_raw_probs])
# Calibrated XGB — apply isotonic per class + renormalize
xgb_cal_probs = np.column_stack([
xgb_iso_calibrators[i].predict(xgb_raw_probs[:, i]) for i in range(num_class)
])
xgb_cal_probs = xgb_cal_probs / xgb_cal_probs.sum(axis=1, keepdims=True)
# Raw LGB
lgb_raw_probs = lgb_model.predict(X_test, num_iteration=lgb_model.best_iteration)
if len(lgb_raw_probs.shape) == 1:
lgb_raw_probs = np.column_stack([1 - lgb_raw_probs, lgb_raw_probs])
# Calibrated LGB — apply isotonic per class + renormalize
lgb_cal_probs = np.column_stack([
lgb_iso_calibrators[i].predict(lgb_raw_probs[:, i]) for i in range(num_class)
])
lgb_cal_probs = lgb_cal_probs / lgb_cal_probs.sum(axis=1, keepdims=True)
# Ensembles
raw_ensemble = (xgb_raw_probs + lgb_raw_probs) / 2
cal_ensemble = (xgb_cal_probs + lgb_cal_probs) / 2
def _eval(probs, label):
preds = np.argmax(probs, axis=1)
acc = accuracy_score(y_test, preds)
ll = log_loss(y_test, probs)
print(f" {label}: Acc={acc:.4f} LogLoss={ll:.4f}")
return {"accuracy": round(float(acc), 4), "logloss": round(float(ll), 4)}
m_xgb_raw = _eval(xgb_raw_probs, "XGB Raw")
m_xgb_cal = _eval(xgb_cal_probs, "XGB Calibrated")
m_lgb_raw = _eval(lgb_raw_probs, "LGB Raw")
m_lgb_cal = _eval(lgb_cal_probs, "LGB Calibrated")
m_ensemble = _eval(raw_ensemble, "Ensemble Raw")
m_cal_ensemble = _eval(cal_ensemble, "Ensemble Calibrated")
# Classification report for ensemble
ens_preds = np.argmax(raw_ensemble, axis=1)
print(f"\n[REPORT] Ensemble Classification Report:")
print(classification_report(y_test, ens_preds))
# ── Phase 6: Save models ─────────────────────────────────────
# Raw models (orchestrator compatible)
xgb_path = os.path.join(MODELS_DIR, f"xgb_v25_{market_name.lower()}.json")
xgb_model.save_model(xgb_path)
print(f"[SAVE] {xgb_path}")
lgb_path = os.path.join(MODELS_DIR, f"lgb_v25_{market_name.lower()}.txt")
lgb_model.save_model(lgb_path)
print(f"[SAVE] {lgb_path}")
# Isotonic calibrators (XGB + LGB)
xgb_cal_path = os.path.join(MODELS_DIR, f"iso_xgb_v25_{market_name.lower()}.pkl")
with open(xgb_cal_path, "wb") as f:
pickle.dump(xgb_iso_calibrators, f)
print(f"[SAVE] {xgb_cal_path}")
lgb_cal_path = os.path.join(MODELS_DIR, f"iso_lgb_v25_{market_name.lower()}.pkl")
with open(lgb_cal_path, "wb") as f:
pickle.dump(lgb_iso_calibrators, f)
print(f"[SAVE] {lgb_cal_path}")
return {
"market": market_name,
"samples": int(len(valid_df)),
"train": int(len(X_train)),
"val": int(len(X_val)),
"cal": int(len(X_cal)),
"test": int(len(X_test)),
"features_used": len(available_features),
"xgb_best_params": xgb_best,
"lgb_best_params": lgb_best,
"xgb_best_iteration": int(xgb_model.best_iteration),
"lgb_best_iteration": int(lgb_model.best_iteration),
"xgb_optuna_best_logloss": round(float(xgb_study.best_value), 4),
"lgb_optuna_best_logloss": round(float(lgb_study.best_value), 4),
"test_xgb_raw": m_xgb_raw,
"test_xgb_calibrated": m_xgb_cal,
"test_lgb_raw": m_lgb_raw,
"test_lgb_calibrated": m_lgb_cal,
"test_ensemble_raw": m_ensemble,
"test_ensemble_calibrated": m_cal_ensemble,
}
def main():
parser = argparse.ArgumentParser(description="V25 Pro Trainer")
parser.add_argument("--markets", type=str, default=None,
help="Comma-separated market names (e.g., MS,OU25,BTTS)")
parser.add_argument("--trials", type=int, default=50,
help="Optuna trials per model per market")
args = parser.parse_args()
print("=" * 60)
print("V25 PRO — Optuna + Isotonic Calibration")
print("=" * 60)
print(f"[INFO] Started: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
print(f"[INFO] Trials per model: {args.trials}")
print(f"[INFO] Total features: {len(FEATURES)}")
df = load_data()
configs = MARKET_CONFIGS
if args.markets:
selected = [m.strip().upper() for m in args.markets.split(",")]
configs = [c for c in configs if c["name"] in selected]
print(f"[INFO] Selected markets: {[c['name'] for c in configs]}")
all_metrics = {
"trained_at": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
"trainer": "v25_pro",
"optuna_trials": args.trials,
"total_features": len(FEATURES),
"markets": {},
}
for config in configs:
target = config["target"]
if target not in df.columns:
print(f"[SKIP] {config['name']}: missing target {target}")
continue
metrics = train_market(
df, target, config["name"], config["num_class"], args.trials,
)
if metrics:
all_metrics["markets"][config["name"]] = metrics
# Save feature list
feature_path = os.path.join(MODELS_DIR, "feature_cols.json")
with open(feature_path, "w") as f:
json.dump(FEATURES, f, indent=2)
# Save full report
report_path = os.path.join(REPORTS_DIR, "v25_pro_metrics.json")
with open(report_path, "w") as f:
json.dump(all_metrics, f, indent=2, default=str)
print(f"\n[SAVE] Report: {report_path}")
# Summary
print("\n" + "=" * 60)
print("[SUMMARY]")
print("=" * 60)
for name, m in all_metrics["markets"].items():
ens = m.get("test_ensemble_calibrated", m.get("test_ensemble_raw", {}))
acc = ens.get('accuracy', '?')
ll = ens.get('logloss', '?')
acc_s = f"{acc:.4f}" if isinstance(acc, float) else str(acc)
ll_s = f"{ll:.4f}" if isinstance(ll, float) else str(ll)
print(f" {name:12s} | Acc={acc_s:>6s} | LL={ll_s:>6s} | "
f"XGB_iter={m.get('xgb_best_iteration','?')} LGB_iter={m.get('lgb_best_iteration','?')}")
print(f"\n[INFO] Completed: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
print("[OK] V25 PRO Training Complete!")
if __name__ == "__main__":
main()
+118 -21
View File
@@ -20,7 +20,7 @@ from sklearn.isotonic import IsotonicRegression
warnings.filterwarnings("ignore")
AI_DIR = Path(__file__).resolve().parent.parent
DATA_CSV = AI_DIR / "data" / "training_data_v27.csv"
DATA_CSV = AI_DIR / "data" / "training_data.csv"
MODELS_DIR = AI_DIR / "models" / "v27"
MODELS_DIR.mkdir(parents=True, exist_ok=True)
@@ -373,15 +373,52 @@ def main():
print("\n" + ""*65)
print(" STAGE A.2: Fundamentals-Only O/U 2.5 Model")
print(""*65)
y_tr_ou = tr["label_ou25"].values
y_va_ou = va["label_ou25"].values
y_tr_ou = tr['label_ou25'].values
y_va_ou = va['label_ou25'].values
mask_tr = ~np.isnan(y_tr_ou)
mask_va = ~np.isnan(y_va_ou)
if mask_tr.sum() > 1000:
ou_models = train_fundamentals_model(
X_tr[mask_tr], y_tr_ou[mask_tr].astype(int),
X_va[mask_va], y_va_ou[mask_va].astype(int),
clean_feats, "ou25")
clean_feats, 'ou25')
# ── STAGE A.3: BTTS Model ──
btts_models = None
if 'label_btts' in tr.columns:
print('\n' + '' * 65)
print(' STAGE A.3: Fundamentals-Only BTTS Model')
print('' * 65)
y_tr_btts = tr['label_btts'].values
y_va_btts = va['label_btts'].values
mask_tr_btts = ~np.isnan(y_tr_btts)
mask_va_btts = ~np.isnan(y_va_btts)
if mask_tr_btts.sum() > 1000:
btts_models = train_fundamentals_model(
X_tr[mask_tr_btts], y_tr_btts[mask_tr_btts].astype(int),
X_va[mask_va_btts], y_va_btts[mask_va_btts].astype(int),
clean_feats, 'btts')
# Quick val accuracy
btts_probs = ensemble_predict(
btts_models,
X_va[mask_va_btts],
clean_feats,
n_class=2,
)
btts_acc = accuracy_score(
y_va_btts[mask_va_btts].astype(int),
btts_probs.argmax(1),
)
btts_ll = log_loss(
y_va_btts[mask_va_btts].astype(int),
btts_probs,
)
print(f'\n BTTS Ensemble Val: acc={btts_acc:.4f}, logloss={btts_ll:.4f}')
# Compare with naive baseline (always predict majority class)
btts_majority = y_va_btts[mask_va_btts].astype(int).mean()
print(f' BTTS baseline: {max(btts_majority, 1-btts_majority):.4f} (majority class)')
print(f' Model vs baseline: {btts_acc - max(btts_majority, 1-btts_majority):+.4f}')
# ── STAGE C: Backtest ──
print("\n" + ""*65)
@@ -422,13 +459,58 @@ def main():
# OU25 backtest
if ou_models:
print("\n --- O/U 2.5 Backtest ---")
print('\n --- O/U 2.5 Backtest ---')
for edge in [0.05, 0.07, 0.10]:
r = backtest_value(ou_models, te, clean_feats, "ou25",
r = backtest_value(ou_models, te, clean_feats, 'ou25',
min_edge=edge, min_odds=1.50, max_odds=3.0,
use_kelly=True)
if r.get("total", 0) > 0:
print_backtest(r, f"OU25 edge>{edge}")
if r.get('total', 0) > 0:
print_backtest(r, f'OU25 edge>{edge}')
# BTTS backtest
if btts_models and 'label_btts' in te.columns:
print('\n --- BTTS Backtest ---')
# Build BTTS odds for backtest
if 'odds_btts_y' in te.columns and 'odds_btts_n' in te.columns:
te_btts = te.copy()
te_btts['odds_btts_y'] = pd.to_numeric(
te_btts['odds_btts_y'], errors='coerce',
).fillna(1.85)
te_btts['odds_btts_n'] = pd.to_numeric(
te_btts['odds_btts_n'], errors='coerce',
).fillna(1.85)
for edge in [0.05, 0.07, 0.10]:
X_test = te_btts[clean_feats].values
probs = ensemble_predict(btts_models, X_test, clean_feats, 2)
y_btts = te_btts['label_btts'].values.astype(int)
odds_arr = te_btts[['odds_btts_n', 'odds_btts_y']].values
m_arr = 1 / odds_arr
impl = m_arr / m_arr.sum(axis=1, keepdims=True)
total_bets = 0
wins = 0
pnl = 0.0
for i in range(len(y_btts)):
for cls in range(2):
e = probs[i, cls] - impl[i, cls]
o = odds_arr[i, cls]
if e < edge or o < 1.50 or o > 3.0:
continue
total_bets += 1
won = (y_btts[i] == cls)
if won:
wins += 1
pnl += 10 * (o - 1)
else:
pnl -= 10
if total_bets > 0:
roi = pnl / (total_bets * 10) * 100
hit = wins / total_bets * 100
print(
f' Edge>{edge:.2f}: {total_bets} bets, '
f'hit={hit:.1f}%, ROI={roi:+.1f}%'
)
# ── Feature importance ──
if "lgb" in ms_models:
@@ -452,25 +534,40 @@ def main():
if ou_models:
for name, m in ou_models.items():
p = MODELS_DIR / f"v27_ou25_{name}.pkl"
with open(p, "wb") as f:
p = MODELS_DIR / f'v27_ou25_{name}.pkl'
with open(p, 'wb') as f:
pickle.dump(m, f)
print(f"{p.name}")
print(f'{p.name}')
if btts_models:
for name, m in btts_models.items():
p = MODELS_DIR / f'v27_btts_{name}.pkl'
with open(p, 'wb') as f:
pickle.dump(m, f)
print(f'{p.name}')
meta = {
"version": "v27-pro", "trained_at": time.strftime("%Y-%m-%d %H:%M:%S"),
"approach": "odds-free fundamentals + value edge detection",
"feature_count": len(clean_feats),
"total_samples": len(df),
"val_acc": round(val_acc, 4), "val_ll": round(val_ll, 4),
"best_config": {k: v for k, v in best_cfg.items() if k != "result"} if best_cfg else {},
"markets": ["ms"] + (["ou25"] if ou_models else []),
'version': 'v27-pro',
'trained_at': time.strftime('%Y-%m-%d %H:%M:%S'),
'approach': 'odds-free fundamentals + value edge detection',
'feature_count': len(clean_feats),
'total_samples': len(df),
'val_acc': round(val_acc, 4),
'val_ll': round(val_ll, 4),
'best_config': {
k: v for k, v in best_cfg.items() if k != 'result'
} if best_cfg else {},
'markets': (
['ms']
+ (['ou25'] if ou_models else [])
+ (['btts'] if btts_models else [])
),
}
with open(MODELS_DIR / "v27_metadata.json", "w") as f:
with open(MODELS_DIR / 'v27_metadata.json', 'w') as f:
json.dump(meta, f, indent=2, default=str)
with open(MODELS_DIR / "v27_feature_cols.json", "w") as f:
with open(MODELS_DIR / 'v27_feature_cols.json', 'w') as f:
json.dump(clean_feats, f, indent=2)
print(f" ✓ metadata + feature_cols")
print(f' ✓ metadata + feature_cols')
print(f"\n Total time: {(time.time()-t0)/60:.1f} min")
print(" DONE!")
+307
View File
@@ -0,0 +1,307 @@
"""
Update Implied Odds in football_ai_features
=============================================
Populates implied_home, implied_draw, implied_away, implied_over25, implied_btts
from real odds data in odd_categories + odd_selections tables.
Also backfills form-based features (home_goals_avg_5, away_goals_avg_5, etc.)
from recent match history.
Usage:
python3 scripts/update_implied_odds.py
"""
import os
import sys
import time
import psycopg2
from dotenv import load_dotenv
load_dotenv()
def get_conn():
db_url = os.getenv("DATABASE_URL", "").split("?schema=")[0]
return psycopg2.connect(db_url)
def update_implied_odds(conn):
"""Update implied probabilities from real odds data."""
cur = conn.cursor()
print("📊 Phase 1: Updating implied odds from real market data...")
t0 = time.time()
# Step 1: Build odds lookup from odd_categories + odd_selections
print(" Loading odds data...")
cur.execute("""
SELECT oc.match_id, oc.name AS cat_name, os.name AS sel_name, os.odd_value
FROM odd_selections os
JOIN odd_categories oc ON os.odd_category_db_id = oc.db_id
WHERE os.odd_value IS NOT NULL
AND CAST(os.odd_value AS FLOAT) > 1.0
""")
odds_by_match = {}
row_count = 0
for match_id, cat_name, sel_name, odd_val in cur.fetchall():
try:
v = float(odd_val)
if v <= 1.0:
continue
except (ValueError, TypeError):
continue
if match_id not in odds_by_match:
odds_by_match[match_id] = {}
cat_lower = (cat_name or "").lower().strip()
sel_lower = (sel_name or "").lower().strip()
# Match Result (1X2)
if cat_lower == 'maç sonucu':
if sel_name == '1':
odds_by_match[match_id]['ms_h'] = v
elif sel_name in ('0', 'X'):
odds_by_match[match_id]['ms_d'] = v
elif sel_name == '2':
odds_by_match[match_id]['ms_a'] = v
# Over/Under 2.5
elif cat_lower == '2,5 alt/üst':
if 'üst' in sel_lower:
odds_by_match[match_id]['ou25_o'] = v
elif 'alt' in sel_lower:
odds_by_match[match_id]['ou25_u'] = v
# BTTS
elif cat_lower == 'karşılıklı gol':
if 'var' in sel_lower:
odds_by_match[match_id]['btts_y'] = v
elif 'yok' in sel_lower:
odds_by_match[match_id]['btts_n'] = v
row_count += 1
print(f" Loaded odds for {len(odds_by_match)} matches ({row_count} selections) in {time.time()-t0:.1f}s")
# Step 2: Calculate implied probabilities and update
print(" Calculating implied probabilities...")
# Get all match_ids in football_ai_features
cur.execute("SELECT match_id FROM football_ai_features")
feature_match_ids = {row[0] for row in cur.fetchall()}
updated = 0
batch_size = 500
updates = []
for match_id in feature_match_ids:
odds = odds_by_match.get(match_id, {})
if not odds:
continue
# Implied MS probabilities (vig-free normalization)
ms_h = odds.get('ms_h', 0)
ms_d = odds.get('ms_d', 0)
ms_a = odds.get('ms_a', 0)
implied_home = 0.33
implied_draw = 0.33
implied_away = 0.33
if ms_h > 1.0 and ms_d > 1.0 and ms_a > 1.0:
raw_sum = (1 / ms_h) + (1 / ms_d) + (1 / ms_a)
if raw_sum > 0:
implied_home = round((1 / ms_h) / raw_sum, 4)
implied_draw = round((1 / ms_d) / raw_sum, 4)
implied_away = round((1 / ms_a) / raw_sum, 4)
# Implied OU25
ou25_o = odds.get('ou25_o', 0)
ou25_u = odds.get('ou25_u', 0)
implied_over25 = 0.50
if ou25_o > 1.0 and ou25_u > 1.0:
raw_sum = (1 / ou25_o) + (1 / ou25_u)
if raw_sum > 0:
implied_over25 = round((1 / ou25_o) / raw_sum, 4)
# Implied BTTS
btts_y = odds.get('btts_y', 0)
btts_n = odds.get('btts_n', 0)
implied_btts = 0.50
if btts_y > 1.0 and btts_n > 1.0:
raw_sum = (1 / btts_y) + (1 / btts_n)
if raw_sum > 0:
implied_btts = round((1 / btts_y) / raw_sum, 4)
# Only update if we have real data (not all defaults)
has_real_data = (ms_h > 1.0 or ou25_o > 1.0 or btts_y > 1.0)
if not has_real_data:
continue
updates.append((
implied_home, implied_draw, implied_away,
implied_over25, implied_btts, match_id
))
if len(updates) >= batch_size:
cur.executemany("""
UPDATE football_ai_features
SET implied_home = %s,
implied_draw = %s,
implied_away = %s,
implied_over25 = %s,
implied_btts_yes = %s
WHERE match_id = %s
""", updates)
updated += len(updates)
updates = []
# Final batch
if updates:
cur.executemany("""
UPDATE football_ai_features
SET implied_home = %s,
implied_draw = %s,
implied_away = %s,
implied_over25 = %s,
implied_btts_yes = %s
WHERE match_id = %s
""", updates)
updated += len(updates)
conn.commit()
print(f" ✅ Updated implied odds for {updated} matches in {time.time()-t0:.1f}s")
return updated
def update_form_features(conn):
"""Backfill form-based features (goals avg, clean sheet rate) from match history."""
cur = conn.cursor()
print("\n📊 Phase 2: Updating form-based features...")
t0 = time.time()
# Load all finished football matches ordered by time
print(" Loading match history...")
cur.execute("""
SELECT id, home_team_id, away_team_id, score_home, score_away, mst_utc
FROM matches
WHERE status = 'FT'
AND score_home IS NOT NULL
AND sport = 'football'
ORDER BY mst_utc ASC
""")
matches = cur.fetchall()
print(f" Loaded {len(matches)} finished matches")
# Build team history incrementally
from collections import defaultdict
team_history = defaultdict(list) # team_id -> [(goals_scored, goals_conceded)]
# Get all feature match IDs
cur.execute("SELECT match_id FROM football_ai_features")
feature_match_ids = {row[0] for row in cur.fetchall()}
updated = 0
batch_size = 500
updates = []
for match_id, home_id, away_id, score_home, score_away, mst_utc in matches:
# Calculate features BEFORE updating history (pre-match features)
if match_id in feature_match_ids:
h_hist = team_history[home_id][-5:] # last 5
a_hist = team_history[away_id][-5:]
# Home team form
if h_hist:
h_goals_avg = sum(g for g, _ in h_hist) / len(h_hist)
h_conceded_avg = sum(c for _, c in h_hist) / len(h_hist)
h_cs_rate = sum(1 for _, c in h_hist if c == 0) / len(h_hist)
h_scoring_rate = sum(1 for g, _ in h_hist if g > 0) / len(h_hist)
else:
h_goals_avg, h_conceded_avg = 1.3, 1.2
h_cs_rate, h_scoring_rate = 0.25, 0.75
# Away team form
if a_hist:
a_goals_avg = sum(g for g, _ in a_hist) / len(a_hist)
a_conceded_avg = sum(c for _, c in a_hist) / len(a_hist)
a_cs_rate = sum(1 for _, c in a_hist if c == 0) / len(a_hist)
a_scoring_rate = sum(1 for g, _ in a_hist if g > 0) / len(a_hist)
else:
a_goals_avg, a_conceded_avg = 1.3, 1.2
a_cs_rate, a_scoring_rate = 0.25, 0.75
updates.append((
round(h_goals_avg, 3), round(h_conceded_avg, 3),
round(h_cs_rate, 3), round(h_scoring_rate, 3),
round(a_goals_avg, 3), round(a_conceded_avg, 3),
round(a_cs_rate, 3), round(a_scoring_rate, 3),
match_id
))
if len(updates) >= batch_size:
cur.executemany("""
UPDATE football_ai_features
SET home_goals_avg_5 = %s,
home_conceded_avg_5 = %s,
home_clean_sheet_rate = %s,
home_scoring_rate = %s,
away_goals_avg_5 = %s,
away_conceded_avg_5 = %s,
away_clean_sheet_rate = %s,
away_scoring_rate = %s
WHERE match_id = %s
""", updates)
updated += len(updates)
updates = []
# Update history AFTER feature extraction (maintains pre-match invariant)
team_history[home_id].append((score_home, score_away))
team_history[away_id].append((score_away, score_home))
# Final batch
if updates:
cur.executemany("""
UPDATE football_ai_features
SET home_goals_avg_5 = %s,
home_conceded_avg_5 = %s,
home_clean_sheet_rate = %s,
home_scoring_rate = %s,
away_goals_avg_5 = %s,
away_conceded_avg_5 = %s,
away_clean_sheet_rate = %s,
away_scoring_rate = %s
WHERE match_id = %s
""", updates)
updated += len(updates)
conn.commit()
print(f" ✅ Updated form features for {updated} matches in {time.time()-t0:.1f}s")
return updated
def main():
print("🚀 Football AI Features — Implied Odds & Form Backfill")
print("=" * 60)
conn = get_conn()
try:
odds_updated = update_implied_odds(conn)
form_updated = update_form_features(conn)
print(f"\n✅ DONE!")
print(f" Implied odds updated: {odds_updated} matches")
print(f" Form features updated: {form_updated} matches")
finally:
conn.close()
if __name__ == "__main__":
main()
+505
View File
@@ -0,0 +1,505 @@
"""
Deterministic betting judge for prediction packages.
The model layer estimates event probabilities. BettingBrain decides whether
those probabilities are trustworthy enough to risk money.
"""
from __future__ import annotations
from typing import Any, Dict, List, Optional, Tuple
class BettingBrain:
MIN_ODDS = 1.30
MIN_BET_SCORE = 72.0
MIN_WATCH_SCORE = 62.0
MIN_BAND_SAMPLE = 8
HARD_DIVERGENCE = 0.22
SOFT_DIVERGENCE = 0.14
EXTREME_MODEL_PROB = 0.85
EXTREME_GAP = 0.30
MARKET_PRIORS = {
"DC": 4.0,
"OU15": 3.0,
"OU25": 2.0,
"BTTS": 0.0,
"MS": -2.0,
"OU35": -2.0,
"HT": -6.0,
"HTFT": -12.0,
"CARDS": -5.0,
"OE": -8.0,
}
def judge(self, package: Dict[str, Any]) -> Dict[str, Any]:
v27_engine = package.get("v27_engine")
if not isinstance(v27_engine, dict):
return package
guarded = dict(package)
rows = self._collect_rows(guarded)
if not rows:
return guarded
judged_rows: Dict[str, Dict[str, Any]] = {}
decisions: List[Dict[str, Any]] = []
for row in rows:
key = self._row_key(row)
judged = self._judge_row(dict(row), guarded)
judged_rows[key] = judged
decisions.append(judged["betting_brain"])
approved = [
row for row in judged_rows.values()
if row.get("betting_brain", {}).get("action") == "BET"
]
watchlist = [
row for row in judged_rows.values()
if row.get("betting_brain", {}).get("action") == "WATCH"
]
approved.sort(key=self._candidate_sort_key, reverse=True)
watchlist.sort(key=self._candidate_sort_key, reverse=True)
original_main = guarded.get("main_pick") or {}
main_pick = None
decision = "NO_BET"
decision_reason = "No candidate passed the betting brain evidence gates."
if approved:
main_pick = dict(approved[0])
main_pick["is_guaranteed"] = bool(main_pick.get("betting_brain", {}).get("score", 0.0) >= 82.0)
main_pick["pick_reason"] = "betting_brain_approved"
decision = "BET"
decision_reason = main_pick.get("betting_brain", {}).get("summary", "Evidence is aligned.")
elif watchlist:
main_pick = dict(watchlist[0])
self._force_no_bet(main_pick, "betting_brain_watchlist")
decision = "WATCHLIST"
decision_reason = main_pick.get("betting_brain", {}).get("summary", "Interesting but not clean enough.")
elif original_main:
main_pick = dict(judged_rows.get(self._row_key(original_main), original_main))
self._force_no_bet(main_pick, "betting_brain_no_safe_pick")
main_key = self._row_key(main_pick) if main_pick else ""
supporting = [
dict(row)
for row in judged_rows.values()
if self._row_key(row) != main_key
]
supporting.sort(key=self._candidate_sort_key, reverse=True)
bet_summary = [
self._summary_item(row)
for row in sorted(judged_rows.values(), key=self._candidate_sort_key, reverse=True)
]
guarded["main_pick"] = main_pick
guarded["value_pick"] = self._pick_value_candidate(judged_rows, main_key)
guarded["supporting_picks"] = supporting[:6]
guarded["bet_summary"] = bet_summary
playable = decision == "BET" and bool(main_pick and main_pick.get("playable"))
advice = dict(guarded.get("bet_advice") or {})
advice["playable"] = playable
advice["suggested_stake_units"] = float(main_pick.get("stake_units", 0.0)) if playable else 0.0
advice["reason"] = "betting_brain_approved" if playable else "betting_brain_no_bet"
advice["decision"] = decision
advice["confidence_band"] = self._decision_band(main_pick)
guarded["bet_advice"] = advice
rejected = [d for d in decisions if d.get("action") == "REJECT"]
guarded["betting_brain"] = {
"version": "judge-v1",
"decision": decision,
"reason": decision_reason,
"main_pick_key": main_key or None,
"approved_count": len(approved),
"watchlist_count": len(watchlist),
"rejected_count": len(rejected),
"top_candidates": self._top_decisions(decisions),
"rules": {
"min_bet_score": self.MIN_BET_SCORE,
"min_watch_score": self.MIN_WATCH_SCORE,
"min_band_sample": self.MIN_BAND_SAMPLE,
"hard_divergence": self.HARD_DIVERGENCE,
"soft_divergence": self.SOFT_DIVERGENCE,
"extreme_model_probability": self.EXTREME_MODEL_PROB,
"extreme_model_market_gap": self.EXTREME_GAP,
},
}
guarded["upper_brain"] = guarded["betting_brain"]
guarded.setdefault("analysis_details", {})
guarded["analysis_details"]["betting_brain_applied"] = True
guarded["analysis_details"]["betting_brain_decision"] = decision
return guarded
def _judge_row(self, row: Dict[str, Any], package: Dict[str, Any]) -> Dict[str, Any]:
market = str(row.get("market") or "")
pick = str(row.get("pick") or "")
model_prob = self._market_probability(row, package)
odds = self._safe_float(row.get("odds"), 0.0) or 0.0
implied = (1.0 / odds) if odds > 1.0 else 0.0
model_gap = (model_prob - implied) if model_prob is not None and implied > 0 else None
calibrated_conf = self._safe_float(row.get("calibrated_confidence", row.get("confidence")), 0.0) or 0.0
play_score = self._safe_float(row.get("play_score"), 0.0) or 0.0
ev_edge = self._safe_float(row.get("ev_edge", row.get("edge")), 0.0) or 0.0
v27_prob = self._v27_probability(market, pick, package.get("v27_engine") or {})
divergence = abs(model_prob - v27_prob) if model_prob is not None and v27_prob is not None else None
triple_key = self._triple_key(market, pick)
triple = self._triple_value(package, triple_key)
band_sample = int(self._safe_float((triple or {}).get("band_sample"), 0.0) or 0.0)
triple_is_value = bool((triple or {}).get("is_value"))
consensus = str((package.get("v27_engine") or {}).get("consensus") or "").upper()
positives: List[str] = []
issues: List[str] = []
vetoes: List[str] = []
score = 0.0
if row.get("playable"):
score += 18.0
positives.append("base_model_playable")
else:
score -= 18.0
issues.append("base_model_not_playable")
is_value_sniper = bool(row.get("is_value_sniper"))
if is_value_sniper:
score += 35.0
positives.append("value_sniper_override")
score += max(0.0, min(20.0, calibrated_conf * 0.22))
score += max(-8.0, min(16.0, ev_edge * 45.0))
score += max(0.0, min(14.0, play_score * 0.12))
score += self.MARKET_PRIORS.get(market, -3.0)
data_quality = package.get("data_quality") or {}
quality_score = self._safe_float(data_quality.get("score"), 0.6) or 0.6
score += max(-8.0, min(6.0, (quality_score - 0.55) * 16.0))
risk = str((package.get("risk") or {}).get("level") or "MEDIUM").upper()
score += {"LOW": 5.0, "MEDIUM": 0.0, "HIGH": -12.0, "EXTREME": -22.0}.get(risk, -4.0)
if odds < self.MIN_ODDS:
vetoes.append("odds_below_minimum")
if calibrated_conf < 38.0 and not is_value_sniper:
vetoes.append("calibrated_confidence_too_low")
if play_score < 50.0 and not is_value_sniper:
vetoes.append("play_score_too_low")
if divergence is not None:
if divergence >= self.HARD_DIVERGENCE and not is_value_sniper:
score -= 42.0
vetoes.append("v25_v27_hard_disagreement")
elif divergence >= self.SOFT_DIVERGENCE:
score -= 18.0
issues.append("v25_v27_soft_disagreement")
else:
score += 11.0
positives.append("v25_v27_aligned")
if isinstance(triple, dict):
if triple_is_value:
score += 18.0
positives.append("triple_value_confirmed")
elif market in {"DC", "MS", "OU25", "BTTS"}:
score -= 18.0
issues.append("triple_value_not_confirmed")
if band_sample >= 25:
score += 8.0
positives.append("strong_historical_sample")
elif band_sample >= self.MIN_BAND_SAMPLE:
score += 3.0
positives.append("usable_historical_sample")
else:
score -= 16.0
issues.append("historical_sample_too_low")
if market == "DC" and not is_value_sniper:
vetoes.append("dc_without_historical_sample")
elif market in {"MS", "DC", "OU25"}:
score -= 10.0
issues.append("missing_triple_value_evidence")
if consensus == "DISAGREE" and market in {"MS", "DC"}:
score -= 12.0
issues.append("engine_consensus_disagree")
if (
model_prob is not None
and model_gap is not None
and model_prob >= self.EXTREME_MODEL_PROB
and model_gap >= self.EXTREME_GAP
and not triple_is_value
and not is_value_sniper
):
score -= 24.0
vetoes.append("extreme_probability_without_evidence")
if market in {"HT", "HTFT", "OE"} and score < 86.0 and not is_value_sniper:
vetoes.append("volatile_market_requires_exceptional_evidence")
score = max(0.0, min(100.0, score))
action = "BET"
if vetoes:
action = "REJECT"
elif score < self.MIN_WATCH_SCORE and not is_value_sniper:
action = "REJECT"
elif score < self.MIN_BET_SCORE and not is_value_sniper:
action = "WATCH"
row["betting_brain"] = {
"action": action,
"score": round(score, 1),
"summary": self._summary(action, market, pick, positives, issues, vetoes),
"positives": positives[:5],
"issues": issues[:6],
"vetoes": vetoes[:6],
"model_prob": round(model_prob, 4) if model_prob is not None else None,
"implied_prob": round(implied, 4),
"model_market_gap": round(model_gap, 4) if model_gap is not None else None,
"v27_prob": round(v27_prob, 4) if v27_prob is not None else None,
"divergence": round(divergence, 4) if divergence is not None else None,
"triple_key": triple_key,
"triple_value": triple,
}
if action != "BET":
self._force_no_bet(row, f"betting_brain_{action.lower()}")
else:
row["is_guaranteed"] = bool(score >= 82.0)
row["pick_reason"] = "betting_brain_approved"
row["stake_units"] = self._brain_stake(row, score)
row["bet_grade"] = "A" if score >= 82.0 else "B"
row["playable"] = True
self._append_reason(row, f"betting_brain_{action.lower()}_{round(score)}")
return row
def _collect_rows(self, package: Dict[str, Any]) -> List[Dict[str, Any]]:
rows: Dict[str, Dict[str, Any]] = {}
for source in ("main_pick", "value_pick"):
item = package.get(source)
if isinstance(item, dict) and item.get("market"):
# print(f"DEBUG: {source} is_value_sniper: {item.get('is_value_sniper')}")
rows[self._row_key(item)] = dict(item)
for source in ("supporting_picks", "bet_summary"):
for item in package.get(source) or []:
if isinstance(item, dict) and item.get("market"):
key = self._row_key(item)
rows[key] = self._merge_row(rows.get(key), item)
return list(rows.values())
@staticmethod
def _merge_row(existing: Optional[Dict[str, Any]], incoming: Dict[str, Any]) -> Dict[str, Any]:
if existing is None:
return dict(incoming)
merged = dict(incoming)
merged.update({k: v for k, v in existing.items() if v is not None})
for key in ("decision_reasons", "reasons"):
reasons = list(existing.get(key) or []) + list(incoming.get(key) or [])
if reasons:
merged[key] = list(dict.fromkeys(reasons))
return merged
def _pick_value_candidate(self, rows: Dict[str, Dict[str, Any]], main_key: str) -> Optional[Dict[str, Any]]:
candidates = [
row for key, row in rows.items()
if key != main_key
and row.get("betting_brain", {}).get("action") in {"BET", "WATCH"}
and (self._safe_float(row.get("odds"), 0.0) or 0.0) >= 1.60
]
candidates.sort(key=self._candidate_sort_key, reverse=True)
return dict(candidates[0]) if candidates else None
def _summary_item(self, row: Dict[str, Any]) -> Dict[str, Any]:
reasons = list(row.get("decision_reasons") or row.get("reasons") or [])
return {
"market": row.get("market"),
"pick": row.get("pick"),
"raw_confidence": row.get("raw_confidence", row.get("confidence")),
"calibrated_confidence": row.get("calibrated_confidence", row.get("confidence")),
"bet_grade": row.get("bet_grade", "PASS"),
"playable": bool(row.get("playable")),
"stake_units": float(row.get("stake_units", 0.0) or 0.0),
"play_score": row.get("play_score", 0.0),
"ev_edge": row.get("ev_edge", row.get("edge", 0.0)),
"implied_prob": row.get("implied_prob", 0.0),
"odds_reliability": row.get("odds_reliability", 0.35),
"odds": row.get("odds", 0.0),
"reasons": reasons[:6],
"betting_brain": row.get("betting_brain"),
}
@staticmethod
def _candidate_sort_key(row: Dict[str, Any]) -> Tuple[float, float, float]:
brain = row.get("betting_brain") or {}
action_boost = {"BET": 2.0, "WATCH": 1.0, "REJECT": 0.0}.get(str(brain.get("action")), 0.0)
return (
action_boost,
float(brain.get("score", 0.0) or 0.0),
float(row.get("play_score", 0.0) or 0.0),
)
@staticmethod
def _row_key(row: Optional[Dict[str, Any]]) -> str:
if not isinstance(row, dict):
return ""
return f"{row.get('market')}:{row.get('pick')}"
def _force_no_bet(self, row: Dict[str, Any], reason: str) -> None:
row["playable"] = False
row["stake_units"] = 0.0
row["bet_grade"] = "PASS"
row["is_guaranteed"] = False
row["pick_reason"] = reason
if row.get("signal_tier") == "CORE":
row["signal_tier"] = "PASS"
self._append_reason(row, reason)
@staticmethod
def _append_reason(row: Dict[str, Any], reason: str) -> None:
key = "decision_reasons" if "decision_reasons" in row else "reasons"
reasons = list(row.get(key) or [])
if reason not in reasons:
reasons.append(reason)
row[key] = reasons[:6]
def _brain_stake(self, row: Dict[str, Any], score: float) -> float:
existing = self._safe_float(row.get("stake_units"), 0.0) or 0.0
odds = self._safe_float(row.get("odds"), 0.0) or 0.0
if odds <= 1.0:
return 0.0
cap = 2.0 if score >= 82.0 else 1.2
if score < 78.0:
cap = 0.8
return round(max(0.25, min(existing if existing > 0 else cap, cap)), 1)
@staticmethod
def _decision_band(main_pick: Optional[Dict[str, Any]]) -> str:
if not main_pick:
return "LOW"
score = float((main_pick.get("betting_brain") or {}).get("score", 0.0) or 0.0)
if score >= 82.0:
return "HIGH"
if score >= 72.0:
return "MEDIUM"
return "LOW"
@staticmethod
def _top_decisions(decisions: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
ordered = sorted(decisions, key=lambda d: float(d.get("score", 0.0) or 0.0), reverse=True)
return [
{
"action": item.get("action"),
"score": item.get("score"),
"summary": item.get("summary"),
"vetoes": item.get("vetoes", []),
"issues": item.get("issues", []),
}
for item in ordered[:5]
]
@staticmethod
def _summary(action: str, market: str, pick: str, positives: List[str], issues: List[str], vetoes: List[str]) -> str:
if action == "BET":
return f"{market} {pick} approved: evidence is aligned enough for a controlled stake."
if action == "WATCH":
return f"{market} {pick} is interesting but not clean enough for stake."
if vetoes:
return f"{market} {pick} rejected: {', '.join(vetoes[:3])}."
if issues:
return f"{market} {pick} rejected: {', '.join(issues[:3])}."
return f"{market} {pick} rejected by evidence score."
def _market_probability(self, row: Dict[str, Any], package: Dict[str, Any]) -> Optional[float]:
direct = self._safe_float(row.get("probability"))
if direct is not None:
return direct
board = package.get("market_board") or {}
payload = board.get(str(row.get("market") or "")) if isinstance(board, dict) else None
probs = payload.get("probs") if isinstance(payload, dict) else None
if not isinstance(probs, dict):
return None
key = self._prob_key(str(row.get("market") or ""), str(row.get("pick") or ""))
return self._safe_float(probs.get(key)) if key else None
def _v27_probability(self, market: str, pick: str, v27_engine: Dict[str, Any]) -> Optional[float]:
predictions = v27_engine.get("predictions") or {}
ms = predictions.get("ms") or {}
ou25 = predictions.get("ou25") or {}
if market == "MS":
return self._safe_float(ms.get({"1": "home", "X": "draw", "2": "away"}.get(pick, "")))
if market == "DC":
home = self._safe_float(ms.get("home"), 0.0) or 0.0
draw = self._safe_float(ms.get("draw"), 0.0) or 0.0
away = self._safe_float(ms.get("away"), 0.0) or 0.0
return {"1X": home + draw, "X2": draw + away, "12": home + away}.get(pick)
if market == "OU25":
key = self._prob_key(market, pick)
return self._safe_float(ou25.get(key)) if key else None
return None
def _triple_value(self, package: Dict[str, Any], key: Optional[str]) -> Optional[Dict[str, Any]]:
if not key:
return None
value = ((package.get("v27_engine") or {}).get("triple_value") or {}).get(key)
return value if isinstance(value, dict) else None
def _triple_key(self, market: str, pick: str) -> Optional[str]:
prob_key = self._prob_key(market, pick)
if market == "MS":
return {"1": "home", "2": "away"}.get(pick)
if market == "DC" and pick.upper() in {"1X", "X2", "12"}:
return f"dc_{pick.lower()}"
if market in {"OU15", "OU25", "OU35"} and prob_key == "over":
return f"{market.lower()}_over"
if market == "BTTS" and prob_key == "yes":
return "btts_yes"
if market == "HT":
return {"1": "ht_home", "2": "ht_away"}.get(pick)
if market in {"HT_OU05", "HT_OU15"} and prob_key == "over":
return f"{market.lower()}_over"
if market == "OE" and prob_key == "odd":
return "oe_odd"
if market == "CARDS" and prob_key == "over":
return "cards_over"
if market == "HTFT" and "/" in pick:
return f"htft_{pick.replace('/', '').lower()}"
return None
@staticmethod
def _prob_key(market: str, pick: str) -> Optional[str]:
norm = str(pick or "").strip().casefold()
if market in {"MS", "HT", "HCAP"}:
return pick if pick in {"1", "X", "2"} else None
if market == "DC":
return pick.upper() if pick.upper() in {"1X", "X2", "12"} else None
if market in {"OU15", "OU25", "OU35", "HT_OU05", "HT_OU15", "CARDS"}:
if "over" in norm or "ust" in norm or "üst" in norm:
return "over"
if "under" in norm or "alt" in norm:
return "under"
if market == "BTTS":
if "yes" in norm or "var" in norm:
return "yes"
if "no" in norm or "yok" in norm:
return "no"
if market == "OE":
if "odd" in norm or "tek" in norm:
return "odd"
if "even" in norm or "cift" in norm or "çift" in norm:
return "even"
if market == "HTFT" and "/" in pick:
return pick
return None
@staticmethod
def _safe_float(value: Any, default: Optional[float] = None) -> Optional[float]:
try:
return float(value)
except (TypeError, ValueError):
return default
+151 -24
View File
@@ -14,11 +14,40 @@ is missing or queries fail.
from __future__ import annotations
import unicodedata
from typing import Any, Dict, Optional, Tuple
from psycopg2.extras import RealDictCursor
# ─── Turkish Name Normalization ──────────────────────────────────
_TR_CHAR_MAP = str.maketrans(
'çÇğĞıİöÖşŞüÜâÂîÎûÛ',
'cCgGiIoOsSuUaAiIuU',
)
def _normalize_name(name: str) -> str:
"""
Normalize a Turkish referee name for fuzzy matching.
Strips accents, lowercases, removes extra whitespace, and maps
Turkish-specific characters to their ASCII equivalents.
"""
if not name:
return ''
# 1. Turkish-specific character mapping
normalized = name.translate(_TR_CHAR_MAP)
# 2. Unicode NFKD decomposition → strip combining marks
normalized = unicodedata.normalize('NFKD', normalized)
normalized = ''.join(
c for c in normalized if not unicodedata.combining(c)
)
# 3. Lowercase + collapse whitespace
return ' '.join(normalized.lower().split())
class FeatureEnrichmentService:
"""Stateless service — all state comes from DB via cursor."""
@@ -380,34 +409,20 @@ class FeatureEnrichmentService:
"""
Referee tendencies: home win bias, avg goals, card rates.
Matches referee by name in match_officials (role_id=1 = Orta Hakem).
Uses Turkish-aware fuzzy matching as a fallback when exact name
lookup returns zero results.
"""
if not referee_name:
return dict(self._DEFAULT_REFEREE)
try:
# Get match IDs officiated by this referee
cur.execute(
"""
SELECT
m.home_team_id,
m.score_home,
m.score_away,
m.id AS match_id
FROM match_officials mo
JOIN matches m ON m.id = mo.match_id
WHERE mo.name = %s
AND mo.role_id = 1
AND m.status = 'FT'
AND m.score_home IS NOT NULL
AND m.score_away IS NOT NULL
AND m.mst_utc < %s
ORDER BY m.mst_utc DESC
LIMIT %s
""",
(referee_name, before_date_ms, limit),
rows = self._query_referee_matches(cur, referee_name, before_date_ms, limit)
# Fuzzy fallback: if exact match fails, try normalized name search
if not rows:
rows = self._fuzzy_referee_lookup(
cur, referee_name, before_date_ms, limit,
)
rows = cur.fetchall()
except Exception:
return dict(self._DEFAULT_REFEREE)
if not rows:
return dict(self._DEFAULT_REFEREE)
@@ -459,6 +474,118 @@ class FeatureEnrichmentService:
'experience': total,
}
def _query_referee_matches(
self,
cur: RealDictCursor,
referee_name: str,
before_date_ms: int,
limit: int,
) -> list:
"""Exact-match referee lookup in match_officials."""
try:
cur.execute(
"""
SELECT
m.home_team_id,
m.score_home,
m.score_away,
m.id AS match_id
FROM match_officials mo
JOIN matches m ON m.id = mo.match_id
WHERE mo.name = %s
AND mo.role_id = 1
AND m.status = 'FT'
AND m.score_home IS NOT NULL
AND m.score_away IS NOT NULL
AND m.mst_utc < %s
ORDER BY m.mst_utc DESC
LIMIT %s
""",
(referee_name, before_date_ms, limit),
)
return cur.fetchall()
except Exception:
return []
def _fuzzy_referee_lookup(
self,
cur: RealDictCursor,
referee_name: str,
before_date_ms: int,
limit: int,
) -> list:
"""
Fuzzy referee lookup using Turkish name normalization.
Strategy: fetch recent distinct referee names from match_officials,
normalize both the query name and each candidate, and pick the
best match. This handles common mismatches like:
- 'Hüseyin Göçek' vs 'Huseyin Gocek'
- 'Ali Palabıyık' vs 'Ali Palabiyik'
- Extra/missing middle initials
"""
normalized_query = _normalize_name(referee_name)
if not normalized_query:
return []
try:
# Fetch candidate referee names (distinct, recent, role=1)
cur.execute(
"""
SELECT DISTINCT mo.name
FROM match_officials mo
JOIN matches m ON m.id = mo.match_id
WHERE mo.role_id = 1
AND m.status = 'FT'
AND m.mst_utc < %s
ORDER BY mo.name
LIMIT 2000
""",
(before_date_ms,),
)
candidates = cur.fetchall()
except Exception:
return []
if not candidates:
return []
# Find best match by normalized name comparison
best_match: Optional[str] = None
best_score = 0.0
for cand_row in candidates:
cand_name = cand_row.get('name', '')
if not cand_name:
continue
normalized_cand = _normalize_name(cand_name)
# Exact normalized match
if normalized_cand == normalized_query:
best_match = cand_name
best_score = 1.0
break
# Substring containment (handles "First Last" vs "First M. Last")
if (
normalized_query in normalized_cand
or normalized_cand in normalized_query
):
containment_score = min(
len(normalized_query), len(normalized_cand)
) / max(len(normalized_query), len(normalized_cand))
if containment_score > best_score and containment_score > 0.6:
best_match = cand_name
best_score = containment_score
if not best_match:
return []
# Re-query with the resolved name
return self._query_referee_matches(
cur, best_match, before_date_ms, limit,
)
# ─── 5. League Averages ─────────────────────────────────────────
def compute_league_averages(
+367
View File
@@ -0,0 +1,367 @@
"""
Match Commentary Generator
===========================
Generates human-readable Turkish commentary from the analysis package.
Reads all engine signals (model, odds band, betting brain, triple value)
and produces a clear, actionable summary for end users.
No LLM required fully template-based.
"""
from __future__ import annotations
from typing import Any, Dict, List, Optional
def generate_match_commentary(package: Dict[str, Any]) -> Dict[str, Any]:
"""
Main entry point. Takes a full analysis package and returns a commentary dict.
Returns:
{
"action": "BET" | "WATCH" | "SKIP",
"headline": "...",
"summary": "...",
"notes": ["...", "..."],
"contradictions": ["...", "..."],
"confidence_label": "YÜKSEK" | "ORTA" | "DÜŞÜK" | "ÇOK DÜŞÜK"
}
"""
match_info = package.get("match_info") or {}
home = match_info.get("home_team", "Ev Sahibi")
away = match_info.get("away_team", "Deplasman")
main_pick = package.get("main_pick") or {}
betting_brain = package.get("betting_brain") or {}
v27_engine = package.get("v27_engine") or {}
market_board = package.get("market_board") or {}
score_pred = package.get("score_prediction") or {}
risk = package.get("risk") or {}
data_quality = package.get("data_quality") or {}
# ── Determine action ──────────────────────────────────────────
brain_decision = str(betting_brain.get("decision") or "NO_BET").upper()
main_playable = bool(main_pick.get("playable"))
main_vetoed = bool((main_pick.get("upper_brain") or {}).get("veto"))
approved_count = int(betting_brain.get("approved_count", 0) or 0)
if main_playable and not main_vetoed and approved_count > 0:
action = "BET"
elif approved_count == 0 and brain_decision == "NO_BET":
action = "SKIP"
else:
action = "WATCH"
# ── Headline ──────────────────────────────────────────────────
headline = _build_headline(action, main_pick, home, away)
# ── Summary paragraph ─────────────────────────────────────────
summary = _build_summary(
action, main_pick, market_board, v27_engine,
score_pred, risk, data_quality, home, away,
)
# ── Quick notes ───────────────────────────────────────────────
notes = _build_notes(market_board, v27_engine, score_pred, risk, home, away)
# ── Contradiction detection ───────────────────────────────────
contradictions = _detect_contradictions(market_board, v27_engine, package)
# ── Overall confidence label ──────────────────────────────────
confidence_label = _overall_confidence_label(main_pick, data_quality)
return {
"action": action,
"headline": headline,
"summary": summary,
"notes": notes[:6],
"contradictions": contradictions[:4],
"confidence_label": confidence_label,
}
# ═══════════════════════════════════════════════════════════════════════
# Headline
# ═══════════════════════════════════════════════════════════════════════
def _build_headline(
action: str,
main_pick: Dict[str, Any],
home: str,
away: str,
) -> str:
if action == "BET":
market = main_pick.get("market", "")
pick = main_pick.get("pick", "")
odds = main_pick.get("odds", 0.0)
conf = main_pick.get("calibrated_confidence", main_pick.get("confidence", 0))
market_tr = _market_to_turkish(market, pick)
return f"🎯 {market_tr} önerisi — Oran: {odds}, Güven: %{conf:.0f}"
if action == "WATCH":
return f"👀 {home} vs {away} — İzlemeye değer sinyaller var"
return f"⚠️ {home} vs {away} — Şu an net bir fırsat görülmüyor"
# ═══════════════════════════════════════════════════════════════════════
# Summary
# ═══════════════════════════════════════════════════════════════════════
def _build_summary(
action: str,
main_pick: Dict[str, Any],
market_board: Dict[str, Any],
v27_engine: Dict[str, Any],
score_pred: Dict[str, Any],
risk: Dict[str, Any],
data_quality: Dict[str, Any],
home: str,
away: str,
) -> str:
parts: List[str] = []
# Who is the favourite?
ms_board = market_board.get("MS") or {}
ms_pick = ms_board.get("pick", "")
ms_conf = float(ms_board.get("confidence", 50) or 50)
if ms_pick == "1" and ms_conf > 45:
parts.append(f"{home} hafif favori görünüyor")
elif ms_pick == "1" and ms_conf > 55:
parts.append(f"{home} net favori")
elif ms_pick == "2" and ms_conf > 45:
parts.append(f"{away} hafif favori görünüyor")
elif ms_pick == "2" and ms_conf > 55:
parts.append(f"{away} net favori")
else:
parts.append("İki takım da birbirine yakın güçte")
# xG expectation
xg_home = float(score_pred.get("xg_home", 0) or 0)
xg_away = float(score_pred.get("xg_away", 0) or 0)
xg_total = xg_home + xg_away
if xg_total > 3.0:
parts.append(f"Gol beklentisi yüksek (toplam xG: {xg_total:.1f})")
elif xg_total < 2.0:
parts.append(f"Düşük gol beklentisi (toplam xG: {xg_total:.1f})")
# Consensus check
consensus = str(v27_engine.get("consensus") or "").upper()
if consensus == "AGREE":
parts.append("Model motorları aynı fikirde")
elif consensus == "DISAGREE":
parts.append("Model motorları farklı sonuçlara ulaşıyor — belirsizlik var")
# Action-specific
if action == "BET":
market_tr = _market_to_turkish(
main_pick.get("market", ""), main_pick.get("pick", "")
)
edge = float(main_pick.get("ev_edge", 0) or 0)
parts.append(
f"{market_tr} yönünde değer tespit edildi (EV edge: {edge:+.1%})"
)
elif action == "SKIP":
parts.append(
"Hiçbir markette piyasanın fiyatlamadığı bir avantaj görülmüyor"
)
# Risk
risk_level = str(risk.get("level") or "MEDIUM").upper()
if risk_level == "HIGH":
parts.append("⚠️ Risk seviyesi yüksek")
elif risk_level == "EXTREME":
parts.append("🔴 Çok yüksek risk — dikkatli olun")
# Data quality
quality_label = str(data_quality.get("label") or "MEDIUM").upper()
if quality_label == "LOW":
parts.append("Veri kalitesi düşük — tahminler daha az güvenilir")
return ". ".join(parts) + "."
# ═══════════════════════════════════════════════════════════════════════
# Quick Notes
# ═══════════════════════════════════════════════════════════════════════
def _build_notes(
market_board: Dict[str, Any],
v27_engine: Dict[str, Any],
score_pred: Dict[str, Any],
risk: Dict[str, Any],
home: str,
away: str,
) -> List[str]:
notes: List[str] = []
triple_value = v27_engine.get("triple_value") or {}
odds_band = v27_engine.get("odds_band") or {}
# MS note
ms = market_board.get("MS") or {}
ms_conf = float(ms.get("confidence", 0) or 0)
if ms_conf < 45:
notes.append("Maç sonucu belirsiz, net favori yok")
elif ms.get("pick") == "1":
notes.append(f"{home} favori ama oran değerli mi kontrol et")
elif ms.get("pick") == "2":
notes.append(f"{away} favori ama oran değerli mi kontrol et")
# OU25 note
ou25 = market_board.get("OU25") or {}
ou25_probs = ou25.get("probs") or {}
over_prob = float(ou25_probs.get("over", 0.5) or 0.5)
if over_prob > 0.58:
notes.append("2.5 Üst yönünde eğilim var")
elif over_prob < 0.42:
notes.append("2.5 Alt yönünde eğilim var")
else:
notes.append("2.5 Üst/Alt dengeli — kesin sinyal yok")
# BTTS note
btts = market_board.get("BTTS") or {}
btts_probs = btts.get("probs") or {}
btts_yes = float(btts_probs.get("yes", 0.5) or 0.5)
if btts_yes > 0.58:
notes.append("Her iki takımın da gol atması bekleniyor")
elif btts_yes < 0.42:
notes.append("KG olasılığı düşük")
# HT note
ht = market_board.get("HT") or {}
ht_pick = ht.get("pick", "")
ht_conf = float(ht.get("confidence", 0) or 0)
if ht_conf > 40 and ht_pick:
ht_label = {"1": f"İY {home}", "2": f"İY {away}", "X": "İY beraberlik"}.get(
ht_pick, f"İY {ht_pick}"
)
notes.append(f"{ht_label} yönünde hafif sinyal (%{ht_conf:.0f})")
# Risk warnings
warnings = risk.get("warnings") or []
for w in warnings[:2]:
notes.append(f"⚠️ {w}")
return notes
# ═══════════════════════════════════════════════════════════════════════
# Contradiction Detection
# ═══════════════════════════════════════════════════════════════════════
def _detect_contradictions(
market_board: Dict[str, Any],
v27_engine: Dict[str, Any],
package: Dict[str, Any],
) -> List[str]:
"""
Detect cases where model prediction and odds band/triple value
point in opposite directions the user's main complaint.
"""
contradictions: List[str] = []
triple_value = v27_engine.get("triple_value") or {}
predictions = v27_engine.get("predictions") or {}
# MS contradiction: model says home but triple_value says away has value
ms_preds = predictions.get("ms") or {}
ms_home = float(ms_preds.get("home", 0) or 0)
ms_away = float(ms_preds.get("away", 0) or 0)
home_triple = triple_value.get("home") or {}
away_triple = triple_value.get("away") or {}
model_favours_home = ms_home > ms_away
away_is_value = bool(away_triple.get("is_value"))
home_is_value = bool(home_triple.get("is_value"))
if model_favours_home and away_is_value:
contradictions.append(
"Model ev sahibini favori görüyor ama oran bandı deplasmanda değer buluyor — "
"bu çelişki nedeniyle MS tahminine dikkatli yaklaş"
)
elif not model_favours_home and home_is_value:
contradictions.append(
"Model deplasmanı favori görüyor ama oran bandı ev sahibinde değer buluyor — "
"bu çelişki nedeniyle MS tahminine dikkatli yaklaş"
)
# HT contradiction
ht_board = market_board.get("HT") or {}
ht_pick = ht_board.get("pick", "")
ht_home_triple = triple_value.get("ht_home") or {}
ht_away_triple = triple_value.get("ht_away") or {}
if ht_pick == "1" and bool(ht_away_triple.get("is_value")):
contradictions.append(
"Model İY ev sahibi diyor ama oran bandı İY deplasmanda değer buluyor — "
"İY tahmini güvenilir değil"
)
elif ht_pick == "2" and bool(ht_home_triple.get("is_value")):
contradictions.append(
"Model İY deplasman diyor ama oran bandı İY ev sahibinde değer buluyor — "
"İY tahmini güvenilir değil"
)
# OU25 contradiction
ou25_board = market_board.get("OU25") or {}
ou25_pick = ou25_board.get("pick", "")
ou25_over_triple = triple_value.get("ou25_over") or {}
ou25_under_triple = triple_value.get("ou25_under") or {}
if ou25_pick == "Üst" and bool(ou25_under_triple.get("is_value")):
contradictions.append(
"Model 2.5 Üst diyor ama oran bandı 2.5 Alt'ta değer buluyor — çelişki var"
)
elif ou25_pick == "Alt" and bool(ou25_over_triple.get("is_value")):
contradictions.append(
"Model 2.5 Alt diyor ama oran bandı 2.5 Üst'te değer buluyor — çelişki var"
)
return contradictions
# ═══════════════════════════════════════════════════════════════════════
# Helpers
# ═══════════════════════════════════════════════════════════════════════
def _overall_confidence_label(
main_pick: Dict[str, Any],
data_quality: Dict[str, Any],
) -> str:
"""Overall confidence label for the entire analysis."""
quality_score = float(data_quality.get("score", 0.5) or 0.5)
main_conf = float(
main_pick.get("calibrated_confidence", main_pick.get("confidence", 0)) or 0
)
main_playable = bool(main_pick.get("playable"))
if main_playable and main_conf >= 60 and quality_score >= 0.8:
return "YÜKSEK"
if main_playable and main_conf >= 45:
return "ORTA"
if main_conf >= 30:
return "DÜŞÜK"
return "ÇOK DÜŞÜK"
_MARKET_TR_MAP = {
"MS": {"1": "Maç Sonucu Ev Sahibi", "2": "Maç Sonucu Deplasman", "X": "Beraberlik"},
"DC": {"1X": "Çifte Şans 1X", "X2": "Çifte Şans X2", "12": "Çifte Şans 12"},
"OU25": {"Üst": "2.5 Üst", "Alt": "2.5 Alt", "Over": "2.5 Üst", "Under": "2.5 Alt"},
"OU15": {"Üst": "1.5 Üst", "Alt": "1.5 Alt", "Over": "1.5 Üst", "Under": "1.5 Alt"},
"OU35": {"Üst": "3.5 Üst", "Alt": "3.5 Alt", "Over": "3.5 Üst", "Under": "3.5 Alt"},
"BTTS": {"KG Var": "Karşılıklı Gol Var", "KG Yok": "Karşılıklı Gol Yok",
"Yes": "Karşılıklı Gol Var", "No": "Karşılıklı Gol Yok"},
"HT": {"1": "İlk Yarı Ev Sahibi", "2": "İlk Yarı Deplasman", "X": "İlk Yarı Beraberlik"},
"HT_OU05": {"Üst": "İY 0.5 Üst", "Alt": "İY 0.5 Alt"},
"HT_OU15": {"Üst": "İY 1.5 Üst", "Alt": "İY 1.5 Alt"},
"OE": {"Tek": "Tek", "Çift": "Çift", "Odd": "Tek", "Even": "Çift"},
"CARDS": {"Üst": "Kart Üst", "Alt": "Kart Alt"},
}
def _market_to_turkish(market: str, pick: str) -> str:
market_map = _MARKET_TR_MAP.get(market, {})
result = market_map.get(pick)
if result:
return result
return f"{market} {pick}"
File diff suppressed because it is too large Load Diff
+27 -14
View File
@@ -1,6 +1,6 @@
# Social Poster Modülü — Otomatik Sosyal Medya Paylaşım Sistemi
Son güncelleme: 1 Mart 2026
Son güncelleme: 5 Mayıs 2026
---
@@ -13,11 +13,11 @@ Top liglerdeki maçların AI tahminlerini **otomatik olarak görselleştirip** I
## 2. Mimari Akış
```
Cron (*/10 dk) → LiveMatch sorgusu (top_leagues.json filtresi)
Cron (*/15 dk) → LiveMatch sorgusu (top_leagues.json filtresi)
→ AI Engine V20+ POST /v20plus/analyze/{match_id}
→ PredictionCardDto oluştur
→ Node Canvas ile 1080x1920 PNG render
→ Gemini ile Türkçe caption üret
→ Node Canvas ile futbol/basketbol 1080x1080 JPEG render
Ollama/Gemini ile Türkçe SEO uyumlu caption üret
→ Twitter / Facebook / Instagram API'ye paylaş
```
@@ -44,20 +44,25 @@ src/modules/social-poster/
### 4.1 SocialPosterService
**Cron:** Her 10 dakikada bir çalışır. 2540 dakika içinde başlayacak maçları `top_leagues.json` filtresiyle bulur.
**Cron:** Her 15 dakikada bir çalışır. Varsayılan olarak 2545 dakika içinde başlayacak futbol ve basketbol maçlarını `top_leagues.json` filtresiyle bulur.
**Tekrar paylaşım koruması:** Başarılı platform paylaşımı alan maç ID'leri `storage/social-poster-posted.json` içinde son 500 kayıt olarak tutulur. Servis restart sonrası aynı maç tekrar paylaşılmaz.
**Pipeline:** `predictAndPost(match)` → Tahmin al → Görsel üret → Caption üret → Paylaş
**AI Engine İsteği:**
```typescript
// POST — GET değil! AI Engine v20plus POST bekler.
axios.post(`${aiEngineUrl}/v20plus/analyze/${matchId}`, null, { timeout: 30000 })
axios.post(`${aiEngineUrl}/v20plus/analyze/${matchId}`, null, {
timeout: 30000,
});
```
**Veri Haritalandırma (V20+ → CardDto):**
| V20+ Response Alanı | CardDto Alanı |
|---|---|
| ----------------------- | ---------------------------------------------- |
| `score_prediction.ht` | `htScore` (ör: "1-1") |
| `score_prediction.ft` | `ftScore` (ör: "2-1") |
| `main_pick.confidence` | `scoreConfidence` (ör: 65) |
@@ -68,7 +73,7 @@ axios.post(`${aiEngineUrl}/v20plus/analyze/${matchId}`, null, { timeout: 30000 }
**Bet Summary Market Kodları:**
| Kod | Türkçe | English |
|---|---|---|
| ------- | --------------- | ----------------- |
| MS | Maç Sonucu | Match Result |
| OU15 | Üst 1.5 Gol | Over 1.5 |
| OU25 | Üst 2.5 Gol | Over 2.5 |
@@ -89,6 +94,7 @@ axios.post(`${aiEngineUrl}/v20plus/analyze/${matchId}`, null, { timeout: 30000 }
**Boyut:** 1080×1920 px (Instagram Story / Reels uyumlu)
**Özellikler:**
- Koyu gradient arka plan (#0a0e27#1a1040#0d1b2a)
- Lig adı + tarih başlık satırı
- Takım logoları (200×200px) — `public/uploads/teams/` altından okunur
@@ -100,6 +106,7 @@ axios.post(`${aiEngineUrl}/v20plus/analyze/${matchId}`, null, { timeout: 30000 }
- Alt bilgi: "⚡ AI Powered by SuggestBet"
**Logo Çözümleme:**
```
1. Yerel dosya varsa → public/uploads/teams/xxx.png oku
2. URL http ile başlıyorsa → HTTP ile indir
@@ -119,7 +126,7 @@ Gemini API kullanarak maç verisi JSON'ından Türkçe post metni üretir.
## 5. API Endpointleri
| Method | Path | Auth | Açıklama |
|---|---|---|---|
| ------ | ------------------------------------- | ------- | ---------------------------------------------------- |
| GET | `/api/social-poster/preview/:matchId` | @Public | Sadece görsel üret + caption üret (paylaşma) |
| POST | `/api/social-poster/post/:matchId` | @Public | Görsel üret + caption üret + tüm platformlara paylaş |
@@ -130,12 +137,18 @@ Gemini API kullanarak maç verisi JSON'ından Türkçe post metni üretir.
## 6. Environment Değişkenleri
| Key | Zorunlu | Varsayılan | Açıklama |
|---|---|---|---|
| --------------------------------------------- | ------- | ------------------------ | -------------------------------------------------------------------- |
| `AI_ENGINE_URL` | ✅ | `http://localhost:8000` | AI Engine base URL |
| `APP_BASE_URL` | ✅ | `http://localhost:3000` | Logo URL çözümleme için |
| `APP_BASE_URL` | ✅ | `http://localhost:3000` | Meta'nın çekebileceği public görsel URL'i ve logo URL çözümleme için |
| `SOCIAL_POSTER_ENABLED` | ❌ | `false` | Cron job'ı aktif/pasif |
| `SOCIAL_POSTER_SPORTS` | ❌ | `football,basketball` | Otomatik paylaşılacak sporlar |
| `SOCIAL_POSTER_WINDOW_MIN` | ❌ | `25` | Başlama zaman penceresi alt sınırı (dakika) |
| `SOCIAL_POSTER_WINDOW_MAX` | ❌ | `45` | Başlama zaman penceresi üst sınırı (dakika) |
| `OLLAMA_BASE_URL` | ❌ | `http://localhost:11434` | Lokal LLM endpoint'i |
| `OLLAMA_MODEL` / `SOCIAL_POSTER_OLLAMA_MODEL` | ❌ | — | Caption üretiminde kullanılacak lokal model |
| `GOOGLE_API_KEY` | ❌ | — | Gemini caption için |
| Twitter API keys | ❌ | — | Twitter paylaşım için |
| Twitter API keys | ❌ | — | X medya upload + `/2/tweets` paylaşımı için OAuth 1.0a user context |
| `META_GRAPH_API_VERSION` | ❌ | `v25.0` | Meta Graph API sürümü |
| Meta API keys | ❌ | — | FB/IG paylaşım için |
---
@@ -166,7 +179,7 @@ RUN apk add --no-cache cairo-dev pango-dev jpeg-dev giflib-dev librsvg-dev
### Port Yönetimi
| Servis | Port |
|---|---|
| -------------- | ------------------------------------------- |
| NestJS Backend | 3000 (production: 150X) |
| AI Engine | 8000 (dev: 8005 — Windows port kısıtlaması) |
@@ -183,7 +196,7 @@ public/
## 9. Bilinen Sorunlar & Çözümler
| Sorun | Sebep | Çözüm |
|---|---|---|
| --------------------------------------- | ------------------------------------ | ----------------------------------------- |
| `WinError 10013` port erişim hatası | Windows Hyper-V port rezervasyonu | Farklı port kullan (8005) |
| `Invalid prisma.liveMatch.findUnique()` | Prisma client eskimiş | `npx prisma generate` çalıştır |
| `405 Method Not Allowed` AI Engine | GET yerine POST gerekiyor | `axios.post()` kullan |
File diff suppressed because one or more lines are too long
+370
View File
@@ -0,0 +1,370 @@
# V28-Pro-Max Model Architecture Documentation
> **Model Version:** `v28-pro-max`
> **Engine File:** `ai-engine/services/single_match_orchestrator.py` (4656 satır)
> **Son Güncelleme:** 2026-04-24
---
## 1. Genel Bakış
V28-Pro-Max, üç bağımsız tahmin katmanını (V25, V27, V28) tek bir orchestrator içinde birleştiren **üçlü hibrit AI tahmin motorudur**. Her maç için 13+ bahis pazarını analiz eder, olasılık hesaplar, risk değerlendirir ve "Value Bet" tespiti yapar.
```
┌─────────────────────────────────────────────────────┐
│ SingleMatchOrchestrator │
│ │
│ ┌──────────┐ ┌──────────┐ ┌────────────────┐ │
│ │ V25 │ │ V27 │ │ V28 │ │
│ │ Ensemble │ │ Dual-Eng │ │ Odds-Band │ │
│ │ (XGB+LGB)│ │ Divergnce│ │ Historical │ │
│ └────┬─────┘ └────┬─────┘ └───────┬────────┘ │
│ │ │ │ │
│ └──────────────┼────────────────┘ │
│ ▼ │
│ FullMatchPrediction │
│ │ │
│ ┌───────────┼───────────┐ │
│ ▼ ▼ ▼ │
│ Market Rows Risk Calc Triple Value │
│ │ │ │ │
│ └───────────┼───────────┘ │
│ ▼ │
│ _build_prediction_package() │
│ → JSON Response (v28-pro-max) │
└─────────────────────────────────────────────────────┘
```
---
## 2. Katman Detayları
### 2.1 V25 — Ensemble ML Katmanı
**Dosya:** `ai-engine/models/v25_ensemble.py`
- **Algoritmalar:** XGBoost + LightGBM ensemble
- **Girdi:** Pre-match feature vektörü (form, elo, odds, kadro, hakem vb.)
- **Çıktı:** Tüm pazarlar için olasılık dağılımları + confidence skorları
- **Özellik:** Odds-aware (bahis oranlarını feature olarak kullanır)
- **Target leakage koruması:** Maç sonucu bilgisi asla feature olarak kullanılmaz
```python
# V25 çağrılma noktası (orchestrator L310-315)
v25_signal = v25_predictor.predict(features)
# Çıktı: {MS: {home: 0.45, draw: 0.28, away: 0.27}, OU25: {...}, BTTS: {...}, ...}
```
### 2.2 V27 — Dual-Engine Divergence Katmanı
**Dosya:** `ai-engine/models/v27_predictor.py`
- **Amaç:** Odds-FREE temel olasılıkları hesaplar (sadece form/elo/kadro)
- **Mekanizma:** V25 (odds-aware) vs V27 (odds-free) karşılaştırması
- **Divergence Tespiti:** İki motor arasındaki fark → bahisçinin fiyatlandırma hatasını tespit eder
- **Çıktı:** `ms_divergence`, `ou25_divergence`, `is_value` sinyalleri
```python
# Divergence hesaplama (orchestrator L830-863)
ms_divergence = {
"home": v25_home_prob - v27_home_prob, # Pozitif = V25 bahisçiyle hemfikir
"away": v25_away_prob - v27_away_prob, # Negatif = Model bahisçiden farklı düşünüyor
}
ms_value = {
"home": {"is_value": v27_home > implied_home and abs(div) > 0.05},
"away": {"is_value": v27_away > implied_away and abs(div) > 0.05},
}
```
### 2.3 V28 — Odds-Band Historical Performance Katmanı
**Dosya:** `ai-engine/features/odds_band_analyzer.py`
- **Amaç:** "Bu oran bandında tarihsel olarak ne oldu?" sorusunu yanıtlar
- **Mekanizma:** Maçın mevcut oranını bir banda yerleştirir (ör: MS Home 1.70-1.90), ardından veritabanındaki aynı banddaki geçmiş maçları sorgular
- **Sorgu:** PostgreSQL üzerinden takım-spesifik tarihsel performans
```python
# OddsBandAnalyzer.compute_all() çıktısı — 18 pazar için band metrikleri:
{
"home_band_ms_win_rate": 0.62, # Ev sahibi bu oran bandında %62 kazanmış
"home_band_ms_sample": 34, # 34 maçlık örneklem
"band_ou25_over_rate": 0.58, # Bu banddaki maçların %58'i 2.5 üst
"band_btts_yes_rate": 0.51, # KG Var oranı
"band_htft_11_rate": 0.28, # İY/MS 1/1 oranı
"band_cards_referee_avg": 4.2, # Hakem kart ortalaması
# ... toplam 60+ feature
}
```
---
## 3. Analiz Edilen Bahis Pazarları (13+)
| # | Pazar | Kod | Olasılık Alanları | Odds Anahtarları |
|---|-------|-----|-------------------|------------------|
| 1 | Maç Sonucu | `MS` | home/draw/away | ms_h, ms_d, ms_a |
| 2 | Çifte Şans | `DC` | 1X/X2/12 | dc_1x, dc_x2, dc_12 |
| 3 | Üst/Alt 1.5 | `OU15` | over/under | ou15_o, ou15_u |
| 4 | Üst/Alt 2.5 | `OU25` | over/under | ou25_o, ou25_u |
| 5 | Üst/Alt 3.5 | `OU35` | over/under | ou35_o, ou35_u |
| 6 | Karşılıklı Gol | `BTTS` | yes/no | btts_y, btts_n |
| 7 | İlk Yarı Sonucu | `HT` | 1/X/2 | ht_h, ht_d, ht_a |
| 8 | İY/MS (9 kombo) | `HTFT` | 1/1, 1/X, 1/2, X/1, X/X, X/2, 2/1, 2/X, 2/2 | htft_11..htft_22 |
| 9 | Tek/Çift | `OE` | odd/even | oe_odd, oe_even |
| 10 | İY Üst/Alt 0.5 | `HT_OU05` | over/under | ht_ou05_o, ht_ou05_u |
| 11 | İY Üst/Alt 1.5 | `HT_OU15` | over/under | ht_ou15_o, ht_ou15_u |
| 12 | Kartlar | `CARDS` | over/under | cards_o, cards_u |
| 13 | Handikap | `HCAP` | 1/X/2 | hcap_h, hcap_d, hcap_a |
---
## 4. Triple Value Detection (V28 Ana Yeniliği)
V28'in en kritik özelliği: **3 bağımsız kaynağı çapraz kontrol ederek "gerçek değer" tespiti yapması.**
```
Triple Value = V27 Divergence + V28 Band Rate + Odds Implied Probability
Koşullar (hepsi sağlanmalı):
1. V27 olasılığı > bahisçi implied olasılığı (v27_confirms)
2. Band tarihsel oranı > implied olasılık (band_confirms)
3. Kombine edge > %5 (edge > 0.05)
4. Band örneklem >= 8 maç (band_sample >= 8)
→ Tüm koşullar sağlanırsa: is_value = True
```
**Örnek:**
```
Galatasaray vs Beşiktaş — MS Home (1.85 oran)
├── Implied Prob: 1/1.85 = 0.54 (%54)
├── V27 (odds-free): 0.61 (%61) → ✅ V27 confirms (0.61 > 0.54)
├── V28 Band Rate: 0.62 (%62, 34 maç) → ✅ Band confirms (0.62 > 0.54)
├── Combined Prob: (0.61 + 0.62) / 2 = 0.615
├── Edge: 0.615 - 0.54 = 0.075 (%7.5) → ✅ Edge > 5%
└── is_value = TRUE → "Bu bahis değerli!"
```
---
## 5. Market Row Dekorasyon Pipeline'ı
Her pazar aşağıdaki pipeline'dan geçer:
```
_build_market_rows() → Ham market row'ları oluştur (13 pazar)
_apply_market_consistency() → Pazarlar arası tutarlılık kontrolü
_decorate_market_row() → Her row'a playability, grading, staking ekle
Sort by (playable, play_score) → En iyi pick'ler başa gelir
```
### 5.1 Decorate Market Row — Quant Hybrid Sistemi
Her market row şu metriklerle dekore edilir:
| Metrik | Formül | Açıklama |
|--------|--------|----------|
| `calibrated_confidence` | `raw_conf × market_calibration` | Kalibre edilmiş güven |
| `ev_edge` | `(prob × odds) - 1.0` | Expected Value edge |
| `simple_edge` | `prob - (1/odds)` | Basit olasılık farkı |
| `play_score` | `cal_conf + (edge × 100 × edge_mult) - penalties` | Oynanabilirlik skoru |
| `stake_units` | Quarter-Kelly Criterion | Önerilen bahis miktarı |
| `bet_grade` | A/B/C/PASS | EV edge bazlı not |
### 5.2 Playability Gates (Güvenlik Kapıları)
Bir market row'un "playable" olması için tüm kapılardan geçmesi gerekir:
1. **Confidence Gate:** `calibrated_conf >= min_conf` (pazar bazlı eşik)
2. **Odds Gate:** Odds-required pazarlarda `odds > 1.01`
3. **Risk-Quality Gate:** HIGH/EXTREME risk + LOW kalite → BLOK
4. **Negative Edge Gate:** `simple_edge < neg_threshold` → BLOK
5. **EV Edge Gate:** `ev_edge < min_edge` → BLOK
6. **Play Score Gate:** `play_score < min_play_score` → BLOK
### 5.3 Kelly Criterion Staking
```python
# Quarter-Kelly (¼ Kelly, 10-unit bankroll)
f* = ((b × p) - q) / b # Full Kelly
stake = f* × 0.25 × 10 # Quarter Kelly × bankroll
stake = min(stake, 3.0) # Cap: max 3 unit
stake = max(stake, 0.25) # Floor: min 0.25 unit
```
---
## 6. Guaranteed Pick Logic (V32 Calibration-Aware)
Ana pick seçimi 4 öncelik sırasıyla yapılır:
```
Priority 1: HIGH_ACCURACY markets (DC, OU15, HT_OU05)
+ Odds >= 1.30 + Confidence >= 44%
→ is_guaranteed = True, reason = "high_accuracy_market"
Priority 2: Any playable + Odds >= 1.30 + Conf >= 44%
→ is_guaranteed = True, reason = "confidence_threshold_met"
Priority 3: Any playable + Odds >= 1.30
→ is_guaranteed = False, reason = "odds_only_fallback"
Priority 4: Best non-playable (last resort)
→ is_guaranteed = False, reason = "last_resort"
```
**Value Pick:** `main_pick`'ten farklı, odds >= 1.60, confidence >= %40 olan en iyi alternatif.
**Aggressive Pick:** HT/FT reversal senaryoları (1/2, 2/1, X/1, X/2) arasından en yüksek olasılıklı.
---
## 7. Risk Assessment Sistemi
```python
risk_score = 100 - max_market_conf + lineup_penalty + referee_penalty + parity_penalty
# Penalty'ler:
lineup_penalty = 12.0 (kadro yok) | 7.0 (probable_xi) | 0.0 (confirmed)
referee_penalty = 6.0 (hakem yok) | 0.0
parity_penalty = 8.0 (|ms_edge| < 0.08) | 0.0
# Risk seviyeleri:
EXTREME: score >= 78
HIGH: score >= 62
MEDIUM: score >= 40
LOW: score < 40
```
### Surprise Risk Tespiti
- `is_surprise_risk = True` → Risk HIGH/EXTREME VEYA draw_prob >= %30
- `surprise_type`: `balanced_match_risk` veya `draw_pressure`
---
## 8. xG ve Skor Tahmini
```python
base_home_xg = (home_goals_avg + away_xga) / 2
base_away_xg = (away_goals_avg + home_xga) / 2
# MS edge ve BTTS etkisiyle düzeltme:
home_xg = base_home_xg + (ms_edge × 0.55) + (btts_prob - 0.5) × 0.18
away_xg = base_away_xg - (ms_edge × 0.55) + (btts_prob - 0.5) × 0.18
# Liga ortalamasıyla ölçekleme:
total_target = league_avg_goals × 0.55 + team_avgs × 0.45 + ou25_signal × 1.15
scale = total_target / (home_xg + away_xg)
final_home_xg = home_xg × scale
final_away_xg = away_xg × scale
# Skor tahmini:
FT = round(home_xg) - round(away_xg)
HT = round(home_xg × 0.45) - round(away_xg × 0.45)
Top5 = Poisson dağılımı ile en olası 5 skor
```
---
## 9. Data Quality Skoru
```python
quality_score = odds_score × 0.35 + lineup_score × 0.35 + ref_score × 0.15 + form_score × 0.15
# Etiketleme:
HIGH: score >= 0.75
MEDIUM: score >= 0.45
LOW: score < 0.45
```
---
## 10. Çıktı JSON Kontratı
```json
{
"model_version": "v28-pro-max",
"match_info": { "match_id", "home_team", "away_team", "league", ... },
"data_quality": { "label", "score", "lineup_source", "flags" },
"risk": { "level", "score", "is_surprise_risk", "warnings" },
"engine_breakdown": { "team", "player", "odds", "referee" },
"main_pick": { "market", "pick", "confidence", "odds", "ev_edge", "bet_grade", "is_guaranteed" },
"value_pick": { ... },
"aggressive_pick": { "market": "HT/FT", "pick": "1/2", ... },
"bet_advice": { "playable", "suggested_stake_units", "reason" },
"bet_summary": [ { "market", "pick", "calibrated_confidence", "bet_grade", "ev_edge", ... } ],
"supporting_picks": [ ... ],
"score_prediction": { "ft", "ht", "xg_home", "xg_away", "xg_total" },
"scenario_top5": [ "1-0", "2-1", ... ],
"market_board": { "MS": {...}, "DC": {...}, "OU25": {...}, ... },
"v25_signal": { "available", "markets", "value_bets" },
"reasoning_factors": [ ... ]
}
```
---
## 11. League-Specific Odds Reliability (V31)
Bazı liglerin bahis oranları daha güvenilirdir. Bu bilgi `_decorate_market_row` içinde edge ağırlıklandırmasında kullanılır:
```python
odds_rel = league_reliability.get(league_id, 0.35) # 0.0 - 1.0
edge_multiplier = 0.60 + (odds_rel × 0.60) # 0.60 - 1.20
# Güvenilir lig → edge daha fazla ağırlık alır
# Güvenilsiz lig → model confidence'a daha çok güvenilir
```
---
## 12. Dosya Haritası
```
ai-engine/
├── services/
│ └── single_match_orchestrator.py ← Ana orchestrator (4656 satır)
├── models/
│ ├── v25_ensemble.py ← XGBoost + LightGBM ensemble
│ └── v27_predictor.py ← Odds-free fundamental predictor
├── features/
│ └── odds_band_analyzer.py ← V28 tarihsel band analizi
└── main.py ← FastAPI endpoint (/predict)
```
---
## 13. Akış Özeti
```
HTTP POST /predict {match_id}
SingleMatchOrchestrator.analyze_match(match_id)
├── _load_match_data() → DB'den maç + odds + kadro + form
├── V25: v25_predictor.predict(features)
│ → 13 pazar olasılık + confidence
├── V27: v27_predictor.predict(features)
│ → Odds-free MS/OU25 olasılıkları
│ → Divergence hesaplama
├── V28: odds_band_analyzer.compute_all()
│ → 18 pazar için tarihsel band metrikleri
├── Triple Value Detection
│ → V27 + V28 + Implied çapraz kontrol
├── _enrich_prediction() → xG, risk, skor tahmini
├── _build_market_rows() → 13+ ham market row
├── _apply_market_consistency()
├── _decorate_market_row() → EV, Kelly, grading
├── Guaranteed Pick Selection → main_pick, value_pick, aggressive_pick
└── _build_prediction_package() → Final JSON kontratı
```
+15 -39
View File
@@ -26,7 +26,7 @@
"@nestjs/swagger": "^11.2.4",
"@nestjs/terminus": "^11.0.0",
"@nestjs/throttler": "^6.5.0",
"@prisma/client": "^5.22.0",
"@prisma/client": "5.22.0",
"axios": "^1.13.6",
"bcrypt": "^6.0.0",
"bullmq": "^5.66.4",
@@ -46,7 +46,7 @@
"passport-jwt": "^4.0.1",
"pino": "^10.1.0",
"pino-http": "^11.0.0",
"prisma": "^5.22.0",
"prisma": "5.22.0",
"reflect-metadata": "^0.2.2",
"rxjs": "^7.8.1",
"twitter-api-v2": "^1.29.0",
@@ -1145,7 +1145,6 @@
"resolved": "https://registry.npmjs.org/@babel/core/-/core-7.28.5.tgz",
"integrity": "sha512-e7jT4DxYvIDLk1ZHmU/m/mB19rex9sv0c2ftBtjSBv+kVM/902eh0fINUzD7UwLLNR+jU585GxUJ8/EBfAM5fw==",
"dev": true,
"peer": true,
"dependencies": {
"@babel/code-frame": "^7.27.1",
"@babel/generator": "^7.28.5",
@@ -3001,7 +3000,6 @@
"resolved": "https://registry.npmjs.org/@nestjs/axios/-/axios-4.0.1.tgz",
"integrity": "sha512-68pFJgu+/AZbWkGu65Z3r55bTsCPlgyKaV4BSG8yUAD72q1PPuyVRgUwFv6BxdnibTUHlyxm06FmYWNC+bjN7A==",
"license": "MIT",
"peer": true,
"peerDependencies": {
"@nestjs/common": "^10.0.0 || ^11.0.0",
"axios": "^1.3.1",
@@ -3095,7 +3093,6 @@
"resolved": "https://registry.npmjs.org/ajv/-/ajv-8.17.1.tgz",
"integrity": "sha512-B/gBuNg5SiMTrPkC+A2+cW0RszwxYmn6VYxB/inlBStS5nx6xHIt/ehKRhIMhqusl7a8LjQoZnjCs5vhwxOQ1g==",
"dev": true,
"peer": true,
"dependencies": {
"fast-deep-equal": "^3.1.3",
"fast-uri": "^3.0.1",
@@ -3262,7 +3259,6 @@
"version": "11.1.11",
"resolved": "https://registry.npmjs.org/@nestjs/common/-/common-11.1.11.tgz",
"integrity": "sha512-R/+A8XFqLgN8zNs2twhrOaE7dJbRQhdPX3g46am4RT/x8xGLqDphrXkUIno4cGUZHxbczChBAaAPTdPv73wDZA==",
"peer": true,
"dependencies": {
"file-type": "21.2.0",
"iterare": "1.2.1",
@@ -3308,7 +3304,6 @@
"resolved": "https://registry.npmjs.org/@nestjs/core/-/core-11.1.11.tgz",
"integrity": "sha512-H9i+zT3RvHi7tDc+lCmWHJ3ustXveABCr+Vcpl96dNOxgmrx4elQSTC4W93Mlav2opfLV+p0UTHY6L+bpUA4zA==",
"hasInstallScript": true,
"peer": true,
"dependencies": {
"@nuxt/opencollective": "0.4.1",
"fast-safe-stringify": "2.1.1",
@@ -3388,7 +3383,6 @@
"version": "11.1.11",
"resolved": "https://registry.npmjs.org/@nestjs/platform-express/-/platform-express-11.1.11.tgz",
"integrity": "sha512-kyABSskdMRIAMWL0SlbwtDy4yn59RL4HDdwHDz/fxWuv7/53YP8Y2DtV3/sHqY5Er0msMVTZrM38MjqXhYL7gw==",
"peer": true,
"dependencies": {
"cors": "2.8.5",
"express": "5.2.1",
@@ -3409,7 +3403,6 @@
"version": "11.1.11",
"resolved": "https://registry.npmjs.org/@nestjs/platform-socket.io/-/platform-socket.io-11.1.11.tgz",
"integrity": "sha512-0z6pLg9CuTXtz7q2lRZoPOU94DN28OTa39f4cQrlZysKA6QrKM7w7z6xqb4g32qjF+LQHFNRmMJtE/pLrxBaig==",
"peer": true,
"dependencies": {
"socket.io": "4.8.3",
"tslib": "2.8.1"
@@ -3784,7 +3777,6 @@
"resolved": "https://registry.npmjs.org/@prisma/client/-/client-5.22.0.tgz",
"integrity": "sha512-M0SVXfyHnQREBKxCgyo7sffrKttwE6R8PMq330MIUF0pTwjUhLbW84pFDlf06B27XyCR++VtjugEnIHdr07SVA==",
"hasInstallScript": true,
"peer": true,
"engines": {
"node": ">=16.13"
},
@@ -3849,7 +3841,6 @@
"version": "1.6.1",
"resolved": "https://registry.npmjs.org/@redis/client/-/client-1.6.1.tgz",
"integrity": "sha512-/KCsg3xSlR+nCK8/8ZYSknYxvXHwubJrU82F3Lm1Fp6789VQ0/3RJKfsmRXjqfaTA++23CvC3hqmqe/2GEt6Kw==",
"peer": true,
"dependencies": {
"cluster-key-slot": "1.1.2",
"generic-pool": "3.9.0",
@@ -4755,7 +4746,6 @@
"resolved": "https://registry.npmjs.org/@types/eslint/-/eslint-9.6.1.tgz",
"integrity": "sha512-FXx2pKgId/WyYo2jXw63kk7/+TY7u7AziEJxJAnSFzHlqTAS3Ync6SvgYAN/k4/PQpnnVuzoMuVnByKK2qp0ag==",
"dev": true,
"peer": true,
"dependencies": {
"@types/estree": "*",
"@types/json-schema": "*"
@@ -4877,7 +4867,6 @@
"version": "22.19.3",
"resolved": "https://registry.npmjs.org/@types/node/-/node-22.19.3.tgz",
"integrity": "sha512-1N9SBnWYOJTrNZCdh/yJE+t910Y128BoyY+zBLWhL3r0TYzlTmFdXrPwHL9DyFZmlEXNQQolTZh3KHV31QDhyA==",
"peer": true,
"dependencies": {
"undici-types": "~6.21.0"
}
@@ -5042,7 +5031,6 @@
"resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-8.52.0.tgz",
"integrity": "sha512-iIACsx8pxRnguSYhHiMn2PvhvfpopO9FXHyn1mG5txZIsAaB6F0KwbFnUQN3KCiG3Jcuad/Cao2FAs1Wp7vAyg==",
"dev": true,
"peer": true,
"dependencies": {
"@typescript-eslint/scope-manager": "8.52.0",
"@typescript-eslint/types": "8.52.0",
@@ -5680,7 +5668,6 @@
"resolved": "https://registry.npmjs.org/acorn/-/acorn-8.15.0.tgz",
"integrity": "sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==",
"dev": true,
"peer": true,
"bin": {
"acorn": "bin/acorn"
},
@@ -5734,7 +5721,6 @@
"resolved": "https://registry.npmjs.org/ajv/-/ajv-6.12.6.tgz",
"integrity": "sha512-j3fVLgvTo527anyYyJOGTYJbG+vnnQYvE0m5mmkc1TK+nxAppkCLMIL0aZ4dblVCNoGShhm+kzE4ZUykBoMg4g==",
"dev": true,
"peer": true,
"dependencies": {
"fast-deep-equal": "^3.1.1",
"fast-json-stable-stringify": "^2.0.0",
@@ -5926,7 +5912,6 @@
"resolved": "https://registry.npmjs.org/axios/-/axios-1.13.6.tgz",
"integrity": "sha512-ChTCHMouEe2kn713WHbQGcuYrr6fXTBiu460OTwWrWob16g1bXn4vtz07Ope7ewMozJAnEquLk5lWQWtBig9DQ==",
"license": "MIT",
"peer": true,
"dependencies": {
"follow-redirects": "^1.15.11",
"form-data": "^4.0.5",
@@ -6240,7 +6225,6 @@
"url": "https://github.com/sponsors/ai"
}
],
"peer": true,
"dependencies": {
"baseline-browser-mapping": "^2.9.0",
"caniuse-lite": "^1.0.30001759",
@@ -6313,7 +6297,6 @@
"version": "5.66.4",
"resolved": "https://registry.npmjs.org/bullmq/-/bullmq-5.66.4.tgz",
"integrity": "sha512-y2VRk2z7d1YNI2JQDD7iThoD0X/0iZZ3VEp8lqT5s5U0XDl9CIjXp1LQgmE9EKy6ReHtzmYXS1f328PnUbZGtQ==",
"peer": true,
"dependencies": {
"cron-parser": "4.9.0",
"ioredis": "5.8.2",
@@ -6387,7 +6370,6 @@
"version": "7.2.7",
"resolved": "https://registry.npmjs.org/cache-manager/-/cache-manager-7.2.7.tgz",
"integrity": "sha512-TKeeb9nSybk1e9E5yAiPVJ6YKdX9FYhwqqy8fBfVKAFVTJYZUNmeIvwjURW6+UikNsO6l2ta27thYgo/oumDsw==",
"peer": true,
"dependencies": {
"@cacheable/utils": "^2.3.2",
"keyv": "^5.5.4"
@@ -6601,7 +6583,6 @@
"resolved": "https://registry.npmjs.org/chokidar/-/chokidar-4.0.3.tgz",
"integrity": "sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA==",
"dev": true,
"peer": true,
"dependencies": {
"readdirp": "^4.0.1"
},
@@ -6651,14 +6632,12 @@
"node_modules/class-transformer": {
"version": "0.5.1",
"resolved": "https://registry.npmjs.org/class-transformer/-/class-transformer-0.5.1.tgz",
"integrity": "sha512-SQa1Ws6hUbfC98vKGxZH3KFY0Y1lm5Zm0SY8XX9zbK7FJCyVEac3ATW0RIpwzW+oOfmHE5PMPufDG9hCfoEOMw==",
"peer": true
"integrity": "sha512-SQa1Ws6hUbfC98vKGxZH3KFY0Y1lm5Zm0SY8XX9zbK7FJCyVEac3ATW0RIpwzW+oOfmHE5PMPufDG9hCfoEOMw=="
},
"node_modules/class-validator": {
"version": "0.14.3",
"resolved": "https://registry.npmjs.org/class-validator/-/class-validator-0.14.3.tgz",
"integrity": "sha512-rXXekcjofVN1LTOSw+u4u9WXVEUvNBVjORW154q/IdmYWy1nMbOU9aNtZB0t8m+FJQ9q91jlr2f9CwwUFdFMRA==",
"peer": true,
"dependencies": {
"@types/validator": "^13.15.3",
"libphonenumber-js": "^1.11.1",
@@ -7497,7 +7476,8 @@
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-2.0.0.tgz",
"integrity": "sha512-5POEcUuZybH7IdmGsD8wlf0AI55wMecM9rVBTI/qEAy2c1kTOm3DjFYjrBdI2K3BaJjJYfYFeRtM0t9ssnRuxw==",
"dev": true
"dev": true,
"peer": true
},
"node_modules/es-object-atoms": {
"version": "1.1.1",
@@ -7555,7 +7535,6 @@
"resolved": "https://registry.npmjs.org/eslint/-/eslint-9.39.2.tgz",
"integrity": "sha512-LEyamqS7W5HB3ujJyvi0HQK/dtVINZvd5mAAp9eT5S/ujByGjiZLCzPcHVzuXbpJDJF/cxwHlfceVUDZ2lnSTw==",
"dev": true,
"peer": true,
"dependencies": {
"@eslint-community/eslint-utils": "^4.8.0",
"@eslint-community/regexpp": "^4.12.1",
@@ -7615,7 +7594,6 @@
"resolved": "https://registry.npmjs.org/eslint-config-prettier/-/eslint-config-prettier-10.1.8.tgz",
"integrity": "sha512-82GZUjRS0p/jganf6q1rEO25VSoHH0hKPCTrgillPjdI/3bgBhAE1QzHrHTizjpRvy6pGAvKjDJtk2pF9NDq8w==",
"dev": true,
"peer": true,
"bin": {
"eslint-config-prettier": "bin/cli.js"
},
@@ -7846,7 +7824,6 @@
"version": "5.2.1",
"resolved": "https://registry.npmjs.org/express/-/express-5.2.1.tgz",
"integrity": "sha512-hIS4idWWai69NezIdRt2xFVofaF4j+6INOpJlVOLDO8zXGpUVEVzIYk12UUi2JzjEzWL3IOAxcTubgz9Po0yXw==",
"peer": true,
"dependencies": {
"accepts": "^2.0.0",
"body-parser": "^2.2.1",
@@ -9051,7 +9028,6 @@
"resolved": "https://registry.npmjs.org/jest/-/jest-30.2.0.tgz",
"integrity": "sha512-F26gjC0yWN8uAA5m5Ss8ZQf5nDHWGlN/xWZIh8S5SRbsEKBovwZhxGd6LJlbZYxBgCYOtreSUyb8hpXyGC5O4A==",
"dev": true,
"peer": true,
"dependencies": {
"@jest/core": "30.2.0",
"@jest/types": "30.2.0",
@@ -9895,7 +9871,6 @@
"version": "5.5.5",
"resolved": "https://registry.npmjs.org/keyv/-/keyv-5.5.5.tgz",
"integrity": "sha512-FA5LmZVF1VziNc0bIdCSA1IoSVnDCqE8HJIZZv2/W8YmoAM50+tnUgJR/gQZwEeIMleuIOnRnHA/UaZRNeV4iQ==",
"peer": true,
"dependencies": {
"@keyv/serialize": "^1.1.1"
}
@@ -10688,6 +10663,7 @@
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/object-hash/-/object-hash-3.0.0.tgz",
"integrity": "sha512-RSn9F68PjH9HqtltsSnqYC1XXoWe9Bju5+213R98cNGttag9q9yAOTzdbsqvIa7aNm5WffBZFpWYr2aWrklWAw==",
"peer": true,
"engines": {
"node": ">= 6"
}
@@ -10920,7 +10896,6 @@
"version": "0.7.0",
"resolved": "https://registry.npmjs.org/passport/-/passport-0.7.0.tgz",
"integrity": "sha512-cPLl+qZpSc+ireUvt+IzqbED1cHHkDoVYMo30jbJIdOOjQ1MQYZBPiNvmi8UM6lJuOpTPXJGZQk0DtC4y61MYQ==",
"peer": true,
"dependencies": {
"passport-strategy": "1.x.x",
"pause": "0.0.1",
@@ -11047,7 +11022,6 @@
"version": "10.1.0",
"resolved": "https://registry.npmjs.org/pino/-/pino-10.1.0.tgz",
"integrity": "sha512-0zZC2ygfdqvqK8zJIr1e+wT1T/L+LF6qvqvbzEQ6tiMAoTqEVK9a1K3YRu8HEUvGEvNqZyPJTtb2sNIoTkB83w==",
"peer": true,
"dependencies": {
"@pinojs/redact": "^0.4.0",
"atomic-sleep": "^1.0.0",
@@ -11077,7 +11051,6 @@
"version": "11.0.0",
"resolved": "https://registry.npmjs.org/pino-http/-/pino-http-11.0.0.tgz",
"integrity": "sha512-wqg5XIAGRRIWtTk8qPGxkbrfiwEWz1lgedVLvhLALudKXvg1/L2lTFgTGPJ4Z2e3qcRmxoFxDuSdMdMGNM6I1g==",
"peer": true,
"dependencies": {
"get-caller-file": "^2.0.5",
"pino": "^10.0.0",
@@ -11286,7 +11259,6 @@
"resolved": "https://registry.npmjs.org/prettier/-/prettier-3.7.4.tgz",
"integrity": "sha512-v6UNi1+3hSlVvv8fSaoUbggEM5VErKmmpGA7Pl3HF8V6uKY7rvClBOJlH6yNwQtfTueNkGVpOv/mtWL9L4bgRA==",
"dev": true,
"peer": true,
"bin": {
"prettier": "bin/prettier.cjs"
},
@@ -11340,7 +11312,6 @@
"resolved": "https://registry.npmjs.org/prisma/-/prisma-5.22.0.tgz",
"integrity": "sha512-vtpjW3XuYCSnMsNVBjLMNkTj6OZbudcPPTPYHqX0CJfpcdWciI1dM8uHETwmDxxiqEwCIE6WvXucWUetJgfu/A==",
"hasInstallScript": true,
"peer": true,
"dependencies": {
"@prisma/engines": "5.22.0"
},
@@ -12479,7 +12450,6 @@
"resolved": "https://registry.npmjs.org/ajv/-/ajv-8.17.1.tgz",
"integrity": "sha512-B/gBuNg5SiMTrPkC+A2+cW0RszwxYmn6VYxB/inlBStS5nx6xHIt/ehKRhIMhqusl7a8LjQoZnjCs5vhwxOQ1g==",
"dev": true,
"peer": true,
"dependencies": {
"fast-deep-equal": "^3.1.3",
"fast-uri": "^3.0.1",
@@ -12794,7 +12764,6 @@
"resolved": "https://registry.npmjs.org/ts-node/-/ts-node-10.9.2.tgz",
"integrity": "sha512-f0FFpIdcHgn8zcPSbf1dRevwt047YMnaiJM3u2w2RewrB+fob/zePZcrOyQoLMMO7aBIddLcQIEK5dYjkLnGrQ==",
"dev": true,
"peer": true,
"dependencies": {
"@cspotcode/source-map-support": "^0.8.0",
"@tsconfig/node10": "^1.0.7",
@@ -12950,7 +12919,6 @@
"resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
"dev": true,
"peer": true,
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
@@ -13298,6 +13266,7 @@
"resolved": "https://registry.npmjs.org/ajv-formats/-/ajv-formats-2.1.1.tgz",
"integrity": "sha512-Wx0Kx52hxE7C18hkMEggYlEifqWZtYaRgouJor+WMdPnQyEK13vgEWyVNup7SoeeoLMsr4kf5h6dOW11I15MUA==",
"dev": true,
"peer": true,
"dependencies": {
"ajv": "^8.0.0"
},
@@ -13315,6 +13284,7 @@
"resolved": "https://registry.npmjs.org/ajv-keywords/-/ajv-keywords-5.1.0.tgz",
"integrity": "sha512-YCS/JNFAUyr5vAuhk1DWm1CBxRHW9LbJ2ozWeemrIqpbsqKjHVxYPyi5GC0rjZIT5JxJ3virVTS8wk4i/Z+krw==",
"dev": true,
"peer": true,
"dependencies": {
"fast-deep-equal": "^3.1.3"
},
@@ -13327,6 +13297,7 @@
"resolved": "https://registry.npmjs.org/eslint-scope/-/eslint-scope-5.1.1.tgz",
"integrity": "sha512-2NxwbF/hZ0KpepYN0cNbo+FN6XoK7GaHlQhgx/hIZl6Va0bF45RQOOwhLIy8lQDbuCiadSLCBnH2CFYquit5bw==",
"dev": true,
"peer": true,
"dependencies": {
"esrecurse": "^4.3.0",
"estraverse": "^4.1.1"
@@ -13340,6 +13311,7 @@
"resolved": "https://registry.npmjs.org/estraverse/-/estraverse-4.3.0.tgz",
"integrity": "sha512-39nnKffWz8xN1BU/2c79n9nB9HDzo0niYUqx6xyqUnyoAnQyyWpOTdZEeiCch8BBu515t4wp9ZmgVfVhn9EBpw==",
"dev": true,
"peer": true,
"engines": {
"node": ">=4.0"
}
@@ -13348,13 +13320,15 @@
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-1.0.0.tgz",
"integrity": "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug==",
"dev": true
"dev": true,
"peer": true
},
"node_modules/webpack/node_modules/mime-db": {
"version": "1.52.0",
"resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
"integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
"dev": true,
"peer": true,
"engines": {
"node": ">= 0.6"
}
@@ -13364,6 +13338,7 @@
"resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
"integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
"dev": true,
"peer": true,
"dependencies": {
"mime-db": "1.52.0"
},
@@ -13376,6 +13351,7 @@
"resolved": "https://registry.npmjs.org/schema-utils/-/schema-utils-4.3.3.tgz",
"integrity": "sha512-eflK8wEtyOE6+hsaRVPxvUKYCpRgzLqDTb8krvAsRIwOGlHoSgYLgBXoubGgLd2fT41/OUYdb48v4k4WWHQurA==",
"dev": true,
"peer": true,
"dependencies": {
"@types/json-schema": "^7.0.9",
"ajv": "^8.9.0",
+5 -5
View File
@@ -22,14 +22,14 @@
"ai:backtest": "python ai-engine/scripts/backtest_v2_runtime.py",
"ai:train:vqwen": "python ai-engine/scripts/train_vqwen_v3.py",
"feeder:historical": "ts-node -r tsconfig-paths/register src/scripts/run-feeder.ts",
"feeder:previous-day": "ts-node -r tsconfig-paths/register src/scripts/run-feeder.ts",
"feeder:repair": "ts-node -r tsconfig-paths/register src/scripts/run-feeder-repair.ts",
"feeder:previous-day": "ts-node -r tsconfig-paths/register src/scripts/run-feeder-previous-day.ts",
"feeder:fill-gaps": "ts-node -r tsconfig-paths/register src/scripts/run-feeder-filtered.ts",
"feeder:basketball": "ts-node -r tsconfig-paths/register src/scripts/run-feeder-basketball.ts",
"feeder:live": "ts-node -r tsconfig-paths/register src/scripts/run-live-feeder.ts",
"cleanup:live": "ts-node -r tsconfig-paths/register src/scripts/cleanup-live-matches.ts",
"swagger:summary": "ts-node -r tsconfig-paths/register src/scripts/export-swagger-endpoints-summary.ts",
"postman:export": "ts-node -r tsconfig-paths/register src/scripts/export-postman-collection.ts"
,
"postman:export": "ts-node -r tsconfig-paths/register src/scripts/export-postman-collection.ts",
"ai:extract:v26": "python3 ai-engine/scripts/extract_training_data_v26.py",
"ai:train:v26": "python3 ai-engine/scripts/train_v26_shadow.py",
"ai:backtest:v26": "python3 ai-engine/scripts/backtest_v26_shadow.py",
@@ -55,7 +55,7 @@
"@nestjs/swagger": "^11.2.4",
"@nestjs/terminus": "^11.0.0",
"@nestjs/throttler": "^6.5.0",
"@prisma/client": "^5.22.0",
"@prisma/client": "5.22.0",
"axios": "^1.13.6",
"bcrypt": "^6.0.0",
"bullmq": "^5.66.4",
@@ -75,7 +75,7 @@
"passport-jwt": "^4.0.1",
"pino": "^10.1.0",
"pino-http": "^11.0.0",
"prisma": "^5.22.0",
"prisma": "5.22.0",
"reflect-metadata": "^0.2.2",
"rxjs": "^7.8.1",
"twitter-api-v2": "^1.29.0",
+33 -2
View File
@@ -543,6 +543,7 @@ model User {
analyses Analysis[]
refreshTokens RefreshToken[]
usageLimit UsageLimit?
subscription Subscription?
coupons UserCoupon[]
totoCoupons TotoCoupon[]
@@ -551,6 +552,27 @@ model User {
@@map("users")
}
model Subscription {
id String @id @default(uuid())
userId String @unique @map("user_id")
paddleSubscriptionId String? @unique @map("paddle_subscription_id")
paddleCustomerId String? @map("paddle_customer_id")
plan SubscriptionStatus @default(free)
billingInterval BillingInterval? @map("billing_interval")
currentPeriodStart DateTime? @map("current_period_start")
currentPeriodEnd DateTime? @map("current_period_end")
cancelledAt DateTime? @map("cancelled_at")
cancelEffectiveDate DateTime? @map("cancel_effective_date")
paddlePriceId String? @map("paddle_price_id")
createdAt DateTime @default(now()) @map("created_at")
updatedAt DateTime @updatedAt @map("updated_at")
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
@@index([paddleSubscriptionId])
@@index([paddleCustomerId])
@@map("subscriptions")
}
model RefreshToken {
id String @id @default(uuid())
token String @unique
@@ -569,6 +591,8 @@ model UsageLimit {
userId String @unique @map("user_id")
analysisCount Int @default(0) @map("analysis_count")
couponCount Int @default(0) @map("coupon_count")
maxAnalyses Int @default(3) @map("max_analyses")
maxCoupons Int @default(1) @map("max_coupons")
lastResetDate DateTime @map("last_reset_date") @db.Date
createdAt DateTime @default(now()) @map("created_at")
updatedAt DateTime @updatedAt @map("updated_at")
@@ -765,8 +789,15 @@ enum UserRole {
enum SubscriptionStatus {
free
active
expired
plus
premium
past_due
cancelled
}
enum BillingInterval {
monthly
yearly
}
enum PlayerPosition {
+1 -1
View File
@@ -24,7 +24,7 @@ async function main() {
firstName: 'Super',
lastName: 'Admin',
role: UserRole.superadmin,
subscriptionStatus: SubscriptionStatus.active,
subscriptionStatus: SubscriptionStatus.free,
isActive: true,
},
});
+267
View File
@@ -0,0 +1,267 @@
[
"3iwftmprsznl6yribr11a8l9m",
"cegl2ivkc25blcatxp4jmk1ec",
"1zp1du9n4rj36p1ss9zbxtqfb",
"bockl24qpr7ryjl8b6obukga",
"byu00jvt1j6csyv4y1lkt2fm2",
"degxm4y6gmvp011ccyrev6z5p",
"c7b8o53flg36wbuevfzy3lb10",
"7ntvbsyq31jnzoqoa8850b9b8",
"581t4mywybx21wcpmpykhyzr3",
"3frp1zxrqulrlrnk503n6l4l",
"287tckirbfj9nb8ar2k9r60vn",
"bgen5kjer2ytfp7lo9949t72g",
"ac112osli9fvox1epcg4ld3t6",
"3is4bkgf3loxv9qfg3hm8zfqb",
"c1d9p6b2e9zr5tqlzx3ktjplg",
"5zr0b05eyx25km7z1k03ca9jx",
"5z8v4mj6cjs9ex6hdrpourjzh",
"scf9p4y91yjvqvg5jndxzhxj",
"3p81ltz6845appgkbgkzxueii",
"b5udgm9vakjqz8dcmy5b2g0xt",
"b1rveez5u792gess9w3e7v5le",
"2ty8ihceabty8yddmu31iuuej",
"8ey0ww2zsosdmwr8ehsorh6t7",
"2nttcoriwf5co73vmz1vr8frm",
"1r097lpxe0xn03ihb7wi98kao",
"2kwbbcootiqqgmrzs6o5inle5",
"907l7wtxdvugdo9i2249wcmr0",
"8o5tv5viv4hy1qg9jp94k7ayb",
"4nidzmunvpvxk1ir9b6m8mpay",
"dkarmrybx9vx10rg7cywumth0",
"a9vrdkelbgif0gtu3wxsr75xo",
"4w7x0s5gfs5abasphlha5de8k",
"8dn0w8zh7nbn2i904603eigwf",
"1gwajyt0pk2jm5fx5mu36v114",
"2o9svokc5s7diish3ycrzk7jm",
"7hl0svs2hg225i2zud0g3xzp2",
"89ovpy1rarewwzqvi30bfdr8b",
"2hsidwomhjsaaytdy9u5niyi4",
"34pl8szyvrbwcmfkuocjm3r6t",
"8r98daokeuzsamu5fmjtblqx5",
"akmkihra9ruad09ljapsm84b3",
"722fdbecxzcq9788l6jqclzlw",
"663a54fmymndjeev47qm7d3nf",
"4zwgbb66rif2spcoeeol2motx",
"9chuiarcjofld1dkj9kysehmb",
"5y0z0l2epprzbscvzsgldw8vu",
"2wolc27r8z03itcvwp43e38c5",
"alpfd99yd3lfv7bhjo0biuq7b",
"ea0h6cf3bhl698hkxhpulh2zz",
"8sdpk4aerruf515yh76ezo7vi",
"6by3h89i2eykc341oz7lv1ddd",
"7r1f93t6ddrsa5n8v1nq6qlzm",
"8yi6ejjd1zudcqtbn07haahg6",
"ein4fkggto3pdh5msp8huafiq",
"b60nisd3qn427jm0hrg9kvmab",
"1qd0wvt30rlswa4g6nu4na660",
"b73zounsynk9d3u1p9nvpu7i2",
"civf31q1inxohs4a03y8reetf",
"bu1l7ckihyr0errxw61p0m05",
"a7247po5qs29o3zsfmt222ydu",
"6lwpjhktjhl9g7x2w7njmzva6",
"4c1nfi2j1m731hcay25fcgndq",
"3ww12jab49q8q8mk9avdwjqgk",
"8y29fg2s85ppcb8uugm5ee8s4",
"82jkgccg7phfjpd0mltdl3pat",
"46b141eaqq9q7o4gz5gtdpikk",
"482ofyysbdbeoxauk19yg7tdt",
"4oogyu6o156iphvdvphwpck10",
"2y8bntiif3a9y6gtmauv30gt",
"e21cf135btr8t3upw0vl6n6x0",
"c0yqkbilbbg70ij2473xymmqv",
"5dycj9wdhxh3n33qubw18ohlk",
"1eruend45vd20g9hbrpiggs5u",
"e1kxdivp5g4cpldgpwvnzl1vv",
"ddyrh5latwfhesgfh4w401n92",
"af79lqrc0ntom74zq13ccjslo",
"3ab1uwtoyjopdj1y1fynyy9jg",
"c0r21rtokgnbtc0o2rldjmkxu",
"e0lck99w8meo9qoalfrxgo33o",
"yv73ms6v1995b5wny16jcfi3",
"5aw6uyw4pz2bpj24t5z8aacim",
"75i269i1ak43magshljadydrh",
"8k1xcsyvxapl4jlsluh3eomre",
"jznihqxle06xych9ygwiwnsa",
"6wubmo7di3kdpflluf6s8c7vs",
"7cwemnr3vi40znjq451zxkus6",
"6ifaeunfdelecgticvxanikzu",
"913mb508il6jzwtlj28fl892h",
"29actv1ohj8r10kd9hu0jnb0n",
"3btdfgw79qiz3jmyfudovtbu2",
"5cwsxtx37les6m10xj71htkgf",
"9nbpdi9q3ywcm4q0j5u0ekwcq",
"dm5ka0os1e3dxcp3vh05kmp33",
"beqqnubkv05mamuwvimeum015",
"57nu0wygurzkp6fuy5hhrtaa2",
"du6jsenbjql5e8f3yk880ox4g",
"cesdwwnxbc5fmajgroc0hqzy2",
"3w1hkk9k9gr8fwssyn4icvdfo",
"65ggsqdi6drpa4m8y3gkll25k",
"4yzidekywejmxxp77gqmdgopg",
"avs3xposm3t9x1x2vzsoxzcbu",
"75434tz9rc14xkkvudex742ui",
"aho73e5udydy96iun3tkzdzsi",
"4qehj8hfxmy6o2ohp4fxinnzo",
"ae1wva3zrzcp2zd15gpvsntg6",
"4d5d3sf6805n5u6jdoa0hdlog",
"3l29w00m506ex93t5bbh9cg2a",
"zs18qaehvhg3w1208874zvfa",
"4mbfidy8zum5u0aqjqo0vuqs2",
"8v97rcbthsxmzqk4ufxws9mug",
"c76z5d6j7dpi1e79tm8fpm39z",
"47s2kt0e8m444ftqvsrqa3bvq",
"9ikchyu9fb8bvx0s673jofj6s",
"6ihotpaocgiovlxw18e9r9prx",
"32n2r9bl6x90psj0wa7bfs6vq",
"zilopfej2h0n3vpan5tcynpo",
"7nmz249q89qg5ezcvzlheljji",
"ajxs0e0g6ryg5ol8qvw3evrcz",
"477yyajzheg2z8u7uick0e13e",
"8t2o4huu2e48ij23dxnl9w5qx",
"1wwro3z1eb3fl601dju6inlc6",
"4yngyfinzd6bb1k7anqtqs0wt",
"1b70m6qtxrp75b4vtk8hxh8c3",
"7af85xa75vozt2l4hzi6ryts7",
"117yqo02rs8dykkxpm274w3bd",
"725gd73msyt08xm76v7gkxj7u",
"f4jc2cc5nq7flaoptpi5ua4k4",
"xwnjb1az11zffwty3m6vn8y6",
"dr2xk7muj8aqcjdz2b3li1c0k",
"1mpjd0vbxbtu9zw89yj09xk3z",
"3428tckxcirwwh3o3jgc1m8ji",
"6sxm2iln2w45ux498pty9miw8",
"6321dlqv4ziuwqte4xpohijtw",
"5c96g1zm7vo5ons9c42uy2w3r",
"ili150pwfuf39f7yfdch9lhw",
"7swf4kpu3v38i2it4h94c5s9k",
"iu1vi94p4p28oozl1h9bvplr",
"5k620c7y6dlbmcm88dt3eb7t",
"f39uq10c8xhg5e6rwwcf6lhgc",
"6lkj3o21cr4g7bql6tb3fk222",
"9ynnnx1qmkizq1o3qr3v0nsuk",
"8usjlmziv3p2re0r2wwzezki9",
"4zwjlzdszduqmxzusysvzymms",
"7mxwwunvot2pi69pj1yr1kh8i",
"5taraea6mqjjldg9zxswo825y",
"9fuwphq8kvugrlc3ckm7k8wes",
"dvstmwnvw0mt5p38twn9yttyb",
"2xg0qvif1rh7du6wmk2eleku3",
"8x3sbh85gc8qir50utw39jl04",
"59tpnfrwnvhnhzmnvfyug68hj",
"1fedahp0rws09tj451onten8r",
"esrunz7rjb0td98mx9e5cedoy",
"2hj3286pqov1g1g59k2t2qcgm",
"55hcphd1ccc6eai1ms77460on",
"40yjcbx2sq6oq736iqqqczwt1",
"eog6knrkfei68si736fpquyzc",
"f47f3717z2vtpxfxrpdd4jl1x",
"3oa9e03e7w9nr8kqwqc3tlqz9",
"apdwh753fupxheygs8seahh7x",
"486rhdgz7yc0sygziht7hje65",
"erpufio3qaujd9gkszcqvb0bf",
"cu0rmpyff5692eo06ltddjo8a",
"eg6s9f1jj7jr6stmbosn0g6c8",
"9p3nnxhdjahfn8qswpzy8oyc3",
"cse5oqqt2pzfcy8uz6yz3tkbj",
"cfesxhzb83yl8b779uv3revz1",
"4rls982p5uzil6x30mhyhv9f3",
"eitf7hulqfv1clb7toewkil24",
"byhmntnl1b4lxw0zz21im3zkd",
"gfskxsdituog2kqp9yiu7bzi",
"ejunkmfhjz9weugd2bqrkgobb",
"bdtat25m14jy85y484z3e6lf",
"ax1yf4nlzqpcji4j8epdgx3zl",
"1j4ehtrbry9depwt6oghaq3lu",
"xaouuwuk8qyhv1libkeexwjh",
"1q4ab2bpg5e8jl1g2udnakrju",
"81txfenlgw75nq3u2nfdkj92o",
"19q13y6ruzo0o84ipblcuouzs",
"3n9mk5b2mxmq831wfmv6pu86i",
"3n5046abeu3x482ds3jwda238",
"2aso72utuctat2ecs6nahjss6",
"2bmwykmdlcc2u1c40ytoc39vy",
"bx57cmq1edfq53ckfk791supi",
"bly7ema5au6j40i0grhl0pnub",
"er5745q30wnr8jv9nr863omzg",
"by5nibd18nkt40t0j8a0j5yzx",
"1ncmha8yglhyyhg6gtaujymqf",
"agpweohvn9tugnyl6ry4rhivp",
"8ztsv3pzrsyq5w1r3a0nfk1y5",
"4davonpqws4a4ejl1awu98zdg",
"6vq8j5p3av14nr3iuyi4okhjt",
"bbajzna018c79opa1kl5kmkqo",
"eu2g5j36zzxiazpd729osx0wm",
"595nsvo7ykvoe690b1e4u5n56",
"1gxlzw2ezkyeykhcaa5x8ozkk",
"2z7257m7hj58zuxcjrsg4erzc",
"392slbmf1kdqlr6sd1ckt71rs",
"6g8hw3acenrw828la7gwx4mvs",
"d9eaigzyfnfiraqc3ius757tl",
"3aa4mumjl6zyetg6o9hwd5hhx",
"6hlw7rhrpe9garwmfoxu4lebc",
"e6vzdkz6l236s9p288mharefy",
"dvtl8sf1262pd2aqgu641qa7u",
"5pq4dbinkmt8ujoepyqzih7iw",
"6qitd9h242qkvjenaytfdnsf2",
"cbdbziaqczfuyuwqsylqi26zd",
"3ymqchdzk8tt6lfphf26xfvh0",
"2rdrisk4vlglfjxwu0precyqd",
"1cnx2c8g3hhp8ssxnwwli0mjb",
"65q4uwm6ol1rkf5dp89m8omny",
"8kt53kt3mfo29gldhkl05u25b",
"5jd0k2txwnq69frs79eulba8j",
"8x62utr2uti3i7kk14isbnip6",
"b3ufcd24wfnnd5j98ped6irfu",
"61fzfjogstjuukzcehighq7mu",
"50ap4sua1xyut3mpu7ehesp63",
"6694fff47wqxl10lrd9tb91f8",
"macko16888165594668885588",
"3e40pestup9xzagsu2o6c0i8u",
"9oqeqyj7swpnl86ytafjwavvo",
"1qt9bfl6dhydf4tpano6n1p7s",
"29lni33vxqrl1tqhadrnfid6t",
"2db0aw1duj2my9l5iey5gm6nq",
"1vyghvhuy6abu4htoemdi79bd",
"4vksk0d2q4c5w0itdl52lzek6",
"193wqkyb0v5jnsblhvd2ocmyo",
"a3egqgf45jqft6y0uoyvw3mbj",
"5liafywveaf56s2nod8hg9nca",
"3a0j0giz3c3ajw9h59evv7lqt",
"2mdmx668tyhy4u4z9zszwjv5v",
"19mr0xdp7li6nkz87oxh53xed",
"8u5w0g8jimye1cu5albkcb3qs",
"2kuyfkulm5lsgjxynrgh3vz70",
"8cit3whr514nnd4zkaovsnqn",
"9mr92dlx7ryaxhi07sgt90ish",
"1dajh9qrda3enawmlt7ogt05w",
"10x5pvhifwo4y7hs3fz9hf245",
"dc4k1xh2984zbypbnunk7ncic",
"e6rl4hongahbihxd3tpudespd",
"2r1hqz453bn9ljzt53kdr2lwb",
"86wrztni4x8tnvq9cr1cetvfu",
"5em08hhvd7komnfdsb1yagpas",
"326jpj7749ojwqhu3ap27zl77",
"bqvy41un7sf86rbse9tv810x7",
"93i7thp7zi0ympyt6l8aa1r2i",
"ahl3vljaignq9ebaos4uqkrvo",
"68zplepppndhl8bfdvgy9vgu1",
"df1o8phtfy4dwhv6n7mmeedvw",
"cj30195079sdep2imeyt7y47p",
"3z6xfyd3ovi5x09orlo4rmskx",
"1n990e5dpi9xwruwf6uslknkq",
"etta63x1t7tnkn4jheisjwk4p",
"2xv6qkye2rsnwram454x8i8f1",
"8c93rclta164ypkno054nkfyt",
"89v3ukjpui1gashsz3i1vphfa",
"8tddm56zbasf57jkkay4kbf11",
"dcgbs1vkp9y3y31li7s95i51f",
"dlf90uty1axvtr1vn2aaw9vqh",
"9gvvndi7vk9fzvpe65pv5x2ir",
"7siumtnmgqfap6nalpu8xcwb6",
"7zsbjmlmhzn0y7923lw4zquud",
"8dxsd8xnjm9n1ogo37yomgl3p",
"arrfx02rdlstdfwdyikwqtwgl",
"afp674ll89oqsbbrqt17xfxlh",
"22euhl6zy56cp651ipq99rooq"
]
+5
View File
@@ -50,6 +50,8 @@ import { LeaguesModule } from "./modules/leagues/leagues.module";
import { AnalysisModule } from "./modules/analysis/analysis.module";
import { CouponsModule } from "./modules/coupons/coupons.module";
import { SporTotoModule } from "./modules/spor-toto/spor-toto.module";
import { AiProxyModule } from "./modules/ai-proxy/ai-proxy.module";
import { SubscriptionsModule } from "./modules/subscriptions/subscriptions.module";
// Services and Tasks
import { ServicesModule } from "./services/services.module";
@@ -76,6 +78,7 @@ const historicalFeederMode = process.env.FEEDER_MODE === "historical";
// Configuration
ConfigModule.forRoot({
isGlobal: true,
envFilePath: [".env.local", ".env"],
validate: validateEnv,
load: [
appConfig,
@@ -201,6 +204,8 @@ const historicalFeederMode = process.env.FEEDER_MODE === "historical";
AnalysisModule,
CouponsModule,
SporTotoModule,
AiProxyModule,
SubscriptionsModule,
// Services and Scheduled Tasks
ServicesModule,
+38 -6
View File
@@ -69,9 +69,8 @@ export class AiEngineClient {
this.defaultTimeoutMs = options.timeoutMs ?? 30000;
this.maxRetries = options.maxRetries ?? 2;
this.retryDelayMs = options.retryDelayMs ?? 750;
this.circuitBreakerThreshold = options.circuitBreakerThreshold ?? 3;
this.circuitBreakerCooldownMs =
options.circuitBreakerCooldownMs ?? 30000;
this.circuitBreakerThreshold = options.circuitBreakerThreshold ?? 5;
this.circuitBreakerCooldownMs = options.circuitBreakerCooldownMs ?? 15000;
this.axiosClient = axios.create({
baseURL: options.baseUrl,
@@ -113,7 +112,9 @@ export class AiEngineClient {
};
}
private async request<T>(config: AiEngineRequestConfig): Promise<AxiosResponse<T>> {
private async request<T>(
config: AiEngineRequestConfig,
): Promise<AxiosResponse<T>> {
this.ensureCircuitAvailable();
const retries = this.resolveRetryCount(config);
@@ -133,7 +134,13 @@ export class AiEngineClient {
const shouldRetry = attempt < retries && this.isRetriableError(error);
if (!shouldRetry) {
// Only register circuit breaker failure for server/network errors, not client errors (4xx)
if (this.isServerError(error)) {
this.registerFailure(error);
} else {
// It's a successful contact with the engine (e.g. 404, 422), so reset failures
this.resetFailures();
}
throw this.toRequestError(error);
}
@@ -162,7 +169,8 @@ export class AiEngineClient {
}
const remainingCooldown =
this.circuitBreakerCooldownMs - (Date.now() - (this.circuitOpenedAt ?? 0));
this.circuitBreakerCooldownMs -
(Date.now() - (this.circuitOpenedAt ?? 0));
if (remainingCooldown > 0) {
throw new AiEngineRequestError("AI engine circuit breaker is open", {
@@ -175,8 +183,11 @@ export class AiEngineClient {
}
this.logger.warn(
`[${this.serviceName}] AI circuit breaker cooldown elapsed, allowing a recovery attempt`,
`[${this.serviceName}] AI circuit breaker cooldown elapsed, allowing a recovery attempt (resetting failures from ${this.consecutiveFailures})`,
);
// Half-open state: reset failures so a single retry failure doesn't
// immediately re-open the circuit at threshold+1
this.consecutiveFailures = 0;
this.circuitOpenedAt = null;
}
@@ -218,6 +229,27 @@ export class AiEngineClient {
return status >= 500 || status === 429 || error.code === "ECONNABORTED";
}
private isServerError(error: unknown): boolean {
if (!axios.isAxiosError(error)) {
return true; // Not an axios error, assume internal/network error
}
if (!error.response) {
return true; // Network error, timeout, etc.
}
// Only count infrastructure-level errors toward circuit breaker:
// - No response (network failure) → already handled above
// - Timeout (ECONNABORTED) → infrastructure
// - 429 (rate limit) → infrastructure
// - 502/503/504 (proxy/gateway errors) → infrastructure
// Do NOT count 500 (app-level crash in AI Engine) — it may be
// match-specific and shouldn't block all other matches.
if (error.code === "ECONNABORTED") {
return true;
}
const status = error.response.status;
return status === 429 || status === 502 || status === 503 || status === 504;
}
private toRequestError(error: unknown): AiEngineRequestError {
if (error instanceof AiEngineRequestError) {
return error;
+1 -4
View File
@@ -14,10 +14,7 @@ function extractDateParts(date: Date, timeZone: string) {
return { year, month, day };
}
export function getDateStringInTimeZone(
date: Date,
timeZone: string,
): string {
export function getDateStringInTimeZone(date: Date, timeZone: string): string {
const { year, month, day } = extractDateParts(date, timeZone);
return `${year}-${String(month).padStart(2, "0")}-${String(day).padStart(2, "0")}`;
}
+19 -1
View File
@@ -22,7 +22,7 @@ export const envSchema = z.object({
// Database
DATABASE_URL: z.string().url(),
// AI Engine
AI_ENGINE_URL: z.string().url().default("http://localhost:8000"),
AI_ENGINE_URL: z.string().url(),
AI_ENGINE_MODE: z.enum(["v28-pro-max", "dual"]).default("v28-pro-max"),
// JWT
@@ -56,13 +56,31 @@ export const envSchema = z.object({
.string()
.transform((val) => val === "true")
.default("false" as any),
SOCIAL_POSTER_SPORTS: z.string().default("football,basketball"),
SOCIAL_POSTER_WINDOW_MIN: z.coerce.number().default(25),
SOCIAL_POSTER_WINDOW_MAX: z.coerce.number().default(45),
SOCIAL_POSTER_OLLAMA_MODEL: z.string().optional(),
APP_BASE_URL: z.string().url().optional(),
TWITTER_API_KEY: z.string().optional(),
TWITTER_API_SECRET: z.string().optional(),
TWITTER_ACCESS_TOKEN: z.string().optional(),
TWITTER_ACCESS_SECRET: z.string().optional(),
META_GRAPH_API_VERSION: z.string().default("v25.0"),
META_PAGE_ACCESS_TOKEN: z.string().optional(),
META_PAGE_ID: z.string().optional(),
META_IG_USER_ID: z.string().optional(),
OLLAMA_BASE_URL: z.string().url().optional(),
OLLAMA_MODEL: z.string().optional(),
// Paddle (Subscription Billing)
PADDLE_API_KEY: z.string().optional(),
PADDLE_WEBHOOK_SECRET: z.string().optional(),
PADDLE_CLIENT_TOKEN: z.string().optional(),
PADDLE_ENVIRONMENT: z.enum(["sandbox", "production"]).default("sandbox"),
PADDLE_PLUS_MONTHLY_PRICE_ID: z.string().optional(),
PADDLE_PLUS_YEARLY_PRICE_ID: z.string().optional(),
PADDLE_PREMIUM_MONTHLY_PRICE_ID: z.string().optional(),
PADDLE_PREMIUM_YEARLY_PRICE_ID: z.string().optional(),
// Optional Features
ENABLE_MAIL: booleanString,
+8 -1
View File
@@ -9,5 +9,12 @@
"serverError": "An unexpected error occurred",
"unauthorized": "You are not authorized to perform this action",
"forbidden": "Access denied",
"badRequest": "Invalid request"
"badRequest": "Bad request",
"SUCCESS_USER_ROLE_UPDATED": "User role updated",
"SUCCESS_USER_SUBSCRIPTION_UPDATED": "User subscription updated",
"SUCCESS_USER_DELETED": "User deleted",
"SUCCESS_USER_STATUS_UPDATED": "User status updated",
"SUCCESS_SETTING_UPDATED": "Setting updated",
"SUCCESS_ALL_LIMITS_RESET": "All usage limits reset",
"SUCCESS_USER_LIMITS_RESET": "User usage limits reset"
}
+9 -1
View File
@@ -10,5 +10,13 @@
"TENANT_NOT_FOUND": "Tenant not found",
"VALIDATION_FAILED": "Validation failed",
"INTERNAL_ERROR": "An internal error occurred, please try again later",
"AUTH_REQUIRED": "Authentication required, please provide a valid token"
"AUTH_REQUIRED": "Authentication required, please provide a valid token",
"USAGE_LIMIT_EXCEEDED": "You have exceeded your daily usage limit. Please upgrade your plan.",
"ANALYSIS_LIMIT_EXCEEDED": "You have exceeded your daily analysis limit. Please upgrade your plan.",
"COUPON_LIMIT_EXCEEDED": "You have exceeded your daily coupon limit. Please upgrade your plan.",
"INVALID_PLAN_TYPE": "Invalid plan type. Must be free, plus, or premium.",
"MATCH_NOT_FOUND": "Match not found",
"PREDICTION_GENERATION_FAILED": "Failed to generate prediction",
"SMART_COUPON_GENERATION_FAILED": "Failed to generate Smart Coupon",
"ANALYSIS_FAILED": "None of the provided matches could be analyzed successfully"
}
+8 -1
View File
@@ -9,5 +9,12 @@
"serverError": "Beklenmeyen bir hata oluştu",
"unauthorized": "Bu işlemi yapmaya yetkiniz yok",
"forbidden": "Erişim reddedildi",
"badRequest": "Geçersiz istek"
"badRequest": "Geçersiz istek",
"SUCCESS_USER_ROLE_UPDATED": "Kullanıcı rolü güncellendi",
"SUCCESS_USER_SUBSCRIPTION_UPDATED": "Kullanıcı aboneliği güncellendi",
"SUCCESS_USER_DELETED": "Kullanıcı başarıyla silindi",
"SUCCESS_USER_STATUS_UPDATED": "Kullanıcı durumu güncellendi",
"SUCCESS_SETTING_UPDATED": "Ayar güncellendi",
"SUCCESS_ALL_LIMITS_RESET": "Tüm kullanıcı limitleri sıfırlandı",
"SUCCESS_USER_LIMITS_RESET": "Kullanıcı limitleri sıfırlandı"
}
+9 -1
View File
@@ -10,5 +10,13 @@
"TENANT_NOT_FOUND": "Kiracı bulunamadı",
"VALIDATION_FAILED": "Doğrulama başarısız",
"INTERNAL_ERROR": "Bir iç hata oluştu, lütfen daha sonra tekrar deneyin",
"AUTH_REQUIRED": "Kimlik doğrulama gerekli, lütfen geçerli bir token sağlayın"
"AUTH_REQUIRED": "Kimlik doğrulama gerekli, lütfen geçerli bir token sağlayın",
"USAGE_LIMIT_EXCEEDED": "Günlük kullanım limitinizi doldurdunuz. Lütfen paketinizi yükseltin.",
"ANALYSIS_LIMIT_EXCEEDED": "Günlük analiz limitinizi doldurdunuz. Lütfen paketinizi yükseltin.",
"COUPON_LIMIT_EXCEEDED": "Günlük kupon limitinizi doldurdunuz. Lütfen paketinizi yükseltin.",
"INVALID_PLAN_TYPE": "Geçersiz paket tipi. (free, plus, premium olmalıdır)",
"MATCH_NOT_FOUND": "Maç bulunamadı",
"PREDICTION_GENERATION_FAILED": "Tahmin oluşturulamadı",
"SMART_COUPON_GENERATION_FAILED": "Akıllı kupon oluşturulamadı",
"ANALYSIS_FAILED": "Sağlanan maçların hiçbiri başarıyla analiz edilemedi"
}
+93 -30
View File
@@ -10,6 +10,7 @@ import {
UseInterceptors,
Inject,
NotFoundException,
BadRequestException,
} from "@nestjs/common";
import {
CacheInterceptor,
@@ -18,7 +19,12 @@ import {
CACHE_MANAGER,
} from "@nestjs/cache-manager";
import * as cacheManager from "cache-manager";
import { ApiTags, ApiBearerAuth, ApiOperation } from "@nestjs/swagger";
import {
ApiTags,
ApiBearerAuth,
ApiOperation,
ApiResponse as SwaggerResponse,
} from "@nestjs/swagger";
import { Roles } from "../../common/decorators";
import { PrismaService } from "../../database/prisma.service";
import { PaginationDto } from "../../common/dto/pagination.dto";
@@ -31,6 +37,8 @@ import {
import { plainToInstance } from "class-transformer";
import { UserResponseDto } from "../users/dto/user.dto";
import { UserRole } from "@prisma/client";
import { SubscriptionsService } from "../subscriptions/subscriptions.service";
import { PlanType } from "../subscriptions/dto/subscription.dto";
@ApiTags("Admin")
@ApiBearerAuth()
@@ -40,12 +48,14 @@ export class AdminController {
constructor(
private readonly prisma: PrismaService,
@Inject(CACHE_MANAGER) private cacheManager: cacheManager.Cache,
private readonly subscriptionsService: SubscriptionsService,
) {}
// ================== Users Management ==================
@Get("users")
@ApiOperation({ summary: "Get all users (admin)" })
@SwaggerResponse({ status: 200, type: [UserResponseDto] })
async getAllUsers(
@Query() pagination: PaginationDto,
): Promise<ApiResponse<PaginatedData<UserResponseDto>>> {
@@ -75,6 +85,7 @@ export class AdminController {
@Get("users/:id")
@ApiOperation({ summary: "Get user by ID" })
@SwaggerResponse({ status: 200, type: UserResponseDto })
async getUserById(
@Param("id") id: string,
): Promise<ApiResponse<UserResponseDto>> {
@@ -98,6 +109,7 @@ export class AdminController {
@Put("users/:id/toggle-active")
@ApiOperation({ summary: "Toggle user active status" })
@SwaggerResponse({ status: 200, type: UserResponseDto })
async toggleUserActive(
@Param("id") id: string,
): Promise<ApiResponse<UserResponseDto>> {
@@ -114,12 +126,13 @@ export class AdminController {
return createSuccessResponse(
plainToInstance(UserResponseDto, updated),
"User status updated",
"common.SUCCESS_USER_STATUS_UPDATED",
);
}
@Put("users/:id/role")
@ApiOperation({ summary: "Update user role" })
@SwaggerResponse({ status: 200, type: UserResponseDto })
async updateUserRole(
@Param("id") id: string,
@Body() data: { role: UserRole },
@@ -131,41 +144,19 @@ export class AdminController {
return createSuccessResponse(
plainToInstance(UserResponseDto, user),
"User role updated",
);
}
@Put("users/:id/subscription")
@ApiOperation({ summary: "Update user subscription" })
async updateUserSubscription(
@Param("id") id: string,
@Body()
data: { subscriptionStatus: string; subscriptionExpiresAt?: string },
): Promise<ApiResponse<UserResponseDto>> {
const user = await this.prisma.user.update({
where: { id },
data: {
subscriptionStatus: data.subscriptionStatus as any,
subscriptionExpiresAt: data.subscriptionExpiresAt
? new Date(data.subscriptionExpiresAt)
: null,
},
});
return createSuccessResponse(
plainToInstance(UserResponseDto, user),
"User subscription updated",
"common.SUCCESS_USER_ROLE_UPDATED",
);
}
@Delete("users/:id")
@ApiOperation({ summary: "Soft delete a user" })
@SwaggerResponse({ status: 200, description: "User deleted" })
async deleteUser(@Param("id") id: string): Promise<ApiResponse<null>> {
await this.prisma.user.update({
where: { id },
data: { deletedAt: new Date() },
});
return createSuccessResponse(null, "User deleted");
return createSuccessResponse(null, "common.SUCCESS_USER_DELETED");
}
// ================== App Settings ==================
@@ -175,6 +166,10 @@ export class AdminController {
@CacheKey("app_settings")
@CacheTTL(60 * 1000)
@ApiOperation({ summary: "Get all app settings" })
@SwaggerResponse({
status: 200,
schema: { type: "object", additionalProperties: { type: "string" } },
})
async getAllSettings(): Promise<ApiResponse<Record<string, string>>> {
const settings = await this.prisma.appSetting.findMany();
const settingsMap: Record<string, string> = {};
@@ -186,6 +181,13 @@ export class AdminController {
@Put("settings/:key")
@ApiOperation({ summary: "Update an app setting" })
@SwaggerResponse({
status: 200,
schema: {
type: "object",
properties: { key: { type: "string" }, value: { type: "string" } },
},
})
async updateSetting(
@Param("key") key: string,
@Body() data: { value: string },
@@ -198,7 +200,7 @@ export class AdminController {
await this.cacheManager.del("app_settings");
return createSuccessResponse(
{ key: setting.key, value: setting.value || "" },
"Setting updated",
"common.SUCCESS_SETTING_UPDATED",
);
}
@@ -206,6 +208,10 @@ export class AdminController {
@Get("usage-limits")
@ApiOperation({ summary: "Get all usage limits" })
@SwaggerResponse({
status: 200,
schema: { type: "array", items: { type: "object" } },
})
async getAllUsageLimits(@Query() pagination: PaginationDto) {
const { skip, take } = pagination;
@@ -233,6 +239,10 @@ export class AdminController {
@Post("usage-limits/reset-all")
@ApiOperation({ summary: "Reset all usage limits" })
@SwaggerResponse({
status: 200,
schema: { type: "object", properties: { count: { type: "number" } } },
})
async resetAllUsageLimits(): Promise<ApiResponse<{ count: number }>> {
const result = await this.prisma.usageLimit.updateMany({
data: {
@@ -244,7 +254,57 @@ export class AdminController {
return createSuccessResponse(
{ count: result.count },
"All usage limits reset",
"common.SUCCESS_ALL_LIMITS_RESET",
);
}
@Post("usage-limits/reset/:userId")
@ApiOperation({ summary: "Reset usage limits for a single user" })
@SwaggerResponse({ status: 200 })
async resetUserUsageLimits(
@Param("userId") userId: string,
): Promise<ApiResponse<null>> {
const user = await this.prisma.user.findUnique({ where: { id: userId } });
if (!user) throw new NotFoundException("USER_NOT_FOUND");
await this.prisma.usageLimit.update({
where: { userId },
data: {
analysisCount: 0,
couponCount: 0,
lastResetDate: new Date(),
},
});
return createSuccessResponse(null, "common.SUCCESS_USER_LIMITS_RESET");
}
@Put("users/:userId/subscription")
@ApiOperation({ summary: "Update a user's subscription tier" })
@SwaggerResponse({ status: 200 })
async updateUserSubscription(
@Param("userId") userId: string,
@Body() data: { plan: string },
): Promise<ApiResponse<null>> {
const user = await this.prisma.user.findUnique({ where: { id: userId } });
if (!user) throw new NotFoundException("USER_NOT_FOUND");
const validPlans = [PlanType.FREE, PlanType.PLUS, PlanType.PREMIUM];
const newPlan = data.plan as PlanType;
if (!validPlans.includes(newPlan)) {
throw new BadRequestException("INVALID_PLAN_TYPE");
}
await this.prisma.user.update({
where: { id: userId },
data: { subscriptionStatus: newPlan },
});
await this.subscriptionsService.syncLimitsWithPlan(userId, newPlan);
return createSuccessResponse(
null,
"common.SUCCESS_USER_SUBSCRIPTION_UPDATED",
);
}
@@ -252,6 +312,7 @@ export class AdminController {
@Get("analytics/overview")
@ApiOperation({ summary: "Get system analytics overview" })
@SwaggerResponse({ status: 200, schema: { type: "object" } })
async getAnalyticsOverview() {
const [
totalUsers,
@@ -263,7 +324,9 @@ export class AdminController {
] = await Promise.all([
this.prisma.user.count(),
this.prisma.user.count({ where: { isActive: true } }),
this.prisma.user.count({ where: { subscriptionStatus: "active" } }),
this.prisma.user.count({
where: { subscriptionStatus: { in: ["plus", "premium"] } },
}),
this.prisma.match.count(),
this.prisma.prediction.count(),
this.prisma.userCoupon.count(),
+2
View File
@@ -1,7 +1,9 @@
import { Module } from "@nestjs/common";
import { AdminController } from "./admin.controller";
import { SubscriptionsModule } from "../subscriptions/subscriptions.module";
@Module({
imports: [SubscriptionsModule],
controllers: [AdminController],
})
export class AdminModule {}
@@ -0,0 +1,20 @@
import { All, Body, Controller, Req } from "@nestjs/common";
import type { Request } from "express";
import { AiProxyService } from "./ai-proxy.service";
@Controller("ai-engine")
export class AiProxyController {
constructor(private readonly aiProxyService: AiProxyService) {}
@All("*path")
proxy(@Req() request: Request, @Body() body: unknown) {
return this.aiProxyService.proxy({
method: request.method,
originalUrl: request.originalUrl,
query: request.query as Record<string, unknown>,
body,
acceptLanguage: request.headers["accept-language"],
});
}
}
+17
View File
@@ -0,0 +1,17 @@
import { Module } from "@nestjs/common";
import { HttpModule } from "@nestjs/axios";
import { AiProxyController } from "./ai-proxy.controller";
import { AiProxyService } from "./ai-proxy.service";
@Module({
imports: [
HttpModule.register({
timeout: 45000,
maxRedirects: 0,
}),
],
controllers: [AiProxyController],
providers: [AiProxyService],
})
export class AiProxyModule {}
+98
View File
@@ -0,0 +1,98 @@
import {
BadGatewayException,
ForbiddenException,
Injectable,
} from "@nestjs/common";
import { HttpService } from "@nestjs/axios";
import { ConfigService } from "@nestjs/config";
import { AxiosError, Method } from "axios";
interface ProxyRequest {
method: string;
originalUrl: string;
query: Record<string, unknown>;
body: unknown;
acceptLanguage?: string | string[];
}
interface AllowedRoute {
method: Method;
pattern: RegExp;
}
const ALLOWED_AI_ROUTES: AllowedRoute[] = [
{ method: "GET", pattern: /^\/$/ },
{ method: "GET", pattern: /^\/health$/ },
{ method: "POST", pattern: /^\/v20plus\/analyze\/[^/]+$/ },
{ method: "GET", pattern: /^\/v20plus\/analyze-htms\/[^/]+$/ },
{ method: "GET", pattern: /^\/v20plus\/analyze-htft\/[^/]+$/ },
{ method: "POST", pattern: /^\/v20plus\/coupon$/ },
{ method: "GET", pattern: /^\/v20plus\/daily-banker$/ },
{ method: "GET", pattern: /^\/v20plus\/reversal-watchlist$/ },
{ method: "GET", pattern: /^\/v2\/health$/ },
{ method: "POST", pattern: /^\/v2\/analyze\/[^/]+$/ },
];
@Injectable()
export class AiProxyService {
constructor(
private readonly httpService: HttpService,
private readonly configService: ConfigService,
) {}
async proxy(request: ProxyRequest) {
const path = this.extractProxyPath(request.originalUrl);
const method = request.method.toUpperCase() as Method;
if (!this.isAllowed(method, path)) {
throw new ForbiddenException("AI_PROXY_ROUTE_NOT_ALLOWED");
}
const baseUrl = this.configService.getOrThrow<string>("AI_ENGINE_URL");
const targetUrl = new URL(path, baseUrl);
try {
const response = await this.httpService.axiosRef.request({
url: targetUrl.toString(),
method,
params: request.query,
data: request.body,
headers: {
"content-type": "application/json",
"accept-language": Array.isArray(request.acceptLanguage)
? request.acceptLanguage[0]
: request.acceptLanguage,
},
timeout: 45000,
maxRedirects: 0,
validateStatus: (status) => status >= 200 && status < 500,
});
return response.data;
} catch (error) {
const axiosError = error as AxiosError;
throw new BadGatewayException({
message: "AI_PROXY_UPSTREAM_FAILED",
status: axiosError.response?.status,
});
}
}
private extractProxyPath(originalUrl: string): string {
const withoutQuery = originalUrl.split("?")[0] || "";
const marker = "/ai-engine";
const markerIndex = withoutQuery.indexOf(marker);
if (markerIndex === -1) {
return "/";
}
const path = withoutQuery.slice(markerIndex + marker.length);
return path.length === 0 ? "/" : path;
}
private isAllowed(method: Method, path: string): boolean {
return ALLOWED_AI_ROUTES.some(
(route) => route.method === method && route.pattern.test(path),
);
}
}
+25 -4
View File
@@ -30,7 +30,18 @@ export class AnalysisController {
@Post("analyze-matches")
@HttpCode(HttpStatus.OK)
@ApiOperation({ summary: "Analyze multiple matches for coupon" })
@ApiResponse({ status: 200, description: "Analysis successful" })
@ApiResponse({
status: 200,
description: "Analysis successful",
schema: {
type: "object",
properties: {
success: { type: "boolean" },
data: { type: "object" },
message: { type: "string" },
},
},
})
@ApiResponse({ status: 400, description: "Invalid input" })
@ApiResponse({ status: 429, description: "Usage limit exceeded" })
async analyzeMatches(
@@ -48,7 +59,7 @@ export class AnalysisController {
);
if (!canProceed) {
throw new ForbiddenException("You have exceeded your daily usage limit");
throw new ForbiddenException("USAGE_LIMIT_EXCEEDED");
}
// Run analysis
@@ -57,7 +68,7 @@ export class AnalysisController {
if (!result) {
return {
success: false,
message: "None of the provided matches could be analyzed successfully",
message: "ANALYSIS_FAILED",
};
}
@@ -92,7 +103,17 @@ export class AnalysisController {
*/
@Get("history")
@ApiOperation({ summary: "Get analysis history" })
@ApiResponse({ status: 200, description: "History retrieved" })
@ApiResponse({
status: 200,
description: "History retrieved",
schema: {
type: "object",
properties: {
success: { type: "boolean" },
data: { type: "array", items: { type: "object" } },
},
},
})
async getHistory(@CurrentUser() user: any) {
const history = await this.analysisService.getAnalysisHistory(user.id);
return { success: true, data: history };
+7 -8
View File
@@ -84,7 +84,7 @@ export class AnalysisService {
}
/**
* Check user usage limit
* Check user usage limit (plan-aware via UsageLimit table)
*/
async checkUsageLimit(
userId: string,
@@ -96,24 +96,23 @@ export class AnalysisService {
});
if (!usageLimit) {
// Create default limit
// Create default limit with free-tier maxes
await this.prisma.usageLimit.create({
data: {
userId,
analysisCount: 0,
couponCount: 0,
maxAnalyses: 3,
maxCoupons: 1,
lastResetDate: new Date(),
},
});
return true;
}
// Check limits (default: 10 analyses, 3 coupons per day)
const user = await this.prisma.user.findUnique({ where: { id: userId } });
const isPremium = user?.subscriptionStatus === "active";
const maxAnalyses = isPremium ? 50 : 10;
const maxCoupons = isPremium ? 10 : 3;
// Use plan-aware limits from DB (set by SubscriptionsService.syncLimitsWithPlan)
const maxAnalyses = usageLimit.maxAnalyses ?? 3;
const maxCoupons = usageLimit.maxCoupons ?? 1;
if (isCoupon) {
return usageLimit.couponCount < maxCoupons;
+11 -1
View File
@@ -67,7 +67,17 @@ export class AuthController {
@Post("logout")
@HttpCode(200)
@ApiOperation({ summary: "Logout and invalidate refresh token" })
@ApiOkResponse({ description: "Logout successful" })
@ApiOkResponse({
description: "Logout successful",
schema: {
type: "object",
properties: {
success: { type: "boolean" },
message: { type: "string" },
data: { type: "null" },
},
},
})
async logout(
@Body() dto: RefreshTokenDto,
@I18n() i18n: I18nContext,
+2 -5
View File
@@ -94,11 +94,8 @@ export class RolesGuard implements CanActivate {
return false;
}
const normalizedUserRoles = (user.roles?.length
? user.roles
: user.role
? [user.role]
: []
const normalizedUserRoles = (
user.roles?.length ? user.roles : user.role ? [user.role] : []
).map((role) => normalizeRole(role));
const normalizedRequiredRoles = requiredRoles.map((role) =>
+72 -3
View File
@@ -53,7 +53,18 @@ export class CouponsController {
@Public()
@HttpCode(HttpStatus.OK)
@ApiOperation({ summary: "Analyze single match with V20 model" })
@ApiResponse({ status: 200, description: "Match analysis" })
@ApiResponse({
status: 200,
description: "Match analysis",
schema: {
type: "object",
properties: {
success: { type: "boolean" },
data: { type: "object" },
message: { type: "string" },
},
},
})
async analyzeMatch(@Body() dto: AnalyzeMatchDto) {
const analysis = await this.smartCouponService.analyzeMatch(dto.matchId);
if (!analysis) {
@@ -99,6 +110,18 @@ export class CouponsController {
@ApiOperation({
summary: "Generate a high-confidence banko combo (2 matches)",
})
@ApiResponse({
status: 200,
description: "Daily banko coupon",
schema: {
type: "object",
properties: {
success: { type: "boolean" },
data: { type: "object" },
message: { type: "string" },
},
},
})
async getDailyBanko(@Body() dto: DailyBankoDto) {
// If no match IDs provided, fetch from system (top 50 upcoming)
let candidateMatches = dto.matchIds || [];
@@ -146,7 +169,18 @@ export class CouponsController {
@Public()
@HttpCode(HttpStatus.OK)
@ApiOperation({ summary: "Suggest Smart Coupon" })
@ApiResponse({ status: 200, description: "Smart Coupon generated" })
@ApiResponse({
status: 200,
description: "Smart Coupon generated",
schema: {
type: "object",
properties: {
success: { type: "boolean" },
data: { type: "object" },
message: { type: "string" },
},
},
})
async suggestCoupon(@Body() dto: SuggestCouponDto) {
// If no match IDs provided, fetch from system (top 50 upcoming)
let candidateMatches = dto.matchIds || [];
@@ -237,6 +271,18 @@ export class CouponsController {
@ApiBearerAuth()
@HttpCode(HttpStatus.CREATED)
@ApiOperation({ summary: "Create and save a user coupon" })
@ApiResponse({
status: 201,
description: "Coupon created",
schema: {
type: "object",
properties: {
success: { type: "boolean" },
data: { type: "object" },
message: { type: "string" },
},
},
})
async createCoupon(@Body() dto: CreateCouponDto, @Req() req: any) {
// req.user is populated by JwtAuthGuard
const coupon = await this.userCouponService.createCoupon(req.user, dto);
@@ -251,6 +297,18 @@ export class CouponsController {
@UseGuards(JwtAuthGuard)
@ApiBearerAuth()
@ApiOperation({ summary: "Get user betting statistics" })
@ApiResponse({
status: 200,
description: "User statistics",
schema: {
type: "object",
properties: {
success: { type: "boolean" },
data: { type: "object" },
message: { type: "string" },
},
},
})
async getUserStats(@Req() req: any) {
const stats = await this.userCouponService.getUserStatistics(req.user.id);
return { success: true, data: stats };
@@ -263,7 +321,18 @@ export class CouponsController {
@Get("history")
@ApiBearerAuth()
@ApiOperation({ summary: "Get coupon history" })
@ApiResponse({ status: 200, description: "History retrieved" })
@ApiResponse({
status: 200,
description: "History retrieved",
schema: {
type: "object",
properties: {
success: { type: "boolean" },
data: { type: "array", items: { type: "object" } },
message: { type: "string" },
},
},
})
async getHistory(@Query("limit") limit?: string) {
// eslint-disable-next-line @typescript-eslint/await-thenable
const results = await this.couponsService.getCouponHistory(
-1
View File
@@ -25,4 +25,3 @@ import { MatchesModule } from "../matches/matches.module";
],
})
export class CouponsModule {}
@@ -109,8 +109,7 @@ export class FrequencyCouponDto {
minSignal?: number;
@ApiPropertyOptional({
description:
"Filter markets: OU1.5, OU2.5, OU3.5, BTTS, MS (default: all)",
description: "Filter markets: OU1.5, OU2.5, OU3.5, BTTS, MS (default: all)",
example: ["OU2.5", "BTTS"],
})
@IsOptional()
@@ -108,8 +108,7 @@ export class FrequencyEngineService {
venue: "home" | "away",
oddsBand: string,
): Promise<TeamFrequencyRow | null> {
const venueColumn =
venue === "home" ? "m.home_team_id" : "m.away_team_id";
const venueColumn = venue === "home" ? "m.home_team_id" : "m.away_team_id";
const oddsSelection = venue === "home" ? "'1'" : "'2'";
const bandRange = this.parseBandRange(oddsBand);
@@ -191,7 +190,7 @@ export class FrequencyEngineService {
// OU 1.5 OVER
const ou15Combined = (homeFreq.ou15_rate + awayFreq.ou15_rate) / 2;
if (ou15Combined >= 0.80) {
if (ou15Combined >= 0.8) {
signals.push({
market: "OU1.5_OVER",
pick: "1.5 UST",
@@ -212,7 +211,7 @@ export class FrequencyEngineService {
// OU 2.5 OVER
const ou25Combined = (homeFreq.ou25_rate + awayFreq.ou25_rate) / 2;
if (ou25Combined >= 0.60) {
if (ou25Combined >= 0.6) {
signals.push({
market: "OU2.5_OVER",
pick: "2.5 UST",
@@ -233,7 +232,7 @@ export class FrequencyEngineService {
// OU 3.5 OVER
const ou35Combined = (homeFreq.ou35_rate + awayFreq.ou35_rate) / 2;
if (ou35Combined >= 0.50) {
if (ou35Combined >= 0.5) {
signals.push({
market: "OU3.5_OVER",
pick: "3.5 UST",
@@ -254,7 +253,7 @@ export class FrequencyEngineService {
// BTTS YES
const bttsCombined = (homeFreq.btts_rate + awayFreq.btts_rate) / 2;
if (bttsCombined >= 0.60) {
if (bttsCombined >= 0.6) {
signals.push({
market: "BTTS_YES",
pick: "KG VAR",
@@ -299,7 +298,7 @@ export class FrequencyEngineService {
const hwCombined = (homeFreq.win_rate + awayFreq.win_rate) / 2;
// awayFreq.win_rate aslında deplasman takımının KAYBETme oranı
// (away takımı o bandda maçları kazanma değil, kaybetme olarak bak)
if (hwCombined >= 0.70 && homeOdds > 1.10 && homeOdds < 3.50) {
if (hwCombined >= 0.7 && homeOdds > 1.1 && homeOdds < 3.5) {
signals.push({
market: "MS_HOME",
pick: "MS 1",
@@ -411,9 +410,7 @@ export class FrequencyEngineService {
/**
* Lig bazlı gol profili.
*/
async getLeagueProfile(
leagueId: string,
): Promise<LeagueProfileRow | null> {
async getLeagueProfile(leagueId: string): Promise<LeagueProfileRow | null> {
const rows = await this.prisma.$queryRawUnsafe<LeagueProfileRow[]>(
`
SELECT
@@ -521,9 +518,7 @@ export class FrequencyEngineService {
return "6.00+";
}
private parseBandRange(
band: string,
): { min: number; max: number } | null {
private parseBandRange(band: string): { min: number; max: number } | null {
const map: Record<string, { min: number; max: number }> = {
"1.00-1.30": { min: 1.0, max: 1.3 },
"1.30-1.50": { min: 1.3, max: 1.5 },
@@ -537,9 +532,7 @@ export class FrequencyEngineService {
return map[band] || null;
}
private calculateLeagueBonus(
profile: LeagueProfileRow | null,
): number {
private calculateLeagueBonus(profile: LeagueProfileRow | null): number {
if (!profile || profile.total_matches < 20) {
return 0;
}
@@ -154,7 +154,8 @@ export class SmartCouponService {
async analyzeMatch(matchId: string): Promise<SingleMatchPredictionPackage> {
let prediction: SingleMatchPredictionPackage;
try {
const response = await this.aiEngineClient.post<SingleMatchPredictionPackage>(
const response =
await this.aiEngineClient.post<SingleMatchPredictionPackage>(
`/v20plus/analyze/${matchId}`,
);
prediction = response.data;
@@ -264,7 +265,7 @@ export class SmartCouponService {
markets?: string[];
}): Promise<FrequencyCouponResult> {
const maxMatches = options.maxMatches ?? 3;
const minSignal = options.minSignal ?? 0.70;
const minSignal = options.minSignal ?? 0.7;
const allowedMarkets = options.markets?.map((m) => m.toUpperCase()) || null;
this.logger.log(
@@ -167,7 +167,7 @@ export class FeederPersistenceService {
const leagueId = this.safeString(league.id);
if (leagueId) {
const logoUrl = `https://file.mackolikfeeds.com/areas/${leagueId}`;
const logoUrl = `https://file.mackolikfeeds.com/competitions/${leagueId}`;
const localPath = `public/uploads/competitions/${leagueId}.png`;
imageDownloads.push(
ImageUtils.downloadImage(logoUrl, localPath)
@@ -853,22 +853,89 @@ export class FeederPersistenceService {
}
async getExistingMatchIds(matchIds: string[]): Promise<string[]> {
const matches = await this.prisma.match.findMany({
where: {
id: { in: matchIds },
AND: [
{ oddCategories: { some: {} } },
{
OR: [
{ footballTeamStats: { some: {} } },
{ basketballTeamStats: { some: {} } },
],
},
],
},
select: { id: true },
});
return matches.map((m) => m.id);
if (matchIds.length === 0) return [];
// Use raw SQL for performance — Prisma's { some: {} } relation filters
// generate heavy correlated subqueries that hang on Raspberry Pi with
// large tables (15M+ odd_selections, 3M+ participations).
const result = await this.prisma.$queryRawUnsafe<Array<{ id: string }>>(
`
SELECT m.id
FROM matches m
WHERE m.id = ANY($1::text[])
AND EXISTS (SELECT 1 FROM odd_categories oc WHERE oc.match_id = m.id)
AND (
(m.sport = 'football'
AND EXISTS (SELECT 1 FROM football_team_stats fts WHERE fts.match_id = m.id)
AND (SELECT count(*) FROM match_player_participation mpp
WHERE mpp.match_id = m.id AND mpp.is_starting = true) >= 18)
OR
(m.sport = 'basketball'
AND EXISTS (SELECT 1 FROM basketball_team_stats bts WHERE bts.match_id = m.id)
AND EXISTS (SELECT 1 FROM basketball_player_stats bps WHERE bps.match_id = m.id))
)
`,
matchIds,
);
return result.map((r) => r.id);
}
/**
* For a list of match IDs that ALREADY exist in DB,
* returns which data scopes are missing per match.
* Only checks completed (Ended) football/basketball matches.
*/
async getMissingScopes(matchIds: string[]): Promise<Map<string, string[]>> {
const result = new Map<string, string[]>();
if (matchIds.length === 0) return result;
// Use raw SQL for performance on Raspberry Pi.
// Note: state is 'postGame' in DB, not 'Ended'.
const rows = await this.prisma.$queryRawUnsafe<
Array<{
id: string;
sport: string;
fts_count: bigint;
pp_count: bigint;
bts_count: bigint;
bps_count: bigint;
oc_count: bigint;
}>
>(
`
SELECT m.id, m.sport::text,
(SELECT count(*) FROM football_team_stats fts WHERE fts.match_id = m.id) as fts_count,
(SELECT count(*) FROM match_player_participation mpp WHERE mpp.match_id = m.id) as pp_count,
(SELECT count(*) FROM basketball_team_stats bts WHERE bts.match_id = m.id) as bts_count,
(SELECT count(*) FROM basketball_player_stats bps WHERE bps.match_id = m.id) as bps_count,
(SELECT count(*) FROM odd_categories oc WHERE oc.match_id = m.id) as oc_count
FROM matches m
WHERE m.id = ANY($1::text[])
AND m.state = 'postGame'
`,
matchIds,
);
for (const m of rows) {
const missing: string[] = [];
if (m.sport === "football") {
if (Number(m.fts_count) === 0) missing.push("stats");
if (Number(m.pp_count) < 18) missing.push("lineups");
} else if (m.sport === "basketball") {
if (Number(m.bts_count) === 0) missing.push("stats");
if (Number(m.bps_count) === 0) missing.push("lineups");
}
if (Number(m.oc_count) === 0) missing.push("odds");
if (missing.length > 0) {
result.set(m.id, missing);
}
}
return result;
}
async hasOdds(matchId: string): Promise<boolean> {
+128 -44
View File
@@ -43,6 +43,14 @@ export class FeederService {
private readonly MAX_RETRIES = 50;
private readonly DAILY_SYNC_TIME_ZONE = "Europe/Istanbul";
/** Watchdog heartbeat updated on every match/date activity */
public lastActivityAt: number = Date.now();
/** Call this to bump the heartbeat */
private heartbeat(): void {
this.lastActivityAt = Date.now();
}
constructor(
private readonly scraperService: FeederScraperService,
private readonly transformerService: FeederTransformerService,
@@ -168,7 +176,7 @@ export class FeederService {
// writing to live_matches. Historical scan should only fill matches table.
endDate.setDate(endDate.getDate() - 2);
const stateKey = `historical_scan_state_${sports.join("_")}${targetLeagueIds.length > 0 ? "_filtered" : ""}_desc`;
const stateKey = `historical_full_data_v2_state_${sports.join("_")}${targetLeagueIds.length > 0 ? "_filtered" : ""}_desc`;
let currentDate: Date | null = null;
// Resume from saved state
@@ -259,6 +267,7 @@ export class FeederService {
): Promise<void> {
const { onlyCompletedMatches = false, refreshExistingMatches = false } =
options;
this.heartbeat();
this.logger.log(`[${sport}] 📅 Processing: ${dateString}`);
try {
@@ -310,9 +319,20 @@ export class FeederService {
const { startTs: targetDateStartTs, endTs: targetDateEndTs } =
this.getDayBoundsForTimeZone(dateString, this.DAILY_SYNC_TIME_ZONE);
// DEBUG: Log sample mstUtc values vs target bounds to diagnose filtering
if (allMatches.length > 0) {
const sample = allMatches.slice(0, 3);
this.logger.warn(
`[${sport}] [${dateString}] DEBUG: bounds=[${targetDateStartTs}, ${targetDateEndTs}] ` +
`(${new Date(targetDateStartTs * 1000).toISOString()} - ${new Date(targetDateEndTs * 1000).toISOString()}) | ` +
`sampleMstUtc=[${sample.map((m) => `${m.mstUtc} (asSec=${new Date(m.mstUtc * 1000).toISOString()}, asMs=${new Date(m.mstUtc).toISOString()})`).join(", ")}]`,
);
}
const dateFilteredMatches = allMatches.filter((m) => {
const matchTs = m.mstUtc;
return matchTs >= targetDateStartTs && matchTs <= targetDateEndTs;
// mstUtc is in milliseconds from API, bounds are in seconds
const matchTsSec = Math.floor(m.mstUtc / 1000);
return matchTsSec >= targetDateStartTs && matchTsSec <= targetDateEndTs;
});
const apiReturnedCount = allMatches.length;
@@ -365,21 +385,76 @@ export class FeederService {
return;
}
// 2. Filter out already existing matches to skip processing
// 2. Filter out already existing matches & patch incomplete ones
const allIds = matchesToProcess.map((m) => m.id);
const existingIds =
await this.persistenceService.getExistingMatchIds(allIds);
const totalCount = matchesToProcess.length;
// ── Patch incomplete existing matches ──────────────────────
// Find matches that ARE in DB but have missing data scopes
const allExistingInDb =
await this.persistenceService.getMissingScopes(allIds);
if (allExistingInDb.size > 0) {
this.logger.log(
`[${sport}] [${dateString}] 🔧 Found ${allExistingInDb.size} existing matches with missing data. Patching...`,
);
for (const [matchId, missingScopes] of allExistingInDb) {
const matchSummary = matchesToProcess.find((m) => m.id === matchId);
if (!matchSummary) continue;
for (const scope of missingScopes) {
await this.delay(500);
try {
const patchScope: "all" | "lineups" | "odds" =
scope === "odds"
? "odds"
: scope === "lineups"
? "lineups"
: "all";
const result = await this.processSingleMatch(
matchSummary,
data.competitions,
sport,
true, // force
patchScope,
);
this.heartbeat();
if (result.success) {
this.logger.log(
`[${sport}] ✅ Patched [${scope}] for ${matchId} ${matchSummary.homeTeam.name} vs ${matchSummary.awayTeam.name}`,
);
} else {
this.logger.warn(
`[${sport}] ⚠️ Patch [${scope}] failed for ${matchId}`,
);
}
} catch (e: any) {
this.logger.warn(
`[${sport}] ❌ Patch [${scope}] exception for ${matchId}: ${e.message}`,
);
}
}
}
}
// ─────────────────────────────────────────────────────────────
// Now filter out COMPLETE existing matches (skip them)
if (!refreshExistingMatches && existingIds.length > 0) {
// Re-check after patching - which ones are now complete?
const updatedExistingIds =
await this.persistenceService.getExistingMatchIds(allIds);
matchesToProcess = matchesToProcess.filter(
(m) => !existingIds.includes(m.id),
(m) => !updatedExistingIds.includes(m.id),
);
}
if (matchesToProcess.length === 0) {
this.logger.log(
`[${sport}] [${dateString}] All ${totalCount} matches already exist. Skipping...`,
`[${sport}] [${dateString}] All ${totalCount} matches processed (${existingIds.length} existed, ${allExistingInDb.size} patched). Done.`,
);
return;
}
@@ -390,7 +465,7 @@ export class FeederService {
);
} else {
this.logger.log(
`[${sport}] [${dateString}] Processing ${matchesToProcess.length}/${totalCount} matches (Skipped ${existingIds.length} existing)`,
`[${sport}] [${dateString}] Processing ${matchesToProcess.length}/${totalCount} new matches (${existingIds.length} existing, ${allExistingInDb.size} patched)`,
);
}
@@ -420,6 +495,7 @@ export class FeederService {
refreshExistingMatches,
);
this.heartbeat();
if (result.success) {
this.logger.log(
`[${sport}] ✅ successful for ${match.id} ${match.homeTeam.name} vs ${match.awayTeam.name}`,
@@ -432,6 +508,7 @@ export class FeederService {
failedMatches.push(match);
}
} catch (e: any) {
this.heartbeat();
this.logger.warn(
`[${sport}] Sequential error for ${match.id}: ${e.message}`,
);
@@ -452,7 +529,7 @@ export class FeederService {
match,
data.competitions,
sport,
refreshExistingMatches,
true, // FORCE: re-fetch incomplete data
);
if (result.success) {
successCount++;
@@ -753,14 +830,12 @@ export class FeederService {
}
// Starting Formation & Substitutes (Always for lineups or all)
// V20 OPTIMIZATION: Disabled to speed up feeder and reduce 502 errors.
// We only use Team Stats for V20 model.
/*
if (scope === 'all' || scope === 'lineups') {
if (scope === "all" || scope === "lineups") {
// Starting Formation
try {
const formationData =
await this.scraperService.fetchStartingFormation(matchId);
const formationData = await fetchResilient("Formation", () =>
this.scraperService.fetchStartingFormation(matchId),
);
if (formationData?.stats) {
this.transformerService.processLineup(
formationData.stats.home || [],
@@ -780,14 +855,15 @@ export class FeederService {
);
}
} catch (e: any) {
if (e.message?.includes('502')) hasCriticalError = true;
if (e.message?.includes("502")) hasCriticalError = true;
this.logger.warn(`[${matchId}] Formation failed: ${e.message}`);
}
// Substitutes
try {
const subsData =
await this.scraperService.fetchSubstitutions(matchId);
const subsData = await fetchResilient("Subs", () =>
this.scraperService.fetchSubstitutions(matchId),
);
if (subsData?.stats) {
this.transformerService.processLineup(
subsData.stats.home || [],
@@ -807,11 +883,10 @@ export class FeederService {
);
}
} catch (e: any) {
if (e.message?.includes('502')) hasCriticalError = true;
if (e.message?.includes("502")) hasCriticalError = true;
this.logger.warn(`[${matchId}] Subs failed: ${e.message}`);
}
}
*/
// Game Stats & Officials
if (scope === "all") {
@@ -869,7 +944,37 @@ export class FeederService {
}
}
// 4. Persist to Database
// ── Pre-save completeness gate ──────────────────────────────
// If a 502 caused missing data, do NOT save. The data exists on
// the API and will be available shortly. Skip and retry instead.
const completedMatch = isMatchCompleted({
state: headerData?.matchStatus ?? matchSummary.state,
status: matchSummary.status,
substate: matchSummary.substate,
statusBoxContent: matchSummary.statusBoxContent,
scoreHome: headerData?.scoreHome ?? matchSummary.score?.home,
scoreAway: headerData?.scoreAway ?? matchSummary.score?.away,
});
const missingParts: string[] = [];
if (scope === "all" && completedMatch) {
if (sport === "football" && !stats) missingParts.push("Stats");
if (sport === "football" && participationData.length < 18)
missingParts.push("Lineups");
if (sport === "basketball" && !basketballTeamStats)
missingParts.push("BoxScore");
if (oddsArray.length === 0) missingParts.push("Odds");
}
// 502 caused missing data → do NOT save, retry later
if (hasCriticalError && missingParts.length > 0) {
this.logger.warn(
`[${matchId}] ⛔ SKIPPED SAVE: 502 errors caused missing [${missingParts.join(", ")}]. Will retry for complete data.`,
);
return { success: false, retryable: true };
}
// 4. SAVE
let saved = false;
if (scope === "lineups") {
saved = await this.persistenceService.saveLineups(
@@ -923,32 +1028,11 @@ export class FeederService {
*/
// ==========================================
const completedMatch = isMatchCompleted({
state: headerData?.matchStatus ?? matchSummary.state,
status: matchSummary.status,
substate: matchSummary.substate,
statusBoxContent: matchSummary.statusBoxContent,
scoreHome: headerData?.scoreHome ?? matchSummary.score?.home,
scoreAway: headerData?.scoreAway ?? matchSummary.score?.away,
});
const missingParts: string[] = [];
if (scope === "all" && completedMatch) {
if (sport === "football" && !stats) missingParts.push("Stats");
if (sport === "basketball" && !basketballTeamStats)
missingParts.push("BoxScore");
if (oddsArray.length === 0) missingParts.push("Odds");
}
if (saved && (hasCriticalError || missingParts.length > 0)) {
const reason = hasCriticalError
? "missing data after upstream errors"
: "incomplete completed-match payload";
// No 502 but data genuinely missing → save anyway, log warning
if (saved && missingParts.length > 0) {
this.logger.warn(
`[${matchId}] Saved with ${reason}. Missing: [${missingParts.join(", ")}]. Scheduled for retry.`,
`[${matchId}] Saved but data genuinely missing (no 502): [${missingParts.join(", ")}]`,
);
return { success: false, retryable: true };
}
return { success: saved, retryable: !saved };
+14 -2
View File
@@ -1,5 +1,5 @@
import { Controller, Get, Res } from "@nestjs/common";
import { ApiTags, ApiOperation } from "@nestjs/swagger";
import { ApiTags, ApiOperation, ApiResponse } from "@nestjs/swagger";
import type { Response } from "express";
import { Public } from "../../common/decorators";
import { PrismaService } from "../../database/prisma.service";
@@ -52,6 +52,17 @@ export class HealthController {
@Get("live")
@Public()
@ApiOperation({ summary: "Liveness check" })
@ApiResponse({
status: 200,
description: "System liveness",
schema: {
type: "object",
properties: {
status: { type: "string" },
timestamp: { type: "string" },
},
},
})
liveness(@Res() response: Response) {
return response
.status(200)
@@ -83,7 +94,8 @@ export class HealthController {
} catch (error: unknown) {
return {
status: "down",
detail: error instanceof Error ? error.message : "Unknown database error",
detail:
error instanceof Error ? error.message : "Unknown database error",
};
}
}
+65 -5
View File
@@ -28,7 +28,11 @@ export class LeaguesController {
@Get("countries")
@Public()
@ApiOperation({ summary: "Get all countries" })
@ApiResponse({ status: 200, description: "List of countries" })
@ApiResponse({
status: 200,
description: "List of countries",
schema: { type: "array", items: { type: "object" } },
})
async getCountries() {
return this.leaguesService.findAllCountries();
}
@@ -40,6 +44,11 @@ export class LeaguesController {
@Get("countries/:id")
@Public()
@ApiOperation({ summary: "Get country by ID with leagues" })
@ApiResponse({
status: 200,
description: "Country details",
schema: { type: "object" },
})
@ApiParam({ name: "id", description: "Country ID" })
async getCountryById(@Param("id") id: string) {
const country = await this.leaguesService.findCountryById(id);
@@ -54,6 +63,11 @@ export class LeaguesController {
@Get()
@Public()
@ApiOperation({ summary: "Get all leagues" })
@ApiResponse({
status: 200,
description: "List of leagues",
schema: { type: "array", items: { type: "object" } },
})
@ApiQuery({
name: "sport",
required: false,
@@ -71,6 +85,11 @@ export class LeaguesController {
@Get("teams/h2h")
@Public()
@ApiOperation({ summary: "Get head-to-head matches between two teams" })
@ApiResponse({
status: 200,
description: "Head-to-head matches",
schema: { type: "array", items: { type: "object" } },
})
@ApiQuery({ name: "team1", required: true })
@ApiQuery({ name: "team2", required: true })
@ApiQuery({ name: "limit", required: false, type: Number })
@@ -93,6 +112,11 @@ export class LeaguesController {
@Get("teams/search")
@Public()
@ApiOperation({ summary: "Search teams by name" })
@ApiResponse({
status: 200,
description: "List of teams matching search",
schema: { type: "array", items: { type: "object" } },
})
@ApiQuery({ name: "q", required: true, description: "Search query" })
@ApiQuery({
name: "sport",
@@ -110,6 +134,11 @@ export class LeaguesController {
@Get("teams/:id")
@Public()
@ApiOperation({ summary: "Get team by ID" })
@ApiResponse({
status: 200,
description: "Team details",
schema: { type: "object" },
})
@ApiParam({ name: "id", description: "Team ID" })
async getTeamById(@Param("id") id: string) {
const team = await this.leaguesService.findTeamById(id);
@@ -124,10 +153,36 @@ export class LeaguesController {
@Get("teams/:id/matches")
@Public()
@ApiOperation({ summary: "Get team's recent matches (paginated)" })
@ApiResponse({
status: 200,
description: "Paginated list of matches",
schema: {
type: "object",
properties: {
data: { type: "array", items: { type: "object" } },
meta: { type: "object" },
},
},
})
@ApiParam({ name: "id", description: "Team ID" })
@ApiQuery({ name: "page", required: false, type: Number, description: "Page number (default: 1)" })
@ApiQuery({ name: "limit", required: false, type: Number, description: "Items per page (default: 20)" })
@ApiQuery({ name: "season", required: false, type: String, description: "Season (e.g. 2024-2025)" })
@ApiQuery({
name: "page",
required: false,
type: Number,
description: "Page number (default: 1)",
})
@ApiQuery({
name: "limit",
required: false,
type: Number,
description: "Items per page (default: 20)",
})
@ApiQuery({
name: "season",
required: false,
type: String,
description: "Season (e.g. 2024-2025)",
})
async getTeamMatches(
@Param("id") id: string,
@Query("page") page?: string,
@@ -138,7 +193,7 @@ export class LeaguesController {
id,
parseInt(page || "1", 10),
parseInt(limit || "20", 10),
season
season,
);
}
@@ -149,6 +204,11 @@ export class LeaguesController {
@Get(":id")
@Public()
@ApiOperation({ summary: "Get league by ID" })
@ApiResponse({
status: 200,
description: "League details",
schema: { type: "object" },
})
@ApiParam({ name: "id", description: "League ID" })
async getLeagueById(@Param("id") id: string) {
const league = await this.leaguesService.findLeagueById(id);
+5 -3
View File
@@ -105,7 +105,7 @@ export class LeaguesService {
teamId: string,
page: number = 1,
limit: number = 20,
season?: string
season?: string,
) {
const skip = (page - 1) * limit;
const where: any = {
@@ -123,7 +123,9 @@ export class LeaguesService {
// Season starts August 1st of startYear
const startDate = new Date(Date.UTC(startYear, 7, 1)).getTime();
// Season ends July 31st of endYear
const endDate = new Date(Date.UTC(endYear, 6, 31, 23, 59, 59, 999)).getTime();
const endDate = new Date(
Date.UTC(endYear, 6, 31, 23, 59, 59, 999),
).getTime();
where.mstUtc = {
gte: startDate,
@@ -186,7 +188,7 @@ export class LeaguesService {
{ homeTeamId: teamId1, awayTeamId: teamId2 },
{ homeTeamId: teamId2, awayTeamId: teamId1 },
],
state: "postGame", // Finished matches are stored as "postGame"
AND: [{ scoreHome: { not: null } }, { scoreAway: { not: null } }],
},
include: {
homeTeam: true,
+12 -1
View File
@@ -71,7 +71,17 @@ export class MatchesController {
@ApiQuery({ name: "page", required: false, type: Number })
@ApiQuery({ name: "limit", required: false, type: Number })
@ApiQuery({ name: "sport", required: false, enum: Sport })
@ApiResponse({ status: 200, description: "Paginated list of matches" })
@ApiResponse({
status: 200,
description: "Paginated list of matches",
schema: {
type: "object",
properties: {
data: { type: "array", items: { type: "object" } },
meta: { type: "object" },
},
},
})
async listMatches(
@Query("page") page?: string,
@Query("limit") limit?: string,
@@ -112,6 +122,7 @@ export class MatchesController {
@ApiResponse({
status: 200,
description: "Match details with lineups, stats, odds, events",
schema: { type: "object" },
})
@ApiResponse({ status: 404, description: "Match not found" })
async getMatchDetails(@Param("id") id: string) {
+406 -43
View File
@@ -20,26 +20,55 @@ import {
@Injectable()
export class MatchesService {
private readonly logger = new Logger(MatchesService.name);
private qualifiedLeagueIds: string[] = [];
private topLeagueIds: string[] = [];
constructor(private readonly prisma: PrismaService) {
this.loadQualifiedLeagues();
this.loadTopLeagues();
}
private loadTopLeagues() {
try {
const topLeaguesPath = path.join(process.cwd(), "top_leagues.json");
if (fs.existsSync(topLeaguesPath)) {
this.topLeagueIds = JSON.parse(fs.readFileSync(topLeaguesPath, "utf8"));
this.logger.log(
`Loaded ${this.topLeagueIds.length} top leagues for filtering.`,
);
const filePath = path.join(process.cwd(), "top_leagues.json");
if (fs.existsSync(filePath)) {
this.topLeagueIds = JSON.parse(fs.readFileSync(filePath, "utf8"));
}
} catch (e) {
this.logger.warn(`Failed to load top_leagues.json: ${e.message}`);
}
}
private loadQualifiedLeagues() {
try {
const filePath = path.join(process.cwd(), "qualified_leagues.json");
if (fs.existsSync(filePath)) {
this.qualifiedLeagueIds = JSON.parse(fs.readFileSync(filePath, "utf8"));
this.logger.log(
`Loaded ${this.qualifiedLeagueIds.length} qualified leagues for filtering.`,
);
}
} catch (e) {
this.logger.warn(`Failed to load qualified_leagues.json: ${e.message}`);
}
}
/**
* Generate URL for the country flag served from Mackolik
*/
private getCountryFlagUrl(countryId?: string | null): string | undefined {
if (!countryId) return undefined;
return `https://file.mackolikfeeds.com/areas/${countryId}`;
}
/**
* Generate URL for the team logo served from local uploads
*/
private getTeamLogoUrl(teamId?: string | null): string | undefined {
if (!teamId) return undefined;
return `https://file.mackolikfeeds.com/teams/${teamId}`;
}
private getLiveFilter(): Prisma.LiveMatchWhereInput {
return {
OR: [
@@ -139,9 +168,9 @@ export class MatchesService {
if (leagueId) {
where.leagueId = leagueId;
} else if (status === "LIVE" && this.topLeagueIds.length > 0) {
// Filter live matches by top leagues by default if no leagueId is provided
where.leagueId = { in: this.topLeagueIds };
} else if (this.qualifiedLeagueIds.length > 0) {
// Only show matches from qualified leagues (leagues with historical data for AI analysis)
where.leagueId = { in: this.qualifiedLeagueIds };
}
if (status === "LIVE") {
@@ -298,7 +327,9 @@ export class MatchesService {
country: {
id: match.league?.country?.id || "",
name: match.league?.country?.name || "",
flagUrl: match.league?.country?.flagUrl || undefined,
flagUrl:
match.league?.country?.flagUrl ||
this.getCountryFlagUrl(match.league?.country?.id),
},
sport: sport,
matches: [],
@@ -353,11 +384,11 @@ export class MatchesService {
htScoreAway: undefined,
homeTeamName: match.homeTeam?.name || "Unknown",
homeTeamLogo: match.homeTeamId
? `https://file.mackolikfeeds.com/teams/${match.homeTeamId}`
? this.getTeamLogoUrl(match.homeTeamId)
: undefined,
awayTeamName: match.awayTeam?.name || "Unknown",
awayTeamLogo: match.awayTeamId
? `https://file.mackolikfeeds.com/teams/${match.awayTeamId}`
? this.getTeamLogoUrl(match.awayTeamId)
: undefined,
leagueName: match.league?.name,
countryName: match.league?.country?.name,
@@ -365,48 +396,62 @@ export class MatchesService {
});
}
return Array.from(leaguesMap.values());
return Array.from(leaguesMap.values()).sort((a, b) => {
const aIdx = this.topLeagueIds.indexOf(a.id);
const bIdx = this.topLeagueIds.indexOf(b.id);
const aPriority = aIdx === -1 ? 999 : aIdx;
const bPriority = bIdx === -1 ? 999 : bIdx;
if (aPriority !== bPriority) return aPriority - bPriority;
return (a.name || "").localeCompare(b.name || "");
});
}
/**
* Get active leagues with match counts
*/
async getActiveLeagues(sport: Sport): Promise<ActiveLeagueDto[]> {
// Start of today in UTC — same reference point as findMatches browse filter
const today = new Date();
today.setUTCHours(0, 0, 0, 0);
const todayMs = BigInt(today.getTime());
// Build finished statuses/states for exclusion (mirroring getBrowseFilter logic)
const finishedStatuses = FINISHED_STATUS_VALUES_FOR_DB;
const finishedStates = FINISHED_STATE_VALUES_FOR_DB;
const liveStatuses = LIVE_STATUS_VALUES_FOR_DB;
const liveStates = LIVE_STATE_VALUES_FOR_DB;
// Use raw query for complex aggregation
// Filter: (mstUtc >= today AND NOT finished) OR is currently live
const leagues = await this.prisma.$queryRaw<any[]>`
SELECT
l.id, l.name, l.code,
c.id as country_id,
c.name as country_name,
c.flag_url as country_flag,
COUNT(lm.id)::int as match_count,
COUNT(CASE WHEN lm.status IN ('LIVE', '1H', '2H', 'HT', '1Q', '2Q', '3Q', '4Q', 'Playing', 'Half Time')
OR lm.state IN ('live', 'firsthalf', 'secondhalf') THEN 1 END)::int as live_count
COUNT(CASE WHEN lm.status IN (${Prisma.join(liveStatuses)})
OR lm.state IN (${Prisma.join(liveStates)}) THEN 1 END)::int as live_count
FROM live_matches lm
JOIN leagues l ON lm.league_id = l.id
LEFT JOIN countries c ON l.country_id = c.id
WHERE lm.sport = ${sport}
${this.topLeagueIds.length > 0 ? Prisma.sql`AND l.id IN (${Prisma.join(this.topLeagueIds)})` : Prisma.empty}
GROUP BY l.id, l.name, l.code, c.name, c.flag_url
${this.qualifiedLeagueIds.length > 0 ? Prisma.sql`AND l.id IN (${Prisma.join(this.qualifiedLeagueIds)})` : Prisma.empty}
AND (
(lm.mst_utc >= ${todayMs} AND lm.status NOT IN (${Prisma.join(finishedStatuses)}) AND COALESCE(lm.state, '') NOT IN (${Prisma.join(finishedStates)}))
OR lm.status IN (${Prisma.join(liveStatuses)})
OR lm.state IN (${Prisma.join(liveStates)})
)
GROUP BY l.id, l.name, l.code, c.id, c.name, c.flag_url
ORDER BY l.name ASC
`;
// Priority sorting (Mackolik style)
const PRIORITY = [
"Trendyol Süper Lig",
"Süper Lig",
"Trendyol 1. Lig",
"1. Lig",
"Premier Lig",
"LaLiga",
"Serie A",
"Bundesliga",
"Ligue 1",
];
return leagues
.filter((l) => l.match_count > 0)
.sort((a, b) => {
const aIdx = PRIORITY.findIndex((p) => a.name?.includes(p));
const bIdx = PRIORITY.findIndex((p) => b.name?.includes(p));
const aIdx = this.topLeagueIds.indexOf(a.id);
const bIdx = this.topLeagueIds.indexOf(b.id);
const aPriority = aIdx === -1 ? 999 : aIdx;
const bPriority = bIdx === -1 ? 999 : bIdx;
@@ -419,7 +464,7 @@ export class MatchesService {
name: l.name,
code: l.code,
countryName: l.country_name,
countryFlag: l.country_flag,
countryFlag: l.country_flag || this.getCountryFlagUrl(l.country_id),
matchCount: l.match_count,
liveCount: l.live_count,
}));
@@ -458,13 +503,9 @@ export class MatchesService {
scoreAway: m.scoreAway,
status: m.status,
homeTeamName: m.homeTeam?.name,
homeTeamLogo: m.homeTeamId
? `https://file.mackolikfeeds.com/teams/${m.homeTeamId}`
: null,
homeTeamLogo: m.homeTeamId ? this.getTeamLogoUrl(m.homeTeamId) : null,
awayTeamName: m.awayTeam?.name,
awayTeamLogo: m.awayTeamId
? `https://file.mackolikfeeds.com/teams/${m.awayTeamId}`
: null,
awayTeamLogo: m.awayTeamId ? this.getTeamLogoUrl(m.awayTeamId) : null,
leagueName: m.league?.name,
countryName: m.league?.country?.name,
})),
@@ -586,7 +627,59 @@ export class MatchesService {
date: new Date(Number(liveMatch.mstUtc)),
// Fill missing relations with empty arrays
teamStats: [],
playerParticipations: [],
playerParticipations: (() => {
const parsed: Array<{
teamId: string;
isStarting: boolean;
shirtNumber: string | number | null;
position: string | null;
player: { id: string; name: string };
}> = [];
const canTrustFeedLineups =
displayStatus === "LIVE" || displayStatus === "Finished";
if (!canTrustFeedLineups) {
return parsed;
}
if (liveMatch.lineups && typeof liveMatch.lineups === "object") {
const lu = liveMatch.lineups as Record<string, any>;
const addPlayers = (teamLu: any, teamId: string | null) => {
if (!teamLu || !teamId) return;
if (teamLu.xi && Array.isArray(teamLu.xi)) {
teamLu.xi.forEach((p: any) => {
parsed.push({
teamId,
isStarting: true,
shirtNumber: p.shirtNumber || p.number,
position: p.position || p.pos,
player: {
id: p.personId || p.id || p.playerId || "unknown",
name:
p.matchName || p.name || p.playerName || "Bilinmiyor",
},
});
});
}
if (teamLu.subs && Array.isArray(teamLu.subs)) {
teamLu.subs.forEach((p: any) => {
parsed.push({
teamId,
isStarting: false,
shirtNumber: p.shirtNumber || p.number,
position: p.position || p.pos,
player: {
id: p.personId || p.id || p.playerId || "unknown",
name:
p.matchName || p.name || p.playerName || "Bilinmiyor",
},
});
});
}
};
addPlayers(lu.home, liveMatch.homeTeamId);
addPlayers(lu.away, liveMatch.awayTeamId);
}
return parsed;
})(),
playerEvents: [],
oddCategories: [], // Will handle odds parsing below
officials: [],
@@ -597,6 +690,65 @@ export class MatchesService {
if (!match) return null;
const detailDisplayStatus = getDisplayMatchStatus({
state: match.state,
status: match.status,
substate: match.substate,
scoreHome: match.scoreHome,
scoreAway: match.scoreAway,
});
const canTrustStoredLineups =
this.canTrustStoredLineups(detailDisplayStatus);
if (Array.isArray(match.playerParticipations)) {
if (!canTrustStoredLineups) {
match.playerParticipations = [];
}
const hasHomeLineup = match.playerParticipations.some(
(p: any) => p.teamId === match.homeTeamId && p.isStarting,
);
const hasAwayLineup = match.playerParticipations.some(
(p: any) => p.teamId === match.awayTeamId && p.isStarting,
);
if (!hasHomeLineup || !hasAwayLineup) {
const sidelined =
match.sidelined && typeof match.sidelined === "object"
? (match.sidelined as Record<string, any>)
: {};
const matchDateMs = Number(match.mstUtc || Date.now());
const probableLineups: any[] = [];
if (!hasHomeLineup && match.homeTeamId) {
probableLineups.push(
...(await this.buildProbableLineupForTeam({
teamId: match.homeTeamId,
beforeDateMs: matchDateMs,
sidelinedTeamData: sidelined.homeTeam,
})),
);
}
if (!hasAwayLineup && match.awayTeamId) {
probableLineups.push(
...(await this.buildProbableLineupForTeam({
teamId: match.awayTeamId,
beforeDateMs: matchDateMs,
sidelinedTeamData: sidelined.awayTeam,
})),
);
}
if (probableLineups.length > 0) {
match.playerParticipations = canTrustStoredLineups
? [...match.playerParticipations, ...probableLineups]
: probableLineups;
match.lineupSource = "probable_xi";
}
}
}
// Structure odds
const odds: Record<
string,
@@ -650,10 +802,19 @@ export class MatchesService {
teamStats: normalizedTeamStats,
mstUtc: Number(match.mstUtc),
date: match.date || new Date(Number(match.mstUtc)),
// Ensure score is in expected format (nested object for frontend if needed, but frontend seems to use match.score.home in some places and match.scoreHome in others.
// The match-detail-content uses match.score.home. Match entity has scoreHome/scoreAway fields.
// Let's ensure compatibility.
score: match.score || { home: match.scoreHome, away: match.scoreAway },
homeTeam: {
...match.homeTeam,
logo: match.homeTeamId
? this.getTeamLogoUrl(match.homeTeamId)
: match.homeTeam?.logoUrl || null,
},
awayTeam: {
...match.awayTeam,
logo: match.awayTeamId
? this.getTeamLogoUrl(match.awayTeamId)
: match.awayTeam?.logoUrl || null,
},
stats: {
home: this.normalizeTeamStat(homeStat, match.sport),
away: this.normalizeTeamStat(awayStat, match.sport),
@@ -699,4 +860,206 @@ export class MatchesService {
return team?.id || null;
}
private async buildProbableLineupForTeam(params: {
teamId: string;
beforeDateMs: number;
sidelinedTeamData?: any;
matchLimit?: number;
lookbackDays?: number;
maxStalenessDays?: number;
}) {
const matchLimit = params.matchLimit ?? 5;
const lookbackDays = params.lookbackDays ?? 370;
const maxStalenessDays = params.maxStalenessDays ?? 120;
const beforeDateMs = params.beforeDateMs || Date.now();
const minDateMs = Math.max(
0,
beforeDateMs - lookbackDays * 24 * 60 * 60 * 1000,
);
const excluded = this.extractSidelinedPlayerIds(params.sidelinedTeamData);
const rows = await this.prisma.$queryRaw<any[]>`
SELECT
mpp.player_id AS "playerId",
p.name AS "playerName",
mpp.position AS "position",
mpp.shirt_number AS "shirtNumber",
m.id AS "matchId",
m.mst_utc AS "mstUtc"
FROM match_player_participation mpp
JOIN matches m ON m.id = mpp.match_id
JOIN players p ON p.id = mpp.player_id
WHERE mpp.team_id = ${params.teamId}
AND mpp.is_starting = true
AND NOT EXISTS (
SELECT 1
FROM match_player_participation later_mpp
JOIN matches later_m ON later_m.id = later_mpp.match_id
WHERE later_mpp.player_id = mpp.player_id
AND later_mpp.team_id <> ${params.teamId}
AND later_m.mst_utc > m.mst_utc
AND later_m.mst_utc < ${BigInt(beforeDateMs)}
AND (
later_m.status = 'FT'
OR later_m.state = 'postGame'
OR (later_m.score_home IS NOT NULL AND later_m.score_away IS NOT NULL)
)
)
AND m.id IN (
SELECT m2.id
FROM matches m2
JOIN match_player_participation recent_mpp
ON recent_mpp.match_id = m2.id
AND recent_mpp.team_id = ${params.teamId}
AND recent_mpp.is_starting = true
WHERE (m2.home_team_id = ${params.teamId} OR m2.away_team_id = ${params.teamId})
AND (
m2.status = 'FT'
OR m2.state = 'postGame'
OR (m2.score_home IS NOT NULL AND m2.score_away IS NOT NULL)
)
AND m2.mst_utc < ${BigInt(beforeDateMs)}
AND m2.mst_utc >= ${BigInt(minDateMs)}
GROUP BY m2.id
HAVING COUNT(recent_mpp.*) >= 9
ORDER BY MAX(m2.mst_utc) DESC
LIMIT ${matchLimit}
)
ORDER BY m.mst_utc DESC
`;
if (!rows.length) return [];
const latestMst = Math.max(...rows.map((row) => Number(row.mstUtc || 0)));
const ageDays =
latestMst > 0
? (beforeDateMs - latestMst) / (24 * 60 * 60 * 1000)
: Number.POSITIVE_INFINITY;
const staleProjection = ageDays > maxStalenessDays;
const matchOrder = new Map<string, number>();
for (const row of rows) {
const matchId = String(row.matchId);
if (!matchOrder.has(matchId)) {
matchOrder.set(matchId, matchOrder.size);
}
}
const playerMap = new Map<
string,
{
playerId: string;
playerName: string;
position: string | null;
shirtNumber: number | null;
score: number;
starts: number;
lastSeenRank: number;
}
>();
for (const row of rows) {
const playerId = String(row.playerId);
if (excluded.has(playerId)) continue;
const rank = matchOrder.get(String(row.matchId)) ?? matchLimit;
const recencyWeight = Math.max(1, matchLimit - rank);
const score = recencyWeight + (rank === 0 ? 3 : rank === 1 ? 1.5 : 0);
const existing = playerMap.get(playerId);
if (!existing) {
playerMap.set(playerId, {
playerId,
playerName: row.playerName || "Bilinmiyor",
position: row.position ?? null,
shirtNumber:
row.shirtNumber === null || row.shirtNumber === undefined
? null
: Number(row.shirtNumber),
score,
starts: 1,
lastSeenRank: rank,
});
} else {
existing.score += score;
existing.starts += 1;
existing.lastSeenRank = Math.min(existing.lastSeenRank, rank);
existing.position = existing.position || row.position || null;
existing.shirtNumber =
existing.shirtNumber ??
(row.shirtNumber === null || row.shirtNumber === undefined
? null
: Number(row.shirtNumber));
}
}
const ranked = [...playerMap.values()]
.sort((a, b) => {
if (b.score !== a.score) return b.score - a.score;
if (b.starts !== a.starts) return b.starts - a.starts;
return a.lastSeenRank - b.lastSeenRank;
})
.slice(0, 11);
const coverage = Math.min(1, ranked.length / 11);
const historyScore = Math.min(1, matchOrder.size / matchLimit);
const stableCore = ranked.filter((p) => p.starts >= 2).length / 11;
const stalenessFactor = Math.max(
0.35,
Math.min(1, maxStalenessDays / Math.max(ageDays, 1)),
);
const confidence = Math.max(
0,
Math.min(
staleProjection ? 0.58 : 0.88,
(coverage * 0.45 + historyScore * 0.25 + stableCore * 0.3) *
stalenessFactor,
),
);
return ranked.map((p) => ({
teamId: params.teamId,
isStarting: true,
shirtNumber: p.shirtNumber,
position: p.position,
isProbable: true,
lineupSource: "probable_xi",
projectionConfidence: Number(confidence.toFixed(3)),
projectionAgeDays: Number(ageDays.toFixed(1)),
projectionStale: staleProjection,
projectionMatchLimit: matchLimit,
projectionLookbackDays: lookbackDays,
projectionMaxStalenessDays: maxStalenessDays,
player: {
id: p.playerId,
name: p.playerName,
},
}));
}
private extractSidelinedPlayerIds(teamData: any): Set<string> {
if (!teamData || typeof teamData !== "object") return new Set();
const players = Array.isArray(teamData.players) ? teamData.players : [];
return new Set(
players
.map((player: any) =>
String(
player?.playerId ??
player?.player_id ??
player?.id ??
player?.personId ??
"",
),
)
.filter(Boolean),
);
}
private canTrustStoredLineups(displayStatus?: string): boolean {
const normalized = String(displayStatus || "").toLowerCase();
return (
normalized === "live" || normalized === "finished" || normalized === "ft"
);
}
}
+45
View File
@@ -148,6 +148,27 @@ export class MatchPickDto {
@ApiProperty({ required: false, default: 0 })
implied_prob?: number;
@ApiProperty({ required: false, default: 0 })
model_probability?: number;
@ApiProperty({ required: false, default: 0 })
model_edge?: number;
@ApiProperty({ required: false, default: 0 })
calibrated_probability?: number;
@ApiProperty({ required: false, default: 0 })
odds_band_probability?: number;
@ApiProperty({ required: false, default: 0 })
odds_band_sample?: number;
@ApiProperty({ required: false, default: 0 })
odds_band_edge?: number;
@ApiProperty({ required: false, default: false })
odds_band_aligned?: boolean;
@ApiProperty()
play_score: number;
@@ -171,6 +192,9 @@ export class MatchPickDto {
enum: ["CORE", "VALUE", "LEAN", "LONGSHOT", "PASS"],
})
signal_tier?: SignalTier;
@ApiProperty({ required: false, default: false })
is_guaranteed?: boolean;
}
export class MatchBetAdviceDto {
@@ -227,6 +251,27 @@ export class MatchBetSummaryItemDto {
@ApiProperty({ required: false, default: 0 })
implied_prob?: number;
@ApiProperty({ required: false, default: 0 })
model_probability?: number;
@ApiProperty({ required: false, default: 0 })
model_edge?: number;
@ApiProperty({ required: false, default: 0 })
calibrated_probability?: number;
@ApiProperty({ required: false, default: 0 })
odds_band_probability?: number;
@ApiProperty({ required: false, default: 0 })
odds_band_sample?: number;
@ApiProperty({ required: false, default: 0 })
odds_band_edge?: number;
@ApiProperty({ required: false, default: false })
odds_band_aligned?: boolean;
@ApiProperty({ required: false, default: 0 })
odds?: number;
@@ -21,12 +21,17 @@ import {
GeneratePredictionDto,
SmartCouponRequestDto,
} from "./dto/predictions-request.dto";
import { Public } from "src/common/decorators";
import { CurrentUser } from "src/common/decorators";
import { AnalysisService } from "../analysis/analysis.service";
import { ForbiddenException } from "@nestjs/common";
@ApiTags("Predictions")
@Controller("predictions")
export class PredictionsController {
constructor(private readonly predictionsService: PredictionsService) {}
constructor(
private readonly predictionsService: PredictionsService,
private readonly analysisService: AnalysisService,
) {}
/**
* GET /predictions/health
@@ -56,6 +61,11 @@ export class PredictionsController {
*/
@Get("test/:id")
@ApiOperation({ summary: "Refetch match data and get prediction" })
@ApiResponse({
status: 200,
description: "Prediction details",
schema: { type: "object" },
})
@ApiParam({ name: "id", description: "Match ID" })
async getTestPrediction(@Param("id") id: string) {
return this.predictionsService.testPrediction(id);
@@ -88,27 +98,43 @@ export class PredictionsController {
* Get prediction for a specific match
*/
@Get(":matchId")
@Public()
@ApiOperation({ summary: "Get prediction for a specific match" })
@ApiParam({ name: "matchId", description: "Match ID" })
@ApiResponse({ status: 200, type: MatchPredictionDto })
@ApiResponse({
status: 200,
description: "Match prediction",
schema: { type: "object" },
type: MatchPredictionDto,
})
@ApiResponse({ status: 404, description: "Match not found" })
@ApiResponse({ status: 403, description: "Daily limit exceeded" })
async getPrediction(
@Param("matchId") matchId: string,
@CurrentUser() user: any,
): Promise<MatchPredictionDto> {
// Check cache first - DISABLED per user request to always fetch from scratch
// const cached = await this.predictionsService.getCachedPrediction(matchId);
// if (cached) {
// return cached;
// }
const canProceed = await this.analysisService.checkUsageLimit(
user.id,
false,
1,
);
if (!canProceed) {
throw new ForbiddenException("ANALYSIS_LIMIT_EXCEEDED");
}
const cached = await this.predictionsService.getCachedPrediction(matchId);
if (cached) {
await this.analysisService.recordUsage(user.id, false);
return cached;
}
// Get from AI Engine
const prediction = await this.predictionsService.getPredictionById(matchId);
if (!prediction) {
throw new NotFoundException(`Match not found: ${matchId}`);
throw new NotFoundException("MATCH_NOT_FOUND");
}
await this.analysisService.recordUsage(user.id, false);
return prediction;
}
@@ -120,17 +146,29 @@ export class PredictionsController {
@HttpCode(HttpStatus.OK)
@ApiOperation({ summary: "Generate prediction with provided match data" })
@ApiResponse({ status: 200, type: MatchPredictionDto })
@ApiResponse({ status: 403, description: "Daily limit exceeded" })
async generatePrediction(
@CurrentUser() user: any,
@Body() dto: GeneratePredictionDto,
): Promise<MatchPredictionDto> {
const canProceed = await this.analysisService.checkUsageLimit(
user.id,
false,
1,
);
if (!canProceed) {
throw new ForbiddenException("ANALYSIS_LIMIT_EXCEEDED");
}
const prediction = await this.predictionsService.getPredictionWithData({
matchId: dto.matchId,
});
if (!prediction) {
throw new NotFoundException("Failed to generate prediction");
throw new NotFoundException("PREDICTION_GENERATION_FAILED");
}
await this.analysisService.recordUsage(user.id, false);
return prediction;
}
@@ -146,8 +184,22 @@ export class PredictionsController {
@ApiResponse({
status: 200,
description: "Smart coupon generated successfully",
schema: { type: "object" },
})
async generateSmartCoupon(@Body() dto: SmartCouponRequestDto): Promise<any> {
@ApiResponse({ status: 403, description: "Daily limit exceeded" })
async generateSmartCoupon(
@CurrentUser() user: any,
@Body() dto: SmartCouponRequestDto,
): Promise<any> {
const canProceed = await this.analysisService.checkUsageLimit(
user.id,
true,
dto.matchIds?.length || 1,
);
if (!canProceed) {
throw new ForbiddenException("COUPON_LIMIT_EXCEEDED");
}
const coupon = await this.predictionsService.getSmartCoupon(
dto.matchIds,
dto.strategy || "BALANCED",
@@ -158,9 +210,10 @@ export class PredictionsController {
);
if (!coupon) {
throw new NotFoundException("Failed to generate Smart Coupon");
throw new NotFoundException("SMART_COUPON_GENERATION_FAILED");
}
await this.analysisService.recordUsage(user.id, true);
return coupon;
}
}
@@ -10,6 +10,7 @@ import { PredictionsQueue } from "./queues/predictions.queue";
import { PredictionsProcessor } from "./queues/predictions.processor";
import { PREDICTIONS_QUEUE } from "./queues/predictions.types";
import { FeederModule } from "../feeder/feeder.module";
import { AnalysisModule } from "../analysis/analysis.module";
const redisEnabled = process.env.REDIS_ENABLED === "true";
@@ -25,6 +26,7 @@ const redisEnabled = process.env.REDIS_ENABLED === "true";
: []),
MatchesModule,
FeederModule,
AnalysisModule,
],
controllers: [PredictionsController],
providers: [
+357 -53
View File
@@ -21,6 +21,10 @@ import {
} from "./dto";
import { Prisma } from "@prisma/client";
import { FeederService } from "../feeder/feeder.service";
import {
isMatchCompleted,
isMatchLive,
} from "../../common/utils/match-status.util";
import * as fs from "node:fs";
import * as path from "node:path";
import {
@@ -49,14 +53,14 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
private queueEvents: QueueEvents | null = null;
private readonly aiEngineUrl: string;
private readonly aiEngineClient: AiEngineClient;
private readonly topLeagueIds = new Set<string>();
private readonly qualifiedLeagueIds = new Set<string>();
private readonly reasonTranslations: Record<string, string> = {
confidence_below_threshold: "Güven eşiğin altında",
confidence_interval_too_wide: "Güven aralığı çok geniş",
confidence_interval_too_wide_for_main_pick:
"Ana seçim için güven aralığı çok geniş",
confidence_band_low: "Güven bandı düşük",
playable_edge_found: "Oynanabilir avantaj bulundu",
playable_edge_found: "Model avantaj sinyali bulundu",
market_signal_dominant: "Piyasa sinyali baskın",
team_form_signal_dominant: "Takım formuna dayalı sinyaller çok baskın",
lineup_signal_strong: "İlk on bir sinyali güçlü",
@@ -73,7 +77,12 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
limited_data_confidence: "Veri kısıtlı olduğu için güven sınırlı",
data_quality_issue: "Veri kalitesi sorunu var",
high_risk_low_data_quality: "Risk yüksek, veri kalitesi düşük",
insufficient_play_score: "Oynanabilirlik puanı yetersiz",
insufficient_play_score: "Model sinyali yetersiz",
odds_band_confirms_value: "Tarihsel oran bandı değeri doğruluyor",
odds_band_sample_too_low: "Tarihsel oran bandı örneklemi yetersiz",
odds_band_missing_probability: "Tarihsel oran bandı olasılığı yok",
odds_band_unavailable: "Tarihsel oran bandı kullanılamıyor",
odds_band_not_aligned: "Model ve tarihsel oran bandı aynı yönde değil",
no_bet_conditions_met: "Bahis koşulları oluşmadı",
market_passed_all_gates: "Market tüm güvenlik kontrollerini geçti",
no_ev_edge_minimum_stake:
@@ -125,10 +134,7 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
private readonly feederService: FeederService,
@Optional() private readonly predictionsQueue?: PredictionsQueue,
) {
this.aiEngineUrl = this.configService.get(
"AI_ENGINE_URL",
"http://localhost:8000",
);
this.aiEngineUrl = this.resolveAiEngineUrl();
this.aiEngineClient = new AiEngineClient({
baseUrl: this.aiEngineUrl,
logger: this.logger,
@@ -137,7 +143,7 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
maxRetries: 2,
retryDelayMs: 750,
});
this.topLeagueIds = this.loadTopLeagueIds();
this.qualifiedLeagueIds = this.loadQualifiedLeagueIds();
}
onModuleInit() {
@@ -155,6 +161,11 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
}
}
private predictionMemCache = new Map<
string,
{ timestamp: number; payload: MatchPredictionDto }
>();
async onModuleDestroy() {
if (this.queueEvents) {
await this.queueEvents.close();
@@ -177,8 +188,7 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
return {
status: response.data?.status || "healthy",
modelLoaded: response.data?.model_loaded ?? true,
predictionServiceReady:
response.data?.prediction_service_ready ?? true,
predictionServiceReady: response.data?.prediction_service_ready ?? true,
aiEngineReachable: true,
circuitState: circuit.state,
consecutiveFailures: circuit.consecutiveFailures,
@@ -223,11 +233,13 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
`/v20plus/analyze/${matchId}`,
{ simulate: true, is_simulation: true, pre_match_only: true },
);
await this.recordPredictionRun(matchId, response.data);
return this.enrichPredictionResponse(
response.data as MatchPredictionDto,
const prediction = this.enrichPredictionResponse(
response.data,
matchContext,
);
await this.recordPredictionRun(matchId, response.data);
await this.cachePrediction(matchId, prediction);
return prediction;
} catch (e: unknown) {
const requestError =
e instanceof AiEngineRequestError
@@ -235,11 +247,69 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
: new AiEngineRequestError("AI Engine request failed");
const status = requestError.status;
const detail = requestError.detail || requestError.message;
// ── Cooldown fallback cascade: memCache → DB stored → DB cached → wait & retry ──
if (
status === HttpStatus.SERVICE_UNAVAILABLE &&
this.hasCooldown(detail)
) {
// 1) In-memory cache (10min TTL)
const memCached = this.predictionMemCache.get(matchId);
if (memCached && Date.now() - memCached.timestamp < 10 * 60 * 1000) {
this.logger.warn(
`AI Engine cooldown for ${matchId}; returning mem-cached prediction`,
);
return memCached.payload;
}
// 2) DB stored prediction (no TTL filter)
const storedPrediction = await this.getStoredPrediction(matchId);
if (storedPrediction) {
this.logger.warn(
`AI Engine cooldown for ${matchId}; returning stored prediction`,
);
return this.enrichPredictionResponse(storedPrediction, matchContext);
}
// 3) DB cached prediction (with model version check)
const cachedPrediction = await this.getCachedPrediction(matchId);
if (cachedPrediction) {
this.logger.warn(
`AI Engine cooldown for ${matchId}; returning cached prediction`,
);
return this.enrichPredictionResponse(cachedPrediction, matchContext);
}
// 4) No cached data at all — return null gracefully
this.logger.warn(
`AI Engine cooldown for ${matchId}; no cached data available — returning null gracefully`,
);
return null;
}
// ── Non-cooldown errors (e.g. AI Engine 500 for this match) ──
// Try DB fallback before giving up
const storedFallback = await this.getStoredPrediction(matchId);
if (storedFallback) {
this.logger.warn(
`AI Engine failed for ${matchId} (status=${status}); returning stored prediction as fallback`,
);
return this.enrichPredictionResponse(storedFallback, matchContext);
}
const cachedFallback = await this.getCachedPrediction(matchId);
if (cachedFallback) {
this.logger.warn(
`AI Engine failed for ${matchId} (status=${status}); returning cached prediction as fallback`,
);
return this.enrichPredictionResponse(cachedFallback, matchContext);
}
this.logger.error(
`Direct AI Engine call failed for ${matchId}: status=${status}, detail=${JSON.stringify(detail)}`,
);
// Forward AI Engine's actual error
// Forward AI Engine's actual error for client-meaningful statuses
if (status === 404) {
throw new HttpException(
`Match not found in AI Engine: ${matchId}`,
@@ -252,10 +322,13 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
HttpStatus.UNPROCESSABLE_ENTITY,
);
}
throw new HttpException(
`AI Engine error: ${typeof detail === "string" ? detail : JSON.stringify(detail)}`,
status || HttpStatus.SERVICE_UNAVAILABLE,
// For server errors (500, 503 etc.) return null instead of throwing
// This prevents the user from seeing raw 503 errors
this.logger.warn(
`AI Engine server error for ${matchId}; returning null gracefully instead of ${status}`,
);
return null;
}
}
@@ -314,37 +387,95 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
match_date_ms: Number(p.match.mstUtc) * 1000,
league: p.match.league?.name || "",
league_id: p.match.leagueId,
is_top_league: this.topLeagueIds.has(p.match.leagueId ?? ""),
is_top_league: this.qualifiedLeagueIds.has(p.match.leagueId ?? ""),
},
} as unknown as MatchPredictionDto;
}),
};
}
private loadTopLeagueIds(): Set<string> {
private loadQualifiedLeagueIds(): Set<string> {
try {
const topLeaguesPath = path.join(process.cwd(), "top_leagues.json");
if (!fs.existsSync(topLeaguesPath)) {
const filePath = path.join(process.cwd(), "qualified_leagues.json");
if (!fs.existsSync(filePath)) {
this.logger.warn(
"qualified_leagues.json not found — all leagues allowed",
);
return new Set<string>();
}
const raw = JSON.parse(fs.readFileSync(topLeaguesPath, "utf8"));
const raw = JSON.parse(fs.readFileSync(filePath, "utf8"));
if (!Array.isArray(raw)) {
return new Set<string>();
}
return new Set(
const ids = new Set(
raw
.map((value) => String(value).trim())
.filter((value) => value.length > 0),
);
this.logger.log(`Loaded ${ids.size} qualified league IDs`);
return ids;
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
this.logger.warn(`Failed to load top_leagues.json: ${message}`);
this.logger.warn(`Failed to load qualified_leagues.json: ${message}`);
return new Set<string>();
}
}
private resolveAiEngineUrl(): string {
const configuredUrl = this.configService.get(
"AI_ENGINE_URL",
"http://localhost:8000",
);
const localEnvUrl = this.readLocalEnvValue("AI_ENGINE_URL");
if (
process.env.NODE_ENV !== "production" &&
localEnvUrl &&
localEnvUrl !== configuredUrl &&
this.isLocalhostUrl(configuredUrl) &&
this.isLocalhostUrl(localEnvUrl)
) {
this.logger.warn(
`AI_ENGINE_URL inherited from parent process (${configuredUrl}) differs from .env.local (${localEnvUrl}); using .env.local for local development`,
);
return localEnvUrl;
}
return configuredUrl;
}
private readLocalEnvValue(key: string): string | null {
const filePath = path.join(process.cwd(), ".env.local");
if (!fs.existsSync(filePath)) {
return null;
}
const line = fs
.readFileSync(filePath, "utf8")
.split(/\r?\n/u)
.find((entry) => entry.trim().startsWith(`${key}=`));
if (!line) {
return null;
}
return line
.slice(line.indexOf("=") + 1)
.trim()
.replace(/^['"]|['"]$/gu, "");
}
private isLocalhostUrl(value: string): boolean {
try {
const url = new URL(value);
return ["localhost", "127.0.0.1", "::1"].includes(url.hostname);
} catch {
return false;
}
}
private async getMatchContext(matchId: string): Promise<MatchContext> {
const match = await this.prisma.match.findUnique({
where: { id: matchId },
@@ -354,7 +485,7 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
if (match) {
return {
leagueId: match.leagueId ?? null,
isTopLeague: this.topLeagueIds.has(match.leagueId ?? ""),
isTopLeague: this.qualifiedLeagueIds.has(match.leagueId ?? ""),
};
}
@@ -365,7 +496,7 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
return {
leagueId: liveMatch?.leagueId ?? null,
isTopLeague: this.topLeagueIds.has(liveMatch?.leagueId ?? ""),
isTopLeague: this.qualifiedLeagueIds.has(liveMatch?.leagueId ?? ""),
};
}
@@ -629,6 +760,7 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
),
confidence_interval: interval,
signal_tier: this.classifySignalTier(record, interval),
is_guaranteed: false,
};
}
@@ -674,6 +806,11 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
odds: this.normalizeDisplayOdds(odds, impliedProb),
implied_prob: impliedProb,
ev_edge: evEdge,
playable: Boolean(record.playable) && interval.threshold_met,
stake_units:
Boolean(record.playable) && interval.threshold_met
? this.asNumber(record.stake_units)
: 0,
reasons: Array.isArray(record.reasons)
? record.reasons.map((reason) => this.translateReason(String(reason)))
: [],
@@ -710,20 +847,27 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
return this.reasonTranslations[normalized];
}
const evMatch = normalized.match(/^ev_edge_([+\-][\d.]+%)_grade_(\w)$/);
const evMatch = normalized.match(/^ev_edge_([-+][\d.]+%)_grade_(\w)$/);
if (evMatch) {
return `Beklenen avantaj ${evMatch[1]} (Not ${evMatch[2]})`;
return `Teorik avantaj sinyali: Not ${evMatch[2]}`;
}
const negativeEdgeMatch = normalized.match(
/^negative_model_edge_([+\-]?[\d.]+)$/,
/^negative_model_edge_([-+]?[\d.]+)$/,
);
if (negativeEdgeMatch) {
return `Model avantajı negatif (${negativeEdgeMatch[1]})`;
}
const bandNoValueMatch = normalized.match(
/^odds_band_no_value_([-+]?[\d.]+)$/,
);
if (bandNoValueMatch) {
return `Tarihsel oran bandı değeri doğrulamadı (${bandNoValueMatch[1]})`;
}
const edgeThresholdMatch = normalized.match(
/^below_market_edge_threshold_([+\-]?[\d.]+)$/,
/^below_market_edge_threshold_([-+]?[\d.]+)$/,
);
if (edgeThresholdMatch) {
return `Piyasa avantaj eşiğinin altında (${edgeThresholdMatch[1]})`;
@@ -919,15 +1063,39 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
return 0;
}
const normalizedPick = pickName.toUpperCase();
const normalizedPick = this.normalizePickKey(pickName);
for (const [key, value] of Object.entries(probabilities)) {
if (key.toUpperCase() === normalizedPick) {
if (this.normalizePickKey(key) === normalizedPick) {
return this.asNumber(value);
}
}
return 0;
}
private normalizePickKey(value: string): string {
const normalized = value.trim().toUpperCase();
const aliases: Record<string, string> = {
ÜST: "OVER",
UST: "OVER",
OVER: "OVER",
ALT: "UNDER",
UNDER: "UNDER",
"KG VAR": "YES",
VAR: "YES",
YES: "YES",
"KG YOK": "NO",
YOK: "NO",
NO: "NO",
TEK: "ODD",
ODD: "ODD",
ÇİFT: "EVEN",
CIFT: "EVEN",
EVEN: "EVEN",
};
return aliases[normalized] ?? normalized;
}
private impliedProbabilityFromOdds(odds: number): number {
if (odds <= 1) {
return 0;
@@ -1026,10 +1194,11 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
// Direct HTTP mode
try {
const response = await this.aiEngineClient.post(
"/smart-coupon",
{ match_ids: matchIds, strategy, ...options },
);
const response = await this.aiEngineClient.post("/smart-coupon", {
match_ids: matchIds,
strategy,
...options,
});
return response.data;
} catch (error: unknown) {
const message =
@@ -1085,8 +1254,26 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
}
async cachePrediction(matchId: string, prediction: MatchPredictionDto) {
this.predictionMemCache.set(matchId, {
timestamp: Date.now(),
payload: prediction,
});
if (this.predictionMemCache.size > 500) {
const firstKey = this.predictionMemCache.keys().next().value;
if (firstKey) this.predictionMemCache.delete(firstKey);
}
const payload = prediction as unknown as Prisma.InputJsonObject;
try {
const existsInMatch = await this.prisma.match.findUnique({
where: { id: matchId },
select: { id: true },
});
if (!existsInMatch) {
return;
}
await this.prisma.prediction.upsert({
where: { matchId },
update: {
@@ -1106,6 +1293,16 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
async getCachedPrediction(
matchId: string,
): Promise<MatchPredictionDto | null> {
const memCached = this.predictionMemCache.get(matchId);
if (memCached) {
if (Date.now() - memCached.timestamp < 10 * 60 * 1000) {
// 10 mins TTL
return memCached.payload;
} else {
this.predictionMemCache.delete(matchId);
}
}
const prediction = await this.prisma.prediction.findUnique({
where: { matchId },
});
@@ -1132,6 +1329,49 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
return prediction.predictionJson as unknown as MatchPredictionDto;
}
private async getStoredPrediction(
matchId: string,
): Promise<MatchPredictionDto | null> {
const prediction = await this.prisma.prediction.findUnique({
where: { matchId },
});
return prediction
? (prediction.predictionJson as unknown as MatchPredictionDto)
: null;
}
private hasCooldown(detail: unknown): boolean {
if (typeof detail === "string") {
return detail.includes("cooldownRemainingMs");
}
if (detail && typeof detail === "object") {
return "cooldownRemainingMs" in detail;
}
return false;
}
private extractCooldownMs(detail: unknown): number {
if (
detail &&
typeof detail === "object" &&
"cooldownRemainingMs" in detail
) {
return (
Number((detail as Record<string, unknown>).cooldownRemainingMs) || 0
);
}
if (typeof detail === "string") {
const match = detail.match(/cooldownRemainingMs[":\s]+(\d+)/);
return match ? Number(match[1]) : 0;
}
return 0;
}
private async ensureSmartCouponDataReady(matchIds: string[]): Promise<void> {
const uniqueMatchIds = [...new Set(matchIds.filter((id) => !!id))];
if (uniqueMatchIds.length === 0) {
@@ -1147,7 +1387,8 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
}
private async ensurePredictionDataReady(matchId: string): Promise<void> {
const [liveMatch, persistedMatch, oddCategoryCount] = await Promise.all([
const [liveMatch, persistedMatch, oddCategoryCount, lineupCount] =
await Promise.all([
this.prisma.liveMatch.findUnique({
where: { id: matchId },
select: {
@@ -1157,6 +1398,7 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
status: true,
scoreHome: true,
scoreAway: true,
leagueId: true,
},
}),
this.prisma.match.findUnique({
@@ -1167,11 +1409,15 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
status: true,
scoreHome: true,
scoreAway: true,
leagueId: true,
},
}),
this.prisma.oddCategory.count({
where: { matchId },
}),
this.prisma.matchPlayerParticipation.count({
where: { matchId },
}),
]);
const hasLiveOdds =
@@ -1188,27 +1434,68 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
);
}
// League qualification gate: reject predictions for leagues without
// sufficient historical training data (odds + lineups + stats)
const leagueId = liveMatch?.leagueId || persistedMatch?.leagueId;
if (
this.qualifiedLeagueIds.size > 0 &&
(!leagueId || !this.qualifiedLeagueIds.has(leagueId))
) {
throw new HttpException(
`Bu lig için yeterli geçmiş veri bulunmuyor. Tahmin yapılamaz.`,
HttpStatus.UNPROCESSABLE_ENTITY,
);
}
const state = liveMatch?.state || persistedMatch?.state;
const status = liveMatch?.status || persistedMatch?.status;
const scoreHome = liveMatch?.scoreHome ?? persistedMatch?.scoreHome;
const scoreAway = liveMatch?.scoreAway ?? persistedMatch?.scoreAway;
const hasScores =
scoreHome !== null &&
scoreHome !== undefined &&
scoreAway !== null &&
scoreAway !== undefined;
const isFinished =
hasScores ||
state === "MS" ||
state === "postGame" ||
["Finished", "Played", "FT", "AET", "PEN", "Ended"].includes(
status as string,
);
const isFinished = isMatchCompleted({
state: state ?? null,
status: status ?? null,
scoreHome,
scoreAway,
});
const isLive = isMatchLive({
state: state ?? null,
status: status ?? null,
});
const hasOdds = hasLiveOdds || oddCategoryCount > 0;
if (hasOdds || isFinished) {
if (hasOdds || isFinished || isLive) {
// ── Lineup guard: fetch lineups if missing before analysis ──
// A proper football lineup has at least 11 starting players (22 total
// with subs). If we have fewer than 11 participation records, the
// lineup data is likely missing — attempt to fetch it from source.
if (lineupCount < 11) {
this.logger.log(
`[${matchId}] ⚠️ Lineups missing (${lineupCount} players in DB). Fetching from source before analysis...`,
);
try {
const refreshResult = await this.feederService.refreshMatch(
matchId,
"lineups",
);
if (refreshResult.success) {
this.logger.log(
`[${matchId}] ✅ Lineups fetched successfully before analysis`,
);
} else {
this.logger.warn(
`[${matchId}] ⚠️ Lineup fetch returned failure — proceeding with existing data. Error: ${refreshResult.error ?? "unknown"}`,
);
}
} catch (err: unknown) {
const message = err instanceof Error ? err.message : String(err);
this.logger.warn(
`[${matchId}] ⚠️ Lineup fetch exception — proceeding with existing data. ${message}`,
);
}
}
return;
}
@@ -1246,7 +1533,9 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
);
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
this.logger.warn(`Prediction run audit skipped for ${matchId}: ${message}`);
this.logger.warn(
`Prediction run audit skipped for ${matchId}: ${message}`,
);
}
}
@@ -1294,8 +1583,15 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
pick: item.pick,
playable: item.playable,
bet_grade: item.bet_grade,
odds: item.odds,
model_edge: item.model_edge,
calibrated_probability: item.calibrated_probability,
calibrated_confidence: item.calibrated_confidence,
ev_edge: item.ev_edge ?? 0,
odds_band_probability: item.odds_band_probability,
odds_band_sample: item.odds_band_sample,
odds_band_edge: item.odds_band_edge,
odds_band_aligned: item.odds_band_aligned,
stake_units: item.stake_units,
}))
: [];
@@ -1311,8 +1607,15 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
pick: payload.main_pick.pick,
playable: payload.main_pick.playable,
bet_grade: payload.main_pick.bet_grade,
odds: payload.main_pick.odds,
model_edge: payload.main_pick.model_edge,
calibrated_probability: payload.main_pick.calibrated_probability,
calibrated_confidence: payload.main_pick.calibrated_confidence,
ev_edge: payload.main_pick.ev_edge ?? 0,
odds_band_probability: payload.main_pick.odds_band_probability,
odds_band_sample: payload.main_pick.odds_band_sample,
odds_band_edge: payload.main_pick.odds_band_edge,
odds_band_aligned: payload.main_pick.odds_band_aligned,
stake_units: payload.main_pick.stake_units,
}
: null,
@@ -1322,14 +1625,15 @@ export class PredictionsService implements OnModuleInit, OnModuleDestroy {
pick: payload.value_pick.pick,
playable: payload.value_pick.playable,
bet_grade: payload.value_pick.bet_grade,
odds: payload.value_pick.odds,
model_edge: payload.value_pick.model_edge,
calibrated_confidence: payload.value_pick.calibrated_confidence,
ev_edge: payload.value_pick.ev_edge ?? 0,
}
: null,
bet_advice: {
playable: payload.bet_advice?.playable ?? false,
suggested_stake_units:
payload.bet_advice?.suggested_stake_units ?? 0,
suggested_stake_units: payload.bet_advice?.suggested_stake_units ?? 0,
reason: payload.bet_advice?.reason ?? null,
},
top_summary: topSummary,
@@ -1,6 +1,8 @@
import { Injectable, Logger } from "@nestjs/common";
import { ConfigService } from "@nestjs/config";
import { GeminiService } from "../gemini/gemini.service";
import { PredictionCardDto } from "./dto/prediction-card.dto";
import axios from "axios";
const SYSTEM_PROMPT = `Sen profesyonel bir spor analisti ve sosyal medya içerik üreticisisin.
Verilen maç tahmin verisini kullanarak kısa, etkili ve ilgi çekici sosyal medya postları yazıyorsun.
@@ -11,6 +13,7 @@ KURALLAR:
- Emoji kullan ama abartma (2-4 emoji yeterli)
- Skor tahminini vurgula
- Güven yüzdesini belirt
- Lig, ülke, takım adları ve ana tahminleri SEO için doğal şekilde geçir
- İlgili hashtag'leri ekle (#PremierLeague, #SüperLig vb.)
- KESİNLİKLE "kesin kazanır", "garanti" gibi ifadeler KULLANMA
- "Tahminimiz", "Beklentimiz", "Analizimiz" gibi ifadeler kullan
@@ -20,13 +23,31 @@ KURALLAR:
@Injectable()
export class CaptionGeneratorService {
private readonly logger = new Logger(CaptionGeneratorService.name);
private readonly ollamaBaseUrl: string;
private readonly ollamaModel: string;
constructor(private readonly geminiService: GeminiService) {}
constructor(
private readonly geminiService: GeminiService,
private readonly configService: ConfigService,
) {
this.ollamaBaseUrl =
this.configService.get<string>("OLLAMA_BASE_URL") ||
"http://localhost:11434";
this.ollamaModel =
this.configService.get<string>("OLLAMA_MODEL") ||
this.configService.get<string>("SOCIAL_POSTER_OLLAMA_MODEL") ||
"";
}
/**
* Generate a social media caption for a match prediction using Gemini AI.
*/
async generateCaption(card: PredictionCardDto): Promise<string> {
if (this.ollamaModel) {
const caption = await this.generateWithOllama(card);
if (caption) return caption;
}
if (!this.geminiService.isAvailable()) {
this.logger.warn("Gemini not available, using template caption");
return this.generateFallbackCaption(card);
@@ -53,6 +74,39 @@ export class CaptionGeneratorService {
}
}
private async generateWithOllama(card: PredictionCardDto): Promise<string> {
const prompt = `${SYSTEM_PROMPT}
${this.buildPrompt(card)}`;
try {
const response = await axios.post(
`${this.ollamaBaseUrl.replace(/\/$/, "")}/api/generate`,
{
model: this.ollamaModel,
prompt,
stream: false,
options: {
temperature: 0.7,
num_predict: 260,
},
},
{ timeout: 20000 },
);
const text = String(response.data?.response || "").trim();
if (!text) return "";
this.logger.log(
`Ollama caption generated for ${card.homeTeam} vs ${card.awayTeam}`,
);
return this.ensureHashtags(text, card);
} catch (error) {
this.logger.warn(`Ollama caption generation failed: ${error.message}`);
return "";
}
}
private buildPrompt(card: PredictionCardDto): string {
const topPicksText = card.topPicks
.map(
@@ -64,9 +118,11 @@ export class CaptionGeneratorService {
return `Aşağıdaki maç tahmin verisini kullanarak bir sosyal medya postu oluştur:
MAÇ: ${card.homeTeam} vs ${card.awayTeam}
SPOR: ${card.sport === "basketball" ? "Basketbol" : "Futbol"}
LİG: ${card.leagueName}
ÜLKE/BÖLGE: ${card.countryName || "-"}
TARİH: ${card.matchDate}
İLK YARI SKOR TAHMİNİ: ${card.htScore}
${card.sport === "basketball" ? "İLK DEVRE" : "İLK YARI"} SKOR TAHMİNİ: ${card.htScore}
MAÇ SONU SKOR TAHMİNİ: ${card.ftScore}
SKOR GÜVEN: %${card.scoreConfidence}
RİSK SEVİYESİ: ${card.riskLevel}
@@ -85,7 +141,8 @@ Sadece post metnini yaz, başka hiçbir şey ekleme.`;
.replace(/[^a-zA-Z0-9üöçşğıİÜÖÇŞĞ]/g, "");
const homeTag = card.homeTeam.replace(/\s+/g, "");
const awayTag = card.awayTeam.replace(/\s+/g, "");
text += `\n\n#${leagueTag} #${homeTag} #${awayTag}`;
const sportTag = card.sport === "basketball" ? "Basketbol" : "Futbol";
text += `\n\n#${leagueTag} #${homeTag} #${awayTag} #${sportTag}`;
}
return text.trim();
}
@@ -99,11 +156,14 @@ Sadece post metnini yaz, başka hiçbir şey ekleme.`;
.replace(/\s+/g, "")
.replace(/[^a-zA-Z0-9üöçşğıİÜÖÇŞĞ]/g, "");
return `${card.homeTeam} vs ${card.awayTeam}
🎯 Tahminimiz: ${card.ftScore} (İY: ${card.htScore})
const sportLabel = card.sport === "basketball" ? "Basketbol" : "Futbol";
const halfLabel = card.sport === "basketball" ? "İD" : "İY";
return `${card.leagueName}${card.countryName ? ` (${card.countryName})` : ""}: ${card.homeTeam} vs ${card.awayTeam}
🎯 ${sportLabel} tahminimiz: ${card.ftScore} (${halfLabel}: ${card.htScore})
📊 Güven: %${card.scoreConfidence}
${topPick ? `🔥 ${topPick.market}: ${topPick.pick} (%${topPick.confidence})` : ""}
#${leagueTag} #SuggestBet #Bahis`.trim();
#${leagueTag} #${sportLabel} #MaçTahmini #iddaai`.trim();
}
}
@@ -21,12 +21,15 @@ export interface TopPick {
export interface PredictionCardDto {
// ─── Match Info ───
matchId: string;
sport: "football" | "basketball";
homeTeam: string;
awayTeam: string;
homeLogo: string;
awayLogo: string;
leagueName: string;
leagueLogo?: string;
countryName?: string;
countryFlag?: string;
/** Formatted date, e.g. "01 Mar 2026 - 21:00" */
matchDate: string;
File diff suppressed because it is too large Load Diff
+16 -3
View File
@@ -10,13 +10,16 @@ export class MetaService {
private readonly pageId: string;
private readonly igUserId: string;
private readonly isEnabled: boolean;
private readonly graphApiBase = "https://graph.facebook.com/v21.0";
private readonly graphApiBase: string;
constructor(private readonly configService: ConfigService) {
this.pageAccessToken =
this.configService.get<string>("META_PAGE_ACCESS_TOKEN") || "";
this.pageId = this.configService.get<string>("META_PAGE_ID") || "";
this.igUserId = this.configService.get<string>("META_IG_USER_ID") || "";
const graphVersion =
this.configService.get<string>("META_GRAPH_API_VERSION") || "v25.0";
this.graphApiBase = `https://graph.facebook.com/${graphVersion}`;
this.isEnabled = !!(this.pageAccessToken && this.pageId);
@@ -63,11 +66,12 @@ export class MetaService {
{
url: imageUrl,
message,
published: true,
access_token: this.pageAccessToken,
},
);
const postId = response.data?.id;
const postId = response.data?.post_id || response.data?.id;
this.logger.log(`✅ Facebook post published: ${postId}`);
return postId || null;
} catch (error) {
@@ -109,6 +113,7 @@ export class MetaService {
{
image_url: imageUrl,
caption,
alt_text: this.buildAltText(caption),
access_token: this.pageAccessToken,
},
);
@@ -156,7 +161,7 @@ export class MetaService {
`${this.graphApiBase}/${containerId}`,
{
params: {
fields: "status_code",
fields: "status_code,status",
access_token: this.pageAccessToken,
},
},
@@ -177,4 +182,12 @@ export class MetaService {
this.logger.warn("Container wait timed out, attempting publish anyway");
}
private buildAltText(caption: string): string {
return caption
.replace(/#[^\s#]+/g, "")
.replace(/\s+/g, " ")
.trim()
.slice(0, 900);
}
}
@@ -19,6 +19,11 @@ import {
// Top leagues loaded once
const TOP_LEAGUES_PATH = path.join(process.cwd(), "top_leagues.json");
const POSTED_STATE_PATH = path.join(
process.cwd(),
"storage",
"social-poster-posted.json",
);
@Injectable()
export class SocialPosterService {
@@ -26,6 +31,9 @@ export class SocialPosterService {
private readonly aiEngineUrl: string;
private readonly appBaseUrl: string;
private readonly isEnabled: boolean;
private readonly sports: string[];
private readonly windowMinMinutes: number;
private readonly windowMaxMinutes: number;
private readonly postedMatchIds = new Set<string>();
private topLeagueIds: Set<string> = new Set();
@@ -44,8 +52,22 @@ export class SocialPosterService {
this.configService.get<string>("APP_BASE_URL") || "http://localhost:3000";
this.isEnabled =
this.configService.get<string>("SOCIAL_POSTER_ENABLED") === "true";
this.sports = (
this.configService.get<string>("SOCIAL_POSTER_SPORTS") ||
"football,basketball"
)
.split(",")
.map((sport) => sport.trim())
.filter(Boolean);
this.windowMinMinutes = Number(
this.configService.get<string>("SOCIAL_POSTER_WINDOW_MIN") || 25,
);
this.windowMaxMinutes = Number(
this.configService.get<string>("SOCIAL_POSTER_WINDOW_MAX") || 45,
);
this.loadTopLeagues();
this.loadPostedState();
}
private loadTopLeagues() {
@@ -59,16 +81,45 @@ export class SocialPosterService {
}
}
private loadPostedState() {
try {
const data = fs.readFileSync(POSTED_STATE_PATH, "utf-8");
const ids = JSON.parse(data);
if (Array.isArray(ids)) {
ids.forEach((id) => this.postedMatchIds.add(String(id)));
}
this.logger.log(
`✅ Loaded ${this.postedMatchIds.size} posted social match IDs`,
);
} catch {
this.logger.warn("⚠️ No social poster state file found yet");
}
}
private savePostedState() {
const dir = path.dirname(POSTED_STATE_PATH);
if (!fs.existsSync(dir)) {
fs.mkdirSync(dir, { recursive: true });
}
fs.writeFileSync(
POSTED_STATE_PATH,
JSON.stringify(Array.from(this.postedMatchIds).slice(-500), null, 2),
);
}
/**
* Cron: Every 10 minutes, check for upcoming matches.
* Cron: Every 15 minutes, check for upcoming matches.
* Posts predictions 30 minutes before kickoff.
*/
@Cron("*/10 * * * *")
@Cron("*/15 * * * *")
async checkAndPostUpcomingMatches() {
if (!this.isEnabled) return;
try {
const matches = await this.getUpcomingMatches(25, 40); // 25-40 min window
const matches = await this.getUpcomingMatches(
this.windowMinMinutes,
this.windowMaxMinutes,
);
this.logger.log(
`📅 Found ${matches.length} upcoming matches in the window`,
);
@@ -77,7 +128,19 @@ export class SocialPosterService {
if (this.postedMatchIds.has(match.id)) continue;
try {
await this.predictAndPost(match);
const result = await this.predictAndPost(match);
const posted =
result.twitterPostId ||
result.facebookPostId ||
result.instagramPostId;
if (!posted) {
this.logger.warn(
`No platform accepted post for match ${match.id}; it will be retried later`,
);
continue;
}
this.postedMatchIds.add(match.id);
// Cleanup: remove old IDs (keep last 500)
@@ -87,6 +150,7 @@ export class SocialPosterService {
.slice(0, arr.length - 500)
.forEach((id) => this.postedMatchIds.delete(id));
}
this.savePostedState();
} catch (error) {
this.logger.error(
`Failed to process match ${match.id}: ${error.message}`,
@@ -113,19 +177,33 @@ export class SocialPosterService {
const minTime = now + minMinutes * 60 * 1000;
const maxTime = now + maxMinutes * 60 * 1000;
const matches = await this.prisma.liveMatch.findMany({
where: {
sport: "football",
leagueId: { in: Array.from(this.topLeagueIds) },
const where: any = {
sport: { in: this.sports },
mstUtc: {
gte: minTime,
lte: maxTime,
},
};
if (this.topLeagueIds.size > 0) {
where.leagueId = { in: Array.from(this.topLeagueIds) };
}
const matches = await this.prisma.liveMatch.findMany({
where: {
...where,
},
include: {
homeTeam: true,
awayTeam: true,
league: true,
league: {
include: {
country: true,
},
},
},
orderBy: {
mstUtc: "asc",
},
});
@@ -233,7 +311,10 @@ export class SocialPosterService {
const ftScore = score.ft || "1-1";
// Extract best bets from bet_summary array
const topPicks = this.extractTopPicks(prediction);
const sport = this.normalizeSport(
match.sport || prediction.match_info?.sport,
);
const topPicks = this.extractTopPicks(prediction, sport);
// Match date formatting
const matchDate = this.formatMatchDate(match.mstUtc);
@@ -246,6 +327,7 @@ export class SocialPosterService {
return {
matchId: match.id,
sport,
homeTeam:
match.homeTeam?.name || prediction.match_info?.home_team || "Home",
awayTeam:
@@ -253,6 +335,12 @@ export class SocialPosterService {
homeLogo: this.resolveLogoUrl(match.homeTeam?.logoUrl || ""),
awayLogo: this.resolveLogoUrl(match.awayTeam?.logoUrl || ""),
leagueName: match.league?.name || prediction.match_info?.league || "",
leagueLogo: this.resolveLogoUrl(match.league?.logoUrl || ""),
countryName:
match.league?.country?.name ||
prediction.match_info?.country ||
this.inferCountryName(match.league?.name || ""),
countryFlag: this.resolveLogoUrl(match.league?.country?.flagUrl || ""),
matchDate,
htScore,
ftScore,
@@ -266,11 +354,14 @@ export class SocialPosterService {
/**
* Extract top 3 picks sorted by confidence from the V20+ bet_summary array.
*/
private extractTopPicks(prediction: any): TopPick[] {
private extractTopPicks(
prediction: any,
sport: "football" | "basketball",
): TopPick[] {
const betSummary: any[] = prediction.bet_summary || [];
// Market code to Turkish/English label mapping
const marketLabels: Record<string, { tr: string; en: string }> = {
const footballLabels: Record<string, { tr: string; en: string }> = {
MS: { tr: "Maç Sonucu", en: "Match Result" },
OU15: { tr: "Üst 1.5 Gol", en: "Over 1.5" },
OU25: { tr: "Üst 2.5 Gol", en: "Over 2.5" },
@@ -282,6 +373,20 @@ export class SocialPosterService {
OE: { tr: "Tek/Çift", en: "Odd/Even" },
HTFT: { tr: "İY/MS", en: "HT/FT" },
};
const basketballLabels: Record<string, { tr: string; en: string }> = {
MS: { tr: "Maç Sonucu", en: "Match Result" },
ML: { tr: "Maç Sonucu", en: "Moneyline" },
WINNER: { tr: "Maç Sonucu", en: "Winner" },
TOTAL: { tr: "Toplam Sayı", en: "Total Points" },
OU: { tr: "Toplam Sayı", en: "Total Points" },
OVER_UNDER: { tr: "Toplam Sayı", en: "Total Points" },
HANDICAP: { tr: "Handikap", en: "Handicap" },
HND: { tr: "Handikap", en: "Handicap" },
SPREAD: { tr: "Handikap", en: "Spread" },
HT: { tr: "İlk Devre Sonucu", en: "First Half Result" },
};
const marketLabels =
sport === "basketball" ? basketballLabels : footballLabels;
const candidates: TopPick[] = betSummary.map((bet) => {
const labels = marketLabels[bet.market] || {
@@ -302,6 +407,32 @@ export class SocialPosterService {
return candidates.slice(0, 3);
}
private inferCountryName(leagueName: string): string {
const normalized = leagueName.toLocaleLowerCase("tr-TR");
if (normalized.includes("süper lig") || normalized.includes("türkiye")) {
return "Türkiye";
}
if (
normalized.includes("premier league") ||
normalized.includes("championship")
) {
return "İngiltere";
}
if (normalized.includes("la liga")) return "İspanya";
if (normalized.includes("serie a")) return "İtalya";
if (normalized.includes("bundesliga")) return "Almanya";
if (normalized.includes("ligue 1")) return "Fransa";
if (normalized.includes("euroleague")) return "Avrupa";
if (normalized.includes("nba")) return "ABD";
return "";
}
private normalizeSport(sport?: string): "football" | "basketball" {
return String(sport).toLowerCase() === "basketball"
? "basketball"
: "football";
}
/**
* Convert relative logo paths to full HTTP URLs.
* On the deployed server, logos exist at public/uploads/teams/...
@@ -351,7 +482,11 @@ export class SocialPosterService {
include: {
homeTeam: true,
awayTeam: true,
league: true,
league: {
include: {
country: true,
},
},
},
});
@@ -373,7 +508,11 @@ export class SocialPosterService {
include: {
homeTeam: true,
awayTeam: true,
league: true,
league: {
include: {
country: true,
},
},
},
});
+11 -3
View File
@@ -20,7 +20,7 @@ export class TwitterService {
void this.initClient(apiKey, apiSecret, accessToken, accessSecret);
} else {
this.logger.warn(
"⚠️ Twitter API keys not configured. Set TWITTER_API_KEY, TWITTER_API_SECRET, TWITTER_ACCESS_TOKEN, TWITTER_ACCESS_SECRET",
"⚠️ X/Twitter API keys not configured. Set TWITTER_API_KEY, TWITTER_API_SECRET, TWITTER_ACCESS_TOKEN, TWITTER_ACCESS_SECRET",
);
}
}
@@ -64,10 +64,10 @@ export class TwitterService {
}
try {
// Step 1: Upload media via v1.1
// Step 1: Upload image media via the X media upload endpoint.
const mediaData = fs.readFileSync(imagePath);
const mediaId = await this.client.v1.uploadMedia(mediaData, {
mimeType: "image/png",
mimeType: this.getMimeType(imagePath),
});
// Step 2: Create tweet via v2
@@ -84,4 +84,12 @@ export class TwitterService {
return null;
}
}
private getMimeType(imagePath: string): string {
const ext = imagePath.toLowerCase().split(".").pop();
if (ext === "jpg" || ext === "jpeg") return "image/jpeg";
if (ext === "webp") return "image/webp";
if (ext === "png") return "image/png";
return "image/jpeg";
}
}
+26 -4
View File
@@ -54,6 +54,7 @@ export class SporTotoController {
@ApiResponse({
status: 200,
description: "Sync result with action (created/updated/unchanged)",
schema: { type: "object" },
})
async syncFromApi() {
const result = await this.sporTotoService.syncFromApi();
@@ -82,6 +83,7 @@ export class SporTotoController {
@ApiResponse({
status: 200,
description: "Array of bulletins with matches and results",
schema: { type: "object" },
})
async listBulletins(
@Query("status") status?: TotoBulletinStatus,
@@ -105,6 +107,7 @@ export class SporTotoController {
@ApiResponse({
status: 200,
description: "Bulletin with matches and results",
schema: { type: "object" },
})
@ApiResponse({ status: 404, description: "Bulletin not found" })
async getBulletin(@Param("id") id: string) {
@@ -123,7 +126,11 @@ export class SporTotoController {
"Creates a new bulletin with 15 matches. Fails if gameCycleNo already exists.",
})
@ApiBody({ type: CreateBulletinDto })
@ApiResponse({ status: 201, description: "Created bulletin with matches" })
@ApiResponse({
status: 201,
description: "Created bulletin with matches",
schema: { type: "object" },
})
@ApiResponse({
status: 409,
description: "Bulletin with this gameCycleNo already exists",
@@ -145,7 +152,11 @@ export class SporTotoController {
})
@ApiParam({ name: "id", description: "Bulletin UUID" })
@ApiBody({ type: UpdateResultsDto })
@ApiResponse({ status: 200, description: "Updated bulletin with results" })
@ApiResponse({
status: 200,
description: "Updated bulletin with results",
schema: { type: "object" },
})
@ApiResponse({ status: 404, description: "Bulletin not found" })
async updateResults(@Param("id") id: string, @Body() dto: UpdateResultsDto) {
const bulletin = await this.sporTotoService.updateResults(id, dto);
@@ -162,7 +173,11 @@ export class SporTotoController {
"Returns pool distribution (35/20/20/25), expected value calculations, and rollover analysis for a bulletin.",
})
@ApiParam({ name: "id", description: "Bulletin UUID" })
@ApiResponse({ status: 200, description: "Pool distribution and EV stats" })
@ApiResponse({
status: 200,
description: "Pool distribution and EV stats",
schema: { type: "object" },
})
async getBulletinStats(@Param("id") id: string) {
const stats = await this.sporTotoService.getBulletinStats(id);
return { success: true, data: stats };
@@ -181,7 +196,11 @@ export class SporTotoController {
type: Number,
description: "Number of results (default: 20)",
})
@ApiResponse({ status: 200, description: "Rollover history with trend data" })
@ApiResponse({
status: 200,
description: "Rollover history with trend data",
schema: { type: "object" },
})
async getRolloverHistory(@Query("limit") limit?: string) {
const history = await this.sporTotoService.getRolloverHistory(
Number(limit) || 20,
@@ -204,6 +223,7 @@ export class SporTotoController {
@ApiResponse({
status: 200,
description: "Generated columns with strategy, cost, and column strings",
schema: { type: "object" },
})
async generateColumns(@Body() dto: GenerateColumnsDto) {
const result = await this.sporTotoService.generateColumns(dto);
@@ -223,6 +243,7 @@ export class SporTotoController {
@ApiResponse({
status: 200,
description: "Evaluation results with correct counts per column",
schema: { type: "object" },
})
async evaluateColumns(@Body() dto: EvaluateColumnsDto) {
const result = await this.sporTotoService.evaluateColumns(
@@ -248,6 +269,7 @@ export class SporTotoController {
status: 200,
description:
"Prediction result with per-match analysis, system coupon, and EV report with play recommendation",
schema: { type: "object" },
})
async generatePrediction(@Body() dto: GenerateSporTotoPredictionDto) {
this.logger.log(
@@ -0,0 +1,178 @@
import {
IsString,
IsOptional,
IsEnum,
IsDateString,
IsInt,
} from "class-validator";
import { ApiProperty, ApiPropertyOptional } from "@nestjs/swagger";
import { Exclude, Expose, Type } from "class-transformer";
export enum PlanType {
FREE = "free",
PLUS = "plus",
PREMIUM = "premium",
}
export enum BillingIntervalType {
MONTHLY = "monthly",
YEARLY = "yearly",
}
/**
* Plan feature limits configuration
*/
export const PLAN_LIMITS: Record<
PlanType,
{ maxAnalyses: number; maxCoupons: number }
> = {
[PlanType.FREE]: { maxAnalyses: 3, maxCoupons: 1 },
[PlanType.PLUS]: { maxAnalyses: 25, maxCoupons: 5 },
[PlanType.PREMIUM]: { maxAnalyses: 999, maxCoupons: 999 },
};
/**
* Plan display information
*/
export interface PlanInfo {
id: PlanType;
name: string;
description: string;
monthlyPrice: number;
yearlyPrice: number;
currency: string;
features: string[];
limits: { maxAnalyses: number; maxCoupons: number };
highlighted: boolean;
}
export const PLANS: readonly PlanInfo[] = [
{
id: PlanType.FREE,
name: "Free",
description: "Temel analiz özellikleri",
monthlyPrice: 0,
yearlyPrice: 0,
currency: "TRY",
features: ["Günlük 3 analiz", "Günlük 1 kupon", "Temel maç istatistikleri"],
limits: PLAN_LIMITS[PlanType.FREE],
highlighted: false,
},
{
id: PlanType.PLUS,
name: "Plus",
description: "Detaylı analiz ve daha fazla kupon",
monthlyPrice: 99,
yearlyPrice: 999,
currency: "TRY",
features: [
"Günlük 25 analiz",
"Günlük 5 kupon",
"AI detaylı analiz",
"H2H karşılaştırma",
"Reklamsız deneyim",
],
limits: PLAN_LIMITS[PlanType.PLUS],
highlighted: true,
},
{
id: PlanType.PREMIUM,
name: "Premium",
description: "Sınırsız erişim ve özel özellikler",
monthlyPrice: 249,
yearlyPrice: 2499,
currency: "TRY",
features: [
"Sınırsız analiz",
"Sınırsız kupon",
"AI detaylı analiz",
"H2H karşılaştırma",
"Kupon Builder",
"Spor Toto analiz",
"Reklamsız deneyim",
"Öncelikli destek",
],
limits: PLAN_LIMITS[PlanType.PREMIUM],
highlighted: false,
},
] as const;
// ── Response DTOs ──
@Exclude()
export class UsageLimitResponseDto {
@Expose()
analysisCount: number;
@Expose()
couponCount: number;
@Expose()
maxAnalyses: number;
@Expose()
maxCoupons: number;
}
@Exclude()
export class SubscriptionResponseDto {
@Expose()
id: string;
@Expose()
plan: string;
@Expose()
billingInterval: string | null;
@Expose()
currentPeriodStart: Date | null;
@Expose()
currentPeriodEnd: Date | null;
@Expose()
cancelledAt: Date | null;
@Expose()
cancelEffectiveDate: Date | null;
@Expose()
paddlePriceId: string | null;
@Expose()
createdAt: Date;
@Expose()
updatedAt: Date;
}
// ── Request DTOs ──
export class CreateCheckoutDto {
@ApiProperty({
enum: PlanType,
example: PlanType.PLUS,
description: "Target plan",
})
@IsEnum(PlanType)
plan: PlanType;
@ApiProperty({
enum: BillingIntervalType,
example: BillingIntervalType.MONTHLY,
description: "Billing interval",
})
@IsEnum(BillingIntervalType)
billingInterval: BillingIntervalType;
}
export class CancelSubscriptionDto {
@ApiPropertyOptional({
description: "Reason for cancellation",
example: "Too expensive",
})
@IsOptional()
@IsString()
reason?: string;
}
+209
View File
@@ -0,0 +1,209 @@
import { Injectable, Logger } from "@nestjs/common";
import { ConfigService } from "@nestjs/config";
import * as crypto from "crypto";
export interface PaddleWebhookEvent {
event_id: string;
event_type: string;
occurred_at: string;
notification_id: string;
data: Record<string, unknown>;
}
interface PaddleTransactionResponse {
data: {
id: string;
customer_id: string;
status: string;
};
}
@Injectable()
export class PaddleService {
private readonly logger = new Logger(PaddleService.name);
private readonly apiKey: string;
private readonly webhookSecret: string;
private readonly environment: "sandbox" | "production";
private readonly baseUrl: string;
constructor(private readonly config: ConfigService) {
this.apiKey = this.config.get<string>("PADDLE_API_KEY", "");
this.webhookSecret = this.config.get<string>("PADDLE_WEBHOOK_SECRET", "");
this.environment = this.config.get<"sandbox" | "production">(
"PADDLE_ENVIRONMENT",
"sandbox",
);
this.baseUrl =
this.environment === "production"
? "https://api.paddle.com"
: "https://sandbox-api.paddle.com";
}
/**
* Verify Paddle webhook signature (Paddle Billing v2)
*/
verifyWebhookSignature(rawBody: string, signatureHeader: string): boolean {
if (!this.webhookSecret) {
this.logger.warn(
"PADDLE_WEBHOOK_SECRET not configured, skipping verification",
);
return false;
}
try {
// Paddle signature format: ts=TIMESTAMP;h1=HASH
const parts = signatureHeader.split(";");
const tsValue = parts
.find((p) => p.startsWith("ts="))
?.replace("ts=", "");
const h1Value = parts
.find((p) => p.startsWith("h1="))
?.replace("h1=", "");
if (!tsValue || !h1Value) {
this.logger.warn("Invalid Paddle signature format");
return false;
}
// Compute expected signature: HMAC-SHA256(ts + ':' + rawBody)
const signedPayload = `${tsValue}:${rawBody}`;
const expectedSignature = crypto
.createHmac("sha256", this.webhookSecret)
.update(signedPayload)
.digest("hex");
return crypto.timingSafeEqual(
Buffer.from(h1Value),
Buffer.from(expectedSignature),
);
} catch (error: unknown) {
const err = error as Error;
this.logger.error(
`Webhook signature verification failed: ${err.message}`,
);
return false;
}
}
/**
* Cancel a Paddle subscription
*/
async cancelSubscription(
paddleSubscriptionId: string,
effectiveFrom:
| "immediately"
| "next_billing_period" = "next_billing_period",
): Promise<void> {
const url = `${this.baseUrl}/subscriptions/${paddleSubscriptionId}/cancel`;
const response = await fetch(url, {
method: "POST",
headers: {
Authorization: `Bearer ${this.apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ effective_from: effectiveFrom }),
});
if (!response.ok) {
const body = await response.text();
this.logger.error(`Paddle cancel failed: ${response.status} ${body}`);
throw new Error(
`Failed to cancel Paddle subscription: ${response.status}`,
);
}
this.logger.log(
`Paddle subscription ${paddleSubscriptionId} cancelled (effective: ${effectiveFrom})`,
);
}
/**
* Get subscription details from Paddle
*/
async getSubscription(
paddleSubscriptionId: string,
): Promise<Record<string, unknown>> {
const url = `${this.baseUrl}/subscriptions/${paddleSubscriptionId}`;
const response = await fetch(url, {
method: "GET",
headers: {
Authorization: `Bearer ${this.apiKey}`,
"Content-Type": "application/json",
},
});
if (!response.ok) {
throw new Error(`Failed to get Paddle subscription: ${response.status}`);
}
const data = (await response.json()) as { data: Record<string, unknown> };
return data.data;
}
/**
* Map Paddle price ID to our internal plan
*/
mapPriceIdToPlan(priceId: string): {
plan: "plus" | "premium";
interval: "monthly" | "yearly";
} | null {
const mapping: Record<
string,
{ plan: "plus" | "premium"; interval: "monthly" | "yearly" }
> = {
[this.config.get<string>("PADDLE_PLUS_MONTHLY_PRICE_ID", "")]: {
plan: "plus",
interval: "monthly",
},
[this.config.get<string>("PADDLE_PLUS_YEARLY_PRICE_ID", "")]: {
plan: "plus",
interval: "yearly",
},
[this.config.get<string>("PADDLE_PREMIUM_MONTHLY_PRICE_ID", "")]: {
plan: "premium",
interval: "monthly",
},
[this.config.get<string>("PADDLE_PREMIUM_YEARLY_PRICE_ID", "")]: {
plan: "premium",
interval: "yearly",
},
};
// Remove empty key (from missing env vars)
delete mapping[""];
return mapping[priceId] ?? null;
}
/**
* Get the Paddle price ID for a given plan and interval
*/
getPriceId(plan: "plus" | "premium", interval: "monthly" | "yearly"): string {
const key = `PADDLE_${plan.toUpperCase()}_${interval.toUpperCase()}_PRICE_ID`;
const priceId = this.config.get<string>(key, "");
if (!priceId) {
throw new Error(
`Price ID not configured for ${plan} ${interval} (env: ${key})`,
);
}
return priceId;
}
/**
* Get the client-side token for Paddle.js
*/
getClientToken(): string {
return this.config.get<string>("PADDLE_CLIENT_TOKEN", "");
}
/**
* Get the Paddle environment
*/
getEnvironment(): "sandbox" | "production" {
return this.environment;
}
}
@@ -0,0 +1,186 @@
import {
Controller,
Get,
Post,
Body,
Req,
Res,
HttpCode,
HttpStatus,
Logger,
ForbiddenException,
} from "@nestjs/common";
import type { RawBodyRequest } from "@nestjs/common";
import {
ApiTags,
ApiBearerAuth,
ApiOperation,
ApiOkResponse,
} from "@nestjs/swagger";
import type { Request, Response } from "express";
import { CurrentUser, Public } from "../../common/decorators";
import type { ApiResponse } from "../../common/types/api-response.type";
import {
createSuccessResponse,
createErrorResponse,
} from "../../common/types/api-response.type";
import { SubscriptionsService } from "./subscriptions.service";
import { PaddleService, PaddleWebhookEvent } from "./paddle.service";
import {
CreateCheckoutDto,
CancelSubscriptionDto,
SubscriptionResponseDto,
PlanInfo,
PlanType,
} from "./dto/subscription.dto";
interface AuthenticatedUser {
id: string;
email: string;
role: string;
}
@ApiTags("Subscriptions")
@Controller("subscriptions")
export class SubscriptionsController {
private readonly logger = new Logger(SubscriptionsController.name);
constructor(
private readonly subscriptionsService: SubscriptionsService,
private readonly paddleService: PaddleService,
) {}
/**
* GET /subscriptions/plans Get all available plans (public)
*/
@Public()
@Get("plans")
@ApiOperation({ summary: "Get all available subscription plans" })
@ApiOkResponse({ description: "List of available plans" })
getPlans(): ApiResponse<readonly PlanInfo[]> {
const plans = this.subscriptionsService.getPlans();
return createSuccessResponse(plans, "Plans retrieved");
}
/**
* GET /subscriptions/me Get current user subscription
*/
@ApiBearerAuth()
@Get("me")
@ApiOperation({ summary: "Get current user subscription status" })
async getMySubscription(
@CurrentUser() user: AuthenticatedUser,
): Promise<ApiResponse<SubscriptionResponseDto | null>> {
const subscription = await this.subscriptionsService.getCurrentSubscription(
user.id,
);
return createSuccessResponse(subscription, "Subscription retrieved");
}
/**
* POST /subscriptions/checkout Get checkout config for Paddle.js
*/
@ApiBearerAuth()
@Post("checkout")
@ApiOperation({
summary: "Get Paddle checkout configuration for a plan",
})
async getCheckoutConfig(
@CurrentUser() user: AuthenticatedUser,
@Body() dto: CreateCheckoutDto,
): Promise<
ApiResponse<{
priceId: string;
clientToken: string;
environment: string;
userId: string;
}>
> {
if (dto.plan === PlanType.FREE) {
throw new ForbiddenException("Cannot checkout for free plan");
}
const config = this.subscriptionsService.getCheckoutConfig(
dto.plan,
dto.billingInterval,
);
return createSuccessResponse(
{
...config,
userId: user.id,
},
"Checkout config ready",
);
}
/**
* POST /subscriptions/cancel Cancel current subscription
*/
@ApiBearerAuth()
@Post("cancel")
@ApiOperation({ summary: "Cancel the current subscription" })
async cancelSubscription(
@CurrentUser() user: AuthenticatedUser,
@Body() _dto: CancelSubscriptionDto,
): Promise<ApiResponse<null>> {
await this.subscriptionsService.cancelSubscription(user.id);
return createSuccessResponse(null, "Subscription cancellation requested");
}
/**
* POST /subscriptions/webhook/paddle Paddle webhook receiver
*
* This endpoint is PUBLIC (no JWT required) Paddle calls it directly.
* Authentication is done via HMAC signature verification.
*/
@Public()
@Post("webhook/paddle")
@HttpCode(HttpStatus.OK)
@ApiOperation({ summary: "Paddle webhook receiver (internal)" })
async handlePaddleWebhook(
@Req() req: RawBodyRequest<Request>,
@Res() res: Response,
): Promise<void> {
const signature = req.headers["paddle-signature"] as string | undefined;
if (!signature) {
this.logger.warn("Paddle webhook received without signature");
res.status(HttpStatus.BAD_REQUEST).json({ error: "Missing signature" });
return;
}
// Get raw body for signature verification
const rawBody = req.rawBody?.toString("utf8");
if (!rawBody) {
this.logger.warn("Paddle webhook received without raw body");
res.status(HttpStatus.BAD_REQUEST).json({ error: "Missing body" });
return;
}
// Verify signature
const isValid = this.paddleService.verifyWebhookSignature(
rawBody,
signature,
);
if (!isValid) {
this.logger.warn("Paddle webhook signature verification failed");
res.status(HttpStatus.UNAUTHORIZED).json({ error: "Invalid signature" });
return;
}
// Parse and process
try {
const event = JSON.parse(rawBody) as PaddleWebhookEvent;
await this.subscriptionsService.handleWebhookEvent(event);
res.status(HttpStatus.OK).json({ received: true });
} catch (error: unknown) {
const err = error as Error;
this.logger.error(`Webhook processing failed: ${err.message}`);
res
.status(HttpStatus.INTERNAL_SERVER_ERROR)
.json({ error: "Processing failed" });
}
}
}
@@ -0,0 +1,13 @@
import { Module } from "@nestjs/common";
import { SubscriptionsController } from "./subscriptions.controller";
import { SubscriptionsService } from "./subscriptions.service";
import { PaddleService } from "./paddle.service";
import { DatabaseModule } from "../../database/database.module";
@Module({
imports: [DatabaseModule],
controllers: [SubscriptionsController],
providers: [SubscriptionsService, PaddleService],
exports: [SubscriptionsService, PaddleService],
})
export class SubscriptionsModule {}
@@ -0,0 +1,334 @@
import { Injectable, Logger, NotFoundException } from "@nestjs/common";
import { PrismaService } from "../../database/prisma.service";
import { PaddleService, PaddleWebhookEvent } from "./paddle.service";
import {
PlanType,
BillingIntervalType,
PLAN_LIMITS,
PLANS,
SubscriptionResponseDto,
} from "./dto/subscription.dto";
import { plainToInstance } from "class-transformer";
@Injectable()
export class SubscriptionsService {
private readonly logger = new Logger(SubscriptionsService.name);
constructor(
private readonly prisma: PrismaService,
private readonly paddleService: PaddleService,
) {}
/**
* Get current subscription for user
*/
async getCurrentSubscription(
userId: string,
): Promise<SubscriptionResponseDto | null> {
const subscription = await this.prisma.subscription.findUnique({
where: { userId },
});
if (!subscription) {
return null;
}
return plainToInstance(SubscriptionResponseDto, subscription);
}
/**
* Get or create subscription record for user
*/
async getOrCreateSubscription(userId: string) {
let subscription = await this.prisma.subscription.findUnique({
where: { userId },
});
if (!subscription) {
subscription = await this.prisma.subscription.create({
data: {
userId,
plan: "free",
},
});
}
return subscription;
}
/**
* Get all available plans
*/
getPlans() {
return PLANS;
}
/**
* Get checkout configuration (client-side token + price ID)
*/
getCheckoutConfig(
plan: PlanType,
billingInterval: BillingIntervalType,
): { priceId: string; clientToken: string; environment: string } {
if (plan === PlanType.FREE) {
throw new Error("Cannot checkout for free plan");
}
const paddlePlan = plan as "plus" | "premium";
const paddleInterval = billingInterval as "monthly" | "yearly";
const priceId = this.paddleService.getPriceId(paddlePlan, paddleInterval);
const clientToken = this.paddleService.getClientToken();
const environment = this.paddleService.getEnvironment();
return { priceId, clientToken, environment };
}
/**
* Handle incoming Paddle webhook event
*/
async handleWebhookEvent(event: PaddleWebhookEvent): Promise<void> {
const eventType = event.event_type;
const data = event.data;
this.logger.log(`Processing Paddle webhook: ${eventType}`);
switch (eventType) {
case "subscription.created":
case "subscription.updated":
await this.handleSubscriptionUpdate(data);
break;
case "subscription.canceled":
await this.handleSubscriptionCancelled(data);
break;
case "subscription.past_due":
await this.handleSubscriptionPastDue(data);
break;
case "subscription.resumed":
await this.handleSubscriptionResumed(data);
break;
case "transaction.completed":
this.logger.log(
`Transaction completed: ${(data as Record<string, unknown>).id}`,
);
break;
case "transaction.payment_failed":
this.logger.warn(
`Payment failed for transaction: ${(data as Record<string, unknown>).id}`,
);
break;
default:
this.logger.debug(`Unhandled Paddle event: ${eventType}`);
}
}
/**
* Cancel subscription for user
*/
async cancelSubscription(userId: string): Promise<void> {
const subscription = await this.prisma.subscription.findUnique({
where: { userId },
});
if (!subscription?.paddleSubscriptionId) {
throw new NotFoundException("No active subscription found");
}
await this.paddleService.cancelSubscription(
subscription.paddleSubscriptionId,
"next_billing_period",
);
this.logger.log(`Cancellation requested for user ${userId}`);
}
// ── Private Handlers ──
private async handleSubscriptionUpdate(
data: Record<string, unknown>,
): Promise<void> {
const paddleSubId = data.id as string;
const customerId = data.customer_id as string;
const status = data.status as string;
const customData = data.custom_data as { userId?: string } | undefined;
const items = data.items as Array<{ price: { id: string } }> | undefined;
const currentBillingPeriod = data.current_billing_period as
| { starts_at: string; ends_at: string }
| undefined;
const userId = customData?.userId;
if (!userId) {
this.logger.warn(
`No userId in custom_data for subscription ${paddleSubId}`,
);
return;
}
// Determine plan from price ID
const priceId = items?.[0]?.price?.id;
let plan: PlanType = PlanType.FREE;
let interval: BillingIntervalType = BillingIntervalType.MONTHLY;
if (priceId) {
const mapped = this.paddleService.mapPriceIdToPlan(priceId);
if (mapped) {
plan = mapped.plan as PlanType;
interval = mapped.interval as BillingIntervalType;
}
}
// Determine effective plan based on Paddle status
const effectivePlan =
status === "active" || status === "trialing" ? plan : PlanType.FREE;
// Upsert subscription record
await this.prisma.subscription.upsert({
where: { userId },
update: {
paddleSubscriptionId: paddleSubId,
paddleCustomerId: customerId,
plan: effectivePlan,
billingInterval: interval,
paddlePriceId: priceId ?? null,
currentPeriodStart: currentBillingPeriod?.starts_at
? new Date(currentBillingPeriod.starts_at)
: null,
currentPeriodEnd: currentBillingPeriod?.ends_at
? new Date(currentBillingPeriod.ends_at)
: null,
cancelledAt: null,
cancelEffectiveDate: null,
},
create: {
userId,
paddleSubscriptionId: paddleSubId,
paddleCustomerId: customerId,
plan: effectivePlan,
billingInterval: interval,
paddlePriceId: priceId ?? null,
currentPeriodStart: currentBillingPeriod?.starts_at
? new Date(currentBillingPeriod.starts_at)
: null,
currentPeriodEnd: currentBillingPeriod?.ends_at
? new Date(currentBillingPeriod.ends_at)
: null,
},
});
// Sync user subscription status
await this.prisma.user.update({
where: { id: userId },
data: { subscriptionStatus: effectivePlan },
});
// Sync usage limits with plan
await this.syncLimitsWithPlan(userId, effectivePlan);
this.logger.log(
`Subscription updated: user=${userId}, plan=${effectivePlan}, interval=${interval}`,
);
}
private async handleSubscriptionCancelled(
data: Record<string, unknown>,
): Promise<void> {
const paddleSubId = data.id as string;
const canceledAt = data.canceled_at as string | undefined;
const currentBillingPeriod = data.current_billing_period as
| { ends_at: string }
| undefined;
const subscription = await this.prisma.subscription.findUnique({
where: { paddleSubscriptionId: paddleSubId },
});
if (!subscription) {
this.logger.warn(`Subscription not found for cancel: ${paddleSubId}`);
return;
}
const effectiveDate = currentBillingPeriod?.ends_at
? new Date(currentBillingPeriod.ends_at)
: new Date();
await this.prisma.subscription.update({
where: { id: subscription.id },
data: {
plan: "cancelled",
cancelledAt: canceledAt ? new Date(canceledAt) : new Date(),
cancelEffectiveDate: effectiveDate,
},
});
// Downgrade user to free
await this.prisma.user.update({
where: { id: subscription.userId },
data: { subscriptionStatus: "free" },
});
await this.syncLimitsWithPlan(subscription.userId, PlanType.FREE);
this.logger.log(
`Subscription cancelled: user=${subscription.userId}, effective=${effectiveDate.toISOString()}`,
);
}
private async handleSubscriptionPastDue(
data: Record<string, unknown>,
): Promise<void> {
const paddleSubId = data.id as string;
const subscription = await this.prisma.subscription.findUnique({
where: { paddleSubscriptionId: paddleSubId },
});
if (!subscription) {
return;
}
await this.prisma.subscription.update({
where: { id: subscription.id },
data: { plan: "past_due" },
});
await this.prisma.user.update({
where: { id: subscription.userId },
data: { subscriptionStatus: "past_due" },
});
this.logger.warn(`Subscription past due: user=${subscription.userId}`);
}
private async handleSubscriptionResumed(
data: Record<string, unknown>,
): Promise<void> {
// Re-process as an update to restore the plan
await this.handleSubscriptionUpdate(data);
}
/**
* Sync usage limits with plan tier
*/
public async syncLimitsWithPlan(
userId: string,
plan: PlanType,
): Promise<void> {
const limits = PLAN_LIMITS[plan] ?? PLAN_LIMITS[PlanType.FREE];
await this.prisma.usageLimit.upsert({
where: { userId },
update: {
maxAnalyses: limits.maxAnalyses,
maxCoupons: limits.maxCoupons,
},
create: {
userId,
analysisCount: 0,
couponCount: 0,
maxAnalyses: limits.maxAnalyses,
maxCoupons: limits.maxCoupons,
lastResetDate: new Date(),
},
});
}
}
+26 -1
View File
@@ -73,7 +73,25 @@ export class ChangePasswordDto {
newPassword: string;
}
import { Exclude, Expose } from "class-transformer";
import { Exclude, Expose, Type } from "class-transformer";
@Exclude()
export class UsageLimitDto {
@Expose()
analysisCount: number;
@Expose()
couponCount: number;
@Expose()
maxAnalyses: number;
@Expose()
maxCoupons: number;
@Expose()
lastResetDate: Date;
}
@Exclude()
export class UserResponseDto {
@@ -95,9 +113,16 @@ export class UserResponseDto {
@Expose()
isActive: boolean;
@Expose()
subscriptionStatus: string;
@Expose()
createdAt: Date;
@Expose()
updatedAt: Date;
@Expose()
@Type(() => UsageLimitDto)
usageLimit?: UsageLimitDto;
}
+39
View File
@@ -0,0 +1,39 @@
/**
* Run Previous-Day Completed Match Sync
* Usage: npm run feeder:previous-day
*/
import { NestFactory } from "@nestjs/core";
import { FeederService } from "../modules/feeder/feeder.service";
import { Logger } from "@nestjs/common";
async function bootstrap() {
process.env.FEEDER_MODE = "historical";
const logger = new Logger("FeederPreviousDayScript");
logger.log("🚀 Starting previous-day completed match sync...");
// Load AppModule after FEEDER_MODE is set so cron imports can be disabled.
// eslint-disable-next-line @typescript-eslint/no-require-imports
const { AppModule } = require("../app.module");
const app = await NestFactory.createApplicationContext(AppModule, {
logger: ["log", "error", "warn"],
});
try {
const feederService = app.get(FeederService);
await feederService.runPreviousDayCompletedMatchesScan();
logger.log("✅ Previous-day completed match sync completed successfully!");
} catch (error: any) {
logger.error(`❌ Feeder failed: ${error.message}`);
logger.error(error.stack);
process.exit(1);
} finally {
await app.close();
}
process.exit(0);
}
void bootstrap();
+123
View File
@@ -0,0 +1,123 @@
/**
* Repair Feeder - Fix incomplete matches
* Usage: npm run feeder:repair
*
* Finds matches in DB that are missing stats or lineups
* and re-fetches them from the API.
*/
import { NestFactory } from "@nestjs/core";
import { Logger } from "@nestjs/common";
import { PrismaService } from "../database/prisma.service";
import { FeederService } from "../modules/feeder/feeder.service";
async function bootstrap() {
process.env.FEEDER_MODE = "historical";
const logger = new Logger("FeederRepair");
logger.log("🔧 Starting feeder repair scan...");
// eslint-disable-next-line @typescript-eslint/no-require-imports
const { AppModule } = require("../app.module");
const app = await NestFactory.createApplicationContext(AppModule, {
logger: ["log", "error", "warn"],
});
const prisma = app.get(PrismaService);
const feederService = app.get(FeederService);
try {
// Find football matches missing stats (no football_team_stats rows)
const matchesMissingStats = await prisma.$queryRaw<
Array<{ id: string; match_name: string }>
>`
SELECT m.id, m.match_name
FROM matches m
LEFT JOIN football_team_stats fts ON fts.match_id = m.id
WHERE m.sport = 'football'
AND m.state = 'Ended'
AND fts.id IS NULL
ORDER BY m.mst_utc DESC
`;
// Find football matches missing lineups (< 18 participation rows)
const matchesMissingLineups = await prisma.$queryRaw<
Array<{ id: string; match_name: string; cnt: bigint }>
>`
SELECT m.id, m.match_name, COUNT(mpp.id) as cnt
FROM matches m
LEFT JOIN match_player_participation mpp ON mpp.match_id = m.id
WHERE m.sport = 'football'
AND m.state = 'Ended'
GROUP BY m.id, m.match_name
HAVING COUNT(mpp.id) < 18
ORDER BY m.mst_utc DESC
`;
// Combine unique match IDs
const repairSet = new Set<string>();
for (const m of matchesMissingStats) repairSet.add(m.id);
for (const m of matchesMissingLineups) repairSet.add(m.id);
logger.log(
`📊 Found ${repairSet.size} incomplete matches (${matchesMissingStats.length} missing stats, ${matchesMissingLineups.length} missing lineups)`,
);
if (repairSet.size === 0) {
logger.log("✅ No incomplete matches found. Everything is clean!");
await app.close();
process.exit(0);
}
let repaired = 0;
let failed = 0;
const matchIds = Array.from(repairSet);
for (let i = 0; i < matchIds.length; i++) {
const matchId = matchIds[i];
// Rate limiting
if (i > 0 && i % 10 === 0) {
logger.log(
`⏸️ Cooldown after 10 repairs... (${repaired} repaired, ${failed} failed, ${matchIds.length - i} remaining)`,
);
await new Promise((r) => setTimeout(r, 5000));
}
await new Promise((r) => setTimeout(r, 500));
try {
const result = await feederService.refreshMatch(matchId, "all");
if (result.success) {
repaired++;
if (repaired % 25 === 0) {
logger.log(`🔧 Progress: ${repaired}/${matchIds.length} repaired`);
}
} else {
failed++;
logger.warn(
`❌ [${matchId}] Repair failed: ${result.error || "unknown"}`,
);
}
} catch (e: any) {
failed++;
logger.error(`❌ [${matchId}] Repair exception: ${e.message}`);
}
}
logger.log(
`🎉 REPAIR COMPLETE: ${repaired} repaired, ${failed} failed out of ${matchIds.length} total`,
);
} catch (error: any) {
logger.error(`❌ Repair failed: ${error.message}`);
logger.error(error.stack);
process.exit(1);
} finally {
await app.close();
}
process.exit(0);
}
void bootstrap();
+52 -5
View File
@@ -1,18 +1,25 @@
/**
* Run Previous-Day Completed Match Sync
* Run Full Historical Feeder
* Usage: npm run feeder:historical
*
* Includes a watchdog that kills the process if no activity
* is detected for 5 minutes (stuck API request), letting PM2
* auto-restart and resume from DB.
*/
import { NestFactory } from "@nestjs/core";
import { FeederService } from "../modules/feeder/feeder.service";
import { Logger } from "@nestjs/common";
const WATCHDOG_INTERVAL_MS = 60_000; // Check every 1 minute
const WATCHDOG_TIMEOUT_MS = 3 * 60_000; // Kill if no activity for 3 minutes
async function bootstrap() {
process.env.FEEDER_MODE = "historical";
const logger = new Logger("FeederScript");
logger.log("🚀 Starting previous-day completed match sync...");
logger.log("🚀 Starting full historical feeder...");
// Load AppModule after FEEDER_MODE is set so cron imports can be disabled.
// eslint-disable-next-line @typescript-eslint/no-require-imports
@@ -21,15 +28,55 @@ async function bootstrap() {
logger: ["log", "error", "warn"],
});
try {
const feederService = app.get(FeederService);
await feederService.runPreviousDayCompletedMatchesScan();
logger.log("✅ Previous-day completed match sync completed successfully!");
// ── Watchdog Timer ──────────────────────────────────────────
// If the feeder hangs on an API call for 3+ minutes, force-kill
// so PM2 can restart and resume from where it left off in DB.
// NOTE: process.exit(1) alone can be blocked by open handles
// (DB connections, HTTP sockets). We use process.kill(SIGKILL)
// as an unconditional fallback.
const watchdog = setInterval(() => {
const idleMs = Date.now() - feederService.lastActivityAt;
if (idleMs > WATCHDOG_TIMEOUT_MS) {
logger.error(
`🐕 WATCHDOG: No activity for ${Math.round(idleMs / 1000)}s. Force-killing for PM2 restart...`,
);
// Try graceful exit first
try {
process.exit(1);
} catch {
// Ignored fallback below
}
// If process.exit didn't work (blocked by open handles),
// schedule an unconditional SIGKILL after 2 seconds
setTimeout(() => {
logger.error("🐕 WATCHDOG: process.exit blocked. Sending SIGKILL...");
process.kill(process.pid, "SIGKILL");
}, 2_000).unref();
}
}, WATCHDOG_INTERVAL_MS);
// Don't let the watchdog timer keep the process alive after scan finishes
watchdog.unref();
try {
const startDate = process.env.FEEDER_START_DATE || "2023-06-01";
const sports = (process.env.FEEDER_SPORTS || "football,basketball")
.split(",")
.map((sport) => sport.trim())
.filter(Boolean) as Array<"football" | "basketball">;
await feederService.runHistoricalScan(sports, startDate);
logger.log("✅ Full historical feeder completed successfully!");
} catch (error: any) {
logger.error(`❌ Feeder failed: ${error.message}`);
logger.error(error.stack);
process.exit(1);
} finally {
clearInterval(watchdog);
await app.close();
}
+1 -3
View File
@@ -44,9 +44,7 @@ export class AiService {
private readonly pythonEngineUrl: string;
private readonly aiEngineClient: AiEngineClient;
constructor(
private readonly configService: ConfigService,
) {
constructor(private readonly configService: ConfigService) {
this.pythonEngineUrl =
this.configService.get("AI_ENGINE_URL") || "http://127.0.0.1:8000";
this.aiEngineClient = new AiEngineClient({
+427 -29
View File
@@ -1,4 +1,4 @@
import { Injectable, Logger } from "@nestjs/common";
import { Injectable, Logger } from "@nestjs/common";
import { Cron } from "@nestjs/schedule";
import { HttpService } from "@nestjs/axios";
import { PrismaService } from "../database/prisma.service";
@@ -9,6 +9,7 @@ import * as path from "path";
import { Prisma } from "@prisma/client";
import { SidelinedResponse } from "../modules/feeder/feeder.types";
import {
deriveStoredMatchStatus,
FINISHED_STATE_VALUES_FOR_DB,
FINISHED_STATUS_VALUES_FOR_DB,
LIVE_STATE_VALUES_FOR_DB,
@@ -74,6 +75,17 @@ interface LiveLineupsJson {
away: { xi: unknown[]; subs: unknown[] };
}
interface PendingPredictionRunForSettlement {
id: bigint;
matchId: string;
engineVersion: string;
payloadSummary: unknown;
scoreHome: number | null;
scoreAway: number | null;
htScoreHome: number | null;
htScoreAway: number | null;
}
type SportType = "football" | "basketball";
// ────────────────────────────────────────────────────────────────
@@ -161,7 +173,8 @@ export class DataFetcherTask {
`Pruned ${deleted.count} stale live matches. Starting full sync...`,
);
} catch (error: unknown) {
const message = error instanceof Error ? error.message : String(error);
const message =
error instanceof Error ? error.message : String(error);
this.logger.error(`Stale live_match cleanup failed: ${message}`);
return;
}
@@ -182,8 +195,11 @@ export class DataFetcherTask {
this.logger.log("syncLiveMatches START");
const today = getDateStringInTimeZone(new Date(), this.timeZone);
const tomorrow = getShiftedDateStringInTimeZone(1, this.timeZone);
await this.syncMatchList(today);
await this.syncMatchList(tomorrow);
await this.updateLiveScores();
await this.settlePredictionRuns();
await this.fetchOddsForMatches();
await this.fillMissingLineups();
@@ -192,12 +208,12 @@ export class DataFetcherTask {
private async syncMatchList(date: string): Promise<void> {
// Football
const footballLeagues = this.loadLeagueFilterSet("top_leagues.json");
const footballLeagues = this.loadLeagueFilterSet("qualified_leagues.json");
if (footballLeagues && footballLeagues.size > 0) {
await this.fetchMatchesForSport("football", date, footballLeagues);
} else {
this.logger.warn(
"top_leagues.json is missing/empty — writing ALL football matches",
"qualified_leagues.json is missing/empty — writing ALL football matches",
);
await this.fetchMatchesForSport("football", date, new Set());
}
@@ -248,7 +264,7 @@ export class DataFetcherTask {
}
this.logger.log(
`📡 Updating scores for ${liveMatches.length} live matches`,
`LIVE Updating scores for ${liveMatches.length} live matches`,
);
for (const match of liveMatches) {
@@ -260,13 +276,23 @@ export class DataFetcherTask {
if (response.data?.data) {
const matchData = response.data.data;
const scoreHome = matchData.homeScore ?? null;
const scoreAway = matchData.awayScore ?? null;
const storedStatus = deriveStoredMatchStatus({
state: matchData.state,
status: matchData.status,
substate: matchData.substate,
scoreHome,
scoreAway,
});
await this.prisma.liveMatch.update({
where: { id: match.id },
data: {
scoreHome: matchData.homeScore ?? null,
scoreAway: matchData.awayScore ?? null,
state: matchData.state || matchData.status,
status: matchData.status,
scoreHome,
scoreAway,
state: matchData.state || null,
substate: matchData.substate || null,
status: storedStatus,
updatedAt: new Date(),
},
});
@@ -276,25 +302,319 @@ export class DataFetcherTask {
}
}
this.logger.log("📡 Live score update complete");
this.logger.log("LIVE Live score update complete");
} catch (error: unknown) {
const message = error instanceof Error ? error.message : String(error);
this.logger.error(`Live score update failed: ${message}`);
}
}
// ────────────────────────────────────────────────────────────
private async settlePredictionRuns(): Promise<void> {
try {
const rows = await this.prisma.$queryRawUnsafe<
PendingPredictionRunForSettlement[]
>(`
SELECT
pr.id,
pr.match_id AS "matchId",
pr.engine_version AS "engineVersion",
pr.payload_summary AS "payloadSummary",
m.score_home AS "scoreHome",
m.score_away AS "scoreAway",
m.ht_score_home AS "htScoreHome",
m.ht_score_away AS "htScoreAway"
FROM prediction_runs pr
JOIN matches m ON m.id = pr.match_id
WHERE pr.eventual_outcome IS NULL
AND m.sport = 'football'
AND m.status = 'FT'
AND m.score_home IS NOT NULL
AND m.score_away IS NOT NULL
ORDER BY pr.generated_at ASC
LIMIT 500
`);
if (rows.length === 0) return;
let settled = 0;
for (const row of rows) {
const result = this.resolvePredictionRunSettlement(row);
if (!result) continue;
const closingOddsSnapshot = await this.getClosingOddsSnapshot(
row.matchId,
);
const settlementSummary = {
settled_at: new Date().toISOString(),
model_version: row.engineVersion,
outcome: result.outcome,
unit_profit: result.unitProfit,
final_score: {
home: row.scoreHome,
away: row.scoreAway,
},
halftime_score: {
home: row.htScoreHome,
away: row.htScoreAway,
},
closing_odds_snapshot: closingOddsSnapshot,
};
await this.prisma.$executeRawUnsafe(
`
UPDATE prediction_runs
SET eventual_outcome = $1,
unit_profit = $2,
payload_summary = payload_summary || jsonb_build_object('settlement', $3::jsonb)
WHERE id = $4
`,
result.outcome,
result.unitProfit,
JSON.stringify(settlementSummary),
row.id,
);
settled++;
}
if (settled > 0) {
this.logger.log(`Settled ${settled} prediction run(s)`);
}
} catch (error: unknown) {
const message = error instanceof Error ? error.message : String(error);
this.logger.warn(`Prediction run settlement skipped: ${message}`);
}
}
private async getClosingOddsSnapshot(
matchId: string,
): Promise<Record<string, unknown>> {
const liveMatch = await this.prisma.liveMatch.findUnique({
where: { id: matchId },
select: {
odds: true,
oddsUpdatedAt: true,
status: true,
state: true,
scoreHome: true,
scoreAway: true,
},
});
if (liveMatch?.odds) {
return {
source: "live_match",
odds: liveMatch.odds,
odds_updated_at: liveMatch.oddsUpdatedAt?.toISOString() ?? null,
status: liveMatch.status ?? null,
state: liveMatch.state ?? null,
score_home: liveMatch.scoreHome,
score_away: liveMatch.scoreAway,
};
}
const categories = await this.prisma.oddCategory.findMany({
where: { matchId },
select: {
name: true,
selections: {
select: {
name: true,
oddValue: true,
position: true,
updatedAt: true,
},
orderBy: { position: "asc" },
take: 12,
},
},
orderBy: { name: "asc" },
take: 24,
});
return {
source: "odd_selections",
category_count: categories.length,
categories: categories.map((category) => ({
name: category.name,
selections: category.selections.map((selection) => ({
name: selection.name,
odd_value: selection.oddValue,
position: selection.position,
updated_at: selection.updatedAt?.toISOString() ?? null,
})),
})),
};
}
private resolvePredictionRunSettlement(
row: PendingPredictionRunForSettlement,
): { outcome: string; unitProfit: number } | null {
const summary = this.asRecord(row.payloadSummary);
const mainPick = this.asRecord(summary.main_pick);
const market = String(mainPick.market || "");
const pick = String(mainPick.pick || "");
const playable = mainPick.playable === true;
const odds = Number(mainPick.odds || 0);
if (
!market ||
!pick ||
!playable ||
!Number.isFinite(odds) ||
odds <= 1.01
) {
return { outcome: "NO_BET", unitProfit: 0 };
}
const won = this.isPredictionPickWon({
market,
pick,
scoreHome: row.scoreHome,
scoreAway: row.scoreAway,
htScoreHome: row.htScoreHome,
htScoreAway: row.htScoreAway,
});
if (won === null) return null;
return {
outcome: `${won ? "WON" : "LOST"}:${market}:${pick}`,
unitProfit: Number((won ? odds - 1 : -1).toFixed(4)),
};
}
private isPredictionPickWon(input: {
market: string;
pick: string;
scoreHome: number | null;
scoreAway: number | null;
htScoreHome: number | null;
htScoreAway: number | null;
}): boolean | null {
const market = input.market.toUpperCase();
const pick = this.normalizePick(input.pick);
const scoreHome = input.scoreHome;
const scoreAway = input.scoreAway;
if (scoreHome === null || scoreAway === null) return null;
if (market === "MS") {
if (pick === "1") return scoreHome > scoreAway;
if (pick === "X" || pick === "0") return scoreHome === scoreAway;
if (pick === "2") return scoreAway > scoreHome;
return null;
}
if (market === "DC") {
const normalized = pick.replace("-", "");
if (normalized === "1X") return scoreHome >= scoreAway;
if (normalized === "X2") return scoreAway >= scoreHome;
if (normalized === "12") return scoreHome !== scoreAway;
return null;
}
if (market === "BTTS") {
const bothScored = scoreHome > 0 && scoreAway > 0;
if (pick.includes("VAR") || pick.includes("YES") || pick === "Y") {
return bothScored;
}
if (pick.includes("YOK") || pick.includes("NO") || pick === "N") {
return !bothScored;
}
return null;
}
const goalLine = this.goalLineForMarket(market);
if (goalLine !== null) {
const total = market.startsWith("HT_")
? this.nullableSum(input.htScoreHome, input.htScoreAway)
: scoreHome + scoreAway;
if (total === null) return null;
if (this.isOverPick(pick)) return total > goalLine;
return total < goalLine;
}
if (market === "HT") {
const htHome = input.htScoreHome;
const htAway = input.htScoreAway;
if (htHome === null || htAway === null) return null;
if (pick === "1") return htHome > htAway;
if (pick === "X" || pick === "0") return htHome === htAway;
if (pick === "2") return htAway > htHome;
}
if (market === "HTFT") {
const htHome = input.htScoreHome;
const htAway = input.htScoreAway;
if (htHome === null || htAway === null || !pick.includes("/"))
return null;
const [htPick, ftPick] = pick.split("/");
return (
this.isResultPickWon(htPick, htHome, htAway) === true &&
this.isResultPickWon(ftPick, scoreHome, scoreAway) === true
);
}
return null;
}
private isResultPickWon(
pick: string,
homeScore: number,
awayScore: number,
): boolean | null {
if (pick === "1") return homeScore > awayScore;
if (pick === "X" || pick === "0") return homeScore === awayScore;
if (pick === "2") return awayScore > homeScore;
return null;
}
private goalLineForMarket(market: string): number | null {
if (market === "OU15") return 1.5;
if (market === "OU25") return 2.5;
if (market === "OU35") return 3.5;
if (market === "HT_OU05") return 0.5;
if (market === "HT_OU15") return 1.5;
return null;
}
private nullableSum(a: number | null, b: number | null): number | null {
if (a === null || b === null) return null;
return a + b;
}
private normalizePick(value: string): string {
return value
.trim()
.toUpperCase()
.replace(/İ/g, "I")
.replace(/Ü/g, "U")
.replace(/Ş/g, "S")
.replace(/Ğ/g, "G")
.replace(/Ö/g, "O")
.replace(/Ç/g, "C");
}
private isOverPick(pick: string): boolean {
return pick.includes("UST") || pick.includes("OVER");
}
private asRecord(value: unknown): Record<string, unknown> {
return value && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: {};
}
// Phase 3: Odds + referee + lineups + sidelined
// ────────────────────────────────────────────────────────────
private async fetchOddsForMatches(): Promise<void> {
this.logger.log("💰 Fetching odds for live matches...");
this.logger.log("MONEY Fetching odds for live matches...");
try {
// Load both league filters
// Load both league filters (data-driven qualified leagues)
const topLeagueIds: string[] = [];
const footballLeagues = this.loadLeagueFilterSet("top_leagues.json");
const footballLeagues = this.loadLeagueFilterSet(
"qualified_leagues.json",
);
if (footballLeagues) topLeagueIds.push(...footballLeagues);
const basketballLeagues = this.loadLeagueFilterSet(
@@ -305,6 +625,9 @@ export class DataFetcherTask {
const allowedLeagueIds = Array.from(new Set(topLeagueIds));
// Get matches needing odds (from 12 hours ago onward)
// CRITICAL: Only fetch odds/lineups for NOT STARTED matches.
// Once a match goes live, odds selections disappear or change,
// corrupting the pre-match data the AI model relies on.
const twelveHoursAgo = new Date(Date.now() - 12 * 60 * 60 * 1000);
const matchesToFetch = await this.prisma.liveMatch.findMany({
@@ -313,6 +636,15 @@ export class DataFetcherTask {
...(allowedLeagueIds.length > 0
? { leagueId: { in: allowedLeagueIds } }
: {}),
// Exclude live and finished matches — preserve their pre-match odds
NOT: {
OR: [
{ status: { in: FINISHED_STATUS_VALUES_FOR_DB } },
{ state: { in: FINISHED_STATE_VALUES_FOR_DB } },
{ status: { in: LIVE_STATUS_VALUES_FOR_DB } },
{ state: { in: LIVE_STATE_VALUES_FOR_DB } },
],
},
},
include: {
homeTeam: { select: { name: true } },
@@ -323,11 +655,13 @@ export class DataFetcherTask {
});
if (matchesToFetch.length === 0) {
this.logger.log("💰 No matches to fetch odds for");
this.logger.log("MONEY No matches to fetch odds for");
return;
}
this.logger.log(`💰 Fetching odds for ${matchesToFetch.length} matches`);
this.logger.log(
`MONEY Fetching odds for ${matchesToFetch.length} matches`,
);
let successCount = 0;
let errorCount = 0;
@@ -356,7 +690,7 @@ export class DataFetcherTask {
// Retry failed matches (502/Timeout)
if (failedMatches.length > 0) {
this.logger.warn(
`⚠️ Retrying ${failedMatches.length} failed matches (502/Timeout)...`,
`Retrying ${failedMatches.length} failed matches (502/Timeout)...`,
);
for (const match of failedMatches) {
@@ -364,7 +698,7 @@ export class DataFetcherTask {
try {
await this.processMatchOdds(match);
successCount++;
this.logger.log(`✅ Retry successful for match ${match.id}`);
this.logger.log(`SUCCESS Retry successful for match ${match.id}`);
} catch (retryErr: unknown) {
const message =
retryErr instanceof Error ? retryErr.message : String(retryErr);
@@ -376,7 +710,7 @@ export class DataFetcherTask {
}
this.logger.log(
`💰 Odds complete: ${successCount} success, ${errorCount} errors (initially)`,
`MONEY Odds complete: ${successCount} success, ${errorCount} errors (initially)`,
);
} catch (error: unknown) {
const message = error instanceof Error ? error.message : String(error);
@@ -432,7 +766,10 @@ export class DataFetcherTask {
for (const match of toUpdate) {
try {
const formation = await this.scraper.fetchStartingFormation(match.id);
const [formation, substitutions] = await Promise.all([
this.scraper.fetchStartingFormation(match.id),
this.scraper.fetchSubstitutions(match.id),
]);
const sidelined = match.matchSlug
? await this.scraper.fetchSidelinedPlayers(
match.id,
@@ -440,11 +777,26 @@ export class DataFetcherTask {
)
: null;
// Normalize to same home.xi/away.xi format used by processMatchOdds
let normalizedLineups: Record<string, unknown> | null = null;
if (formation || substitutions) {
normalizedLineups = {
home: {
xi: formation?.stats?.home || [],
subs: substitutions?.stats?.home || [],
},
away: {
xi: formation?.stats?.away || [],
subs: substitutions?.stats?.away || [],
},
};
}
await this.prisma.liveMatch.update({
where: { id: match.id },
data: {
lineups: formation
? JSON.parse(JSON.stringify(formation))
lineups: normalizedLineups
? JSON.parse(JSON.stringify(normalizedLineups))
: Prisma.JsonNull,
sidelined: sidelined
? JSON.parse(JSON.stringify(sidelined))
@@ -670,6 +1022,15 @@ export class DataFetcherTask {
// Safe score parsing
const sHome = this.asInt(match.homeScore ?? match.score?.home);
const sAway = this.asInt(match.awayScore ?? match.score?.away);
const storedStatus = deriveStoredMatchStatus({
state: match.state,
status: match.status,
substate: match.substate,
statusBoxContent: match.statusBoxContent,
scoreHome: sHome,
scoreAway: sAway,
score: match.score,
});
// Handle postponed matches (ERT = Erteledendi)
if (match.statusBoxContent === "ERT") {
@@ -698,7 +1059,7 @@ export class DataFetcherTask {
leagueId: leagueId,
state: match.state || null,
substate: match.substate || null,
status: match.status || match.state || "NS",
status: storedStatus,
scoreHome: sHome,
scoreAway: sAway,
homeTeamId: homeTeamId,
@@ -713,7 +1074,7 @@ export class DataFetcherTask {
leagueId: leagueId,
state: match.state || null,
substate: match.substate || null,
status: match.status || match.state || "NS",
status: storedStatus,
mstUtc: BigInt(match.mstUtc || Date.now()),
scoreHome: sHome,
scoreAway: sAway,
@@ -810,8 +1171,8 @@ export class DataFetcherTask {
const matchTime = Number(match.mstUtc);
const diffHours = (matchTime - now) / (1000 * 60 * 60);
// Fetch if between -3 hours (started) and +4 hours (upcoming)
if (diffHours < 4 && diffHours > -3) {
// Fetch if between -3 hours (started) and +24 hours (upcoming)
if (diffHours < 24 && diffHours > -3) {
// Lineups
try {
const [startingFormation, substitutions] = await Promise.all([
@@ -871,7 +1232,44 @@ export class DataFetcherTask {
}
}
// ALWAYS update oddsUpdatedAt to ensure rotation
// Guard: If match already has pre-match odds and is now live/finished,
// do NOT overwrite odds/lineups/sidelined — the model needs stable pre-match data.
const matchState = match.state?.toLowerCase() ?? "";
const matchStatus = match.status?.toLowerCase() ?? "";
const liveStates = LIVE_STATE_VALUES_FOR_DB.map((s) => s.toLowerCase());
const liveStatuses = LIVE_STATUS_VALUES_FOR_DB.map((s) => s.toLowerCase());
const finishedStates = FINISHED_STATE_VALUES_FOR_DB.map((s) =>
s.toLowerCase(),
);
const finishedStatuses = FINISHED_STATUS_VALUES_FOR_DB.map((s) =>
s.toLowerCase(),
);
const isLiveOrFinished =
liveStates.includes(matchState) ||
liveStatuses.includes(matchStatus) ||
finishedStates.includes(matchState) ||
finishedStatuses.includes(matchStatus);
const existingOdds = match.odds as Record<string, unknown> | null;
const hasExistingOdds =
!!existingOdds &&
typeof existingOdds === "object" &&
Object.keys(existingOdds).length > 0;
if (isLiveOrFinished && hasExistingOdds) {
// Match is live/finished and already has pre-match odds — skip data update
this.logger.debug(
`🛡️ Preserving pre-match data for ${match.matchName} (status: ${matchStatus}, state: ${matchState})`,
);
await this.prisma.liveMatch.update({
where: { id: match.id },
data: { oddsUpdatedAt: new Date() },
});
return;
}
// Update odds/lineups/sidelined for pre-match (NS) matches
await this.prisma.liveMatch.update({
where: { id: match.id },
data: {
@@ -892,7 +1290,7 @@ export class DataFetcherTask {
sidelined.awayTeam.totalSidelined > 0))
) {
this.logger.log(
`✅ Loop update: ${match.matchName} | Odds: ${Object.keys(odds).length} | Ref: ${refereeName || "N/A"} | Lineups: ${lineups ? "Yes" : "No"} | Sidelined: ${sidelined ? "Yes" : "No"}`,
`SUCCESS Loop update: ${match.matchName} | Odds: ${Object.keys(odds).length} | Ref: ${refereeName || "N/A"} | Lineups: ${lineups ? "Yes" : "No"} | Sidelined: ${sidelined ? "Yes" : "No"}`,
);
} else {
this.logger.debug(
+44 -10
View File
@@ -141,7 +141,7 @@ export class LimitResetterTask {
}
/**
* Reset subscription status for expired users
* Downgrade cancelled subscriptions that have passed their cancel effective date
*/
@Cron("0 0 * * *", { timeZone: "Europe/Istanbul" })
async checkSubscriptions() {
@@ -155,21 +155,55 @@ export class LimitResetterTask {
try {
const now = new Date();
const result = await this.prisma.user.updateMany({
// Find subscriptions with passed cancel effective date
const expiredSubs = await this.prisma.subscription.findMany({
where: {
subscriptionStatus: "active",
subscriptionExpiresAt: { lt: now },
plan: "cancelled",
cancelEffectiveDate: { lt: now },
},
data: {
subscriptionStatus: "expired",
select: { id: true, userId: true },
});
for (const sub of expiredSubs) {
// Downgrade to free
await this.prisma.user.update({
where: { id: sub.userId },
data: { subscriptionStatus: "free" },
});
// Sync limits to free tier
await this.prisma.usageLimit.upsert({
where: { userId: sub.userId },
update: { maxAnalyses: 3, maxCoupons: 1 },
create: {
userId: sub.userId,
analysisCount: 0,
couponCount: 0,
maxAnalyses: 3,
maxCoupons: 1,
lastResetDate: new Date(),
},
});
if (result.count > 0) {
this.logger.log(`${result.count} subscriptions marked as expired`);
// Reset subscription to free
await this.prisma.subscription.update({
where: { id: sub.id },
data: {
plan: "free",
cancelledAt: null,
cancelEffectiveDate: null,
},
});
}
} catch (error: any) {
this.logger.error(`Subscription check failed: ${error.message}`);
if (expiredSubs.length > 0) {
this.logger.log(
`${expiredSubs.length} cancelled subscriptions downgraded to free`,
);
}
} catch (error: unknown) {
const err = error as Error;
this.logger.error(`Subscription check failed: ${err.message}`);
}
},
this.logger,
BIN
View File
Binary file not shown.