# AI Context: Deployment & Feeder Optimization Changelog

**Date:** 2026-01-12
**Component:** Backend / DevOps / Feeder
**Author:** AI Agent (Antigravity)

## 1. Infrastructure & Deployment (EC2 & GitHub Actions)

### 🚀 Zero-to-Hero Deployment (`deploy-feeder.yml`)

- **Automated Setup:** The pipeline now handles full server provisioning (installing Node.js v20, Git, Docker, PM2) and repository cloning if not present.
- **Private Repo Access:** Switched to using `GH_PAT` (Personal Access Token) for `git clone` and `git pull`, resolving "Username not found" errors on private repositories.
- **Secure Environment Management (Senior Approach):**
  - Moved away from insecure/fragile `.env` manipulation (sed/cp).
  - Implemented dynamic `.env` generation from **GitHub Secrets** (`DATABASE_URL`, `REDIS_HOST`, `JWT_SECRET`).
  - **Critical Config:** `DATABASE_URL` is configured to `localhost:15432` for the Host-based PM2 process to access Dockerized Postgres, while Docker containers use internal networking.

### 💾 Data Persistence

- **Docker Volumes:** Switched from named volumes to **Bind Mounts**:
  - Postgres: `./data/postgres:/var/lib/postgresql/data`
  - Redis: `./data/redis:/data`
- **Result:** Data persists directly on the EC2 host file system, surviving container recreation and allowing easier backups.

## 2. Feeder Service Optimization (`feeder.service.ts`)

### ⚡ Performance Tuning (Turbo Mode)

- **Concurrency:** Increased from `5` to **`20`** parallel requests.
- **Request Delay:** Reduced from `500ms` to **`50ms`** per batch.
- **Throughput:** ~5-10x speed improvement for historical data ingestion.

### 🧠 Enhancements

- **Smart Resume:** The service checks `AppSetting` to resume from the last successfully processed date.
- **ETA Logging:** Added real-time calculation logic:
  - Tracks `AvgTimePerDay`.
  - Projects `RemainingTime` based on remaining days.
  - Logs nice status: `⏱️ PROGRESS: [X days done] | Avg/Day: Ys | Remaining: Z days | 🏁 ETA: HH:MM:SS`
- **Clean Code:** Removed unused variables (`dayStartTime`, `totalDaysInRange`) for better maintainability.

## 3. Stability & Persistence Fixes (`feeder-persistence.service.ts`)

### 🛡️ Race Condition Handling

- **Country Upsert:** Wrapped `prisma.country.upsert` in a `try-catch` block to silently ignore `P2002` (Unique Constraint) errors. This fixes crashes caused by multiple parallel workers trying to create the same country simultaneously.

### 🧹 Data Deduplication

- **Match Officials:** Implemented in-memory deduplication (using `Set`) before insertion.
- **Problem:** Source data (Mackolik) sometimes lists the same official twice for a match, causing DB constraint failures.
- **Fix:** `name + role` combinations are checked, and duplicates are filtered out before hitting the database.

## 4. Current Architecture Overview

- **App Runtime:** `PM2` (Host) -> Runs `npm run feeder:historical`.
- **Database:** `Docker` (Postgres 16) -> Mapped to Host `15432`.
- **Cache:** `Docker` (Redis 7) -> Mapped to Host `6379`.
- **Flow:** Feeder (Host) connects to -> localhost:15432 (Postgres) & localhost:6379 (Redis).

---

_This document serves as a memory checkpoint for future context. Do not delete._

---

# AI Context: Ransomware Saldırısı ve Güvenlik Sıkılaştırması

**Date:** 2026-01-16
**Component:** Security / Infrastructure / DevOps
**Author:** AI Agent (Antigravity)

## 1. Olay Özeti

### 🚨 Ransomware Saldırısı Tespit Edildi

- **Tarih:** 15 Ocak 2026
- **Sorun:** PostgreSQL veritabanı (`boilerplate_db`) silindi
- **Sebep:** Port 15432 internete açıktı + default credentials (`postgres/postgres`)
- **Fidye Notu:** `readme_to_recover` database'inde Bitcoin talebi

### 🔍 Saldırı Vektörü

1. Saldırgan açık 15432 portunu taradı
2. Default `postgres/postgres` ile giriş yaptı
3. `boilerplate_db` silindi
4. `readme_to_recover` fidye notu bırakıldı

## 2. Uygulanan Güvenlik Önlemleri

### A. Veritabanı Güvenliği

| Önlem     | Eski         | Yeni                |
| --------- | ------------ | ------------------- |
| Kullanıcı | `postgres`   | `suggestbet`        |
| Şifre     | `postgres`   | `SuGGesT2026SecuRe` |
| Port 15432 | Herkese açık | Sadece localhost    |

### B. AWS Security Group

- ❌ Port 15432 **kapatıldı**
- ❌ Port 22 **kapatıldı** (SSH yerine SSM)
- ✅ Port 80/443 açık (Nginx)

### C. SSM Session Manager

- SSH yerine AWS SSM kullanılıyor
- IAM Role: `EC2-SSM-Role` ile `AmazonSSMManagedInstanceCore` policy
- Port forwarding ile DBeaver bağlantısı (lokal port: 15432)

### D. UFW Firewall (EC2)

```bash
# Aktif kurallar:
80/tcp      ALLOW
443/tcp     ALLOW
15432        ALLOW 127.0.0.1  # Sadece localhost
```

### E. Docker Güvenliği (`docker-compose.yml`)

```yaml
# Portlar sadece localhost'a bind
ports:
  - '127.0.0.1:15432:15432' # Postgres
  - '127.0.0.1:6379:6379' # Redis
```

### F. Redis Güvenliği

- Şifre eklendi: `RedisSecure2026`

## 3. Dosya Değişiklikleri

| Dosya                                 | Değişiklik                               |
| ------------------------------------- | ---------------------------------------- |
| `docker-compose.yml`                  | Yeni credentials, localhost-only binding |
| `.env`                                | Yeni DB user/password, Redis password    |
| `.github/workflows/deploy-feeder.yml` | Database varlık kontrolü, Redis password |
| `mds/SERVER_SECURITY_GUIDE.md`        | **YENİ** - Bağlantı kılavuzu             |

## 4. Bağlantı Yöntemi

### DBeaver Bağlantısı (Mac/Windows)

```bash
# Terminal'de SSM port forwarding başlat:
dbconnect  # alias

# DBeaver ayarları:
# Host: localhost
# Port: 15432
# Database: boilerplate_db
# User: suggestbet
# Password: SuGGesT2026SecuRe
```

## 5. GitHub Secrets Güncellenmeli

| Secret           | Değer                                                                                   |
| ---------------- | --------------------------------------------------------------------------------------- |
| `DATABASE_URL`   | `postgresql://suggestbet:SuGGesT2026SecuRe@localhost:15432/boilerplate_db?schema=public` |
| `REDIS_PASSWORD` | `RedisSecure2026`                                                                       |

## 6. Kalan İşler

- [x] GitHub Secrets güncelle
- [x] Değişiklikleri push'la
- [ ] EC2'de `docker compose up -d` (yeni config için)
- [ ] Feeder'ı yeniden başlat

---

# AI Context: Database Sample Export Scripts

**Date:** 2026-01-16
**Component:** Developer Tools / AI Context
**Author:** AI Agent (Antigravity)

## Amaç

AI asistanların veritabanı yapısını ve içeriğini anlaması için örnek veri export script'leri oluşturuldu.

## Dosyalar

| Dosya                           | Platform  | Açıklama                  |
| ------------------------------- | --------- | ------------------------- |
| `scripts/export-db-samples.sh`  | Mac/Linux | Bash script               |
| `scripts/export-db-samples.ps1` | Windows   | PowerShell script         |
| `mds/DATABASE_SAMPLES.md`       | -         | Oluşturulan çıktı dosyası |

## Kullanım

### Mac/Linux

```bash
# 1. SSM port forwarding başlat
dbconnect

# 2. Yeni terminal'de script çalıştır
bash scripts/export-db-samples.sh
```

### Windows

```powershell
# 1. SSM port forwarding başlat
dbconnect

# 2. Yeni PowerShell'de script çalıştır
.\scripts\export-db-samples.ps1
```

## Çıktı

Script şu bilgileri `mds/DATABASE_SAMPLES.md` dosyasına yazar:

- Tüm tabloların kayıt sayıları
- Her önemli tablodan 5-10 örnek kayıt (JSON formatında)
- Matches, Leagues, Teams, Countries, Predictions, Stats, Odds

## Ne Zaman Çalıştırılmalı

- Yeni AI oturumu başlamadan önce
- Veritabanı yapısı değiştiğinde
- Önemli veri değişikliklerinden sonra

---

---

# AI Context: V20 Ensemble & Feeder Optimization

**Date:** 2026-02-08
**Component:** AI Engine / Data Feeder / Stability
**Author:** AI Agent (Antigravity)

## 1. V20 Ensemble "Beast" Deployment

- **Architecture:** Synthesis of 4 engines (Team, Player, Odds, Referee).
- **Surprise Detection:** Added `UpsetEngine` to track motivation and position-based risks.
- **Enhanced Predictions:** Added xG (Expected Goals), Top 5 correct scores, and Smart Value recommendations.

## 2. Core Stability Patches

- **Null-Safety:** Exhaustive `is not None` checks added to `ContextEngine`, `UpsetEngine`, and `V20EnsemblePredictor` to prevent crashes when standings/stats are missing.
- **Environment Parity:** Patched hardcoded production IPs (`13.49.226.80`) with `localhost` across all AI sub-engines via `patch-ips.js`.

## 3. Feeder & Data Fetching

- **Top Leagues Filter:** Implementation of `top_leagues.json` reduced processing load by ~85% (~160 matches vs 1200+).
- **Lineup Coverage:** Expanded fetch window (4h pre-match, 3h post-match) ensures 11-man starting lineups (XI) are captured for major leagues.
- **Retry Logic:** Added 502/Timeout handling in `DataFetcherTask` for resilient data ingestion.

---

_This document serves as a memory checkpoint. For deep technical details, see [mds/V20_AI_ENGINE_AND_FEEDER_EVOLUTION.md](file:///c:/Users/fahri/Documents/GitHub/Suggest-Bet-BE/mds/V20_AI_ENGINE_AND_FEEDER_EVOLUTION.md)._