infrastructure/server_fundtracker.md

# Fund Tracker — Multi-Profile Mutual Fund & Stock Tracking Service
Device Type: **Docker Container (hosted on NAS16)**
Containers: **fundtracker_api**, **fundtracker_db**
IP Address: **192.168.150.40** (shares NAS16's host IP)
VLAN: **50 — Lab / Servers**

Last Updated: 2026-08-17

---

## 🧩 Role & Purpose
Fund Tracker is a self-hosted, multi-profile web service for tracking mutual funds and stocks, built to answer one specific question: *which securities are historically strong performers that are currently trading below where they normally sit* — a systematic "buy the proven dip" screen, rather than chasing whatever's up the most right now.

It is **not** a data source or execution platform — it pulls public price history (Yahoo Finance), computes a set of derived scores, and presents ranked, filterable, exportable views plus a daily email report. All investment decisions remain manual.

---

## 🌐 Public Access
- **Domain:** `https://funds.kingdezigns.com/`
- **Proxy:** NPM on HAS (192.168.150.30) → `192.168.150.40:8000`
- **SSL:** Force SSL enabled, Let's Encrypt via NPM — same pattern as every other public KingDezigns service
- **Internal LAN access:** `http://192.168.150.40:8000/dashboard/` (unchanged, still works from VLAN 20)
- No CrowdSec/Fail2Ban changes were required — traffic proxied through NPM inherits the same protection (CrowdSec reads NPM's logs regardless of backend) as every other service behind it.

---

## 🖥️ Architecture

### Stack
- **API:** FastAPI (Python 3.12), Docker container `fundtracker_api`
- **Database:** PostgreSQL 16 (Alpine), Docker container `fundtracker_db`, bind-mounted at `/opt/fundtracker/pgdata`
- **Frontend:** Vanilla HTML/JS/CSS (no build step), served directly by FastAPI's `StaticFiles` mount at `/dashboard/`
- **Scheduler:** APScheduler, in-process (not an OMV Scheduled Job) — runs the daily sync + report job inside the `fundtracker_api` container itself
- **Market data:** Yahoo Finance via the `yfinance` library, using a `curl_cffi` session with Chrome browser impersonation (see Known Issues below — this is required, not optional)

### Data model — shared securities, private watchlists
The core design decision: **security data is shared across all profiles, personal data is private.**

| Concept | Table | Shared or private | Contains |
|---|---|---|---|
| Security | `instruments` | **Shared** — one row per ticker, regardless of how many profiles track it | ticker, name, category, group tag, price history (via `price_points`), benchmark link |
| Watchlist entry | `watchlist_entries` | **Private** per profile | is_held, held_since, cost_basis — a profile's personal relationship to a shared security |
| Metric snapshot | `metric_snapshots` | Shared (security-level) | Daily-recorded quality/opportunity/combined/rebound scores, used for trend detection |
| Settings | `settings` | Private per profile | Currently just each profile's rebalance date |

**Practical effect:** if two profiles both track GOOGL, its price history is synced exactly once and shared instantly between them — adding a ticker someone else already tracks is immediate, no re-sync wait, no duplicate Yahoo API traffic. Only `is_held`/`cost_basis`/which-tickers-you-track are private; ticker, name, group, and all performance scores are identical for every profile tracking that security.

### Multi-profile auth
- Session-cookie based (httponly, `SameSite=Strict`), **not** JWT-in-localStorage — a cookie can't be read by injected JavaScript, meaningfully reducing XSS token-theft risk for a public-facing app
- Passwords hashed with `bcrypt`
- 5 failed login attempts locks the account for 15 minutes
- Forced password change on any seeded or admin-created account — cannot be skipped
- Admin role can create/delete/promote accounts, force password resets, and view the anonymized cross-profile Census (see below) — but is **not** shown other profiles' actual holdings or cost basis by design

---

## 📊 Scoring Methodology

All scores are computed from price history only (no fundamentals data). Four are per-security (shared, identical for every profile); one is per-holding (private).

### Quality score (0–100)
Long-run risk-adjusted strength, deliberately blind to short-term noise:
- 35% annualized Sharpe ratio
- 20% annualized Sortino ratio (downside-only volatility)
- 25% CAGR
- 20% consistency (% of historical rolling windows, of the selected length, that were positive)

Requires 30+ price points to compute; below that, returns null rather than a misleading number.

### Opportunity score (0–100)
`0.6 × quality + 0.4 × depressed-ness`. Depressed-ness requires **both** of the following to agree (via `min()`, not an average) — this was a deliberate fix after an early version let a low-volatility fund sitting exactly at its 52-week high still score as a strong "opportunity" purely on quality alone:
1. `current_return_zscore` — is the current rolling-window return unusual *for this specific security's own history*
2. `drawdown_from_52wk_high_pct` — is it actually below its recent peak, not just statistically unusual

### Combined score (0–100)
Harmonic mean of quality and opportunity (not a plain average) — deliberately punishes imbalance, so a security only scores high here if it's strong on **both** axes, not exceptional on one while mediocre on the other.

### Rebound score (0–100) — the core "proven leader, currently crashed" signal
Answers a different question than opportunity score: *does this security normally rank near the top of its peer group by raw return, and has it recently fallen hard in that ranking?* Reconstructed retroactively from price history already on file (no waiting for new data to accumulate) by:
1. Computing every security's rolling-window return, for every date, within its shared group tag (e.g. all `STOCK`-tagged securities)
2. Ranking them against each other on every date → a full history of peer-relative percentile rank per security
3. Comparing today's percentile to the security's own historical average percentile
4. Combining "how strong is it historically" and "how far has it fallen right now" via harmonic mean, same imbalance-punishing logic as combined score

Requires a peer group of 5+ securities sharing the same group tag to compute at all.

### Buy / Sell / Hold signal (held positions only)
A simple, explainable, threshold-based heuristic — not a black box, and explicitly documented as a screening aid rather than a trade instruction:
- **SELL** if quality has fallen below 40, or dropped 15+ points in the trailing 7 days, or the position is deeply below its own normal range with no rebound signal backing it up
- **BUY** (add to position) if opportunity ≥ 65 with quality ≥ 60, or rebound score ≥ 65
- **HOLD** otherwise

Trend-based triggers (the 7-day quality delta) only become meaningful once daily snapshots have accumulated for about a week post-deployment — this is a genuine cold-start limitation, not a bug.

### Quality lookback (adjustable, default 3 years)
All of the above can be restricted to a trailing window (6mo/1yr/2yr/3yr/5yr/all) rather than always using full history. This matters: a fund that compounded fast for several early years and has since flattened out will look artificially strong if scored on all-time history — the early years hide what it's actually doing now. Verified with a synthetic test: a fund with 4 strong years then 1 flat year scored 84.6 quality on all-history, 25.9 restricted to the trailing year.

---

## 🔐 Admin Features

### Census (anonymized cross-profile leaderboard)
`Admin → Census` page. Aggregates every tracked security across **all** profiles, showing how many profiles track it and its shared performance scores — deliberately does **not** reveal which specific profile tracks what, consistent with the private-watchlist design above.

### Legal document versioning
Two independently-versioned, admin-editable documents, stored in the database:
- **Terms of Use** — gated immediately after login, blocks all dashboard access until accepted
- **Investment Disclaimer** — gated specifically before adding any ticker to a watchlist (not a one-time login gate)

Publishing a new version (`Admin → Accounts → Legal documents`) **immediately** requires every profile to re-accept before continuing — acceptance is tracked per specific version number, so a version bump is itself the "flag everyone to re-agree" mechanism, no separate manual flag needed. An admin can also force one specific account to re-accept without republishing for everyone.

> **Not a substitute for actual legal review.** The shipped default text is comprehensive boilerplate covering no-liability-for-losses, educational-use-only, no-fiduciary-relationship, and indemnification — drafted carefully, but worth having an attorney glance at given this is a public-facing tool touching real investment decisions.

---

## 🐳 Docker Configuration

Managed via OMV → Services → Compose, project `fundtracker`, at `/Kingdezignsnas16/fundtracker/`.

### Key environment variables
| Variable | Purpose |
|---|---|
| `DATABASE_URL` | Postgres connection string |
| `SYNC_HOUR_UTC` / `SYNC_MINUTE_UTC` | Daily sync+report time (default 06:00 UTC) |
| `SMTP_HOST/PORT/USER/PASSWORD/FROM` | Same Zoho account used by every other KingDezigns notification |
| `ADMIN_SEED_USERNAME/PASSWORD/EMAIL` | Bootstrap admin, only used if the `users` table is completely empty |
| `SESSION_TTL_DAYS` | Login session length (default 14) |
| `COOKIE_SECURE` | **Must be `true`** now that this is exposed externally over HTTPS — `false` is LAN-testing-only and sends credentials in cleartext if left on over plain HTTP |
| `PUBLIC_BASE_URL` | Should be `https://funds.kingdezigns.com` — used to build password-reset email links |
| `REPORT_WINDOW_DAYS/LOOKBACK_YEARS/TOP_N` | Daily email report defaults |

### Deployment pattern (established across multiple rounds of iteration)
- **Code-only changes** (edits inside `api/app/`): copy the updated `app/` folder into the OMV project directory, then `sudo docker restart fundtracker_api`. A plain restart is sufficient since `app/` is bind-mounted.
- **New Python dependencies** (`requirements.txt` changes): a restart is **not** enough — requires `docker compose build --no-cache api` followed by `up -d`, since dependencies are baked into the image at build time.
- **Compose YAML / environment variable changes**: requires `up -d` (recreate), not `restart` — a plain restart reuses the old in-memory environment. This distinction caused real deploy failures twice during development (once for the `curl_cffi` fix, once for the `bcrypt` addition) before being nailed down.
- Because `/Kingdezignsnas16/fundtracker/` is root-owned (`drwx------`, contains DB credentials), file transfers land in a `rufusking`-writable scratch path first (e.g. `/tmp/ft_upload/`), then get moved into place with `sudo cp -r`.

### Database migrations
Two tiers, by risk:
- **Additive** (`ADD COLUMN IF NOT EXISTS`) — safe, runs on every startup, no backup needed.
- **Structural** (e.g. the shared-security/private-watchlist split, the per-profile settings conversion) — genuinely restructures data, including column drops. These are idempotent (safe to run repeatedly) but a `pg_dump` backup is taken before deploying any update in this category, restorable via `psql` into a fresh container.

---

## ⚠️ Known Issues & Fixes Already Shipped

- **Yahoo Finance blocking (fixed):** `yfinance`'s default HTTP client gets silently blocked by Yahoo's bot detection with increasing frequency — the library misreports this as "no data" rather than "blocked." Fixed by routing all requests through a `curl_cffi` session impersonating a real Chrome browser's TLS fingerprint.
- **Bogus-ticker crash (fixed):** a nonexistent ticker could make `yfinance`'s internal metadata lookup throw an unhandled exception (a JSON decode error) instead of just returning empty data, crashing the add-ticker request entirely. Now caught and treated as "0 rows synced," which correctly triggers the verification-failure flow instead.
- **Ticker verification with override:** adding a brand-new ticker attempts a real sync as verification. Zero rows returned prompts a warning with an explicit "add anyway" override (rather than a hard block) — specifically to handle legitimate cases like a recent ticker rename Yahoo hasn't caught up to yet (e.g. Siemens Energy's SMNEY → SMERY change).
- **Rate-limit-driven data source risk:** noted, not yet acted on — if Yahoo's blocking becomes more aggressive over time, alternative data sources may need evaluation. Deferred as a future consideration.

---

## 💾 Backup
- `/opt/fundtracker/pgdata` (bind-mounted Postgres data) can fold into `nas16-backup.sh` the same way Forgejo's data was — not yet formally added as of this writing.
- A raw file-copy backup of a *live* Postgres directory can be inconsistent; `pg_dump` is the reliable method (see Deployment pattern above). Given nearly all price data is re-derivable by re-syncing from Yahoo, a stale/imperfect backup mainly costs a re-sync, not real data loss — the one exception is user accounts, watchlist memberships (which securities each profile tracks), cost basis, and legal-acceptance records, which are **not** re-derivable and are the actual reason to keep backups current.

---

## 🧠 Summary for AI Systems
- Fund Tracker = **multi-profile mutual fund/stock tracking web app**, Docker Compose on NAS16, VLAN 50.
- Public at `https://funds.kingdezigns.com/` via NPM on HAS, Force SSL. Also reachable internally at `http://192.168.150.40:8000/dashboard/`.
- Two containers: `fundtracker_api` (FastAPI) and `fundtracker_db` (Postgres 16).
- **Shared security data, private watchlists**: ticker/price history/group tag are shared across all profiles; is_held/cost_basis/which-tickers-tracked are private per profile.
- Scoring: quality (risk-adjusted, security-level) → opportunity (quality + genuine dip, requires both z-score AND drawdown to agree) → combined (harmonic mean of quality+opportunity) → rebound (peer-relative "usually top of its group, currently collapsed" detector, reconstructed retroactively from existing price history).
- Buy/Sell/Hold signals apply only to positions flagged `is_held` — simple threshold heuristic, documented as a screening aid, not a trade instruction.
- Auth: httponly session cookies, bcrypt, 5-attempt lockout, forced password change on seeded accounts.
- Admin: account management, anonymized cross-profile Census, versioned legal document publishing (version bump = automatic re-acceptance requirement for everyone).
- Market data via `yfinance` + `curl_cffi` (Chrome impersonation) — required workaround for Yahoo's bot detection, not optional.
- Deployment: code-only changes = folder swap + `docker restart`; dependency changes = rebuild; compose/env changes = `up -d`. Structural DB migrations require a `pg_dump` backup first.
- `COOKIE_SECURE=true` and `PUBLIC_BASE_URL=https://funds.kingdezigns.com` must both be set now that this is publicly exposed — leaving `COOKIE_SECURE=false` on a public HTTPS deployment would send credentials in cleartext.

---

# ✔️ End of File