| 1 | # Persisted OHLCV technical features (TECHNICAL_FEATURES_V2) |
| 2 | |
| 3 | ## Integration and source selection |
| 4 | |
| 5 | `TechnicalFeatureEngine.compute(..., daily_bar_history=...)` evolves the existing |
| 6 | pure engine. No provider, database or network call occurs inside computation. |
| 7 | Stage-B `GlobalScanner.enrich_candidates` reads NSE DailyMarketBar rows in bounded |
| 8 | candidate-ID batches, then passes grouped histories to this input. It retains |
| 9 | batched close reads for fallback and the unchanged sector engine. Phase-1 scan, |
| 10 | preScore, eligibility, sector formulas, and Stage-B score weights are unchanged. |
| 11 | |
| 12 | Source policy: only REAL persisted NSE candles for the requested canonical UUID, |
| 13 | matching requested currency/provider allowlist, and known at as_of. Exchange |
| 14 | DATEs are compared to Asia/Kolkata DATE; retrieval timestamps remain UTC. History |
| 15 | trading-date fields expose actual DATEs. Display history timestamps are exchange |
| 16 | midnight metadata only and never become candle-series keys. |
| 17 | |
| 18 | A fresh NSE series with at least 20 usable observations takes precedence. When |
| 19 | NSE is shorter than 20 or stale and a current non-conflicting close-only series |
| 20 | has at least 20 observations, the entire close-only series is selected instead, |
| 21 | with `NSE_DAILY_HISTORY_BELOW_20` or `NSE_DAILY_HISTORY_STALE` diagnostics. This |
| 22 | prevents four old candles from disabling hundreds of usable closes. If neither |
| 23 | source can support overall readiness, available NSE evidence remains visible |
| 24 | without fabricating sufficient history. Missing NSE history uses close fallback. |
| 25 | No extension of NSE history with another provider's older closes, and no mixed |
| 26 | high/low/close/volume candles. A selected short NSE series can compute ATR at 15 |
| 27 | candles even though overall technical scoring still requires 20. |
| 28 | |
| 29 | Within NSE, latest known retrieval wins a date correction. Identical ties |
| 30 | collapse; conflicting same-retrieval OHLCV ties fail that date. A conflicting |
| 31 | latest date blocks current features and fallback. Older conflicts are excluded |
| 32 | from closes with diagnostics; OHLC warmup restarts after them. Other providers, |
| 33 | wrong currencies, future retrievals/dates and DEMO rows cannot contaminate NSE. |
| 34 | Provider symbols are not identity. Missing candle close is treated as a conflict, |
| 35 | not substituted with another provider's price. |
| 36 | |
| 37 | Provenance: `DAILY_MARKET_BAR_NSE` or `CLOSE_ONLY_FALLBACK`; conflict diagnostics |
| 38 | include `MIXED_NOT_ALLOWED`. Daily-bar count is reported even when fallback is |
| 39 | selected. Source URLs/cookies are not added to the technical output. |
| 40 | |
| 41 | ## Formula and readiness contracts |
| 42 | |
| 43 | ATR14 uses actual prior chronological candle close, ignoring provider-supplied |
| 44 | previousClose. TR is max(high-low, abs(high-prior close), abs(low-prior close)). |
| 45 | The first candle supplies dependencies only, not an invented TR. Seed ATR with |
| 46 | the mean of 14 TR values (15 complete candles); subsequent ATR is |
| 47 | (previous ATR * 13 + current TR) / 14. ATR percent is 100*ATR/latest close. |
| 48 | Zero volatility is zero, not null. Missing OHLC restarts warmup rather than |
| 49 | compressing out a missing candle and bridging it. |
| 50 | |
| 51 | ADX14: upMove=current high-prior high; downMove=prior low-current low. Positive |
| 52 | DM is upMove only when positive and strictly greater than downMove; negative DM |
| 53 | is analogous. Ties give both zero. Wilder-smooth TR/+DM/-DM over 14 changes; |
| 54 | DI = 100*smoothed DM/smoothed TR. DX = 100*abs(+DI - -DI)/(+DI + -DI). |
| 55 | Zero denominators yield zero DX. Seed ADX with 14 DX values, then Wilder-smooth. |
| 56 | Minimum 28 complete chronological candles. Output is bounded 0–100. Flat markets |
| 57 | yield ADX=0 after warmup. No ADX approximation from closes. |
| 58 | |
| 59 | Inputs are Decimal-validated; ATR/ADX use the engine's established float numeric |
| 60 | convention and round only at output. Independent tests use hand calculations |
| 61 | and closed-form Decimal weighted sums rather than duplicating the recursion. |
| 62 | Formula references: [Fidelity ATR](https://www.fidelity.com/learning-center/trading-investing/technical-analysis/technical-indicator-guide/atr) |
| 63 | and [Fidelity DMI](https://www.fidelity.com/learning-center/trading-investing/technical-analysis/technical-indicator-guide/DMI). |
| 64 | |
| 65 | Readiness tiers remain <20, 20–49, 50–99, 100–199 and >=200. Feature readiness |
| 66 | separately reports RSI14 (15 closes), ATR14 (15 candles), ADX14 (28 candles), |
| 67 | MA20/50/100/200, breakout (prior 20 + current), VOLUME20 and VOLUME_CONFIRMATION. |
| 68 | Diagnostics distinguish missing OHLC/volume, insufficient history, conflicts, |
| 69 | zero volume baseline and staleness. The existing seven-day technical price-age |
| 70 | policy remains; acquisition freshness settings are not technical-score settings. |
| 71 | |
| 72 | ## Volume and scoring |
| 73 | |
| 74 | Current volume retains its actual BIGINT integer. Average20 uses the prior 20 |
| 75 | selected candle observations, excluding current; ratio therefore requires 21 |
| 76 | candles. Sum/average/ratio normalization uses Decimal before float output, with |
| 77 | no turnover-based inference. Missing prior volume makes the baseline unavailable; |
| 78 | missing current volume leaves ratio unavailable. Current zero gives ratio zero |
| 79 | against a positive baseline. All-zero baseline leaves average=0 and ratio=null |
| 80 | with ZERO_BASELINE diagnostics. Conflicted dates cannot be compressed out of the |
| 81 | 21-observation volume window. |
| 82 | |
| 83 | Expansion: ratio >=1.5 (existing confirmation threshold). Contraction: ratio |
| 84 | <=0.75 (explicit TechnicalConfig threshold). Otherwise normal. Thresholds are |
| 85 | validated to bracket one. Confirmation is true/false only when an existing |
| 86 | price breakout/reversal signal and a valid volume ratio are available; missing |
| 87 | volume stays null. Reversal confirmation is diagnostic only. Without a price |
| 88 | signal, confirmation is NOT_APPLICABLE. Legacy explicit PersistedVolumeObservation |
| 89 | input remains supported for compatibility; Stage-B fallback does not supply it. |
| 90 | |
| 91 | Existing close-based state rules, MA/RSI/MACD/returns/slopes, close-based rolling |
| 92 | support/resistance/extrema and their ordering remain unchanged. Score stays |
| 93 | 0–100: trend alignment/momentum/price position weights 50/30/20, renormalized over |
| 94 | available components; overextension penalty 15; existing confirmed-breakout |
| 95 | bonus 5. ATR is volatility context and ADX is non-directional strength: neither |
| 96 | adds directional score. No reversal bonus or weak-volume penalty is introduced. |
| 97 | Core confidence remains unchanged; separate ohlcvFeatureCoverage reports ATR, |
| 98 | ADX and volume-ratio availability without penalizing close-only fallback. |
| 99 | |
| 100 | Audited state precedence (unchanged): OVEREXTENDED when DMA20 distance >=10% |
| 101 | and RSI >=70; otherwise BREAKOUT above the prior-20-close resistance by >1%; |
| 102 | otherwise PULLBACK_IN_UPTREND when broad trend is up, price >DMA50, retreat from |
| 103 | the prior five-close maximum is >=2%, and DMA20 or DMA50 proximity is <=3%; |
| 104 | otherwise REVERSAL_CANDIDATE when slope50 <-0.02%, slope20 >0.02% and price |
| 105 | >DMA20; otherwise UPTREND on broad-up and price >DMA50; otherwise DOWNTREND on |
| 106 | price <DMA50 and slope50 <-0.02%; otherwise BASE_BUILDING on absolute slope20 |
| 107 | <=0.02% and the latest 20-close range <=5%; otherwise RANGE_BOUND. Broad-up |
| 108 | requires positive slope50 >0.02% and, when DMA200 exists, DMA50 >DMA200. |
| 109 | Unavailable/stale history keeps INSUFFICIENT_DATA. Price breakdown below the |
| 110 | prior-20 support by >1% remains separate breakout-state evidence. |
| 111 | |
| 112 | A deterministic Stage-B engineering test with identical closes changes technical |
| 113 | and Stage-B scores from 84 to 89 only after actual expansion volume confirms an |
| 114 | existing price breakout; preScore and the fallback candidate remain unchanged. |
| 115 | This synthetic comparison is not an investment conclusion. |
| 116 | |
| 117 | ## Persistence reads and runtime evidence |
| 118 | |
| 119 | At default batch size 250, no benchmark references: empty candidates = 0 queries; |
| 120 | 1 candidate = 1 NSE daily read + 1 close read; 18 candidates = the same 2 reads. |
| 121 | Larger sets retain bounded batching. Benchmarks participate only in close batches. |
| 122 | Tests count SQLite statements, check deterministic grouping, forbid provider |
| 123 | acquisition and verify Phase-1 objects remain unchanged. |
| 124 | |
| 125 | Persisted-only smoke on 2026-09-13 used the prior local backfill SQLite database |
| 126 | (opened read-only) and a read-only snapshot of existing local PostgreSQL closes. |
| 127 | NILKAMAL and POLYCAB each have four persisted NSE candles, September 1–4: not |
| 128 | enough for ATR14, ADX14 or volume20. Candle-only results correctly remain null. |
| 129 | Their longer current close histories are retained through explicit fallback: |
| 130 | |
| 131 | | Instrument | NSE bars | Selected closes | Technical score before/after | State | |
| 132 | |---|---:|---:|---|---| |
| 133 | | NILKAMAL | 4 | 272 | 85.67599167 / 85.67599167 | PULLBACK_IN_UPTREND | |
| 134 | | POLYCAB | 4 | 341 | 25.07289151 / 25.07289151 | DOWNTREND | |
| 135 | | PERSISTENT | 0 | 341 | 52.19912039 / 52.19912039 | UPTREND | |
| 136 | |
| 137 | All repeated outputs, including reversed input order, were identical. Provider |
| 138 | calls were zero; network connection creation was forbidden during computation. |
| 139 | Numerical ATR/ADX/volume validation uses independent sufficient-history tests, |
| 140 | not invented runtime candles. No live NSE request or new history persistence. |
| 141 | Runtime artifacts are under ignored `.tmp/`. |
| 142 | |
| 143 | The local deployed PostgreSQL schema currently lacks the daily-bar table, so |
| 144 | this is not deployed PostgreSQL application validation. Richer live-data sample |
| 145 | conclusions require persisted backfill; this phase does not acquire it. |
| 146 | |
| 147 | Remaining scope: deployed PostgreSQL validation, sector benchmark mapping/history, |
| 148 | broad-market benchmark history, and recommendation/ranker/prediction phases. |