Fixing Gold Market Overfitting: A Predictive Machine Studying Method with ONNX and Gradient Boosting
Case Research: The “Golden Gauss” Structure
Creator: Daglox Kankwanda
ORCID: 0009-0000-8306-0938
Technical Paper: Zenodo Repository (DOI: 10.5281/zenodo.18646499)
Contents
- Introduction
- The Core Issues in Algorithmic Buying and selling
- Methodology
- System Structure
- Characteristic Engineering
- Validation and Outcomes
- Commerce Administration
- Sincere Limitations
- Conclusion
- Implementation & Availability
- References
1. Introduction
The algorithmic buying and selling house, notably in retail markets, faces a elementary credibility drawback. The sample is predictable and pervasive: programs exhibit spectacular backtest efficiency, adopted by speedy degradation in ahead testing, culminating in account destruction throughout dwell deployment. This failure mode stems from a single root trigger—optimization for in-sample efficiency with out rigorous out-of-sample validation.
The mathematical actuality is simple: given ample levels of freedom, any mannequin can “memorize” historic worth patterns. Such memorization produces spectacular backtest metrics whereas offering zero predictive energy for future market habits. The mannequin has realized the noise, not the sign.
Past overfitting, conventional indicator-based approaches undergo from a elementary timing deficiency. Technical indicators, by building, are reactive—they course of historic knowledge to generate alerts after worth actions have already begun.
Core Thesis: A very helpful buying and selling system should determine the situations previous vital worth exercise, not the exercise itself. The purpose is prediction, not affirmation.
This text presents a technique that synthesizes machine studying analysis insights right into a sensible, deployable buying and selling system for XAUUSD (Gold) markets, demonstrated by means of the “Golden Gauss” structure.
2. The Core Issues in Algorithmic Buying and selling
2.1 The Overfitting Disaster
The proliferation of “AI-powered” buying and selling programs in retail markets has created a credibility disaster, with most programs exhibiting catastrophic failure when deployed on unseen knowledge on account of extreme overfitting.
Determine 1: Conceptual illustration of the standard Knowledgeable Advisor lifecycle. Fashions optimized for historic efficiency steadily fail catastrophically when deployed on unseen market situations.
2.2 The Latency Drawback in Technical Evaluation
Technical indicators are inherently reactive:
- By the point RSI crosses the overbought threshold, the worth has already moved considerably
- By the point a MACD crossover confirms, the optimum entry window has handed
- By the point a breakout is “confirmed,” stop-loss necessities have expanded considerably
Determine 2: Comparability of timing between reactive technical indicators and predictive machine studying approaches. Conventional indicators verify strikes after optimum entry has handed, whereas predictive programs determine setup situations earlier than execution.
2.3 Literature Context
The applying of machine studying to monetary time-series prediction has advanced considerably. A number of constant findings are related:
| Discovering | Implication |
|---|---|
| Gradient Boosting Dominance on Tabular Information | Regardless of advertising attraction of “deep studying,” ensemble strategies constantly outperform neural networks on structured monetary knowledge |
| Characteristic Engineering Criticality | High quality of engineered options sometimes determines mannequin success greater than architectural decisions |
| Temporal Validation Necessities | Normal cross-validation that shuffles knowledge is inappropriate for monetary time-series on account of lookahead bias |
| Cross-Asset Info | Monetary devices don’t commerce in isolation; correlated devices present priceless context |
3. Methodology
3.1 The Predictive Labeling Methodology
Normal approaches to coaching buying and selling fashions label knowledge on the level the place worth motion happens. This creates a elementary drawback: if the mannequin learns options calculated from the identical bars which can be labeled, it successfully learns to acknowledge strikes which can be already occurring fairly than strikes which can be about to occur.
The Golden Gauss structure employs a technique that maintains temporal separation between characteristic calculation and label placement:
- The labeling course of identifies worthwhile zones the place worth moved considerably in a selected path
- All options are calculated from market knowledge that occurred earlier than the labeled zone begins
Determine 3: Guide labeling interface displaying XAUUSD worth motion with recognized directional zones. The labeled BUY and SELL areas characterize worthwhile strikes used as coaching targets; the mannequin learns to foretell these strikes utilizing options calculated from previous market knowledge.
Implications: This temporal separation ensures the mannequin learns to acknowledge preconditions—the market microstructure patterns that precede vital strikes—fairly than traits of the strikes themselves.
3.2 High quality-Filtered Coaching Labels
Not all worth actions are significant or tradeable. Many are:
- Too small to beat transaction prices (unfold + fee)
- Too erratic to execute cleanly
- A part of bigger consolidation patterns with out directional follow-through
The labeling course of applies strict filtering standards, figuring out solely zones the place worth moved with ample magnitude and directional consistency. This ensures the mannequin learns solely from setups that exceeded minimal profitability thresholds.
3.3 Twin-Mannequin Directional Structure
Market dynamics exhibit elementary asymmetry between bullish and bearish habits:
- Accumulation patterns differ structurally from distribution patterns
- Worry-driven promoting sometimes executes quicker than greed-driven shopping for
- Assist habits differs from resistance habits
- Quantity traits differ between advances and declines
To respect this asymmetry, the structure employs two unbiased binary fashions:
| Mannequin | Output | Coaching Information |
|---|---|---|
| BUY Mannequin | P(Bullish Transfer Imminent) | Skilled solely on bullish labels |
| SELL Mannequin | P(Bearish Transfer Imminent) | Skilled solely on bearish labels |
Every mannequin is a binary classifier detecting solely its respective directional setup. This prevents the confusion that happens when a single mannequin makes an attempt to study contradictory patterns concurrently.
3.4 Stroll-Ahead Validation Protocol
Normal machine studying cross-validation, which shuffles knowledge randomly, is inappropriate for monetary time-series on account of temporal dependencies and lookahead bias dangers.
The system makes use of strict walk-forward validation with full chronological separation:
- Coaching knowledge extends by means of December 31, 2024
- All architectural choices, hyperparameters, and have engineering decisions had been finalized utilizing solely this knowledge
- The mannequin was then frozen and validated on a 13-month out-of-sample interval (January 2025 by means of January 2026)
Determine 4: Temporal knowledge separation for walk-forward validation. Coaching knowledge extends by means of finish of 2024; all 2025-2026 analysis represents strictly out-of-sample efficiency on knowledge not used for coaching.
Essential Guidelines:
- No shuffling of time-series knowledge
- Analysis interval evaluation solely in any case mannequin choices finalized
- No iterative “peeking” at analysis outcomes to regulate parameters
4. System Structure
The system contains two distinct however built-in elements:
- Coaching Pipeline — applied in Python for mannequin growth and validation
- Execution Engine — applied in MQL5 for real-time deployment inside MetaTrader 5
Determine 5: Excessive-level structure of the system. The coaching pipeline (prime) processes historic knowledge by means of characteristic engineering and mannequin coaching, exporting by way of ONNX. The execution engine (backside) calculates options instantaneously, obtains likelihood scores, and applies commerce administration logic for place execution.
4.1 Mannequin Structure Choice
The selection of mannequin structure was pushed by empirical analysis in opposition to standards particular to monetary time-series prediction:
| Criterion | Precedence |
|---|---|
| Efficiency on structured/tabular knowledge | Essential |
| Robustness to noise and outliers | Essential |
| Dealing with of regime adjustments | Excessive |
| Coaching knowledge effectivity | Excessive |
| Inference velocity for dwell deployment | Excessive |
| Interpretability (characteristic significance) | Medium |
Primarily based on intensive testing, Gradient Boosting Resolution Timber (GBDT) had been chosen. This selection aligns with constant findings within the machine studying literature that GBDT architectures outperform deep studying approaches on structured monetary knowledge.
Why Not Neural Networks?
Whereas “Neural Community” generates advertising attraction, the technical actuality for tabular monetary knowledge:
- GBDTs deal with characteristic interactions naturally with out express specification
- GBDTs are extra strong to noise and outliers in monetary knowledge
- GBDTs require considerably much less coaching knowledge
- GBDTs present interpretable characteristic significance rankings
- GBDTs practice quicker, enabling extra intensive hyperparameter search
4.2 ONNX Deployment
The mannequin is exported by way of ONNX (Open Neural Community Change) for platform-agnostic deployment, enabling Python-trained fashions to execute at C++ speeds inside MT5.
A vital requirement is training-serving parity: characteristic calculations in MQL5 have to be mathematically similar to these carried out throughout Python coaching. Any discrepancy creates “training-serving skew” that degrades mannequin efficiency.
4.3 The MQL5-ONNX Interface
The bridge between Python coaching and MQL5 execution depends on the native ONNX API launched in MetaTrader 5 Construct 3600. The first engineering problem is guaranteeing the enter tensor form matches the Python export precisely, and accurately decoding the classifier’s dual-output construction.
Under is the structural logic used to initialize and run inference with the Gradient Boosting mannequin throughout the Knowledgeable Advisor:
Mannequin Initialization
#useful resource "InformationBULLISH_Model.onnx" as uchar ExtModelBuy() lengthy g_onnx_buy; const int SNIPER_FEATURES = 239; bool InitializeONNXModels() { Print("Loading ONNX fashions..."); g_onnx_buy = OnnxCreateFromBuffer(ExtModelBuy, ONNX_DEFAULT); if(g_onnx_buy == INVALID_HANDLE) { Print("(FAIL) Didn't load BUY mannequin"); return false; } ulong input_shape_buy() = {1, SNIPER_FEATURES}; if(!OnnxSetInputShape(g_onnx_buy, 0, input_shape_buy)) { Print("(FAIL) Didn't set BUY mannequin enter form"); return false; } Print(" (OK) BUY mannequin loaded efficiently"); return true; }
Chance Inference
The classifier outputs two tensors: predicted labels and sophistication possibilities. For probability-based execution, we extract the likelihood of the goal class:
bool GetBuyPrediction(const float &options(), double &likelihood) { likelihood = 0.0; if(g_onnx_buy == INVALID_HANDLE) { Print("(FAIL) BUY mannequin not loaded"); return false; } float input_data(); ArrayResize(input_data, SNIPER_FEATURES); ArrayCopy(input_data, options); lengthy output_labels(); float output_probs(); ArrayResize(output_labels, 1); ArrayResize(output_probs, 2); ArrayInitialize(output_labels, 0); ArrayInitialize(output_probs, 0.0f); if(!OnnxRun(g_onnx_buy, ONNX_NO_CONVERSION, input_data, output_labels, output_probs)) { int error = GetLastError(); Print("(FAIL) BUY ONNX inference failed: ", error); return false; } likelihood = (double)output_probs(0); return true; }
Key Implementation Particulars:
- Twin-Output Construction: Gradient Boosting classifiers exported by way of ONNX produce two outputs—the expected label and the likelihood distribution throughout lessons. The likelihood output is used for threshold-based execution.
- Class Mapping: Class 0 represents the goal situation (BULLISH for the BUY mannequin). The likelihood output_probs(0) straight signifies mannequin confidence in an imminent bullish transfer.
- Form Validation: Strict form checking at initialization catches training-serving mismatches instantly fairly than producing silent prediction errors throughout dwell buying and selling.
4.4 Execution Configuration
| Parameter | Worth |
|---|---|
| Image | XAUUSD solely |
| Timeframe | M1 (characteristic calculation) |
| Energetic Hours | 14:00–18:00 (dealer time, configurable) |
| Chance Threshold | 88% |
| Cease Loss | Fastened preliminary; dynamically managed |
| Take Revenue | Goal-based with ratchet safety |
| Prohibited Methods | No grid, no martingale |
5. Characteristic Engineering
The system processes 239 engineered options throughout a number of research-backed domains. These options had been developed by means of educational literature evaluate, area experience in market microstructure, and iterative empirical testing with strict validation protocols.
5.1 Characteristic Classes Overview
| Class | Conceptual Focus |
|---|---|
| Volatility Regime | Market state classification, tradeable vs. non-tradeable situations |
| Momentum | Multi-scale fee of change, pattern persistence |
| Quantity Dynamics | Participation ranges, uncommon exercise detection |
| Value Construction | Assist/resistance proximity, vary place |
| Cross-Asset | Correlated instrument alerts, correlation regime shifts |
| Microstructure | Directional stress and short-horizon stress proxies |
| Temporal | Session timing, cyclical patterns |
| Sequential | Sample recognition, run-length evaluation |
5.2 Key Driving Options
The next options constantly ranked among the many most influential in response to international SHAP significance evaluation:
- ADX Pattern Power (14-period): Measuring pattern power, unbiased of path
- VWAP Volatility Deviation: Distance of worth from intraday VWAP, normalized by latest volatility
- Volatility Regime Classifier: ATR relative to its shifting common, indicating low-, normal-, or high-volatility states
- MACD Histogram Momentum: Capturing short-term momentum and potential reversals
- 60-minute Gold/DXY Rolling Correlation: Rolling correlation between XAUUSD and DXY returns
- 60-minute Gold/USDJPY Rolling Correlation: Rolling correlation between XAUUSD and USDJPY returns
- Directional Volatility Regime: Signed volatility characteristic combining EMA-based pattern power with present ATR regime
- Order-Movement Persistence: Proxy for a way lengthy directional strikes persist throughout latest candles
- EMA Unfold Dynamics: Distances and slopes between quick and gradual EMAs
The presence of well-known indicators (ADX, MACD) alongside proprietary regime and correlation options demonstrates that the mannequin enhances, fairly than replaces, established market relationships with higher-resolution timing alerts.
5.3 Cross-Asset Intelligence
Gold (XAUUSD) doesn’t commerce in isolation. Its worth motion is influenced by:
- US Greenback Dynamics: Usually inverse correlation; greenback power usually pressures gold costs
- Protected-Haven Flows: Correlation with different safe-haven belongings throughout risk-off durations
- Yield Expectations: Relationship with actual rate of interest proxies
The characteristic set incorporates lagged returns from correlated devices, rolling correlations at a number of time scales, divergence detection, and regime change alerts.
6. Validation and Outcomes
The validation strategy follows a single precept: exhibit generalization, not memorization. Any mannequin can obtain spectacular outcomes on knowledge it has seen. The one significant analysis is efficiency on strictly unseen knowledge.
6.1 Out-of-Pattern Efficiency
All 2025 efficiency represents true out-of-sample (OOS) outcomes. The mannequin structure, hyperparameters, and have set had been frozen earlier than any 2025 knowledge was evaluated.
Determine 6: Backtest fairness and stability curves from Jan 2021 to Jan 2026. The interval Jan 2021–Dec 2024 represents knowledge included in mannequin coaching; the interval Jan 2025–Jan 2026 constitutes strictly out-of-sample analysis.
| Metric | Full Interval (Jan 2021– Jan 2026) | OOS Solely (Jan 2025–Jan 2026) |
|---|---|---|
| Win Fee | 88.71% | 83.67% |
| Whole Trades | 1,030 | 319 |
| Revenue Issue | 1.77 | 1.50 |
| Sharpe Ratio | 9.90 | 13.9 |
| Max Drawdown (0.01 lot) | ~$500 | ~$313 |
| Restoration Issue | 11.57 | 3.66 |
| Avg Holding Time | 30 min 30 sec | 30 min 30 sec |
Interpretation: The out-of-sample interval demonstrates continued profitability with metrics that degrade gracefully from the coaching interval:
- Win fee decreases from 88.71% to 83.67%—a managed 5% discount indicating the mannequin generalizes fairly than memorizes
- Revenue issue stays above 1.50, confirming constructive expectancy on unseen knowledge
- The upper OOS Sharpe ratio (13.9 vs 9.90) gives robust proof in opposition to overfitting
This efficiency hole is anticipated and wholesome. The managed degradation confirms real sample generalization.
6.2 Chance Threshold Evaluation
The mannequin outputs steady likelihood scores. Evaluation reveals the connection between likelihood ranges and commerce outcomes:
| Chance Vary | Trades | Win Fee |
|---|---|---|
| 0.880 – 0.897 | 231 | 88.3% |
| 0.897 – 0.923 | 167 | 90.4% |
| 0.923 – 0.950 | 190 | 93.2% |
| 0.950 – 0.976 | 107 | 87.9% |
| 0.976 – 0.993 | 27 | 96.3% |
Why 88% Minimal Threshold? The 88% threshold was decided by means of systematic analysis because the optimum entry level balancing commerce frequency in opposition to high quality. Under this threshold, false-positive charges improve considerably.
6.3 Exit Composition Evaluation
| Exit Sort | Proportion | Interpretation |
|---|---|---|
| Ratchet Revenue (SL_WIN) | 87.1% | Dynamic revenue seize |
| Take Revenue (TP) | 3.2% | Full goal reached |
| Cease Loss (SL_LOSS) | 9.7% | Managed losses |
The overwhelming majority of successful trades exit by way of the ratchet system, capturing earnings dynamically fairly than ready for full TP.
6.4 Temporal Consistency
| 12 months | Trades | Win Fee | Standing |
|---|---|---|---|
| 2021 | 172 | 93.6% | Coaching |
| 2022 | 125 | 93.6% | Coaching |
| 2023 | 64 | 87.5% | Coaching |
| 2024 | 124 | 93.5% | Coaching |
| 2025 | 237 | 85.2% | Out-of-Pattern |
| 2026 | — | — | — |
All years worthwhile with constant efficiency patterns throughout coaching and out-of-sample durations.
7. Commerce Administration
The system implements a complete commerce administration layer that extends past easy entry execution.
7.1 Chance-Primarily based Resolution Making
In contrast to programs that generate discrete “purchase” or “promote” alerts, the structure calculates likelihood scores instantaneously on every new bar:
- Entry Resolution: Chance should exceed 88% threshold earlier than place opening
- Route Choice: Larger likelihood between BUY and SELL fashions determines path
- Exit Timing: Chance adjustments inform place closure choices
- Maintain/Shut Logic: Steady likelihood monitoring throughout open positions
7.2 Entry Validation and Filtering
- Twin-Mannequin Affirmation: Each BUY and SELL mannequin possibilities are assessed to verify directional bias and filter ambiguous situations
- Regime Filtering: Further filters detect unfavorable market regimes (excessive volatility occasions, low liquidity durations)
- Conditional Execution: Commerce execution proceeds solely after likelihood thresholds are happy and regime filters verify favorable situations
7.3 Ratchet Revenue Safety
Drawback Addressed: Value could transfer 80% towards the take-profit degree, then reverse—with out lively administration, this unrealized revenue can be misplaced.
Ratchet Answer: As worth strikes favorably, the system progressively locks in revenue by tightening exit situations, guaranteeing that vital favorable strikes are captured even when the total take-profit will not be reached.
7.4 Ratchet Loss Minimization
Drawback Addressed: Even high-confidence predictions often fail; ready for the mounted stop-loss leads to most loss on each shedding commerce.
Ratchet Answer: When worth strikes adversely, the system actively manages the exit to attenuate loss fairly than passively ready for stop-loss execution, lowering common loss per unsuccessful commerce.
8. Sincere Limitations
8.1 What This System Is NOT
- Not infallible: Roughly 15–18% of alerts end in suboptimal entries relying on market situations
- Not common: Skilled solely for XAUUSD with its particular market microstructure and session dynamics
- Not static: Periodic retraining (3–6 months) is required as markets evolve
- Not assured: Out-of-sample validation demonstrates methodology soundness however doesn’t assure future efficiency
8.2 Recognized Threat Components
| Threat | Description | Mitigation |
|---|---|---|
| Regime Change | Market construction evolves by means of coverage shifts and geopolitical occasions | Periodic retraining protocol |
| Execution Threat | Slippage throughout volatility can degrade realized outcomes | Session-aware execution, lively hours restriction |
| Edge Decay | Predictive edges face decay as markets evolve | Retraining with methodology preservation |
| Focus | Unique XAUUSD focus gives no diversification | Consumer duty for portfolio allocation |
8.3 Execution Assumptions
All reported outcomes are based mostly on historic simulations. No further slippage mannequin has been utilized, and real-world execution could result in materially totally different efficiency. These statistics needs to be interpreted as estimates underneath very best execution situations.
9. Conclusion
This text offered a technique for fixing two elementary failures that characterize retail algorithmic buying and selling—overfitting to historic noise and reactive sign technology—by means of rigorous machine studying practices.
The core improvements demonstrated within the Golden Gauss structure embody:
- Predictive labeling that permits real anticipation of worth strikes
- Twin-model directional specialization that respects market asymmetry
- Chance-driven execution that quantifies confidence earlier than commerce entry
- Clever commerce administration that minimizes losses when predictions show suboptimal
On strictly out-of-sample 2025 knowledge—collected in any case mannequin choices had been finalized—the system demonstrates roughly 83.67% directional accuracy on the 88% likelihood threshold. The managed efficiency differential from coaching metrics signifies real sample studying fairly than memorization.
Key Takeaways for Practitioners
- By no means shuffle time-series knowledge throughout validation—this creates lookahead bias and knowledge leakage
- Out-of-sample efficiency is the one significant metric for evaluating dwell buying and selling potential
- Chance thresholds allow accuracy/frequency tradeoffs—increased thresholds yield fewer however higher-quality alerts
- Twin binary fashions respect the asymmetry between bullish and bearish market dynamics
- Commerce administration amplifies edge—ratchet mechanisms maximize wins and reduce losses
- All programs have limitations—trustworthy acknowledgment permits applicable deployment and threat administration
The retail algorithmic buying and selling business suffers from systematic misalignment between vendor incentives and person outcomes. The methodology offered right here—strict temporal separation, documented efficiency degradation, bounded confidence claims—gives a template for trustworthy system analysis that prioritizes sustainable operation over advertising attraction.
Knowledgeable critique of the validation methodology and underlying assumptions is welcomed. Progress in algorithmic buying and selling requires programs designed to outlive scrutiny fairly than keep away from it.
10. Implementation & Availability
The structure described on this paper—particularly the predictive labeling engine and the ONNX likelihood inference—has been totally applied within the Golden Gauss AI system.
To assist additional analysis and validation, the entire system is on the market for testing within the MQL5 Market. The package deal contains the “Visualizer” mode, which renders the likelihood cones and “Kill Zones” straight on the chart, permitting merchants to look at the mannequin’s decision-making course of in real-time.
Threat Disclaimer: Buying and selling foreign exchange and CFDs entails substantial threat of loss and isn’t appropriate for all traders. Previous efficiency, whether or not in backtesting or dwell buying and selling, doesn’t assure future outcomes. The validation outcomes offered characterize historic evaluation underneath particular market situations that won’t persist. Merchants ought to solely use capital they will afford to lose and may take into account their monetary scenario earlier than buying and selling.
References
- Cao, L. J. and Tay, F. E. H. (2001). Monetary forecasting utilizing assist vector machines. Neural Computing & Functions, 10(2), 184-192.
- Chen, T. and Guestrin, C. (2016). XGBoost: A scalable tree boosting system. Proceedings of the twenty second ACM SIGKDD Worldwide Convention on Information Discovery and Information Mining, 785-794.
- López de Prado, M. (2018). Advances in Monetary Machine Studying. Wiley.
- Bailey, D. H. and López de Prado, M. (2014). The likelihood of backtest overfitting. Journal of Computational Finance, 17(4), 39-69.
- Pardo, R. (2008). The Analysis and Optimization of Buying and selling Methods (2nd ed.). Wiley.
- Krauss, C., Do, X. A., and Huck, N. (2017). Deep neural networks, gradient-boosted timber, random forests: Statistical arbitrage on the S&P 500. European Journal of Operational Analysis, 259(2), 689-702.
- Baur, D. G. and McDermott, T. Okay. (2010). Is gold a secure haven? Worldwide proof. Journal of Banking & Finance, 34(8), 1886-1898.
- ONNX Runtime Builders (2021). ONNX Runtime: Excessive efficiency inference and coaching accelerator. Accessible: https://onnxruntime.ai/