Ensemble Model

Ensemble models are fundamental to modern machine learning and particularly valuable in trading because of their robustness against overfitting and noisy signals.

The core insight: individual models make errors. Different models make different errors. If their errors are uncorrelated (or partially so), averaging or combining their predictions cancels out individual errors and leaves the consistent signal. Ensembles are mathematically guaranteed to have variance no greater than the average base-model variance (and often substantially lower).

**Bagging (Bootstrap Aggregating):** - Train K base models, each on a different bootstrap sample of the training data - For prediction: average (regression) or majority vote (classification) of base-model predictions - Reduces variance from data sampling; doesn't address bias - Example: Random Forest = bagging of decision trees, with additional feature-bagging at each tree split

**Boosting:** - Train base models sequentially; each new model focuses on the errors of previous models - Final prediction: weighted sum of base-model predictions (weights derived during training) - Reduces both variance and bias - Examples: AdaBoost, Gradient Boosting (XGBoost, LightGBM, CatBoost), the dominant approach in tabular ML today

**Stacking:** - Train K base models on the training data - Generate predictions on held-out data - Train a meta-model to combine base-model predictions optimally - More flexible than fixed averaging; can capture model-specific strengths - Risk: more parameters means more overfitting potential; requires careful cross-validation

**Voting / Averaging:** - Simplest ensembles: equal-weight average (regression) or majority vote (classification) - Effective when base models are diverse and roughly equally accurate - Used as a baseline against more sophisticated combination methods

**Application patterns in trading:**

1. **Cross-algorithm ensembling**: train Random Forest, XGBoost, and Neural Network on the same problem. Average predictions or use stacking meta-model. Each algorithm captures different aspects of the data structure.

2. **Cross-feature ensembling**: train models on different feature subsets. One model uses technical indicators, another uses microstructure features, another uses regime classifications. Combine predictions to capture signal across feature classes.

3. **Cross-timeframe ensembling**: train separate models on different timeframe features (M5, H1, D1). The H1 model captures intra-day patterns; the D1 model captures regime context. Combine for trades that have multi-timeframe support.

4. **Cross-instrument ensembling**: train models for each major pair (EURUSD, GBPUSD, USDJPY) separately. For a portfolio strategy, the ensemble of per-instrument models produces both per-instrument signals and a portfolio-level confidence metric.

Validation challenges: - Computational cost: K base models take K× training time. For neural networks or large ML models, this becomes substantial. - Diversity requirement: ensembles work only if base models are diverse. K copies of the same model give no improvement. Ensure variation via different algorithms, features, hyperparameters, or training data. - Overfitting at the meta-level: stacking ensembles add a meta-model that can itself overfit. Use cross-validation when training the meta-model. - Interpretability decreases: ensembles are harder to explain than single models. For commercial EAs where vendors must explain logic, this can be a constraint.

For trading EAs: - Simple ensembles (averaging) are easy to implement in MQL5 (run multiple sub-strategies, average their signals) - Sophisticated ensembles (boosting, stacking) typically require external Python services running the ML - Inference latency: ensemble inference is K× single-model latency. For HF strategies, ensemble size must be balanced against latency budget.

In FxRobotEasy: our AI flagship EAs use multi-timeframe ensembles where signals from different timeframes (M5, M15, H1) must align for trade entry. This is a simple form of ensembling that increases signal quality at the cost of trade frequency.