Ensemble Model
Definition
An ensemble model combines predictions from multiple base models to produce a stronger overall prediction. Common patterns: bagging (random forest), boosting (XGBoost), stacking (meta-model combining base-model outputs). In trading, ensembles reduce overfitting risk by averaging out individual model variances and capture different aspects of the same signal.
In-depth: Ensemble Model
Ensemble models are fundamental to modern machine learning and particularly valuable in trading because of their robustness against overfitting and noisy signals.
The core insight: individual models make errors. Different models make different errors. If their errors are uncorrelated (or partially so), averaging or combining their predictions cancels out individual errors and leaves the consistent signal. Ensembles are mathematically guaranteed to have variance no greater than the average base-model variance (and often substantially lower).
**Bagging (Bootstrap Aggregating):** - Train K base models, each on a different bootstrap sample of the training data - For prediction: average (regression) or majority vote (classification) of base-model predictions - Reduces variance from data sampling; doesn't address bias - Example: Random Forest = bagging of decision trees, with additional feature-bagging at each tree split