XGBoost for Return Direction: Class Imbalance and Decision Thresholds

Train a classifier to predict whether the next hour's return exceeds +0.5%, and you have not built a balanced binary problem. During quiet crypto regimes 70--80% of bars fall below the threshold, so the model can score 75% accuracy by never predicting a single positive. Accuracy is useless here. So is the default 0.5 decision boundary, which nobody chose deliberately — it is the value predict() happens to use.

There are three standard responses: reweight the loss with scale_pos_weight, replace the loss with focal loss, or leave the loss alone and move the decision threshold after the fact. They are usually presented as interchangeable. They are not. They produce different precision/recall trade-offs and, more importantly, different post-cost PnL, because trading cares about precision far more than recall — a missed trade costs nothing, a bad one costs spread plus fees.

This article measures all three on the same purged walk-forward folds of hourly BTC and ETH data, then covers two things that follow from it: why min_child_weight is a Hessian threshold rather than a row count (which is why the "right" value shifts when you reweight the loss), and where XGBoost, LightGBM, and CatBoost actually differ.

Same folds, same features, same cost model for every row. Sharpe is deflated for the number of configurations tried, per deflated Sharpe and multiple testing; costs follow slippage and cost models.

Handling Class Imbalance in Return Prediction

Approach 1: scale_pos_weight

The simplest method. Set scale_pos_weight = n_negative / n_positive:

n_pos = y_train.sum()
n_neg = len(y_train) - n_pos
scale_pos_weight = n_neg / n_pos  # e.g., 3.0 if 75% negative

model = xgb.XGBClassifier(
    scale_pos_weight=scale_pos_weight,
    ...
)

This scales the gradient for positive-class samples, telling the model that misclassifying a positive example is $k$ times worse than misclassifying a negative one.

The side effect is easy to miss: it also scales the Hessians. Since min_child_weight is a threshold on the sum of Hessians in a leaf (see below), reweighting the loss silently changes how aggressively the tree splits. A min_child_weight tuned at scale_pos_weight=1 no longer means the same thing at scale_pos_weight=3. Retune the two together, not sequentially.

The second side effect: the output probabilities are no longer calibrated. predict_proba returns something monotone in the true probability but not equal to it, which makes any downstream position sizing based on those numbers wrong.

Approach 2: Focal Loss

Focal loss down-weights easy examples regardless of class and concentrates training on hard, ambiguous samples. That is an appealing framing for return prediction, where the boundary between a +0.5% bar and a +0.4% bar is mostly noise — though "focus on the ambiguous cases" and "focus on the unlearnable cases" are the same instruction when the labels are this noisy, which is the reason to measure rather than assume.

$\text{FL}(p_t) = -\alpha_t (1 - p_t)^{\gamma} \log(p_t)$

where $p_t$ is the predicted probability for the true class, $\alpha_t$ balances class weights, and $\gamma$ (typically 1--3) controls how strongly easy examples are down-weighted.

def focal_loss_objective(y_true, y_pred, gamma=2.0, alpha=0.25):
    """
    Custom focal loss for XGBoost. Returns gradient and hessian.
    """
    p = 1.0 / (1.0 + np.exp(-y_pred))  # sigmoid

    g1 = alpha * y_true * (1 - p)**gamma * (gamma * p * np.log(p + 1e-9) + p - 1)
    g2 = (1 - alpha) * (1 - y_true) * p**gamma * (
        -gamma * (1 - p) * np.log(1 - p + 1e-9) - p
    )
    grad = -(g1 + g2)

    hess = np.maximum(grad * (1 - grad), 1e-6)

    return grad, hess


model = xgb.XGBClassifier(objective=focal_loss_objective, ...)

Custom objectives need a gradient and a positive Hessian for the leaf-weight formula to be a descent step. The exact second derivative of focal loss is not positive everywhere, which is why implementations reach for a surrogate — but a surrogate off by a scale factor changes step sizes, and through the Hessian sum it changes what min_child_weight prunes. Only the comparison table shows whether that costs anything in practice.

Approach 3: Threshold Optimization

Leave the loss alone, train a calibrated model, and move the decision threshold on the validation set:

from sklearn.metrics import precision_recall_curve

def optimize_threshold(y_true, y_proba, min_precision=0.55):
    """
    Find the threshold maximizing F1 subject to a minimum precision.
    """
    precisions, recalls, thresholds = precision_recall_curve(y_true, y_proba)
    f1_scores = 2 * (precisions * recalls) / (precisions + recalls + 1e-9)

    valid = precisions[:-1] >= min_precision
    if not valid.any():
        return 0.5  # fallback

    best_idx = np.argmax(f1_scores[:-1] * valid)
    return thresholds[best_idx]

The precision floor is what matters for trading. Maximizing F1 alone trades precision for recall, which in PnL terms means more marginal trades each paying the spread. Constraining precision and maximizing recall underneath it maps onto "trade less, but be right when you do."

Two rules keep this honest: fit the threshold on validation data that sits chronologically after the training window, and refit it per fold. A threshold chosen once on the whole sample is a look-ahead leak of the kind catalogued in the look-ahead bias taxonomy. Per-fold refitting also gives a free diagnostic — if the optimal threshold jumps around across folds, calibration is unstable and the threshold is fitting noise.

Why min_child_weight Is a Hessian Sum, Not a Row Count

Gradient boosting builds an additive ensemble. At step $t$ it adds a tree $f_t$ minimizing a regularized objective, which XGBoost approximates with a second-order Taylor expansion:

$\mathcal{L}^{(t)} \approx \sum_{i=1}^{n} \Bigl[ g_i f_t(\mathbf{x}_i) + \frac{1}{2} h_i f_t^2(\mathbf{x}_i) \Bigr] + \Omega(f_t)$

with $g_i = \partial l / \partial \hat{y}_i^{(t-1)}$, $h_i = \partial^2 l / \partial (\hat{y}_i^{(t-1)})^2$, and

$\Omega(f) = \gamma T + \frac{1}{2}\lambda \sum_{j=1}^{T} w_j^2$

where $T$ is the leaf count, $w_j$ the leaf weight, $\gamma$ the per-leaf penalty, and $\lambda$ the L2 term. Solving for the optimal leaf weight gives $w_j^* = -\sum_{i \in j} g_i / (\sum_{i \in j} h_i + \lambda)$.

The Hessian sum sits in the denominator, and that is the point. min_child_weight thresholds that sum, not the number of rows in the leaf. For log loss, $h_i = p_i(1 - p_i)$, which is largest at $p = 0.5$ and collapses toward zero as predictions approach certainty. So a leaf full of confidently-classified bars has a tiny Hessian sum and gets pruned, while a leaf holding a handful of genuinely ambiguous bars can survive.

For noisy financial labels this is the behavior you want, and it explains the practical consequences:

• Raising min_child_weight prunes leaves that rest on few uncertain observations — precisely the ones most likely to be fitting noise. This is a sharper instrument than min_child_samples-style row counting.
• Anything that rescales the loss rescales the Hessians. scale_pos_weight, custom objectives, and sample weights all shift the effective min_child_weight even though the number you typed did not change.
• As boosting proceeds and predictions sharpen, Hessians shrink globally, so a fixed min_child_weight prunes more aggressively in later rounds. That is a built-in annealing effect, and it is why lower learning rates with more trees behave differently from fewer, larger steps.

LightGBM's min_child_samples is a row count, i.e. a genuinely different parameter with a similar name. Its Hessian analogue is min_sum_hessian_in_leaf. Porting a configuration between the two libraries by matching names is a common way to accidentally change the model.

XGBoost vs. LightGBM vs. CatBoost: The Engineering Differences

All three implement gradient boosted decision trees. The differences are in tree construction, and they are what actually shows up in training time and out-of-sample score.

XGBoost grows level-wise (breadth-first): all leaves at a given depth are split before going deeper. This produces balanced trees, makes max_depth a meaningful complexity control, and is more predictable to tune. It also wastes work splitting leaves that have little loss left to reduce.

LightGBM grows leaf-wise: it splits whichever leaf offers the largest loss reduction, wherever that leaf sits. Fewer splits reach the same training loss, but trees become deep and unbalanced, so max_depth stops being the right control knob — num_leaves is. Two further tricks drive its speed:

• GOSS (Gradient-based One-Side Sampling) keeps all large-gradient instances and randomly subsamples the small-gradient ones, upweighting the survivors to keep the gradient estimate unbiased. On financial data, large-gradient instances are the bars the model currently gets wrong — which is also where the label noise lives, so GOSS concentrates sampling exactly where noise is worst. Worth checking against plain subsampling rather than assuming.
• EFB (Exclusive Feature Bundling) packs mutually exclusive sparse features into a single bin-space feature. Effective on one-hot encodings; near-useless on dense continuous features, which is most of a technical feature matrix.

CatBoost grows symmetric (oblivious) trees: every node at a given depth uses the same split condition. This is a strong regularizer and makes inference very fast — the tree becomes an index lookup — at the cost of expressiveness per tree. Its two distinguishing mechanisms:

• Ordered boosting. Standard boosting computes residuals for a sample using a model that was trained on that sample, which biases the residual — "prediction shift." CatBoost estimates residuals for each sample using a model fitted only on samples preceding it in a random permutation. The structure is a natural fit for time-series thinking, and it matters most when data is limited.
• Ordered target statistics for categorical encoding, which computes target statistics from preceding samples only, avoiding the target leakage that naive mean-encoding introduces. This is the honest reason to use CatBoost when the feature set carries exchange, asset tier, or regime labels.

The qualitative rows are library facts. The last two rows are the only ones that answer "which should I use," and they have to come from a run on your own data — differences between well-tuned implementations are typically small enough that dataset shape decides.

Switching libraries is mostly a rename. One training function, with the parameters that differ listed rather than triplicated:

import xgboost as xgb

def train_xgb_model(X_train, y_train, X_val, y_val, class_weight_ratio=1.0):
    """Train XGBoost classifier for return direction prediction."""
    model = xgb.XGBClassifier(
        n_estimators=2000,
        max_depth=5,
        learning_rate=0.01,
        subsample=0.7,
        colsample_bytree=0.7,
        min_child_weight=10,   # Hessian sum, not row count
        gamma=1.0,
        reg_alpha=0.1,
        reg_lambda=1.0,
        scale_pos_weight=class_weight_ratio,
        objective='binary:logistic',
        eval_metric='logloss',
        tree_method='hist',
        random_state=42,
        early_stopping_rounds=50,
    )
    model.fit(X_train, y_train, eval_set=[(X_val, y_val)], verbose=False)
    return model

SHAP Importance Across Folds as an Alpha-Decay Detector

The blog already covers SHAP on a gradient-boosting model — TreeExplainer, summary plots, and how to read them. What is not covered is using SHAP longitudinally: one explainer per walk-forward fold, tracking each feature's mean absolute attribution over time.

def shap_over_time(models, test_sets, feature_names) -> pd.DataFrame:
    """
    Track SHAP-based feature importance across walk-forward folds.
    Rows are folds, columns are features.
    """
    importance_over_time = []
    for fold_idx, (model, X_test) in enumerate(zip(models, test_sets)):
        explainer = shap.TreeExplainer(model)
        shap_values = explainer.shap_values(X_test)
        mean_abs_shap = np.abs(shap_values).mean(axis=0)
        importance_over_time.append(
            pd.Series(mean_abs_shap, index=feature_names, name=fold_idx)
        )
    return pd.DataFrame(importance_over_time)

The output is a fold × feature matrix, and there are three distinguishable shapes in it:

1. Monotone decline — the feature's edge is decaying. Candidate for removal, or for investigating what changed in market structure.
2. High variance, no trend — the feature is noise that the model latches onto in some regimes. This is the same signal that plateau analysis quantifies for parameters, applied to features.
3. Regime-switched step change — importance drops at a specific fold and stays down. Usually traceable to an exchange, listing, or fee-structure event rather than to alpha decay.

The trap is that mean absolute SHAP is not comparable across folds when the model's overall confidence changes: a model that predicts closer to 0.5 everywhere produces smaller attributions for every feature at once. Normalize each fold's importances to sum to 1 before comparing, so you are reading relative importance shifts rather than confidence shifts.

Why Trees, Briefly

Grinsztajn, Oyallon, and Varoquaux (NeurIPS 2022) tested tree ensembles against deep learning across 45 tabular datasets and isolated three structural properties that favor trees — all three describe financial data:

1. Irregular target functions. Returns are not smooth; they have discontinuities, regime changes, and threshold effects. Axis-aligned splits capture these without approximating a smooth surface.
2. Uninformative features. An alpha pipeline generates hundreds of candidates and most are noise. Trees select at each split; neural networks spread capacity across all inputs, spending parameters on noise.
3. Non-rotational invariance. Volume is not interchangeable with volatility. Neural networks are rotationally invariant by default, treating linear combinations of features as equivalent to the originals — which is simply wrong for features with distinct semantic meaning.

For the practical side of this trade-off — data size, latency, feature-engineering effort, interpretability, regime adaptation — the blog already has a full decision table in spread modeling with machine learning.

Feature Engineering

import pandas as pd
import numpy as np

def build_features(df: pd.DataFrame) -> pd.DataFrame:
    """
    Build trading features from OHLCV data.

    Expects columns: open, high, low, close, volume, timestamp
    """
    feat = pd.DataFrame(index=df.index)

    feat['return_1'] = df['close'].pct_change(1)
    feat['return_5'] = df['close'].pct_change(5)
    feat['return_15'] = df['close'].pct_change(15)
    feat['return_60'] = df['close'].pct_change(60)

    log_ret = np.log(df['close'] / df['close'].shift(1))
    feat['volatility_20'] = log_ret.rolling(20).std()
    feat['volatility_60'] = log_ret.rolling(60).std()
    feat['vol_ratio'] = feat['volatility_20'] / feat['volatility_60']

    feat['parkinson_vol'] = np.sqrt(
        (1 / (4 * np.log(2)))
        * (np.log(df['high'] / df['low']) ** 2).rolling(20).mean()
    )

    feat['volume_sma_ratio'] = df['volume'] / df['volume'].rolling(20).mean()
    feat['volume_std_20'] = df['volume'].rolling(20).std()
    feat['obv'] = (np.sign(df['close'].diff()) * df['volume']).cumsum()
    feat['obv_slope'] = feat['obv'].diff(5) / feat['obv'].shift(5)

    feat['high_low_range'] = (df['high'] - df['low']) / df['close']
    feat['close_position'] = (df['close'] - df['low']) / (df['high'] - df['low'])
    feat['gap'] = df['open'] / df['close'].shift(1) - 1

    delta = df['close'].diff()
    gain = delta.clip(lower=0).rolling(14).mean()
    loss = (-delta.clip(upper=0)).rolling(14).mean()
    feat['rsi_14'] = 100 - 100 / (1 + gain / loss)

    ema_12 = df['close'].ewm(span=12).mean()
    ema_26 = df['close'].ewm(span=26).mean()
    feat['macd'] = (ema_12 - ema_26) / df['close']
    feat['macd_signal'] = feat['macd'].ewm(span=9).mean()
    feat['macd_hist'] = feat['macd'] - feat['macd_signal']

    for window in [10, 20, 50]:
        sma = df['close'].rolling(window).mean()
        feat[f'distance_sma_{window}'] = (df['close'] - sma) / sma
        std = df['close'].rolling(window).std()
        feat[f'bb_position_{window}'] = (df['close'] - sma) / (2 * std)

    return feat

Every feature above is causal by construction — rolling and expanding operations only, no whole-sample statistics. That is deliberate: whole-series z-scoring is the single most common leak in this kind of pipeline, and its effect on reported Sharpe is measured in the look-ahead bias taxonomy.

Hyperparameter Ranges for XGBoost on Financial Data

The goal is not maximum in-sample performance but maximum out-of-sample stability. Regularization first, complexity second:

For the search procedure itself — TPE, study persistence, and why Bayesian search beats coordinate descent — see Optuna vs. coordinate descent. Whatever trial count you use there has to be carried into the Sharpe deflation afterwards.

What This Article Deliberately Leaves to Other Posts

The scaffolding around a gradient boosting model is already covered here with measurements attached:

• Validation splits. Walk-forward optimization for anchored vs. rolling windows and degradation rate; purged_walk_forward in spread modeling with machine learning for a splitter with a real purge/embargo gap. Do not ship one without a purge — overlapping forward-window targets leak across fold boundaries.
• Overfitting controls. Train/test gap and cross-fold feature stability, quantified in plateau analysis and the probability of backtest overfitting.
• Predictions to PnL. A vectorized signal * shifted-return backtest with flat bps costs manufactures Sharpe from a handful of lucky trades — see objective function design and PnL per active time. Deflate any post-search Sharpe for the trial count: deflated Sharpe, and the honest negative for what that looks like when the edge is not real.
• Transaction costs. Slippage cost models and maker-taker fees. A 53%-accuracy direction model lives or dies on which cost model you pick.

Three further pitfalls are stated here as hypotheses, not results: survivorship bias from training only on listed assets; non-stationarity of raw return targets, where beta-residual returns may behave better; and retraining cadence, where the SHAP-over-folds diagnostic is the natural drift monitor.

Conclusion

Return-direction classification is an imbalanced problem with a decision threshold nobody chose, and the three standard fixes are not interchangeable. scale_pos_weight is one line but decalibrates probabilities and silently moves min_child_weight. Focal loss needs a Hessian that most implementations approximate. Threshold optimization leaves the model alone and puts the trading-relevant constraint — precision — where it belongs, but only if the threshold is refitted per fold on data that comes after training.

Which one wins is an empirical question about your data, your threshold, and your cost model. The table at the top is the answer for one dataset; running it on yours costs less than the tuning time spent guessing.

Further reading:

• Grinsztajn et al., "Why do tree-based models still outperform deep learning on typical tabular data?" (NeurIPS 2022)
• Wang et al., "Imbalance-XGBoost: leveraging weighted and focal losses for binary label-imbalanced classification"
• Chen & Guestrin, "XGBoost: A Scalable Tree Boosting System" (KDD 2016)
• Ke et al., "LightGBM: A Highly Efficient Gradient Boosting Decision Tree" (NeurIPS 2017)
• Prokhorenkova et al., "CatBoost: unbiased boosting with categorical features" (NeurIPS 2018)
• XGBoost documentation: Notes on Parameter Tuning