Inside Our House Algorithm: HRP + Long/Short + CVaR with Hull-White

📄 This article grew into a research paper. The HRP allocation at the heart of this pipeline is put to a controlled test against Markowitz, Ledoit–Wolf shrinkage, and 1/N under known covariance (4,800 experiments). Read the paper online (interactive version + PDF) at hrp.marketmaker.cc, code and data at github.com/suenot/hrp-validation.

In our overview «12 Portfolio Optimization Algorithms, Compared» we raced a dozen allocation methods side by side. Eleven of them are textbook classics. The twelfth, Pipeline, is ours — and it got exactly one bullet point in that post. This article is the deep dive: what is inside it, where each formula comes from, and how the specification turns into Rust.

Pipeline does not invent a new way to compute weights. It takes the most robust known recipe — Hierarchical Risk Parity (HRP) — and wraps it in the two layers a live trading account actually needs but plain HRP lacks: direction (long/short from strategy signals) and a hard risk budget (CVaR adjusted for the current volatility regime). That gives four stages.

The four stages

• I — log returns of every asset.
• II — base weights from HRP.
• III — a long/short split from agent signals, with risk shares set by confidence.
• IV — a CVaR correction with Hull-White volatility; the surplus risk goes to cash.

Let's walk them in order.

Stage I. Log returns

Everything starts with the move from prices to log returns:

$r_{i,t} = \ln\!\left(\frac{S_{i,t}}{S_{i,t-1}}\right)$

where $i$ is the asset and $t$ the time step. Log returns add up over time and are more symmetric than plain percentage changes — the standard input for any covariance math.

Stage II. HRP as the foundation

HRP, proposed by Marcos López de Prado in 2016, sidesteps Mean-Variance Optimization's central disease — inverting an ill-conditioned covariance matrix. It never inverts it at all. Instead it works with the structure of the correlations.

Covariance and correlation

From the returns we build the covariance matrix $\Sigma$ and normalize it into the correlation matrix $C$:

$\Sigma_{i,j} = \mathrm{Cov}(r_i, r_j), \qquad C_{i,j} = \rho_{i,j} = \frac{\mathrm{Cov}(r_i, r_j)}{\sigma_i \sigma_j}$

Distance matrix

We turn correlation into a distance metric, so that strongly correlated assets land "close together":

$d_{i,j} = \sqrt{\frac{1 - \rho_{i,j}}{2}}$

The closer $\rho_{i,j}$ is to 1, the closer $d_{i,j}$ is to 0 — and the more likely the assets share a cluster.

Dendrogram and leaf order

From the distance matrix we build a cluster hierarchy via average linkage and read off the leaf order $\pi = (\pi_1, \ldots, \pi_N)$ — a permutation of the assets in which similar ones sit adjacent.

Optional step: the optimal number of clusters can be chosen by the silhouette coefficient $s_i = \dfrac{b_i - a_i}{\max(b_i, a_i)}$, where $a_i$ is the mean distance within a cluster and $b_i$ the mean distance to the nearest neighboring one. The base pass does not need it — recursive bisection already respects the hierarchy.

Quasi-diagonalization

We permute the rows and columns of $\Sigma$ by $\pi$, gathering large values along the diagonal:

$\Sigma^{q}_{i,j} = \Sigma_{\pi_i, \pi_j}$

Recursive bisection

Then the recursion runs top-down. At each step a cluster is split in half into $L$ and $R$, and capital is allocated between the halves inversely proportional to their variances:

$w_L = \frac{1/\sigma_L^2}{\dfrac{1}{\sigma_L^2} + \dfrac{1}{\sigma_R^2}}, \qquad w_R = 1 - w_L$

A cluster's variance is computed on its covariance sub-block as $\sigma_C^2 = \tfrac{1}{m^2}\sum_{i,j \in C}\Sigma^{q}_{i,j}$. The descent continues until each node holds a single asset. Weights are long-only, non-negative, summing to 1.0.

In our implementation this is the function hrp_from_cov(cov) -> Vec<f64>: correlation → distance → average linkage → leaf order → quasi-diagonalization → recursive bisection. Pipeline calls it as its base — and it is also the public optimize() for the no-signals case.

Stage III. The long/short overlay

Plain HRP is a "buy-only" portfolio. But a strategy often says not just how much but which way. Stage III takes per-asset signals (Long/Short) from the agent and builds two sub-portfolios.

1. Assets are split into long and short baskets by signal.
2. Inside each basket, weights are computed with the same HRP (on the covariance sub-block of those assets), summing to 1 per basket.
3. If the agent also emits a confidence $p_i$, the risk shares between the sides are set by total confidence:

$\xi_L = \sum_{i \in L} p_i, \quad \xi_S = \sum_{i \in S} p_i, \qquad \lambda_L = \frac{\xi_L}{\xi_L + \xi_S}, \quad \lambda_S = \frac{\xi_S}{\xi_L + \xi_S}$

With no confidence, the shares fall back to the asset count in each basket. The final signed weight is $w_i = \lambda_L \cdot w_i^{\text{HRP}}$ for longs and $w_i = -\lambda_S \cdot w_i^{\text{HRP}}$ for shorts, after which the whole gross exposure is normalized to 1.

An honest note about the code. The original spec carries correction factors $\alpha_L = \sqrt{\lambda_L}/\sigma_L$, $\alpha_S = \sqrt{\lambda_S}/\sigma_S$ — but it also marks them with a "do we even need this step?". The implementation does not apply them: the two sides are combined directly by the risk shares $\lambda$, which keeps gross exposure exactly at 1 and creates no hidden leverage. That is a deliberate simplification of the spec, not an oversight.

Stage IV. CVaR with a Hull-White adjustment

HRP balances risk structurally, but knows nothing about the absolute level of risk in money terms. The final stage puts a hard ceiling on tail risk — and makes it sensitive to a change of market regime.

Portfolio return and EWMA volatility

First we collapse the weights into a portfolio return and estimate conditional volatility with EWMA:

$r_{p,t} = \sum_{i=1}^{n} w_i\, r_{i,t}, \qquad \sigma_{p,t}^2 = \lambda\, \sigma_{p,t-1}^2 + (1 - \lambda)\, r_{p,t-1}^2$

with $\lambda = 0.94$ (the classic RiskMetrics value). EWMA gives "today's" volatility rather than one averaged over all history.

Hull-White rescaling

The key idea: past returns cannot be taken as-is — they happened under a different volatility. The Hull-White method rescales each past return to the current level:

$\widetilde{r}_{p,s} = \frac{\sigma_{p,t+1}}{\sigma_{p,s}}\, r_{p,s}, \qquad s = t - N + 1, \ldots, t$

A calm month is "stretched", a turbulent one "compressed", and the distribution is brought into the current regime.

VaR and CVaR

On the rescaled distribution we take the loss quantile and the mean loss in the tail:

$VaR_\alpha^{HW} = -q_{1-\alpha}(\widetilde{r}_p)$

$CVaR_\alpha^{HW} = -\mathbb{E}\!\left[\widetilde{r}_p \mid \widetilde{r}_p \le q_{1-\alpha}(\widetilde{r}_p)\right]$

CVaR (a.k.a. Expected Shortfall) answers not "how bad is a typical bad day" but "how bad is it on average across the worst $\alpha$ percent" — so it sees the thickness of the tail, not just its edge.

Risk budget and cash

If CVaR exceeds the acceptable threshold, every risky position is shrunk by a single factor, and the freed capital moves to cash:

$w_i^{new} = \gamma\, w_i, \quad \gamma = \frac{CVaR_{\max}}{CVaR_\alpha^{HW}}, \qquad w_{cash} = 1 - \sum_{i=1}^{n} |w_i^{new}|$

So the portfolio de-risks itself when tail risk grows and re-enters the market when it calms down.

From spec to code

The whole algorithm lives in a single Rust crate, portfolio-pipeline, and obeys the workspace's uniform contract:

pub fn optimize(prices: &[Vec<f64>]) -> Vec<f64>

This is the long-only projection (stages I, II, IV with no signals) — the exact same prices -> weights interface as the other eleven algorithms, so Pipeline is a drop-in replacement for any of them. The full version with every stage is a separate function:

pub fn run(
    prices: &[Vec<f64>],
    signals: Option<&[Side]>,      // Long / Short per asset
    confidence: Option<&[f64]>,    // agent confidence → risk shares λ
    cfg: &PipelineConfig,          // CVaR / Hull-White parameters
) -> PipelineResult                // signed weights + cash + cvar + σ

The overlay defaults: tail cvar_alpha = 0.05, budget cvar_max = 0.05, EWMA ewma_lambda = 0.94, Hull-White window hw_window = 0 (all history). The implementation has no external dependencies and is deliberately defensive: on short histories (fewer than 4 price points) it returns equal weights, and the CVaR overlay only kicks in at ≥8 return observations — otherwise there is nothing to estimate a tail from.

Why Rust: one deterministic codebase for both backtest and production, with no "Python in research, something else in prod" drift, and fast enough to run all twelve algorithms in a single request through the comparison backend.

What it costs in time

How fast is "fast enough"? We pulled the HRP core (log returns → covariance → average linkage → quasi-diagonalization → recursive weights) into a standalone benchmark and ran the exact same math across seven languages — C, C++, Rust, Zig, Python, Node.js and Bun — under identical conditions: Apple Silicon, single thread, 365 daily observations per asset, synthetic prices, asset counts $N$ from 10 to 10,000.

A word on complexity, because it sets the whole shape. The textbook version of average linkage rescans the full distance matrix for the closest pair on every merge — that is $O(N^3)$ and becomes the bottleneck at a few thousand assets. The benchmark instead uses the $O(N^2)$ nearest-neighbour-chain algorithm (Müllner 2011) — the same one behind SciPy's linkage(method='average'). With that in place, clustering is no longer the dominant stage: at $N = 2000$ it is ~15 ms out of a ~0.5 s pass. The cost is now dominated by the covariance matrix, $O(N^2 \cdot T)$ — the one stage no HRP-style method can avoid.

What the runs show (full per-language tables and a one-command reproduction script live in the project repository):

• On realistic portfolios it's free. A crypto basket is dozens of assets, rarely more than a hundred. At $N \le 100$ a full HRP pass is single-digit milliseconds even in Node and microseconds in Rust/C. Recomputing weights on every tick is a non-issue.
• Rust is within ~1.0–1.3× of C — the same order of magnitude, both compiled, and effectively tied once $N$ reaches the thousands. C is a touch faster on raw arithmetic, but Rust gives the same predictability with no garbage collector and no UB.
• It scales to thousands of assets. With $O(N^2)$ linkage a full pass is ~0.5 s at $N = 2000$ and a few seconds at $N = 5000$ in the compiled languages; even interpreted Node clears $N = 2000$ in under two seconds. What sets the ceiling now is the covariance stage, not the clustering.

The pragmatic takeaway: at our portfolio sizes, choosing Rust is not about "beating C" (C is slightly faster here) — it's about one deterministic codebase for research and production, with no GC pauses and years of performance headroom. The full seven-language benchmark, with results and a reproduction script, is open in the project repository.

Where Pipeline sits among the twelve

In our comparison on a single (deliberately rigged) basket, Pipeline behaved like HRP — because through the long-only optimize() entry point it is HRP with a CVaR overlay. Its directional machinery only comes alive when you feed it strategy signals. That is the whole point: Pipeline is not "yet another optimizer for backtesting weights" but the execution layer between strategy signals and real orders — it takes your buy/sell calls, lays out capital by HRP within each side, balances the sides by confidence, and clips tail risk down to a set budget.

For the full context — which other eleven methods exist and how they differ — see the overview, «12 Portfolio Optimization Algorithms, Compared». And you can try all of it live at portfolio-optimizer.marketmaker.cc.

References

1. López de Prado, M. (2016). Building Diversified Portfolios that Outperform Out of Sample. The Journal of Portfolio Management.
2. López de Prado, M. (2018). Advances in Financial Machine Learning. Wiley.
3. Hull, J., & White, A. (1998). Incorporating Volatility Updating into the Historical Simulation Method for Value at Risk. Journal of Risk.
4. Rockafellar, R. T., & Uryasev, S. (2000). Optimization of Conditional Value-at-Risk. Journal of Risk.
5. RiskMetrics Group (1996). RiskMetrics — Technical Document. J.P. Morgan.
6. Marketmaker.cc: marketmaker.cc

Citation

@article{soloviov2026pipeline,
  author = {Soloviov, Eugen and Zhuravleva, Marina and Kiselev, Kirill},
  title = {Inside Our House Algorithm: HRP + Long/Short + CVaR with Hull-White Adjustment},
  year = {2026},
  url = {https://marketmaker.cc/en/blog/post/portfolio-pipeline-hrp-cvar},
  description = {A deep dive into Pipeline, a composite portfolio allocation algorithm built on Hierarchical Risk Parity with a signal-driven long/short overlay and a Hull-White CVaR risk-budget correction, with the full specification and its Rust implementation.}
}