Kalman Filters for Dynamic Hedge Ratios: Tracking a Beta That Refuses to Sit Still

EWA (Australia) and EWC (Canada) are the canonical cointegration pair — two commodity-heavy developed markets whose daily returns correlate at 0.83. Regress EWC on EWA over the full sample and you get one hedge ratio for fifteen years: $\mathbf{\beta }=1.57$, with an ADF p-value of 0.038 on the residual spread — cointegrated, by the book (Engle–Granger: regress one price on the other, then unit-root-test the residual). The problem is the word one. A hedge ratio is an estimate of an economic relationship — commodity mix, currency betas, index composition — and every one of those drifted between 2010 and 2024. A rolling window is the standard fix, and it limps: every observation inside the window carries equal weight, so year-old data moves today's estimate as much as yesterday's, and each point falling out of the window jerks the estimate — the window cliff.

The Kalman filter starts from a different premise: the regression coefficients are unobserved statesthat evolve through time, and each day's prices are a noisy measurement of them. Two equations define the model. The state equation says the hedge ratio and intercept follow a random walk — $[{\mathbf{\beta }}_{\mathbf{t}},{\mathbf{\alpha }}_{\mathbf{t}}]=[{\mathbf{\beta }}_{\mathbf{t}-1},{\mathbf{\alpha }}_{\mathbf{t}-1}]+{\mathbf{\omega }}_{\mathbf{t}}$— tomorrow's relationship is today's, plus noise. The observation equation says ${\mathrm{EWC}}_{\mathbf{t}}={\mathbf{\beta }}_{\mathbf{t}}{\mathrm{EWA}}_{\mathbf{t}}+{\mathbf{\alpha }}_{\mathbf{t}}+{\mathbf{\epsilon }}_{\mathbf{t}}$. Two variances close the model: the state noise $\mathbf{Q}=\frac{\mathbf{\delta }}{1-\mathbf{\delta }}\mathbf{I}$ with $\mathbf{\delta }=10^{-5}$ (the standard parameterization from Chan), and observation noise $\mathbf{R}=10^{-3}$.

Delta is the single real knob, and it replaces the window size entirely: it is a forgetting rate. Larger δ lets the states wander faster (adaptive but noisy); smaller δ pins them down (smooth but laggy); δ = 0 collapses to recursive least squares — a static beta refined forever. Both values here are textbook defaults, deliberately not tuned on this sample: tune δ to the backtest and you are optimizing the strategy through the back door.

No library, no black box — the whole filter is a predict step and an update step, looped over the sample. Predict: with a random-walk transition the state estimate is unchanged and its covariance grows by Q (uncertainty leaks in). Update: compare the observed EWC to the prediction, and shift the states toward the error in proportion to the Kalman gain — the ratio of state uncertainty to total uncertainty.

def kalman_hedge(x, y, delta=1e-5, r_obs=1e-3):
    q = (delta / (1 - delta)) * np.eye(2)     # trans_cov
    state, p_cov = np.zeros(2), np.eye(2)     # [beta, alpha], diffuse start
    betas, alphas = np.zeros(len(x)), np.zeros(len(x))
    for t in range(len(x)):
        h = np.array([x[t], 1.0])             # observation map
        p_cov = p_cov + q                     # predict   (F = I)
        e = y[t] - h @ state                  # innovation
        s = h @ p_cov @ h + r_obs             # innovation variance
        k = p_cov @ h / s                     # Kalman gain
        state = state + k * e                 # update
        p_cov = p_cov - np.outer(k, h @ p_cov)
        betas[t], alphas[t] = state
    return betas, alphas

The gain is the elegance. When the filter is uncertain (large p_cov), it learns aggressively from each observation; once confident, new data barely moves it — unless Q keeps injecting doubt, which is exactly what lets β keep adapting forever. An exponentially-weighted regression, derived from first principles rather than picked from a menu of window sizes.

Slice off the first year (rolling-OLS warm-up, Kalman burn-in) and compare the three estimators of the same quantity. The static line says the answer is 1.57, forever. The rolling OLS swings between 0.19 and 2.75 — whipping around every regime change a full window late. The Kalman path covers 0.71 to 1.49, moving early and smoothly: no cliff, because no window.

Note what the rolling estimator does around 2020–2021: the COVID shock enters the window, distorts the regression for exactly 252 trading days, then falls out and the estimate jumps again — two artefacts from one event. The filter digests the same shock in weeks and moves on.

The spread is ${\mathrm{EWC}}_{\mathbf{t}}-{\mathbf{\beta }}_{\mathbf{t}}{\mathrm{EWA}}_{\mathbf{t}}-{\mathbf{\alpha }}_{\mathbf{t}}$, z-scored on a trailing 60-day window. Rules, identical for both variants: enter long the spread (long EWC, short β·EWA) when $\mathbf{z}<-2$, short when $\mathbf{z}>+2$, exit when z crosses zero. Positions are sized to $1 gross notional at entry, the Kalman variant re-hedges the EWA leg to the current β daily, and every unit of traded notional pays 10 bp. Signals use the close and P&L starts the next day — no lookahead in the rule. The static beta itself, of course, is one giant lookahead: it was fit on all fifteen years, including the future of every trade it takes.

spread = ewc - beta * ewa - alpha              # beta_t, alpha_t from the filter
z = (spread - spread.rolling(60).mean()) / spread.rolling(60).std()

# enter |z| > 2, exit when z crosses 0, hold in between
if p == 0:
    p = 1 if z[t] < -2 else (-1 if z[t] > 2 else 0)
elif p == 1 and z[t] >= 0: p = 0               # long leg reverted
elif p == -1 and z[t] <= 0: p = 0              # short leg reverted

Here is the honest scoreboard, and it is more interesting than a clean win. Gross of costs, the Kalman spread is the better signal on every risk-adjusted axis: Sharpe 0.55 vs 0.47, volatility 2.4% vs 5.0%, max drawdown -6.0% vs -12.3% — half the risk, more reward per unit of it. But the filtered spread mean-reverts fast, so it trades 152 round trips to the static variant's 57 — and at 10 bp per unit of traded notional, that turnover consumes the entire edge and then some.

So did the Kalman beta “improve” the strategy? As an estimator, unambiguously — better gross Sharpe, half the drawdown, and no lookahead, against a static baseline that was handed the answer key. As a net P&L line at retail costs, no: the same adaptivity that tracks the relationship also generates signals faster than 10 bp round trips can pay for. Halve the cost and the gap halves; at institutional frictions of 1–2 bp the Kalman variant pulls level and ahead. Estimation quality and implementability are different axes, and a backtest that reports only one is hiding the other.