2026-03-17
The affine bridge in Part 5 and the package scaffold in Part 6 were the prerequisites for the first paper-driven cluster of the series. We now turn to the problem of long-run expectations and term premia through two closely related references: Don Kim and Athanasios Orphanides, Term Structure Estimation with Survey Data on Interest Rate Forecasts, and Don Kim and Jonathan Wright, An Arbitrage-Free Three-Factor Term Structure Model and the Recent Behavior of Long-Term Yields and Distant-Horizon Forward Rates. The current Federal Reserve implementation is summarized on the Three-Factor Nominal Term Structure Model page.
These papers address a basic problem that shows up whenever one tries to estimate a flexible Gaussian term-structure model from yields alone. Interest rates are persistent, samples are finite, and the model can fit the cross section of yields while producing implausible long-run expectations of future short rates. In economic language, the model can confuse the expectations component and the term-premium component of long yields. Kim and Orphanides use survey forecasts of short rates to discipline the latent state. Kim and Wright then use the resulting model to study long-term yields and forward rates.
The contribution of this post is to derive the logic of that survey anchor in the notation of this series. The main point is simple. Survey data do not replace the no-arbitrage term-structure model. They add an observation equation for objects that yields alone leave weakly identified.
Let x_t \in \mathbb{R}^n denote the latent factor vector. Under the physical measure, write the discrete-time state evolution as
x_{t+1} = c_P + F_P x_t + \eta_{t+1}
where \eta_{t+1} is Gaussian with mean zero and covariance matrix Q_P. Under the pricing measure, yields are still affine in the same latent state:
y_t(\tau) = a_Q(\tau) + b_Q(\tau)^\top x_t + \varepsilon_t(\tau)
where \varepsilon_t(\tau) is a measurement error. This is the same state-space structure introduced in the previous two posts. The difference is that we now add survey observables that speak directly to the expected path of the short rate under P.
The short rate itself is written as
r_t = \delta_0 + \delta_1^\top x_t
The problem in yields-only estimation is not that this formula is wrong. The problem is that the parameters governing the dynamics under P can be very weakly pinned down by yields alone, especially when those yields are persistent and measured over a relatively short sample. Kim and Orphanides state this point directly in their abstract: the small-sample problem is severe in dynamic no-arbitrage term-structure models with flexible market-price-of-risk specifications.
Suppose we observe a survey forecast of the short rate h periods ahead. Call that survey variable s_t(h). The natural measurement equation is
s_t(h) = \mathbb{E}_t^P[r_{t+h}] + \nu_t(h)
where \nu_t(h) captures survey noise or imperfect measurement. We now derive this expectation as an affine function of the current latent state.
Iterating the physical transition equation gives
\mathbb{E}_t^P[x_{t+h}] = F_P^h x_t + \sum_{j=0}^{h-1} F_P^j c_P
Substituting this into the short-rate equation yields
\mathbb{E}_t^P[r_{t+h}] = \delta_0 + \delta_1^\top F_P^h x_t + \delta_1^\top \sum_{j=0}^{h-1} F_P^j c_P
Therefore the survey equation is itself affine in the latent state:
s_t(h) = a_s(h) + b_s(h)^\top x_t + \nu_t(h)
with
a_s(h) = \delta_0 + \delta_1^\top \sum_{j=0}^{h-1} F_P^j c_P
and
b_s(h) = F_P^{h \top} \delta_1
This is the key derivation in the cluster. A survey observation is not an alien object that sits outside the term-structure model. It is simply another linear observation on the same latent state. Once we see that, the estimation problem becomes conceptually straightforward: stack yields and surveys into one joint observation equation and let the filter use both.
The long-run expectation object that practitioners often care about is not one single future short rate but an average over a distant horizon. Let the long-run expectation from year T_1 to year T_2 be
\bar r_t(T_1, T_2) = \frac{1}{T_2 - T_1}\int_{T_1}^{T_2}\mathbb{E}_t^P[r_{t+u}]du
In discrete time, if the observation interval is one quarter and the corresponding steps are h_1 and h_2, the same object becomes
\bar r_t(h_1, h_2) = \frac{1}{h_2 - h_1}\sum_{j=h_1+1}^{h_2}\mathbb{E}_t^P[r_{t+j}]
By the previous derivation, this again reduces to an affine function of x_t. This is what it means to anchor long-run expectations in a state-space model. We are not asserting that surveys are perfect. We are asserting that the model should not be free to treat every long-run movement in yields as a term premium when an external measurement of expected short rates says otherwise.
Once the physical expectation of future short rates is identified more sensibly, the term-premium decomposition becomes more credible. The term premium in a yield of maturity \tau is defined on the Federal Reserve page as the departure from the expectations hypothesis, namely
\operatorname{TP}_t(\tau) = y_t(\tau) - \frac{1}{\tau}\mathbb{E}_t^P\left[\int_t^{t+\tau} r_s ds\right]
In discrete time this is
\operatorname{TP}_t(h) = y_t(h) - \frac{1}{h}\sum_{j=1}^h \mathbb{E}_t^P[r_{t+j}]
The decomposition is therefore only as good as the expected-short-rate component. This is why the survey equation is not a side issue. It is the stabilizer that stops the model from forcing too much of long-yield variation into the term premium.
Kim and Orphanides report that adding survey forecasts of the 3-month Treasury bill rate helps overcome the small-sample problem, produces a stable estimate of the expected path of the short rate, reproduces familiar failures of the expectations hypothesis, and captures part of the short-run variation in survey forecasts of longer-rate changes. Kim and Wright then apply a related three-factor model to U.S. Treasury yields since 1990 and conclude that a large portion of the decline in long-term yields and distant-horizon forward rates after the middle of 2004 reflected lower term premia rather than a collapse in expected short rates alone.
Those results matter because they change the economic reading of the yield curve. Without an anchor, a flexible Gaussian model can produce implausibly flat long-run expectations and mechanically label almost all variation in long-horizon forwards as term premia. With an anchor, long-run expectations are allowed to move, but not arbitrarily.
The important modeling choice in this literature is not merely the number of factors. It is the decision to combine three ingredients: a no-arbitrage Gaussian yield model, a flexible physical law of motion, and survey data as noisy measurements of expectations. Each ingredient solves a different problem. The no-arbitrage restriction keeps the cross section of yields coherent. The physical transition system generates expected short-rate paths. The survey equation disciplines those expectations when the sample is too short to identify them from yields alone.
The main difficulty is that every one of these choices is fragile in finite samples. Survey data are noisy and available only for certain horizons. The market-price-of-risk parameters remain weakly identified. Latent factors are statistically convenient but economically opaque. Extending the sample backward can help, but it can also mix regimes with different inflation anchors and different monetary-policy behavior. The Federal Reserve Q&A on the current three-factor model is very clear on this point: the problem is not solved simply by sampling yields more frequently or by fitting more factors.
The mathematical core of the long-run-expectations models is the survey measurement equation. Once we derive that equation, the rest of the cluster follows naturally. Yields tell us about prices under Q. Surveys tell us about expectations under P. The state-space system holds those two pieces together without forcing one to masquerade as the other.
In the next post, we will implement this idea in the package. The goal will not be a perfect re-estimation of the Federal Reserve model from raw data in one step. The goal will be to build a reusable survey-augmented Gaussian scaffold that can compute long-run anchors, expectations paths, and term-premium decompositions in the style of Kim-Orphanides and Kim-Wright.