Skip to content

Model card — Structural breaks at unknown dates

bai_perron · sup_f_test

"Did the relationship change — and when?" is a different question from "did it change at this date I picked in advance?". Testing at a date chosen by looking at the data invalidates the usual F distribution; the tools here pay the correct statistical price for searching over dates. sup_f_test is the Andrews (1993) sup-F (Quandt likelihood ratio) test for a single break at an unknown date; bai_perron is the Bai-Perron (1998, 2003) machinery for multiple breaks — a dynamic program finds the globally optimal break dates, a sequential sup-F procedure decides how many there are, and Bai (1997) gives confidence intervals for the dates themselves.

Two known-date complements live on the specification-tests card: use chow_test when theory hands you the candidate date before looking at the data (it is sharper there), and cusum_test for a general "is this regression stable at all?" monitoring scan without a break-date alternative in mind.

sup_f_test — Andrews-Quandt test for one unknown break

What it estimates. The supremum of the Chow F statistic over every admissible break date in the trimmed interior of the sample,

supF = max_{k in [trim*T, (1-trim)*T]} F(k),

testing H0: one coefficient vector governs the whole sample against H1: the coefficients change once, at some unknown date. The p-value uses Hansen's (1997) approximation to the non-standard asymptotic distribution (the fixed- date chi-square is wrong here — the statistic is a maximum over many correlated tests, which is exactly what Andrews tabulated).

Assumptions. A linear regression with well-behaved (stationary, weakly dependent) regressors within each regime; homoskedastic errors in the classical F form; a break, if any, in the coefficients (a pure variance break is not this alternative). The trimming keeps enough observations on each side for the split regressions to exist.

When to use (and when not). Use it as the first, honest question — "is there a break somewhere?" — when you have no pre-committed date. If you have a theory-given date, use chow_test (more power, standard distribution). If one break is found and you suspect more, go straight to bai_perron, which nests this test as its first stage. Do not use f_path's argmax as a precisely-known date: it is a point estimate whose uncertainty bai_perron's confidence intervals quantify.

Key arguments and defaults (and why). trim = 0.15 (search over the central 70% of dates) — Andrews' recommended default; smaller trims search more dates but destabilize the endpoint regressions and inflate the critical values. x is your full design including your constant.

How to read the output. stat is the supF; p_value is the Hansen (1997) approximation (small ⇒ reject stability); break_date is the argmax date (the last observation of the pre-break regime); dates/f_path trace the whole Chow-F profile — plot it: a sharp single peak is one clean break, a plateau is genuine date uncertainty, twin peaks hint at two breaks. h is the minimum segment length implied by the trim.

Failure modes. Persistent regressors and unit roots break the asymptotics (this is a stationary-regression test, not a test against a random walk). Heteroskedastic or serially correlated errors distort the classical F form — prewhiten or difference first. A gradual drift in coefficients triggers the test too, at a diffuse peak: rejection means "not stable", not "one sharp break". And the trimmed window means breaks in the first or last 15% of the sample are invisible by construction.

Validated against. Statistics, F paths, and Hansen p-values pinned at 1e-8 (p-values 1e-10) to a numpy/scipy-only reference (fixtures/tsecon-breaks.json, generated by fixtures/generate_tsecon-breaks_fixtures.py); a property test confirms the supF equals the maximum of an independently computed Chow-F path.

References. Andrews (1993, Econometrica 61:821-856); Hansen (1997, Journal of Business & Economic Statistics 15:60-67); Quandt (1960).

bai_perron — multiple breaks: how many, when, how sure

What it estimates. Up to max_breaks coefficient breaks in the linear model y_t = x_t' b_j + u_t (regime j), with all coefficients switching (a "pure structural change" model — include your constant in x). Three stages, all returned:

  1. Where. For each m = 1..max_breaks, the dynamic program of Bai-Perron (2003) finds the break dates that globally minimize the total SSR over all admissible partitions — no greedy approximation.
  2. How many. The sequential supF(l+1 | l) procedure tests l+1 against l breaks at the Bai-Perron published 5% critical values, stopping at the first non-rejection.
  3. How sure. Per-regime OLS coefficients with standard errors, and Bai (1997) confidence intervals for each break date.

The homogeneity caveat, stated plainly. The shipped break-date confidence intervals are the Bai (1997) homogeneous case: the same regressor second-moment matrix and error variance are assumed on both sides of each break, so a single scale (ci_scale) maps the shift magnitude into the argmax distribution of Picard/Bai (the one with the published anchors P(|xi| <= 7.7) ≈ 0.90, P(|xi| <= 11.03) ≈ 0.95). The heterogeneity-robust variant — separate moments and variances per side — is not shipped. If the regressor distribution or the error variance visibly differs across regimes (compare regime_ssr per observation), read the CIs as approximate and lean on the point dates, which do not depend on that assumption.

Assumptions. Stationary, weakly dependent regressors within regimes; breaks separated by at least trim * T observations (the minimum segment length h); classical errors for the supF stages, homogeneity as above for the CIs. The number of breaks found is capped by both max_breaks and what the trimming geometry admits.

When to use (and when not). Use it whenever "the Great Moderation question" arises — did a mean, a slope, a policy rule shift, possibly more than once, at dates you do not know? It is the standard tool for dating regime changes in coefficient space. Do not use it for smooth parameter drift (a time-varying-parameter model fits better), for variance-only breaks, for unit-root regressors, or with a known candidate date (chow_test). For "many small frequent shifts", the trimming makes segments artificially long.

Key arguments and defaults (and why). max_breaks = 5 — the published critical values cover the sequential tests up to this order. trim = 0.15 sets the minimum regime length h = ceil(trim * T); smaller trims allow shorter regimes but weaken every per-regime estimate and push the supF critical values up. x is T × q with every coefficient switching — to hold some coefficients fixed across regimes (partial structural change) you would need the partial model, which this function does not implement.

How to read the output. n_breaks is the sequentially selected count; break_dates the chosen dates (each the last observation of its regime); compare sup_f_seq to sup_f_crit element-wise to see why selection stopped where it did. break_dates_by_m and ssr_path expose the whole DP surface — ssr_path[m] is the best achievable SSR with m breaks, and its flattening is the informal elbow check on n_breaks. Per-regime: regime_starts/regime_ends, params/bse (row per regime, column order of x), regime_ssr. Break-date uncertainty: ci_lower_90/95 and ci_upper_90/95 are inclusive integer date bounds; ci_scale is the per-break scale discussed above; a CI spanning only a few periods means the break is sharply identified.

Failure modes. Reading the CIs as robust when regimes have visibly different variance (the caveat above). Trusting n_breaks when consecutive sup_f_seq entries sit near their critical values — selection is a sequence of 5% tests, not a posterior. Outliers masquerade as short regimes when trim is small. And a break within h of the sample edge cannot be found, no matter how real.

Validated against. The dynamic program is pinned to exact brute-force enumeration of all admissible partitions (same minimal SSR at 1e-8 relative, same dates) on small samples up to three breaks; the full output — ssr_path, dates for every m, supF(l+1|l), the selection, per-regime coefficients/SEs (1e-8), and the CIs (exact integers) — is pinned on two-break and no-break DGPs (fixtures/tsecon-breaks.json). Seeded Monte-Carlo property tests recover a two-break DGP within ±2 periods and confirm the sequential procedure's ~5% false-break rate on stable data.

References. Bai & Perron (1998, Econometrica 66:47-78); Bai & Perron (2003, Journal of Applied Econometrics 18:1-22, the algorithm and critical values); Bai (1997, Review of Economics and Statistics 79:551-563, break-date CIs); Andrews (1993).

The DGP has two genuine breaks — the intercept jumps at t = 100, the slope flips at t = 200 — and both stages should find them, with the artificial third break firmly rejected:

import numpy as np, tsecon

rng = np.random.default_rng(0)
n = 300
x1 = rng.standard_normal(n)
X = np.column_stack([np.ones(n), x1])        # include YOUR constant; all coefs switch
y = np.zeros(n)
for t in range(n):
    if t < 100:   a, b = 0.0, 0.5            # regime 1
    elif t < 200: a, b = 1.5, 0.5            # regime 2: intercept jumps at t=100
    else:         a, b = 1.5, -0.5           # regime 3: slope flips at t=200
    y[t] = a + b * x1[t] + 0.5 * rng.standard_normal()

# ---- sup_f_test: is there a break at some unknown date?
sf = tsecon.sup_f_test(y, X, trim=0.15)
print(f"supF = {sf['stat']:.1f}   p = {sf['p_value']:.4f}   "
      f"argmax date = {sf['break_date']}")

# a stable series for contrast — the test should NOT reject
y0 = 0.5 * x1 + 0.5 * rng.standard_normal(n)
sf0 = tsecon.sup_f_test(y0, X, trim=0.15)
print(f"stable series: supF = {sf0['stat']:.1f}   p = {sf0['p_value']:.4f}")

# ---- bai_perron: how many breaks, where, with what uncertainty?
bp = tsecon.bai_perron(y, X, max_breaks=5, trim=0.15)
print(f"\nn_breaks = {bp['n_breaks']}   "
      f"break_dates = {[int(d) for d in bp['break_dates']]}")
print("supF(l+1|l) sequence:", np.round(np.asarray(bp["sup_f_seq"]), 1),
      "\n  vs 5% critical    :", np.asarray(bp["sup_f_crit"]))
params = np.asarray(bp["params"]); bse = np.asarray(bp["bse"])
for j in range(bp["n_breaks"] + 1):
    s, e = bp["regime_starts"][j], bp["regime_ends"][j]
    print(f"regime {j+1} [{s:3d}..{e:3d}]: const={params[j,0]:+.3f} ({bse[j,0]:.3f})"
          f"  slope={params[j,1]:+.3f} ({bse[j,1]:.3f})")
print("95% CIs for the break dates:",
      [(int(l), int(u)) for l, u in zip(bp["ci_lower_95"], bp["ci_upper_95"])])
# supF = 423.0   p = 0.0000   argmax date = 99
# stable series: supF = 3.7   p = 0.7740
#
# n_breaks = 2   break_dates = [99, 199]
# supF(l+1|l) sequence: [423.  192.2   3.9   2.9   1.4]
#   vs 5% critical    : [11.47 12.95 14.03 14.85 15.29]
# regime 1 [  0.. 99]: const=-0.017 (0.047)  slope=+0.470 (0.048)
# regime 2 [100..199]: const=+1.507 (0.055)  slope=+0.518 (0.057)
# regime 3 [200..299]: const=+1.508 (0.046)  slope=-0.438 (0.041)
# 95% CIs for the break dates: [(96, 102), (195, 203)]

Both true break dates are found exactly (99 and 199, the last pre-break observations), the sequential supF collapses from 192.2 to 3.9 the moment a third break is proposed (against a critical value of 14.03), the per-regime estimates recover the true (0, 0.5) → (1.5, 0.5) → (1.5, -0.5) path, and the 95% date CIs span about ±3 periods. On the stable series, supF = 3.7 with p = 0.77 — the search over dates has been correctly paid for.