Stage 5, Phase 3 — 0th-Order Residual Dispersion (5.3-Zeroth_Order_Dispersion)¶

Per CLAUDE.md's Stage 5 Phase 3 plan. The physical model's m=0 grating term predicts zero wavelength dependence for the 0th order, but the actual ground-test 0th-order data is measurably wavelength-dependent: zeroth_order_centers (dispcraft/measurement.py) reduces each spectrum's 0th order to a two-blob model -- a rank-1 (blue, 1206 nm) blob and a rank-2 (red, 1892 nm) blob, the RGS passband's transmission edges (doi:10.1051/0004-6361/202555859) -- whose separation is real dispersion, not noise. This is produced by an optical effect the physical model's hypotheses exclude by construction (chromatic behaviour of the grism substrate/prism surviving the undiffracted beam, or chromatic aberration in collimator/camera -- root cause not identified yet).

Step 1 (this notebook, so far): residual-structure diagnostic. Before modeling anything, characterize the two-blob separation itself -- across configs and field position -- using *_zeroth.csv data that has never been fed into any physical fit (Stage 2's compare_orders was exploratory only).

1. Loading 0th-Order Data — a Real Bug in zeroth_order_centers¶

load_spectra's original quality filter required both sig_y < sig_max and sig_z < sig_max. In every *_zeroth.csv, sig_z (e_PSF_centroid[1]) is 100% NaN — it is never measured for the 0th order, only sig_y is. A NaN < sig_max comparison is always False, so the combined filter dropped every single row: zeroth_order_centers silently returned an empty DataFrame on all 7 real configs. This went unnoticed because its only caller so far was Stage 2's exploratory compare_orders (never fed into a fit), and its unit test used synthetic sig_z values that happened to be valid.

Fixed in dispcraft/measurement.py: load_spectra now takes a sig_columns parameter (defaults to both axes, for first-order data); zeroth_order_centers passes sig_columns=("sig_y",). Covered by two new tests in tests/test_measurement.py (all-NaN sig_z, and restricting sig_columns directly).

2. Outlier Investigation — Why Some Spectra Give a ~30 mm "Separation"¶

Computing dz = cent_z(rank2) - cent_z(rank1) per spectrum, most configs have a tight, physically sensible distribution (sub-mm), but 5 of 7 show a handful of spectra with |dz| of 20-30 mm — an order of magnitude larger than the detector itself would make plausible for a chromatic effect. Median-collapsing repeated exposures (Section 1) does not remove these: they turn out to come from spectra where one rank has only a single, unreplicated measurement, and that lone measurement is itself a bad crossmatch (mismatched to a different, nearby PSF), e.g.:

spectra_id 433994101 (rgs000_0):
  rank=1: 4 rows, three at cent_z≈6.10, one at cent_z=36.71  -> median≈6.10 (protected)
  rank=2: 1 row  at cent_z=37.08                              -> no redundancy to protect it
  dz = 37.08 - 6.10 = 30.98 mm

This is a data-quality issue in the underlying nearest-match crossmatch (upstream of this project), not measurement noise — and not something a larger sig_max cut would catch, since the bad row's own sig_y is small (it's a confident measurement of the wrong source). A robust MAD-based clip on the separation, per config, isolates exactly these cases: the clean population has MAD*1.4826 of ~0.01-0.02 mm, so a >5*MAD cut is a wide margin against the true signal while catching >20 mm outliers cleanly.

3. Per-Config Baseline and Field-Position Dependence¶

With outliers removed, check two things per config/axis: the near-constant baseline separation (does it split by grism identity, as offset_z_mm did in Stage 4 Phase 2?), and how much of the remaining scatter a plain linear separation ~ y0 + z0 term explains — the same quantitative lower bound Stage 5 Phase 2 Step 1 used before considering any NN.

4. Findings¶

The two-blob separation is real, structured, and dominated by z, not y. dz (rank2−rank1, i.e. red−blue) has a near-constant baseline 2-3 orders of magnitude larger than dy:

• dz splits cleanly by grism identity, the same tilt+180°-mounting pattern Stage 4 Phase 2 traced offset_z_mm's sign to: rgs000_* ≈ +0.21 to +0.23 mm across all three tilts (0/m4/p4 agree to ~0.015 mm), rgs180_* ≈ -0.21 to -0.22 mm. bgs000_0 (a different device — broad-band grism, not part of the 6-config RGS set) sits at +0.126 mm, roughly half the RGS magnitude — consistent with it being a materially different optical path, not a data-quality artifact.
• dy's baseline is near-zero for the two _0 (no-tilt) configs (~0.0005 mm) but ~0.014-0.016 mm for every ±4°-tilt config, with sign depending on both tilt direction and grism instance (rgs000_m4=+0.016, rgs000_p4=-0.014, rgs180_m4=-0.016, rgs180_p4=+0.015) — a tilt-grism interaction, not a pure grism-identity split like dz.
• Field position explains a substantial share of what's left: a plain linear separation ~ y0 + z0 term gives R²=0.34-0.77 across all config/axis combinations (only rgs180_0's dy is weak, R²=0.15 — its baseline is already ~0, so there's less linear signal to explain to begin with) and cuts RMS 25-53%. This is stronger field dependence than Stage 5 Phase 2 Step 1 found for the first-order residual (R²=29-60%) — the 0th order's field structure is, if anything, easier to model, not harder.
• A real, upstream data-quality issue was found and isolated, not papered over: 0-52 spectra per config (rgs180_m4: 0, bgs000_0: 52) have a >20mm "separation" caused by a single unreplicated, mismatched-crossmatch measurement — traced to a concrete example (Section 2), removed via a per-config 5×MAD clip (clean population MAD≈0.006-0.017mm, so the margin against the true ~0.2mm signal is wide). bgs000_0's unusually high 52/301 (17%) outlier rate is itself notable — this device may have a systematically worse crossmatch than the RGS configs, worth keeping in mind if bgs000_0 is carried into modeling.

Implication for a Phase 3 model: like Phase 2, the natural structure is tiered — a per-grism-instance (or per-config, given the tilt-dy interaction) near-constant baseline, plus a field-dependent term on top, with bgs000_0 likely needing separate treatment as a different device rather than folding into the 6-config RGS set. Nothing promoted to dispcraft/ yet — this step is diagnostic only, matching Phase 2 Step 1's own precedent. Next step (not started): pick and compare candidate model families for the field-dependent term, the same explore-before-commit approach Phase 2 Step 2 used.

5. Step 2 Setup — One Point per Field Position¶

Multiple spectra share (nearly) the same exact (y_nisp, z_nisp) — e.g. rgs000_0 has 339 clean spectra but only 167 distinct field positions. Group by exact position (rounded to 3 d.p., same convention dispcraft. field_calibration.aggregate_by_position uses) and average dy/dz within each — the same treatment Phase 2 Step 2 gave its residuals, and for the same reason: repeated measurements at one field position should be averaged down for noise, not treated as independent samples.

bgs000_0 is excluded from here on — Step 1 flagged it as a different device (broad-band grism) with a much higher outlier rate (17% vs 0-4.5% for the RGS configs), so it doesn't belong pooled or compared alongside the 6-config RGS set without separate treatment.

6. Architecture Grid — Input Encoding × Output Structure (rgs000_0)¶

Same protocol as Phase 2 Step 2: 5-fold KFold over field positions (each row already one position, no grouping needed), rgs000_0 as the representative single dataset, every candidate logged to a new zeroth_field_dependent_arch MLflow experiment. Two encoding × output-mode axes plus a hidden_layer_sizes/alpha grid:

• n_bands: 0 = plain standardized (y0,z0); 4 = plus 4 Fourier bands per coordinate.
• output_mode: independent (one MLP per axis) vs joint (one two-output MLP).

Baselines: baseline_constant (predicts the training fold's mean dy/dz — the "no field dependence, just Step 1's per-config baseline" assumption) and baseline_linear (Step 1's separation ~ y0 + z0 term). baseline_constant replaces Phase 2's baseline_zero, since here the target is the raw separation (with a real, large, non-zero baseline), not an already-baseline-subtracted residual.

7. Sharing Granularity — Per-Dataset vs. Per-Instance Pooled Network¶

Same question Phase 2 Step 2 asked: fit the field-dependent network per-dataset, or shared per grism instance (pooling {rgs000_0, rgs000_m4, rgs000_p4} / {rgs180_0, rgs180_m4, rgs180_p4})? Step 1 already found a concrete reason to expect per-dataset to win here even more clearly than in Phase 2: dy's baseline sign depends on tilt direction, not just grism identity (rgs000_m4=+0.016mm vs rgs000_p4=-0.014mm) — pooling across tilts would average away that real, tilt-specific signal, not just smooth out noise.

Tested directly: for each instrument, the position-level intersection across its 3 configs, 5-fold KFold over positions, comparing a per-dataset network (trained on that config's own training-fold positions) against a pooled network (trained on all 3 configs' training-fold positions/targets together) — both evaluated on each config's own held-out positions. Uses the winning plain-coordinate joint MLP from Section 6 (Fourier encoding's one nominal win there is flagged as fragile — see Section 8 — so it's not carried forward here).

8. Findings¶

Aggregation (Section 5) helps, more modestly than Phase 2. Within- position std is 0.0007-0.0012 mm (dy) / 0.0055-0.0073 mm (dz); between-position std is 0.0018-0.0032 mm (dy) / 0.0074-0.0145 mm (dz) — a ~2-2.3x gap, not Phase 2's 20-30x. Makes sense: unlike Phase 2's residual (pure field-position signal on top of near-zero noise), this separation's "between" spread already contains its own baseline-driven variation, so the gap here specifically isolates the repeated-exposure noise reduction, which is real but smaller.

Architecture grid (Section 6). Every plain-coordinate (n_bands=0) NN candidate lands in a tight 0.0037-0.0063 mm test-RMSE band, beating baseline_constant (0.0091 mm, ~-59%) clearly but baseline_linear (0.0040 mm) only modestly (best plain-coordinate candidate: 0.0037 mm, ~6-14% better) — a much smaller NN-over-linear margin than Phase 2's ~75%, consistent with Step 1 already finding a strong linear component here (R²=0.34-0.77 vs. Phase 2's 29-60%): there's simply less non-linear structure left for an NN to earn its complexity on.

• Output structure is a wash here, unlike Phase 2's decisive joint win: best plain-coordinate candidate is joint (hidden_layer_sizes= (32,16), alpha=1e-3, 0.00374 mm) but the best independent candidates are statistically tied with it (0.00377 mm, <1% apart) — closer to Stage 4's original wash than Phase 2's clear joint win. Plausible reason: Phase 2's r_y/r_z shared one corner feature with one likely physical cause; here dz's baseline splits by grism identity while dy's splits by tilt and grism identity — different physical drivers per axis, so less cross-axis structure for a joint model to exploit.
• Fourier encoding is not a reliable win, despite one nominal top result. The single best row overall is n_bands=4, hidden=(32,16), alpha=0.1, independent (0.00344 mm) — but every other n_bands=4 configuration with lighter regularization (alpha=1e-3) catastrophically overfits (test RMSE 0.013-0.040 mm, 3-10x worse than any plain-coordinate candidate) on this ~134-position training set. A single win surrounded by that much instability reads as a fold-specific fluke, not a real architecture edge — plain coordinates are the robust choice, same conclusion Phase 2 reached, now for a second, independent residual target.
• Winning architecture: standardized (y0,z0), joint MLP, hidden_layer_sizes=(32,16), alpha=1e-3 — identical hyperparameters to Phase 2's winner, arrived at independently on a different target (0th-order two-blob separation vs. first-order MLP residual).

Sharing granularity (Section 7): per-dataset wins decisively, 6/6 configs (vs. Phase 2's 5/6), pooled RMSE 1.4-3.7x worse — sharpest for the tilt configs (rgs000_m4/rgs000_p4/rgs180_m4/rgs180_p4: pooled 2.6-3.7x worse), consistent with Step 1's finding that dy's baseline sign flips with tilt direction, not just grism identity — pooling across the 3 tilts of one instrument averages away that real tilt-specific signal, more severely than Phase 2's parameter pooling did.

Nothing promoted to dispcraft/ yet — architecture settled (matching Phase 2 Step 2's own precedent, no promotion until the tier is actually fit). Next (not started): fit the field-dependent NN into the two-blob separation per config (Phase 2 Step 3's pattern), then decide how this 0th-order correction composes with the existing physical/hybrid model — open question, not yet resolved, since the 0th order isn't part of dispcraft.calibration's current predict_centroids at all.

9. Step 3 — Fitting the Field-Dependent Model per Config¶

Unlike Phase 2 Step 3, there's no existing physical parameter this competes with or replaces: the physical model's m=0 grating term already predicts exactly zero 0th-order separation, and nothing in dispcraft.calibration is fit against 0th-order data at all. So the NN here is a standalone model of the separation itself (baseline + field dependence together, both learned in one fit), not a residual against a physical prediction — no alternating fit or "replace, not add" degeneracy concern (Phase 2's tilt_deg/offset coupling) applies.

Promoted to dispcraft/, the two-phase workflow's real logic move:

• dispcraft.measurement.zeroth_order_separation — the per-spectrum dy/dz computation from Sections 1-2 (median-collapsing repeats per rank first). Also fixed the same latent bug in zeroth_order_centers while touching this code: it averaged raw rows over spectra_id without median-collapsing per-rank repeats first, so it was just as exposed to Section 2's single-bad-exposure crossmatch outliers as the separation was — median-collapsing now happens before combining ranks in both functions. Covered by new tests in tests/test_measurement.py.
• dispcraft.zeroth_dispersion (new module) — mad_outlier_mask, aggregate_by_position, fit_zeroth_dispersion_model, predict_zeroth_dispersion, composing measurement's data with dispcraft.ml's NN machinery, mirroring how field_calibration composes measurement with calibration. Tests in tests/test_zeroth_dispersion.py, including a synthetic-recovery check (known field-dependent + constant-baseline ground truth, recovered from generated data).

Fits one model per config (all 7, including bgs000_0 — Step 1/2's "needs separate treatment" means not pooled with the RGS instances, which a per-config fit already respects), using Step 2's winning architecture.

10. Residuals After the Field-Dependent Fit¶

Same diagnostic as Section 4's plot, now for data - NN(y0,z0). If the field-dependent model genuinely absorbed the structure Steps 1-2 found, this should look like unstructured noise at a much smaller color scale.

11. Findings¶

A clean, decisive win across all 7 configs, no bugs or surprises this time (unlike Phase 2 Step 3's sign bug) — likely because there's no alternating fit here to get a sign backwards in: RMSE dropped 24-54% relative to the per-config constant baseline (rgs000_0: 0.0137→ 0.0063mm; weakest reduction is bgs000_0 at 24%, consistent with it being the noisiest config throughout Steps 1-2).

The NN's mean prediction recovers the constant baseline almost exactly (e.g. rgs000_0: nn_mean_dz=0.2170 vs. median_dz=0.2185; rgs180_m4: nn_mean_dy=-0.0158 vs. median_dy=-0.0155) — confirming the single combined fit correctly learns "baseline + field shape" together without needing a separate scalar term, across every config including the sign-flipped rgs180_* ones and the tilt-dependent dy baselines.

Residuals after the fit (Section 10) lose the field-position structure Steps 1-2 found — no visible gradient or corner feature remains at a much smaller color scale than Section 4's pre-fit plot, for every config.

Nothing wired into the physical/hybrid model yet — this is a standalone predictive model of the 0th-order separation, not a correction inside predict_centroids. The physical model doesn't represent the 0th order at all currently (only dispcraft.measurement.zeroth_order_centers/ zeroth_order_separation extract it from data); composing this NN with the rest of dispcraft.calibration — e.g. using it to predict where the 0th-order center itself should land, or combining it with the first-order hybrid model in a shared training/reporting pipeline — is an open design question for a later step, not resolved here.

12. Step 4 — Joining 0th and 1st Order via a Shared material_k¶

Steps 1-3 built a standalone model of the two-blob separation, with no connection to the physical model already fit against first-order data. CLAUDE.md's Phase 3 plan names two candidate physical causes for the "unmodeled" separation: chromatic behavior of the prism material, and chromatic aberration in the collimator/camera. Neither had been tested.

dispcraft.calibration.GroundTestModel's m_order is a plain dataclass field, so evaluating the existing, unmodified predict_centroids at m_order=0 (dataclasses.replace) with each config's already-frozen first-order parameters (models/joint_specific_fit_<cfg>.toml) requires no new physics code. The Grism's deviation is prism.deviation(λ) - grating.deviation(λ); at m=0 the grating term is zero, but the prism's Cauchy chromatic dispersion is not — so the model already predicts a non-zero rank2-rank1 separation, from parameters fit purely on first-order data. Below: how much of the observed separation that alone explains.

13. Fitting a Shared material_k¶

Stage 3 found material_k "not identifiable from centroid-position data alone" and fixed it at nominal (0.004) — but that used only first-order, single-narrow-wavelength-range data. 0th order's two widely-separated wavelengths (1206nm/1892nm) supply exactly the baseline first-order data lacks. Fit one shared material_k across all 6 RGS configs at once (dispcraft.zeroth_dispersion.fit_shared_material_k), holding every other parameter at its already-frozen first-order value, m_order=0. Also fit material_k independently per config as a consistency check (mirrors Stage 4 Phase 3's shared-vs-specific methodology) — if a shared material property is really being identified, independent per-config fits should land close to the joint one and to each other.

Fit against the separation, not absolute position. The first version of this fit reused dispcraft.calibration.cost_joint directly on absolute 0th-order centroids — a real bug, caught before trusting the result: it landed on a physically meaningless material_k≈-0.017 with several-mm RMSE (rgs000_0 alone: ~7mm), because the 0th order's absolute position doesn't match the shared-physics prediction at all (a much bigger, unrelated registration problem Stage 2 explicitly deferred and this step doesn't reopen). Fixed by fitting the separation instead (dispcraft.zeroth_dispersion.predict_zeroth_order_separation_physical), which cancels the absolute-registration mismatch out and isolates the actual chromatic effect — recovering the material_k≈0.0143 found while planning this step.

14. Residual After the Shared-Physics Baseline¶

With material_k fit, recompute the residual (actual separation - physics prediction with the fit material_k) and rerun Section 3's plain linear residual ~ y0 + z0 diagnostic on it. If what's left is now relatively more field-position-dependent than before (Section 3's R²), that's evidence for the second candidate cause (chromatic aberration in the collimator/camera, which would vary across the field) rather than the prism material (which doesn't).

15. Retraining the Field-Dependent NN on the Smaller Residual¶

Retrain Step 3's field-dependent NN (dispcraft.zeroth_dispersion.fit_zeroth_dispersion_model, architecture unchanged) on this smaller residual instead of the raw separation, and compare final RMSE to Step 3's own per-config numbers (Section 9's tier_results).

16. Findings¶

A single shared material_k explains the baseline, decisively. Fitting material_k jointly across all 6 RGS configs against the two-blob separation (not absolute position — see the caught-bug note in Section 13) lands at 0.0143 (vs. nominal 0.004), consistent to 3 significant figures across independent per-config fits (0.0140-0.0147) — as clean a confirmation as this project has produced that a physical parameter is identifiable and shared. Separation-prediction RMSE drops ~90-93% (0.15-0.16mm → 0.011-0.016mm) for every config. The nominal-material_k baseline (Section 12) explains a consistent ~27-29% of the observed separation across configs (rgs180_0's 70% dy figure is a division artifact — its dy baseline is ~0, so the ratio is meaningless there, not a real outlier).

The remaining field-dependence is untouched, and that's informative, not a gap. R² of the linear residual ~ y0+z0 diagnostic (Section 14) is identical before and after fitting material_k, to 4 decimal places, for every config/axis. This isn't a bug — it's mathematically guaranteed: material_k's effect is a per-config constant (Section 12 confirmed dy_phys/dz_phys don't vary with field position at all), and subtracting a constant shifts a linear regression's intercept without changing its R² (variance explained relative to the residual's own variance is invariant under a constant shift). So this cleanly separates the two candidate causes CLAUDE.md named: the prism material accounts for the field-independent baseline and nothing else; whatever field-dependence exists (R²=0.15-0.77, Phase 3 Step 1's finding) must come from a different, additive mechanism — consistent with the second candidate, collimator/camera chromatic aberration (a field-position-dependent effect by nature), though this step doesn't test that hypothesis directly.

Retraining the field-dependent NN on the smaller residual doesn't meaningfully change its own accuracy (Section 15): RMSE is within ±2% of Step 3's raw-target number for 5/6 configs, and better by 23% for rgs000_p4 (0.0088mm → 0.0068mm). This makes sense once R²'s invariance above is understood: the NN could already represent a per-config constant perfectly via its own bias term (Step 3 already confirmed this), so removing that constant via physics doesn't make the NN's job easier in absolute terms. The value Step 4 adds is interpretability, not accuracy: the ~27-29% baseline is now attributed to a real, shared, newly-identified material property (material_k, shared with the first-order model) instead of being folded invisibly into an opaque per-config learned constant — directly answering one of CLAUDE.md's two open candidate causes for the "unmodeled" 0th-order dispersion, with the other (collimator/camera chromatic aberration) now isolated as the remaining, not-yet-explained field-dependent share.

Promoted to dispcraft/: calibration.py's material_k is now genuinely fittable (nominal_params, no longer hardcoded to model.material.k); zeroth_dispersion.py gained predict_zeroth_order_physical (the shared-physics baseline, m_order=0), predict_zeroth_order_separation_physical (the two-blob separation this step actually fits against), and fit_shared_material_k. A real bug was caught and fixed before trusting the result (Section 13) — the module docstring documents it, and the promoted fit_shared_material_k structurally can't repeat it (its dfs are separation-shaped, no absolute cent_y/ cent_z columns to accidentally fit against). bgs000_0 still excluded (no frozen first-order fit) — stays on Step 3's standalone empirical model, an open gap. fit_zeroth_dispersion_model/predict_zeroth_dispersion (Steps 1-3) are unchanged code, now documented as expecting a baseline-subtracted target from Step 4 onward.

Still open: whether collimator/camera chromatic aberration is really the remaining mechanism is not tested here (would need independent evidence -- e.g. a wavelength-and-field-position model of the first-order residual that Phase 2 didn't consider, or optical modeling of lateral color); no models/*.toml recording of the fit material_k (this step establishes it computationally, not as a frozen, versioned result yet); and the "joined model" is still two separate function calls (predict_zeroth_order_physical + a residual NN), not a single unified prediction entry point -- left for whenever this composes with the rest of dispcraft.calibration.

17. Step 5 — Testing the Collimator/Camera Chromatic-Aberration Hypothesis Directly¶

Section 16 left one candidate cause untested: the prism material accounts for the field-independent baseline and nothing else, so whatever field-dependence remains (R²=0.15-0.77, Section 4) must come from a different, additive mechanism -- CLAUDE.md's second candidate, collimator/camera chromatic aberration.

Deriving a testable, not guessed, functional form. At m_order=0, Grism.forward reduces to: rotate the incoming angle, add the prism's (field-independent) deviation, rotate back. The only field-dependence in the whole chain comes from Collimator.forward (theta = -x/f_coll) and Camera.forward (x = -f_cam*theta), which are currently achromatic -- dispcraft.optics.elements.Collimator/Camera take one scalar f, no wavelength dependence. Composing the two (and using R.T@R=I) gives:

pos_cam(y0,z0,λ) = (cam_f(λ)/coll_f(λ)) * pos_foc(y0,z0)  -  cam_f(λ) * R.T@[0, prism.deviation(λ)]

The second term is field-independent (already Section 12-13's material_k baseline). The first term is a magnification ratio M(λ) = cam_f(λ)/coll_f(λ) multiplying the field position directly -- exactly the textbook signature of lateral color / transverse chromatic aberration: a field-position-proportional, wavelength-dependent image shift, with zero intercept (unlike the general y0+z0 linear fit Section 4 used, which allows one).

Stage 4 Phase 3 already found coll_f/cam_f individually non-identifiable (only their ratio matters, 99.8% bootstrap-correlated) -- so parametrizing two separate chromatic coefficients would just reintroduce that degeneracy. Instead: one shared chromaticity coefficient mu on the ratio, in the same Cauchy form already used for the prism material (n(λ) = n0 + k/λ²), applied to cam_f alone (equivalent to putting it on the ratio, since coll_f stays fixed):

cam_f_eff(λ) = cam_f * (1 + mu / λ_um²)

mu is fit shared across all 6 RGS configs (same bench collimator/camera, same reasoning Stage 4 Phase 3 gave coll_f/cam_f global Tier-1 status), against the Section 14 residual (dy_resid/dz_resid, already material_k-baseline-subtracted) -- diagnostic/notebook-local for now, no new dispcraft/ code, matching every other Step in this phase before its architecture was validated.

18. Fitting a Shared Chromatic Coefficient mu¶

Two fits, mirroring Section 13's shared-vs-per-config check for material_k:

1. mu alone, holding material_k at Section 13's joint value (k_fit_joint) -- fit against the Section 14 residual (dy_resid/dz_resid), the direct test of "does this specific functional form explain what's left."
2. material_k and mu refit jointly from the raw separation -- an identifiability check: if the two are genuinely orthogonal (one a pure field-independent constant, the other a pure field-proportional term through the origin), material_k should barely move from Section 13's k_fit_joint.

19. Residual After the Chromatic Term¶

Same diagnostic as Section 14: recompute the residual after subtracting the fitted chromatic prediction, and rerun the linear residual ~ y0+z0 probe. Unlike material_k (a pure constant, mathematically guaranteed to leave R² unchanged, Section 16), the chromatic term is field-proportional -- if it's really the missing mechanism, R² should drop substantially toward 0.

20. Isotropic mu Is Not Enough — Testing an Anisotropic Variant¶

Section 19's single, shared, isotropic mu (one magnification-ratio chromaticity applied equally to y and z) gives a genuinely mixed result, not a clean confirmation: for dz, R² drops substantially (e.g. rgs000_0: 0.72→0.48) and RMS improves 16-27% in all 6 configs -- real explanatory power. But for dy, RMS gets worse in 5 of 6 configs (-6% to -92%) and R² mostly goes up -- the same scalar mu that helps z overcorrects y in the wrong shape entirely.

An isotropic magnification chromaticity is the simplest version of the hypothesis, not the only one: real lateral color can be anisotropic (chromatic difference of astigmatism, or the two axes' effective optical paths differing after the grism's tilt). Testing a version with independent mu_y, mu_z isolates whether the mechanism is "collimator/camera chromatic aberration, just not isotropic" vs. "not this mechanism at all for y."

21. Findings¶

The collimator/camera chromatic-aberration hypothesis is confirmed for z, not for y -- a mixed, axis-asymmetric result, not a clean yes/no. The functional form was derived, not guessed: dispcraft.optics.elements .Collimator/Camera are currently achromatic (one scalar f each, no λ dependence); giving their ratio a chromatic magnification M(λ)=cam_f(λ)/coll_f(λ)=M0*(1+mu/λ_um²) (mirroring the Cauchy form already used for the prism material) predicts a field-proportional, zero-intercept separation term from the existing, unmodified Grism.forward -- the textbook lateral-color signature, and structurally orthogonal to material_k's field-independent baseline by construction (confirmed: jointly refitting material_k and mu together moves material_k by only 0.6%, from 0.01430 to 0.01439, Section 18).