1. Why Ground Calibration?

1. The NISP Ground-Test Campaign

The Ground Test Pipeline

Input Coordinates: the Telescope Simulator Frame

Telescope simulator
The source position in the telescope field of view is given by two spherical angles:

• theta — azimuthal angle [degrees]
• phi — elevation angle [degrees]

Input: angular source position in the telescope field of view
(θ, φ) measured from the optical axis

Output Coordinates: the Detector Mosaic Frame (R_MOS)

2. The Datasets

The calibration data was collected for 7 optical configurations, combining grism choice and grism wheel angle (GWA).

1st-Order Spectra (Fabry-Pérot)

0th-Order Spectra (Two-Blob Model)

3. Data Format — 1st-Order Files (*_first.csv)

Each row is one identified emission line from one source observation.

Data Format — 0th-Order Files (*_zeroth.csv)

Each row is one blob (rank 1 or 2) from one source observation.

A Quick Word on pandas

Reading the Data

import pandas as pd

df = pd.read_csv("data/rgs000_0_first.csv")
print(df.shape)     # (16501, 11)
print(df.columns.tolist())
# ['psf_id', 'spectra_id', 'y_nisp', 'z_nisp', 'wavelength',
#  'cent_y', 'cent_z', 'sig_y', 'sig_z', 'theta', 'phi']
print(df.head())

print(df.describe())
# Number of unique source positions
positions = df[["theta", "phi"]].drop_duplicates()
print(f"{len(positions)} unique positions")
# Wavelength range
print(f"lambda: {df['wavelength'].min():.0f} - {df['wavelength'].max():.0f} nm")

Working with the datasets

# All emission lines from one source observation
spectrum = df[df["spectra_id"] == df["spectra_id"].iloc[0]]
print(spectrum[["wavelength", "cent_y", "cent_z"]])

sig_max = 0.05   # mm
clean = df[(df["sig_y"] < sig_max) & (df["sig_z"] < sig_max)]
print(f"{len(clean)} / {len(df)} rows kept")

dz = pd.read_csv("data/rgs000_0_zeroth.csv")
print(dz.columns.tolist())
# ['psf_id','spectra_id','rank','wavelength','y_nisp','z_nisp',
#  'theta','phi','cent_y','cent_z','sig_y','sig_z']

# The two blobs of each source
blob1 = dz[dz["rank"] == 1]   # blue edge (1250 nm)
blob2 = dz[dz["rank"] == 2]   # red edge  (1850 nm)

4. Exercise — Load and Explore

Write a function that loads one 1st-order dataset, filters outliers, and returns a clean DataFrame.

import pandas as pd
import numpy as np

def load_spectra(filepath: str, sig_max: float = 0.05) -> pd.DataFrame:
    """Load and clean a 1st-order spectra dataset.

    Parameters
    ----------
    filepath : str
        Path to a *_first.csv file (e.g. 'data/rgs000_0_first.csv').
    sig_max : float
        Maximum allowed centroid uncertainty (mm).
        Rows with sig_y or sig_z above this are removed.

    Returns
    -------
    pd.DataFrame
        Cleaned DataFrame with columns:
        spectra_id, theta, phi, wavelength, cent_y, cent_z, sig_y, sig_z.

    Hints
    -----
    - Use pd.read_csv to load the file.
    - Build a boolean mask: (sig_y < sig_max) & (sig_z < sig_max).
    - Return df[mask].reset_index(drop=True).
    """
    # TODO: implement this function
    ...

Exercise (continued) — Median per Spectrum

Compute the median centroid per (spectra_id, wavelength) to reduce noise from repeated exposures.

def median_per_spectrum(df: pd.DataFrame) -> pd.DataFrame:
    """Compute the median PSF centroid per (spectra_id, wavelength).

    Parameters
    ----------
    df : pd.DataFrame
        Cleaned DataFrame returned by load_spectra().

    Returns
    -------
    pd.DataFrame
        One row per (spectra_id, wavelength), with columns:
        spectra_id, theta, phi, wavelength, cent_y, cent_z.

    Hints
    -----
    - Use df.groupby(["spectra_id", "wavelength"]).median().
    - Reset the index after groupby.
    - theta and phi are the same for all rows with the same spectra_id.
    """
    # TODO: implement this function
    ...

Exercise (continued) — 0th-Order Centers

Load a 0th-order dataset and compute the 0th-order center for each source.

def zeroth_order_centers(filepath: str) -> pd.DataFrame:
    """Compute the 0th-order center from the two-blob model.

    Parameters
    ----------
    filepath : str
        Path to a *_zeroth.csv file.

    Returns
    -------
    pd.DataFrame
        One row per source with columns:
        spectra_id, theta, phi, cent_y, cent_z
        where cent_y and cent_z are the mean of the two blob centroids.

    Hints
    -----
    - Load with pd.read_csv.
    - Group by spectra_id and compute mean of cent_y, cent_z.
    - The mean over rank 1 and rank 2 gives the 0th-order center.
    - theta and phi are the same for both ranks: keep with .first().
    """
    # TODO: implement this function
    ...

5. Data Visualization

Key questions to answer visually:

• Where are the sources in the field of view (θ, φ) in the telescope simulator frame?
• Where do their PSFs land in R_MOS (cent_y, cent_z)?
• Is the mapping smooth and regular?
• How does the GWA angle shift the spectra on the detector?
• What do the 1st-order and 0th-order positions look like relative to each other?

Exercise 1 — Plot Source Positions

Plot the unique source positions in the telescope simulator frame.

import matplotlib.pyplot as plt

def plot_field(df: pd.DataFrame, title: str = "") -> None:
    """Plot unique source positions in the simulator frame.

    Parameters
    ----------
    df : pd.DataFrame
        DataFrame with theta and phi columns.
    title : str
        Plot title.

    Hints
    -----
    - Extract unique positions: df[["theta","phi"]].drop_duplicates()
    - Use plt.scatter(pos["theta"], pos["phi"], s=10).
    - Label the axes: "theta [degrees]", "phi [degrees]".
    - Add plt.grid(True) and plt.title(title).
    - What is the angular range of the field of view?
    """
    # TODO: implement this function
    ...

Exercise 2 — Map to Detector

Plot the PSF centroids in R_MOS, colored by wavelength or source angle.

def plot_centroids(df: pd.DataFrame, color_by: str = "wavelength") -> None:
    """Plot PSF centroid positions in R_MOS.

    Parameters
    ----------
    df : pd.DataFrame
        DataFrame with cent_y, cent_z columns.
    color_by : str
        Column name for color coding.
        Try: "wavelength", "theta", "phi".

    Hints
    -----
    - plt.scatter(df["cent_y"], df["cent_z"],
                  c=df[color_by], cmap="rainbow", s=4, theta=0.5)
    - Add plt.colorbar(label=color_by).
    - Label the axes: "cent_y [mm]", "cent_z [mm]".
    - When colored by wavelength, each source traces a line = its spectrum.
    """
    # TODO: implement this function
    ...

Exercise 3 — Compare GWA Configurations

Overlay multiple configurations to see the effect of GWA angle on spectrum positions.

def compare_gwa(configs: list[str], labels: list[str]) -> None:
    """Overlay centroid maps from different GWA angle configurations.

    Parameters
    ----------
    configs : list[str]
        List of *_first.csv paths.
        e.g. ['data/rgs000_0_first.csv',
               'data/rgs000_m4_first.csv',
               'data/rgs000_p4_first.csv']
    labels : list[str]
        Legend labels for each dataset.

    Hints
    -----
    - For each file: load_spectra -> median_per_spectrum.
    - Plot cent_y vs cent_z for each on the same axes.
    - Use distinct colors and add plt.legend().
    - The GWA angle shifts all spectra along the dispersion direction.
    - What is the shift in mm between _0, _m4 and _p4?
    """
    # TODO: implement this function
    ...

Exercise 4 — 1st Order vs 0th Order

Overlay 1st-order spectra and 0th-order blob positions for the same configuration.

def compare_orders(config: str) -> None:
    """Compare 1st-order and 0th-order positions on the detector.

    Parameters
    ----------
    config : str
        Configuration name, e.g. 'rgs000_0'.

    Hints
    -----
    - Load data/<config>_first.csv  -> 1st-order centroids.
    - Load data/<config>_zeroth.csv -> 0th-order blob positions.
    - Plot both on the same axes with distinct colors/markers.
    - The 0th-order blobs should appear as two compact clusters,
      offset from the dispersed 1st-order spectra (~800 px away).
    - Use plt.legend(['1st order', '0th blob (rank1)', '0th blob (rank2)']).
    """
    # TODO: implement this function
    ...

Final Goal — Spectra in R_MOS

Plot 1st-order PSF centroids in R_MOS colored by wavelength.

def plot_spectra(df: pd.DataFrame) -> None:
    """Plot 1st-order PSF centroids in R_MOS colored by wavelength.

    Hints
    -----
    - plt.scatter(df["cent_y"], df["cent_z"],
                  c=df["wavelength"], cmap="rainbow", s=6, theta=0.5)
    - Add colorbar labeled "wavelength [nm]".
    - Try coloring by "theta" or "phi" to see the field structure.
    """
    # TODO: implement this function
    ...