7. The data reduction recipes

Most of the recipes of the DRS are written entirely in C to ensure a satisfactory execution speed. Some recipes group together different reduction steps. This minimizes the number of recipes that one needs to execute to complete the data reduction process, while keeping the reduction blocks logically and operationally separated. This allows the astronomer to carry out intermediate checks on the quality of the data reduction process using the provided utilities (see The SIPGI reduction cookbook).

Each recipe is a standalone program, whose detailed behavior is regulated via a set of input parameters. SIPGI is released with a default configuration for all the input parameters which depends on the instrument (see parameters files and Workspace).

Important

We urge users to check these parameters and to set the configuration that best suits their reduction purposes.

The recipes outputs are FITS files, and to avoid increasing the number of produced files, the different reduction mid-products together with the various calibration tables useful for the reduction are appended as extensions to the original FITS files, instead of creating independent ones. This is not done to save disk space, but to simplify the astronomer’s work, as the data reduction process would otherwise lead to the creation of thousands of files.

All of the pipeline recipes can be executed via a very simple point and click mode in the graphic interface, taking advantage of the files organization made by the built-in data-organizer. For practical examples see the Cookbook section.

7.1. Create Bad Pixels Image

It creates the bad pixel map which stores the information on the position of dead and hot pixels. This recipe is able to automatically detects 2 kinds of bad pixels: the dead and the hot pixels.

The recipe detects dead pixels using slitless flats. It combines the input frames using a median \(F = median\{f_1, ..., f_n\}\) and creates a smoothed version of this image \(\bar{F}\) using a median filter with a window size defined by the parameter WIN_SIZE. It divides the two frames \(\hat{F} = F/\bar{F}\) and estimates the median value \(m_{\hat{F}}\) and the standard deviation \(s_{\hat{F}}\) of the image \(\hat{F}\). In case of MODS observations, this operation is performed for each quadrant.

The parameter DEAD_SIGMA_FACTOR defines the detection threshold factor \(d\): every pixel below \(m_F-d\cdot s_F\) is labeled as dead.

If dark frames are provided (in LUCI reduction), the recipe also looks for hot pixels. Using a median, it combines dark frames and estimates the median value \(m_D\) and standard deviation \(s_D\) of this combined image. The parameter HOT_SIGMA_FACTOR defines the threshold factor \(h\): every pixel above the level \(m_D + h\cdot s_D\) is labeled as hot.

All the pixels labeled as hot or dead are stored in the bad pixel image, i.e. an image with the good pixels values set at 1 and the bad pixels values set at 0.

The recipe offers the possibility to exclude the edges of the frame from the detection of bad pixels. In particular, median values, standard deviations and the detection of bad pixels is performed within the rectangle identified by the two points (LLX, LLY) and (URX, URY).

Table 7.1 **The Create Bad Pixels Image Recipe Summary**. In bold mandatory inputs
	MODS	LUCI
INPUTS	Slitless flat frames	Slitless flat frames and/or dark frames
OUTPUTS	Bad pixel image	Bad pixel image

7.2. Append Bad Pixels Image

It loads bad pixels from the already created bad pixel image and overwrites the bad pixels list in the CCD Table of each frame. The user must append the bad pixel image to the frames to clean. If the UPDATE parameter is True, the recipe merges the list of bad pixels already present in the CCD table with the new one provided.

Table 7.2 **The Append Bad Pixels Image Recipe Summary**. In bold mandatory inputs
	MODS	LUCI
INPUTS	Frames to polish;	Frames to polish;
INPUTS	Bad pixel image	Bad pixel image
OUTPUTS	Input frames with updated CCD extension	Input frames with updated CCD extension

7.3. Create Master Bias

It creates the Master Bias, i.e. a FITS file containing the bias level to subtract to scientific frames. This recipe is available only for MODS. For LUCI reduction, the bias level is removed with the Master Dark (see Create Master Dark section for more details).

The recipe takes in input a list of bias frames. If desired, activating the flag CLEAN_COSMIC in the parameter file, the recipe polishes input biases from cosmic rays.

The cosmic rays polishing algorithm is similar to the one used by the MIDAS command FILTER/COSMIC.

The input image is smoothed using a median filter. The Poisson error \(P\) of this image is estimated. Then a loop is done on original pixels searching for cosmic rays candidates; a candidate is a pixel whose original value differs from the median filtered one by more than COSMIC_THRESHOLD \(\cdot\)\(P\).

Contiguous candidates are then grouped together; each group is then processed to verify that it is not a real object. The recipe identifies the pixel of the group with the maximum count and the 8 ones around it. The mean \(M\) of the values of these 8 pixels is computed. A group is marked as a cosmic-ray hit if COSMIC_RATIO \(\cdot\)\(M\) is lower than the maximum value itself. The recipe returns an error if the percentage of cosmic rays is greater than COSMIC_MAX_FRACTION. Finally cosmic-rays hits are cleaned using the same algorithm as for the bad pixels cleaning (see Create Master Dark).

If the parameter CLEAN_NEGATIVE_PIX is True negative pixels are treated as cosmic rays.

After the cosmic ray cleaning procedure, frames are combined according to the selected COMB_METHOD:

MEDIAN: median of input frames;

AVERAGE: average of input frames;

KSIGMA: the sigma clipped average of input frames. K_SIGMA_LOW and K_SIGMA_HIGH are the lower and upper sigma factors of the clipping;

REJECT: for each pixel (x,y) of the input frames, computes a mean value after removing the lowest and highest values based on the percentages defined by the MIN_REJECTION and MAX_REJECTION parameters.

Finally, if the entry MOCK_BIAS is set to True, the recipe creates a mock Master Bias frame. This is an image full of 0 that doesn’t remove the bias level from the frames, but is used by the Preliminary Reduction to trim the prescan/overscan regions.

The user can save the log messages in a separated extension of the Master Bias and/or can record all the files used as input flagging the DUMP_LOG and/or the DATA_LINAGE entry, respectively.

Table 7.3 **The Create Master Bias Recipe Summary**. In bold mandatory inputs
	MODS
INPUTS	Bias frames
OUTPUTS	Master Bias

Note

Together with Master Bias, SIPGI can estimate the bias level from the prescan/overscan regions of each frame. In this case, the bias level is not stored in a Master file but it is estimated on the fly for each frame during the Preliminary Reduction). In this case, SIPGI estimates eight bias levels: two for each quadrant, one for each quadrant channel. The single bias level is computed as the 1-sigma clipped average of the pixel values in the current channel/quadrant.

7.4. Create Master Dark

It creates the Master Dark, i.e. a FITS file with the dark level to subtract to scientific frames. This recipe is available only for LUCI. It is standard practice at LBT to provide a set of dark frames with the same DIT, NDIT, READMODE and SAVEMODE as the scientific frames (both science and telluric ones). The availability of dark frames with the same DIT as the scientific ones simplifies the reduction process, since they can be used to simultaneously remove both bias and dark levels. For this reason, LUCI calibrations do not include bias frames and, consequently SIPGI is not design to treat bias in LUCI Workspaces.

Important

Take in consideration that for a correct removal of bias and dark level using SIPGI for LUCI reductions, Master Dark must be obtained on frames with the same DIT and READMODE of scientific frames.

The recipe takes in input a list of dark frames \(d_i\). If desired, activating the flag CLEAN_COSMIC in the parameter file, the recipe polishes input dark frames from cosmic rays as described in Create Master Bias section. If the parameter CLEAN_NEGATIVE_PIX is True, negative pixels will be handled as cosmic rays by the recipe and cleaned at the same time. After the optional cosmic ray correction, frames are combined using the COMB_METHOD provided by the user (see Create Master Bias section for more details on the COMB_METHOD).

If the NDIT_NORM flag is activated, the recipe normalizes input dark frames by their NDIT value (retrieved from the header) \(\bar{d}_i=d_i/{NDIT}_{d_{i}}\) before combining them.

Note

Remember that \(t_{exp} = DIT \cdot NDIT\)

When this normalized Master Dark is used in the reduction process (see e.g. Preliminary Reduction), the recipe automatically multiplies it by science NDIT values, before subtracting it. This allows the user to subtract the dark level in case of science frames with the same DIT, READMODE, SAVEMODE values but different NDIT with respect to the dark frames.

If the entry CLEAN_BAD_PX is True, the recipe polishes the combined image from bad pixels contained in the CCD table. The bad pixel correction replaces the bad pixel value interpolating the values of the first good pixels around the bad pixel. For recipe details see the Appendix E

Table 7.4 **The Create Master Dark Recipe Summary**. In bold mandatory inputs
	LUCI
INPUTS	Dark frames
OUTPUTS	Master Dark

7.5. Create Pix2pix Variation Image

This recipe uses flat slit frames obtained without any mask (e.g. slitless flat frames) to create an image which stores the information on the pixel to pixel variation. In the MODS case this image is also used to correct the different gain responses between the 4 MODS quadrants.

The recipe takes as input slitless flats. In MODS reduction, if requested, it subtracts the bias level from the input frames using the BIAS_METHOD provided by the user. The choice is between: SKIP, MASTER, and PRESCAN. With the SKIP option the user skips the bias subtraction. In case of MASTER choice, the user must also supply the Master Bias. This is the canonical subtraction of the Master Bias for each frame. With the PRESCAN method the bias level is computed from the prescan/overscan regions of the same input flats as described in Create Master Bias. In LUCI reduction, if a Master Dark is provided, the recipe subtracts the dark level from flat frames.

If the CLEAN_COSMIC entry is set to True, the recipe polishes frames from cosmic rays. After this, the recipe combines flat frames with the COMB_METHOD provided by the user. For more details on the cosmic cleaning and the combining methods see Create Master Bias.

A smoothed version of the combined image is created quadrant by quadrant using the SMOOTH_METHOD. The recipe has two smoothing method: a MEDIAN filter and a GAUSSIAN kernel. The median filter replaces the value of each pixel with the median estimated in a box of size SMOOTH_SIZE_BOX centered on that pixel. The GAUSSIAN method smooths the image using a Gaussian kernel with the sigma defined by SMOOTH_SIGMA (more details about this algorithm can be found in scipy.ndimage.gaussian_filter).

MODS shows different gain levels for each quadrant as shown in the Fig. 7.1.

_images/blue_ff_gains.png — Fig. 7.1 A zoom on a flat frame at the center of the MODS detector.

These differences can be corrected using the parameter GAIN_WIN_SIZE. If GAIN_WIN_SIZE is greater than zero, the recipe selects for each quadrant of the smoothed image a square region at the quadrant junction (see Fig. 7.2) and collapses it along the cross dispersion direction. An example of the collapsed signal for the 4 quadrants is shown in Fig. 7.3 (see dashed lines).

_images/gain_win_size.png — Fig. 7.2 The red square is the `GAIN_WIN_SIZE` region used to compute the normalization factor.

Then it fits a second order polynomial to model the flat variation (solid lines in the Fig. 7.3). The values of these polynomial in the junction are the scaling factor applied to each quadrant (dots in the Fig. 7.3).

_images/pix2pix.png — Fig. 7.3 An example of the procedure for the 4 different gain levels adjustment in MODS data. The jagged lines are the collapsed flat signal along cross dispersion direction in the original frame; each line highlights the different channels response which affects the MODS data. The dashed lines are the collapsed flat signal in the smoothed images; the solid thin lines are the second order polynomials fitted to the dashed lines; the dots are the value of this polynomial at the junction edges. These values are used as normalization levels.

This image stores the large scale variation of the pixels response. Finally, the original combined image is divided by the smoothed gain-corrected image in order to recover the pixel to pixel variation map.

This frame is an optional input for the Create Master Flat recipe. If it is not provided, pixel-to-pixel variation will be directly computed on the through-slit flats provided as input to the recipe. Although it is not mandatory, it is relevant in two cases:

In MODS reduction, both in LS and MOS observations, the slit(s) pattern on through-slit flats does not allow a proper correction for different responses of different quadrants (keep in mind that even the MODS LS observations present 5 slits). The Pix2Pix Variation Image provides this correction.
In LUCI reduction, it offers the possibility to correct for the pixel-to-pixel variation in all of the cases in which through-slit flats are not aligned with science frames.

Table 7.5 **The Create Pix2Pix Variation Image Recipe Summary**. In bold mandatory inputs
	MODS	LUCI
INPUTS	Slitless flat frames	Slitless flat frames
OUTPUTS	Pix2Pix variation image	Pix2Pix variation image

7.6. Adjust First Guesses

This is an interactive task, which allows the user to refine on real data the theoretical OPT, CRV and IDS Models stored in the Paf file. For more details see adjust first guesses paragraph.

The task requires in input a flat field frame and a lamp frame, in its standard execution. For more details on peculiar cases see the Cookbook.

The task loads the Lines catalog appended by the organizer to the lamp frame and selects lines in the extraction range defined by the WLEN START end WLEN END keywords of the Grism table (contained into the flat frame).

Using the Models solutions contained in the header of the flat field frame, the task creates DS9 regions for the expected line positions and spectra edges and displays them on the selected lamp frame.

The user can check the current Models solutions and, in case they do not fit the emission lines pattern on the lamp frame or the spectra edges, he/she can “adjust” the solutions moving the DS9 regions on the real line position/edge and refitting new solutions. The goal of this interactive process is to obtain the final Models solutions that best matches the real lines/edges from a visual point of view. These final solutions will be considered as “first guess” for the more sophisticated recipes Create Master Flat and Create Master Lamp which compute the definitive spectra location and the wavelength solution, respectively.

Attention

The task uses lamp frame to check and adjust the solution contained into the flat frame. This solution is then applied to science frames to extract 2D spectra and wavelength calibrate them. For this reason, the position and the tracing of dispersed spectra in all of the three kinds of frames (flat, lamp, science) must match (see adjust first guesses for more details in case this condition is not verified).

Table 7.6 **The Adjust First Guess Recipe Summary**. In bold mandatory inputs
	MODS	LUCI
INPUTS	Through-slit flat or science frames	Through-slit flat or science frames
INPUTS	Through-slit lamp	Through-slit lamp
OUTPUTS	Input flat or science with updated Models	Input flat or science with updated Models

7.7. Create Master Flat

This recipe creates the Master Flat, a FITS file that contains:

the information on the slits location and on the spectral tracing on the raw frames;

the pixel to pixel variations image.

The recipe takes in input a (list of) through-slit flat(s) (or science frames, see cookbook). In MODS reduction, if requested, it subtracts from frames the bias level according to the BIAS_METHOD provided by the user (see Create Pix2pix Variation Image for more details on the methods). In LUCI reduction, if a Master Dark is provided, dark level subtraction is performed. After this, if the CLEAN_COSMIC entry is set to True, the recipe polishes frames from cosmic rays according to the COSMIC_RATIO and COSMIC_THRESHOLD settings, otherwise it directly combines the frames with the COMB_METHOD provided (see Create Master Bias for more details on cosmic rays detection/polishing and on the combining method). The combined image is corrected for bad pixels (see Create Master Dark section).

The accurate tracing of the spectra curvatures is performed fitting the edges of the spectra on the combined and polished frame. Starting from the position of the edge defined by the first guesses, the recipe computes the real location of the spectrum left edge. The main steps of the edges computing are:

SIPGI uses the solution of CRV Model obtained by the adjust procedure as first guess for the left edge location;
in order to increase the S/N of the spectrum edge, the recipe stacks together multiple rows. In this case, it moves along this edge and cuts thumbnails of the spectrum centered on the first guess value and with dimensions TRACING_BIN \(\cdot\)EXTRA_PIXELS (blue boxes in Fig. 7.4);
the software collapses these thumbnails along the dispersion direction and computes the edge profile (the red curve in the figure) in each thumbnail.
using the profile, SIPGI measures the edge position as the peak of the numerical derivative of the profile;
these edge values are used by SIPGI to fit a 1D polynomial which describes the spectra curvature along the dispersion direction.

_images/master_flat_left.png — Fig. 7.4 This figure shows how the master flat recipe works on the edge of the stacked flat frame. Blue boxes indicate the flat region collapsed by the recipe to maximize the edge signal that is shown in the inset box (red curve). The dimension of these boxes is set by the two parameters `TRACING_BIN` \(\cdot\)`EXTRA_PIXELS`. The center of the box is defined by the “first guesses” stored in the *Paf file* of the input flat.

Once fitted the left edge of each spectrum in the frame, the recipe determines their right edges by shifting the left ones by the slit length (quantity read from the header of the file). The right edge position can be refined by activating the entry RIGHT_EDGE_REF. In this case, the recipe uses the determined right edges as first guesses and proceeds as for the computation of the left edges. If the distance between the two edges so estimated is less than the slit length, the solution is saved, otherwise the recipe takes as solution the first guess. The tracing solution will be stored in a new extension of the Master Flat, the Extraction Table (EXR). This extension contains the grism table header taken from input files, the best-fitted solution of the tracing and the first guesses of the IDS Model (the last ones will be refined with the Create Master Lamp recipe).

The master flat tracing limits (in pixels) are defined by the keywords: LLEN LO and LLEN HI stored in the Grism Table header. In standard condition tracing limits include extraction range limits (see grism table note). However distortions can be such that this condition is no met (the Show Spectra Location utility allows the user to check this). This implies that the wavelength solution will be computed only within the tracing limits. To avoid this, user can push the tracing limits of ADJUST_TRACING_LIMITS pixels over the extraction limits.

After the spectra tracing, the recipe computes the pixel-to-pixel variation. If a Pix2Pix Variation Image is provided, the recipe substitutes the image of the combined flat with it. In this case the user can skip all the setting parameters in the box “Pixel to pixel variation”. If the Pix2Pix Variation Image is not provided, the recipe will compute the pixel-to-pixel variation on the combined through-slit flat following the same approach described in Create Pix2pix Variation Image. Starting from the tracing solution stored in the EXR extension, the recipe extracts the spectra and smooths their image using the method defined by SMOOTH_METHOD. This is a MEDIAN or AVERAGE smoothing box filter, with box size defined by the SMOOTH_BOX_SIZE parameter. Then it obtains the pixel 2 pixel map slit by slit, dividing the original spectra by their smoothed version.

Important

If the user decides to derive the pixel-to-pixel variation from through-slit flats, some caveats should be remembered:

for MODS data, this approach does not remove the different responses of the four quadrants (for more details see Create Pix2pix Variation Image);

flat field frames and science must be aligned.

If through-slit flats and science frames are not aligned (a quite frequent case in LUCI observations), the user can decide to perform the spectral tracing directly on science frames. In this case the pixel-to-pixel variation cannot be computed on input frames and the user has two possibilities: either he/she provides in input a Pix2Pix Variation Image or performs a mock Master Flat clicking the entry MOCK_FF. In the last option, the Master Flat image has all the values set to 1, thus the file stores the information on the tracing but not on the pixel-to-pixel variation.

Both the Pix2Pix Variation Image and the Create Master Flat recipe address only the issue of the pixel-to-pixel variation. None of them takes into account the large scales variations on the frames. Large scales variations along the dispersion direction will be compensated by the sensitivity function. Our recipes do not take into account for different responses of the detector along the spatial direction. For this reason, it is strongly recommended to plan observations with targets and telluric/standard in the same position in order to minimize the effects.

Table 7.7 **The Create Master Flat Recipe Summary**. In bold mandatory inputs
	MODS	LUCI
INPUTS	Through-slit flat or science frames	Through-slit flat or science frames
	Master Bias	Master Dark
	Pix2Pix variation image	Pix2Pix variation image
OUTPUTS	Master Flat	Master Flat

7.8. Create Master Lamp

The recipe starts from the information contained in the Master Flat, namely the spectra position and tracing, and refines on real data the wavelength solution predicted by the first guesses and stored in the EXR extension of the Master Flat. SIPGI allows the user to compute the wavelength solution both on arc lamp frames and science frames (see Adjust First Guesses). Science frames can be very useful in case instrument distortions on calibration frames do not match science frames ones.

The Master Lamp is fundamental to achieve the 2D spectra extraction: the solution contained into this frame allows to obtain 2D spectra corrected by the distortions and linearly calibrated in wavelength.

The recipe starts by cleaning up the input frames from the instrument signature. In MODS reduction, if requested, it removes the bias level accordingly with the BIAS METHOD (see Create Pix2pix Variation Image for more details). In LUCI reductions, if a Master Dark is provided as input, the recipe subtracts dark current from the input frames. Then it polishes the input frames for effects like cosmic rays, if the parameters CLEAN_COSMIC and CLEAN_NEGATIVE_PIX are activated (see Create Master Dark).

Starting from these polished frames, the recipe uses the solution contained into the EXR extension of the Master Flat to locate the 2D spectrum on the CCD. If the EXTRA_PIXELS parameter is greater than 0 the recipe refines the spectrum location within a searching window of EXTRA_PIXELS size. For each 2D spectra (i.e. for each slit), the CreateMasterLamp recipe cuts the 2D spectrum in N slices: one slice per pixel along the cross dispersion direction. Namely, if the 2D spectrum is 11 pixels large, the recipe will create 11 slices of the 2D spectrum (see Fig. 7.5).

_images/spectra_slicing.png — Fig. 7.5 This picture shows the slicing of the 2D spectrum performed to compute the wavelength solution on the 2D spectrum. The wavelength solution is computed in each 1px slice of the 2D spectrum.

The recipe moves along each slice following the spectral curvature defined by the Master Flat and it handles each slice like a 1D spectrum. Moving along the slice it computes the real line positions. Then it computes the slice wavelength solution using these measured lines positions.

More in details, for each slice the following iterative process is carried out:

the recipe picks the nominal lines positions from the input Lines catalog. It uses every line in the catalog in the wavelength range defined by the WLEN START end WLEN END keywords in the Extraction Table;

it uses the IDS Model (in the first iteration the one stored in the Master Flat) to have a first guess in pixel of the position of the given lines.

it measures the real line position by computing the barycenter of the flux detected in a window of dimension EXTR_WINDOW centered on the expected line position (see Fig. 7.6). If the line measurement fails (e.g line too flat, …) the line is flagged as lost, otherwise it is considered good;

it computes a new IDS Model using only good lines; all the lines deviating more than 2.5 sigma from the best-solution are flagged ad bad;

if no bad lines are found, the recipe exits the iterative loop;

if bad lines are found, the recipe repeats the loop using the new IDS Model as “first guess”. This procedure is repeated till the recipe finds the same bad lines in two consecutive loops;

in case the number of good lines is lower than twice the IDS polynomial order, the slice is flagged as invalid.

_images/wave_refinement1.png — Fig. 7.6 Zoom in of a single slice spectrum (grey dashed line) around the spectral region where a line is expected (vertical dotted line; from the *Line Catalog*). The position of the observed line (full blue circle) is derived as the barycenter of the spectrum within a window of center the first guess position of the lines and width defined by the EXTR_WINDOW parameter (red line). This barycenter position is compared to the expected position to derive the correction needed for the wavelength calibration.

Typically, different arc lamps (Xe, Ne, Ar) are acquired. In case the emission lines of a single lamp does not cover the whole spectral range of science frames,multiple lamp frames can be merged together. This could be done before running the Create Master Lamp recipe using the utility Merge Lamps. In this case a dedicated Lines catalog must be provided to the recipe.

The final solution is stored in the Extraction Table extension.

Table 7.8 **The Create Master Lamp Recipe Summary**. In bold mandatory inputs
	MODS	LUCI
INPUTS	Through-slit lamp or science frames	Through-slit lamp or science frames
	Master Flat	Master Flat
	Master Bias	Master Dark
OUTPUTS	Master Lamp	Master Lamp

7.9. Preliminary Reduction

This recipe takes in input a Master Flat and raw frames (optionally a Master Bias/Dark). It corrects input frames for bad pixels and cosmic rays, removes dark/bias level, and applies the flat field correction.

For MODS reduction, if requested, the recipe subtracts the bias level according to the BIAS_METHOD (see Create Pix2pix Variation Image for more details). In LUCI reduction, if a Master Dark is provided, the recipe subtracts the dark level. After this, it trims the prescan/overscan regions from input frames. If the keyword CLEAN_COSMIC is activated, it polishes raw frames from cosmic rays (see Create Master Bias for more details) while CLEAN_BAD_PIX activates the polishing of bad pixels.

The pixel-to-pixel variation will be corrected according to the image stored in the Master Flat primary header.

Note

If a mock Master Flat is provided, no pixel-to-pixel variation will be performed.

The output frames will have the same name of the raw frames with a final extension indicating the type of correction applied:

D for dark subtraction;

B for bias subtraction;

F for flat field correction (actually since Master Flat is mandatory, the “F” suffix will be always present, also in case of a mock Master Flat);

C for cosmic rays treatment.

As example, the file raw_file_name_BFC.fits is the raw_file_name.fits file bias subtracted, flat fielded and with cosmic rays treated.

The BFC file contains all the extensions of the input file, but the Primary extension now contain the polished frame. If ERROR_MAP is True, the BFC file will contain also an ERROR extension including the error on the polished frame. For more details on the errors propagation see Appendix C.

Table 7.9 **The Preliminary Reduction Recipe Summary**. In bold mandatory inputs
	MODS	LUCI
INPUTS	Raw science frames	Raw science frames
	Master Flat	Master Flat
	Master Bias	Master Dark
OUTPUTS	Pre-reduced frames	Pre-reduced frames

7.10. Reduce Observations

This recipe extracts 2D spectra wavelength calibrated, free of distortions, optionally sky-subtracted and flux-calibrated, determines their extraction profiles and extracts the 1D spectra.

The recipe takes in input the pre-reduced frames and a Master Lamp.

If CHECK_DITHER is activated, the first step performed by the recipe is to control whether the input frames are dithered one respect to the other before to move on. This is an important cross-check to be performed when ABBA or DAVIES method are used (see later).

7.10.1. Refining the spectral tracing & wavelength solution on science frames

The pipeline gives the possibility to refine the slits position and the wavelength calibration using the sky lines in the science frame. If the USE_LINES parameter is activated, the recipe checks on each science frame the spectral tracing and the wavelength solution stored in the Master Lamp. In details:

the recipe reads from the EXT table of the Master Lamp a series of sky lines that it will use for the two checks;

using the solution stored in the Master Lamp, it determines the edge position of the 2D spectrum at the position of these sky lines;

it recomputes the edge location starting from these positions and comparing them to the real ones in a window with width EXTRA_PIXELS;

it computes the shifts between the old and new edge positions (for all sky lines, in all slits), derives the median value of these shifts, and applies a shift along the cross dispersion direction (X axis) equal to the median value to the spectral tracing solution stored in the Master Lamp.

A similar procedure is followed to refine the wavelength calibration. The recipe computes the shift between the sky lines position predicted by the Master Lamp and the one re-computed on real data. The fitting procedure looks for the line in a window of LINE_WIDTH dimension, slice per slice, on each slit (like SIPGI does to create the Master Lamp). Then the recipe estimates the median value of all of these shifts and applies a shift along the dispersion direction (Y axis) equal to median value to the EXR solution.

Both adjustments are done twice, in the first iteration the recipe computes the shifts and in the second one it checks the new solution and applies a further (smaller) refinement.

Since the wavelength solutions are computed slice by slice the solution along the slit can be slightly jagged. If FIT_IDS_COEFFS is positive the recipe applies a smoothing polynomial fit (of order FIT_IDS_COEFFS) to the model coefficients, in order to reduce this effect. In case FIT_IDS_COEFFS is negative no smoothing is performed. This parameter could be also used to repair invalid slices (see Create Master Lamp). The IDS model of such a slice is extrapolated from the smoothed fit.

7.10.2. Extraction of 2D spectra

The extraction of the 2D spectra is a set of procedures that start from the pre-reduced frames (in which the dispersion runs on the y-axis), re-bins them and provide in output the 2D spectra rectified by distortion and wavelength calibrated (hereafter EXR2D spectra) (see Fig. 7.7 and for more details Appendix D). In the EXR2D spectra extraction, SIPGI also “flips” the spectra: the dispersion direction is aligned along the x-axis with wavelength increasing from left to right, and cross dispersion is aligned along y-axis. This product is stored in a new extension called EXR2D.

_images/2Dextraction.png — Fig. 7.7 Scheme showing the transformation carried out during the extraction of the 2D spectra. In this transformations, the spectrum is flipped, wavelength calibrated and corrected by distortions.

The recipe extracts the EXR2D spectra in the range [WLEN START, WLEN END] with a WLEN INC sampling. By default, these values are read from the Master Lamp. The user can change the extraction limits and the spectral sampling at run-time, using the parameters: WLEN_START, WLEN_END, and WLEN_INC.

SIPGI is able to extract EXR2D spectra before or after the sky subtraction, depending on the user’s requests (see next paragraph).

7.10.3. Sky subtraction

SIPGI can perform sky subtraction directly on raw frames and on EXR2D. Sky subtraction on raw frames is suggested only on spectra considered distortions-free, i.e. for raw frames in which lines of constant wavelength are perfectly orthogonal to the dispersion direction. The pipeline distinguishes between “distorted” and “un-distorted” spectra on the basis of the parameter SLIT_TOLERANCE. SIPGI defines “un-distorted” those spectra for which the maximum drift in pixels of the IDS solution along the cross dispersion direction is lower than the SLIT_TOLERANCE parameter. If the SLIT_TOLERANCE is negative, SIPGI will consider all of the slits as “distorted”.

SIPGI offers two main procedures for the sky subtraction. These procedures can work both on raw frames (for the “un-distorted” spectra) and on re-binned EXR2D (for the “distorted” spectra).

The SKY_METHOD: in this case the sky level is computed (and subtracted) row by row on the pre-reduced frames or column by column in EXR2D spectra using the method defined by the parameter SKY_METHOD. The choice is between:

SKIP the recipe does not subtract the sky level;

MEDIAN the recipe computes the median of input pixels values;

FIT the recipe performs a fit of POLY_ORDER order to the pixels values;

WFIT the recipe performs a weighted fit of POLY_ORDER order to pixels values.

In all the methods, the recipe excludes from the sky-level estimate the pixels interested by the objects signal. These pixels are identified using the objects identification procedure described below applied on not sky-subtracted frames.

The user can decide to also exclude from the sky-level estimate the SLIT_MARGIN pixels near the spectrum edges. If RSLIT_MARGIN is positive, the recipe excludes SLIT_MARGIN pixels from the bottom (left in EXR2D) edge and RSLIT_MARGIN pixels from the top (right in EXR2D) edge. In the sky level estimate, whatever the method, the recipe excludes from the computation all of the pixels occupied by (detected) sources. On the “un-distorted” slits, the recipe allows to control the minimum number of pixels to be used for sky subtraction through the parameter N_SKY_MIN.

The ABBA method: it is activated flagging the parameter ABBA. In this case the recipe removes the sky level subtracting from each frame the next dithered frame in the exposure sequence. The exposures must be dithered in order to avoid the source removal. Even in this case the A-B subtraction is performed on raw frames for “un-distorted” slits and on EXR2D spectra for distorted slits

Hint

If the PI wanted to apply ABBA on raw frames for all slit, he/she can set a huge insane value of SLIT_TOLERANCE, i.e. 1000.

In the LUCI Workspaces the DAVIES method is provided too [1]: it can be activated flagging the parameter DAVIES. If DAVIES is selected, all slits are treated as “distorted” by SIPGI. As the ABBA method, DAVIES is based on the subtraction of two dithered exposures (see Davies 2007 for more details).

If the DAVIES method is active two additional parameters can be used to control the sky subtraction: DAVIES_THERMAL_SUB and DAVIES_SCALE_FACTORS. If DAVIES_THERMAL_SUB is True SIPGI fits a black-body function to the thermal background in the sky spectrum, and subtracts it (see Davies 2007). If DAVIES_SCALE_FACTORS is True, the recipe computes a scaling factor for each vibrational transition band of the OH spectrum.

Caution

ABBA and DAVIES methods cannot be used at the same time.

SKY_METHOD and ABBA or DAVIES can be mixed according to the PI needs. In Fig. 7.8 and Fig. 7.9 two schemes of all the sky subtraction possibilities offered by SIPGI are reported.

_images/sky_subtraction_flow.png — Fig. 7.8 This diagram shows the sky subtraction flow in case the `ABBA` method is selected.

Caution

If DAVIES parameter is true all slits will be considered “distorted”.

_images/davies_subtraction_flow.png — Fig. 7.9 This diagram shows the sky subtraction flow in case the `DAVIES` method is selected.

In MODS reduction, if an Extinction Table is selected, the extinction correction is performed during the 2D extraction.

7.10.4. Fringing Correction

If FRINGING_CORR is True, the recipe corrects input frames for the fringing pattern.

Caution

SIPGI uses a very simple approach for fringing correction. Moreover, the procedure is applied on wavelength calibrated frames (EXR2D). Users are strongly encouraged to evaluate if the correction procedure is suitable for their purposes.

To correct for the fringing pattern a global residual image is computed from the sky-subtracted EXR2D frames. In each frame the areas covered by the (detected) objects are masked; these areas can be enlarged using the FRING_PIXELS parameter which determines the number of pixels that are added at each side.

If the FRING_INTERP parameter is False, the pixel in the objects areas will be ignored in the following combination step. If FRING_INTERP is True, a linear interpolation is computed in each column between the values at the edges of the areas; the pixel values of the objects area are then substituted with the interpolated values (that are used in the combination step).

The resulting objects-masked images are combined using a median filter to estimate the global residual image; this image is then subtracted from all the sky-subtracted images to remove the fringing pattern.

Important

It is not recommended to use the fringing correction in case of objects that overlap in all exposures (i.e. no or small offsets)

7.10.5. Flux Calibration & 1D extraction

Once the sky subtraction and the wavelength calibration have been carried out, the recipe moves to the flux-calibration. SIPGI offers the possibility to have:

1D spectra and EXR2D spectra not calibrated in flux (if no sensitivity function is provided);

flux calibrated 1D spectra and EXR2D spectra not calibrated in flux, if the sensitivity function is provided and the parameter CALIB_EXR2D is False;

1D and EXR2D spectra calibrated in flux if a sensitivity function is provided and parameter CALIB_EXR2D is True.

Table 7.10 Scheme of the Reduce Observations recipe output units on the basis of input provided. Column 1: Is the sensitivity function provided?; Column 2: setting of the `CALIB_EXR2D` parameter; Column 3: unit of the 1D spectra; Column 4: unit of 2D spectra.
Sens. Funct.	`CALIB_EXR2D`	Output 1D unit	Output EXR2D unit
No	False	ADU	ADU
Yes	False	erg cm^-2 s^-1Å^-1	ADU
Yes	True	erg cm^-2 s^-1Å^-1	erg cm^-2 s^-1Å^-1

The extraction of 1D spectra requires the estimate of the extraction profile. This procedure relies on the EXR2D spectra (sky-subtracted and flux calibrated following the user’s choices), and computes the profile following these steps:

for each row \(i\) of the EXR2D the recipe sorts the values of all pixels;

in case SPEC_WSTART and SPEC_WEND parameters are provided, the recipe sorts pixels between SPEC_WSTART and SPEC_WEND. This allows to select a specific spectral region for sources detection (see Fig. 7.10).

given this sorted array, the recipe computes the average (\(m_i\)) on the central fraction of the array defined by SPEC_FRACT. This allow to reject outliers.

\(m_i\) as a function of spectrum row is the extraction profile (see Fig. 7.11);

the mean and the std of the profile are estimated. All profile points above mean + std \(\cdot\) DETECTION_LEVEL are defined as possible object candidates.

_images/ProfileCreation0.png — Fig. 7.10 The Figure show the behavior of the `SPEC_WSTART` and `SPEC_WEND` parameters on an EXR2D frame. If these parameters are activated, the recipe collapses the 2D spectrum regions highlighted in the blue box.

_images/ProfileCreation3.png — Fig. 7.11 The Figure shows the extraction profile (blue line). The grey dashed line indicates the mean value of the profile, the yellow dashed line the threshold, and the red line the spectrum rows where a spectrum is expected to be (red line does not starts from yellow line due to the finite pixel dimension). In case the detection is confirmed (see `MIN_OBJ_SIZE` and `MAX_OBJ_SIZE` parameter), the recipe collapses all of the rows highlighted in red.

If the number of contiguous points is greater than MAX_OBJ_SIZE, the recipe tries to deblend the detection in more sources. If the number of contiguous points is between MIN_OBJ_SIZE and MAX_OBJ_SIZE, the recipe considers the detection as a single object; if the number of contiguous points is lower then MIN_OBJ_SIZE, it rejects the detection. If no detection is found, and REFINE_DETECTION is True, the recipe tries to identify sources with a parabolic fit of the extraction profile. All the information on the detected object are stored in the Window Table.

Once the detections have been identified, the recipe extracts the 1D spectrum. The recipe sums up the signal in all of the rows identified through the analysis of the extraction profile. It is to highlight that the 1D extraction procedure in SIPGI does not take into account any curvature in the object trace. For this reason, for a good extraction, it is crucial that the object signal is parallel to pixel rows(see Sec. 4.1).

Note

The user can activate an Horne extraction (Horne 1986) flagging the parameter HORNE_EXTRACTION only if the objects trace in 2D spectra are perfectly aligned with the pixels rows. If the object trace is not perfectly aligned with the pixel rows, the Horne extraction is compromised.

If ERROR_MAP is True, the recipes will propagate the errors for all the steps of the reduction (i.e. sky subtraction, extraction of 2D and 1D spectra) . For more details on the errors computation see Appendix C. If the ERROR_MAP flag was also activated during the preliminary reduction, the recipe also takes into account the contribution to the error derived from the preliminary reduction.

The output frames replace the pre-reduced input frames and their names are modified by adding an R as final extension. As example, the file raw_file_name_BFC.fits is replaced by the raw_file_name_BFCR.fits file which contains - among other things - the preliminary and fully reduced 1D and 2D spectra.

According to the choices made by the user different extensions will be appended to the output file:

Primary: the preliminary reduced frame.

ABBASUB: the sky values subtracted directly on the preliminary reduced frames by the ABBA method.

ABBASUB_ERROR: the error estimate on the ABBASUB extension.

ABBASUBRAW: the preliminary reduced frame ABBA subtracted

ABBASUBRAW_ERROR: the error estimate on the ABBASUBRAW extension.

SKYSUB: the sky values subtracted directly on the preliminary reduced frames.

SKYSUB_ERROR: the error estimate on the SKYSUB extension.

SKYSUBRAW: the preliminary reduced frame sky subtracted

SKYSUBRAW_ERROR: the error estimate on the SKYSUBRAW extension.

WIN: the window table. It contains geometry information about slits and about detected object (see the Show Window Table paragraph for more details)

EXR2D: 2D extracted spectra wavelength calibrated and corrected by distortions.

EXR2D_ERROR: the error estimate on the EXR2D extension.

SKY2D: the subtracted sky on EXR2D.

SKY2D_ERROR: the error estimate on the SKY2D extension

ABBA2D: the subtracted sky on by the ABBA method.

ABBA2D_ERROR: the error estimate on the ABBA2D extension

DAVIES2D: the sky subtracted by the Davies procedure.

EXR2D_FRINCOR: The EXR2D extension fringing corrected

EXR2D_FRINCOR_ERROR: the error estimate on the EXR2D_FRINCOR extension.

EXR2DFLUX: 2D extracted spectra flux and wavelength calibrated, corrected by distortions.

EXR2DFLUX_ERROR: the error estimate on the EXR2DFLUX extension.

EXR1D: 1D extracted spectra wavelength calibrated

EXR1D_ERROR: the error estimate on the EXR1D extension.

EXR1DFLUX: 1D extracted spectra wavelength and flux calibrated

EXR1DFLUX_ERROR: the error estimate on the EXR1DFLUX extension.

SPH2D: the Sensitivity-function applied on data (see next paragraph)

The EXR2D extension, in each of its declinations (e.g. EXR2DFLUX, EXR2D_FRINCOR), contains the 2D extracted spectra of all the slits except alignment and reference slits (see Fig. 7.12). The 2D extracted spectra of “scientific” slits are shown all together in the EXR2D extension and re-arranged from bottom to top of the image according to their increasing SIPGI slit number (for an example, see Fig. 7.10).

_images/exr2d_exr1d_slitnum.png — Fig. 7.12 An example of a raw imported frame (top panel) for a MODS MOS case, and of the EXR2D (bottom left panel) and EXR1D (bottom right panel) extension in its corresponding reduced frame. Green lines and numbers in the raw imported frames indicate the slits positions and their relative SIPGI slit numbers, respectively (see the Show Slit Position utility for more details). SIPGI does not enumerate reference and alignment slits.

In MOS data, SIPGI enumerates slits following the slit numbering order in the LMS/MMS file (see Fig. 7.12, top panel). SIPGI does not enumerate alignment and reference slits (by default square slits are considered reference slits).

SIPGI enumeration starts from #1. The relation between the SIPGI and the LMS/MMS slit numbering is described by the set of keywords LBT INS SLIT# ID in the header of the files. For example: LBT INS SLIT1 ID = ‘07_M31’ means: the first slit extracted by SIPGI (SLIT1) is the slit number 07 in the LMS/MMS file and its original object name is M31.

Note

Since the SIPGI slits numbers follow the slit numbering order of the LMS/MMS file, the order of the 2D spectra sequence in the EXR2D extension from the bottom to the top of the frame could not match the sequential order of 2D spectra from left to right in the imported frame (see Fig. 7.12). For example, if two slits located at the mask extremes have two sequential numbers in the LMS/MMS file, their 2D spectra will be close each other in the EXR2D, even if other “scientific” slits are present between them on the mask. To know the SIPGI slit number of each slit in imported frames, and hence identify the corresponding 2D spectra in the EXR2D extension, the user can use the Show Slit Position utility. For more details see also the Show Window Table section.

The EXR1D (or the EXR1DFLUX) extension is an image in which each row is a 1D spectrum. From the bottom to top, the image contains the 1D spectra of all the detected objects in the same order these objects are located in the EXR2D image, starting from the slit number 1 (i.e. the one at the bottom of the frame) (see Fig. 7.12).

Table 7.11 **The Reduce Observations Recipe Summary**. In bold mandatory inputs
	MODS	LUCI
INPUTS	Pre-reduced frames	Pre-reduced frames
	Master Lamp	Master Lamp
	Sensitivity Function	Sensitivity Function
	Extinction Curve
	“Sky” frames (for ABBA method)	“Sky” frames (for ABBA/DAVIES methods)
OUTPUTS	Reduced frames	Reduced frames

7.11. Create Sensitivity

This recipe creates the sensitivity function, i.e. the function which converts spectra from ADU to erg cm^-2 s^-1Å^-1. It takes in input the wavelength calibrated standard/telluric spectra (see Reduce Single Observations), compares them with the relative Stellar template/Spectro-photometric standard and computes a curve as function of lambda which perform the conversion.

Given the different observing strategy for MODS and LUCI observations, the recipe follows slightly different approaches for the two instruments.

7.11.1. MODS sensitivity function

The recipe takes in input the wavelength calibrated spectra of the standard star, the relative Spectro-photmetric standard and optionally the relative Star template.

Starting from the wavelength calibrated frames of the standard star (e.g. BFCR files sky-subtracted but non flux calibrated) the recipe takes the 1D extracted spectra of the star. Since reduced frames can have many detections, the recipe identifies the star detection as that with the highest counts and takes the relative 1D spectrum. However spurious detections could be mis-classified by the recipe, for this reason, it is good practice that the user identifies the star detection and provide it to the recipe. Once 1D spectra of standard star have been identified, the recipe stacks these spectra together using a median.

Then it reads the input Spectro-photometric standard. If the parameter MAG is True, the recipe assumes that the Spectro-photometric standard is expressed in magnitudes, otherwise it assumes the Spectro-photometric standard in fluxes units. The Spectro-photometric standards released with SIPGI cover the full MODS wavelength range. However, the user can provide a customized Spectro-photometric standard with a shorter wavelength coverage with respect to the data. Considering this, the recipe offers to the user the opportunity to provide in input a Stellar template (of the same spectral class of the observed standard star) that is used to extend the “shorter” customized Spectro-photometric standard. If a template is provided, the recipe re-bins the template to the same binning of the Spectro-photometric standard, scales it to the same flux level of the Spectro-photometric standard at the HEAD_JOIN (or TAIL_JOIN) point and substitutes the Spectro-photometric standard with the template at wavelength blueward (or redward) the HEAD_JOIN (or TAIL_JOIN). If HEAD_JOIN or TAIL_JOIN parameters are negative, the recipe will not join the template and the Spectro-photometric standard.

After this, the recipe interpolates the Spectro-photometric standard to match the sampling of the observed stars and divides the median spectrum of observed star for the Spectro-photometric standard (possibly extended) creating a raw sensitivity function. If EDIT_ABS is True, the recipe edits the raw sensitivity function replacing its values in the four wavelength ranges [6820-6960] Å, [7150-7350] Å, [7580-7750] Å, and [9280-9850] Å with straight lines that join the extremes of the ranges. Finally the recipe smooths the raw sensitivity. If the parameter METHOD is SMOOTH, the recipe smooths the raw sensitivity function using a median filter of size WIN_SIZE and then if the parameter MEAN_WIN_SIZE is greater than zero, it refines the smooth with a mean filter of size MEAN_WIN_SIZE. If the METHOD is SPLINE the recipe fits the raw sensitivity function with a B-spline. The kind of spline is defined by the parameter SPLINE_MODE, the function computes a B-splines of order k=SPLINE_MODE+1. If SPLINE_MODE is 0, a cubic spline (k=4) is computed (for more details see GSL documentation). The user can provide the break points of the spline with the parameter SPLINE_BREAK. Otherwise the recipe uses the parameter SPLINE_ORDER to uniformly break the spline (as described by the gsl_bspline_alloc help).

7.11.2. LUCI sensitivity function

Differently from MODS observations, the recipe that derives the sensitivity function for LUCI data is more complex and foresees a different approach for LS and MOS data. The first steps done by the recipe are aimed at identifying the spectral type and the magnitude of the observed star.

This can be done in automatic by the recipe, if TARGET_NAME=NULL. In this case the recipe retrieves the telluric name looking for the OBJECT keyword in the observed frames (e.g. HIP-56147). If FILTER=AUTO and STELLAR_TYPE=NULL, the recipe uses this name to extract from the Hipparcos Catalog the spectral type and the nominal magnitude of the star. In the Hipparcos Catalog provided by SIPGI there are many magnitude for a given telluric. The recipe automatically retrieves the nominal magnitude in the filter closer to the one of the observations (f_obs). Once magnitude and spectral type have been identified, the recipe scales the the Stellar template corresponding to the telluric spectral type to the nominal magnitude of the star in the filter f_obs.

Note

This approach requires that the OBJECT keyword of the input files is filled in with the Hipparcos name of the telluric (i.e. HIP-56147 not HD-99966), and that the observed telluric is present in the Hipparcos Catalog. If the OBJECT keyword is not properly filled in the user must provide it to the recipe as TARGET_NAME = HIP 56147 (or just 56147).

If the telluric is not present in the Hipparcos Catalog, the user must provide:

the STELLAR_TYPE

the MAG_VALUE

the FILTER in which the MAG_VALUE is estimated.

These operations produces the “normalized” template that is interpolated in order to match the sampling of observed data and then is compared with the observed star.

MOS LUCI sensitivity function

The standard practice for telluric observations in LUCI MOS program is to observe the telluric through the blue-most and red-most slit of the mask to maximize the spectral coverage. This means that to obtain a Sensitivity function which covers the whole spectra range, the recipe joins the spectra obtained in the two slits. For this reason in the MODS case the parameter RED_AND_BLUE must be True.

The joining procedure is driven by the parameter AUTO_RED_AND_BLUE. If it is True, the recipe tries to automatically recognize the blue-most and red-most 1D input spectra. If the entry is False, the user must provide input frames in even number n and the first n/2 frames of the input list are assumed to be the blue-most spectra and the other n/2 frames are assumed to be the red-most spectra. Once blue-most and the red-most spectra have been recognized, the recipe computes the two median spectra and join them at the point defined by the parameters LAMBDA_JOIN or PIXEL_JOIN:

if LAMBDA_JOIN is positive it defines the joining point in Å;

if LAMBDA_JOIN is negative, but PIXEL_JOIN is positive the joining point is defined in pixels;

if both the parameters are negative the recipe joins the two spectra at the middle point.

If the parameter RED_AS_MASTER is True, the recipe normalizes the blue-most spectrum to the red-most at the joining point and join the spectra, otherwise the recipe will normalize the red-most spectrum to the blue-most.

Once obtained the joined median observed spectrum, the recipe divides it by the “normalized” template obtaining a raw sensitivity function.

In the LUCI case the smoothing of the sensitivity function is driven by the parameter METHOD. If the METHOD parameter is FIT a polynomial FIT of order defined by POLY_ORDER if fitted producing the sensitivity function. If the METHOD is INTERP the input raw sensitivity function is divided in a number of intervals of length INTERP_LEN [Å]. For each interval the median value of the raw sensitivity function is computed, then a Cubic spline is interpolated between median values of these intervals. If the parameters LAMBDA_MIN and LAMBDA_MAX are zero, the spline is interpolated along all the points of the raw sensitivity function. Otherwise this range can be limited by the parameters LAMBDA_MIN and LAMBDA_MAX.

LS LUCI sensitivity function

For LS observations, the parameter RED_AND_BLUE must be False since just one slit is available. In this case, the recipe computes the median of all 1D spectra detected as the star. Then it divides the median spectrum by the “normalized” template obtaining a raw sensitivity function and smooth it like in the LUCI-MOS case

Table 7.12 **The Create Sensitivity Recipe Summary**. In bold mandatory inputs
	MODS	LUCI
INPUTS	Wavelength calibrated frames	Wavelength calibrated frames
	Spettro-photometric standars	Stellar template
	Stellar template
OUTPUTS	Sensitivity function	Sensitivity function

7.12. Combine Observations

This recipe combines 2D spectra contained in the BFCR / DFRC frames and yields:

stacked 2D spectra for each slit

1D calibrated spectrum for each detected object

error estimated spectra for each product.

It takes in input at least two BFCR / DFRC frames and optionally a sensitivity function. The recipe combines the EXR2D extensions of input frames or EXR2DFLUX in case they were calibrated. If USE_EXR2D_FRINCOR is True the recipe combines the EXR2D_FRINCOR extensions. Combine methods are defined by the parameter COMB_METHOD. Available options are: AVERAGE, MEDIAN and KSIGMA (for details on methods see Create Master Bias). EXR2D spectra can be normalized by their exposure time using the entry EXPTIME_NORM.

Before combining frames the recipe shifts them if offsets are provided.

7.12.1. Compute Offsets

The recipe offers different methods to compute offsets. They can be selected in the Combine Observation panel, which appears at the recipe launch. The choices are:

Get from Headers: the recipe computes offsets using RA and Dec information contained into the files header;

Compute from WIN: the recipe uses the OBJ_POS of the detected object (in the window table) to estimate relative shift between frames. If some slits contain multiple detection, the recipe evaluates the distance between these detections and checks that it does not change between frames within an error defined by MULTI_DET_STD. The user can set a minimum number of common detections by the parameter MIN_OBJ_OFFSET, as well as, the slits to be used for the computation (SLITS);

Compute from EXR2D: the recipe recomputes the objects detection on the EXR2D extensions as in the Reduce Observations. Then it performs the same computation of Compute from WIN.

The user can also compute the offset manually and provide them in the dedicated panel (see Combine Observations paragraph in the Cookbook section).

7.12.2. 1D extraction

The recipe recomputes object detection on 2D stacked frames as the Reduce Observations recipe does. If the parameter ALLOW_NO_DETECTION is True and no detection is found the recipe exits without errors.

In case the Sensitivity function is selected, and the input frames are not already flux calibrated the recipe calibrates the 1D extracted spectra.

Table 7.13 Scheme of the Combine Observations recipe output units on the basis of input provided. Column 1: Is the sensitivity function provided?; Column 2: unit of input EXR2D spectra; Column 3: unit of the 1D combined spectrum; Column 4: unit of 2D combined spectrum.
Sens. Funct.	Input EXR2D unit	Output 1D unit	Output EXR2D unit
No	ADU	ADU	ADU
No	erg cm^-2 s^-1Å^-1	erg cm^-2 s^-1Å^-1	erg cm^-2 s^-1Å^-1
Yes	ADU	erg cm^-2 s^-1Å^-1	ADU

Table 7.14 **The Combine Observations Recipe Summary**. In bold mandatory inputs
	MODS	LUCI
INPUTS	Reduced BFCR frames	Reduced DFCR frames
INPUTS	Offsets between exposures	Offsets between exposures
OUTPUTS	Combined frame	Combined frame

7.13. Other data reduction utilities

7.13.1. Science to Flat

This button allows the user to convert a science frame into a flat frame that can be read by the Adjust First Guesses recipe and by the create Master Flat. Sometimes LS/MOS flats are not perfectly aligned with science frames. This implies that if the user adopts a flat frame to refine the OPT and CRV Models, he/she obtains a Master Flat in which the Extraction table does not well describe the location of the spectra in science frames. In this situation, the user can decide to use the same science frame as flat.

Important

If the user uses such kind of flat to create the Master Flat, the Master Flat should be MOCK, unless a dedicated Pix2Pix Variation Image is provided.

7.13.2. Science to Lamp

This button allows the user to convert a science frame into a lamp frame that can be used as input lamp by the Adjust First Guesses and to create a Master Lamp. This facility is provided to compensate misalignment between science and lamps (science to flat).

Bright objects in the science frames can affect the solution of the Master Lamp: the fit of the sky line positions can be biased by the object. To avoid this problem, the user can select two science frames with the object in 2 different positions and SIPGI will create a lamp frame object free.

By default the task appends to the output lamp the Lines catalog found in the first input frame. A custom Lines catalog can be provided to this recipe.

Table 7.15 **The Science to Lamp Utility Summary**. In bold mandatory inputs
	MODS	LUCI
INPUTS	Through-slit science frames	Through-slit science frames
INPUTS	Line catalog	Line catalog
OUTPUTS	Lamp frame	Lamp frame

7.13.3. Merge Lamps

It allows to sum together different lamps and to append to the resulting file a Lines catalog provided by the user. This task is useful to create a lamp which covers the whole lambda range to use in the adjust first guesses step.

Table 7.16 **The Merge Lamps Utility Summary**. In bold mandatory inputs
	MODS	LUCI
INPUTS	Through-slit science frames	Through-slit science frames
INPUTS	Line catalog	Line catalog
OUTPUTS	New lamp frame	New lamp frame

7.13.4. Append Table

It allows user to append: Lines catalogs, CCD Table or Grism Table to files. In case table already exists it will be overwritten without any warning.

7.13.5. SIPGI To Molecfit

During his/her own reduction, the user could decide to correct his/her spectra for atmospheric absorption features using the software Molecfit (see ESO).

SIPGI offers three utilities to interact with Molecfit: SIPGI to Molecfit, Apply Molecfit Tra and Revert Molecfit Tra

The SIPGI to Molecfit utility “exports” reduced data out of the SIPGI environment so that the user can process them with Molecfit. In addition to this, this utility creates all the necessary files to run the standalone version of Molecfit 2.0.1. In particular:

the param file. SIPGI fills in the configuration parameters file with all the information that can be acquired by the FITS header of the reduced file. The user must fills all the other entries on the basis of his/her own reduction purposes.
the “exclude_p” file. This is an empty file that the user can customize;
the “exclude_w” file. This is an empty file that the user can customize;
the “include” file. This file contains the regions expressed in μm to be processed by Molecfit. By default, SIPGI fills this file with the full default extraction range.

We remand to Molecfit documentation for more details on the software functioning.

7.13.6. Apply Molecfit Tra

Once the user has estimated the transmission function with Molecfit, this utility allows user to apply it to his/her spectra (1D and 2D, if present) in SIPGI.

7.13.7. Revert Molecfit Tra

Remove the Molecfit transmission function previously applied and restore the frame to their original form.