3. The SIPGI reduction cookbook

The reduction procedure within SIPGI consists in the following steps:

Each step is described in detail in the following sections.

Important

This SIPGI release is able to handle data acquired with the following instrument configurations:

  • For MODS data with 1x1 binning mode: any configuration with the grism/dichroic/mask listed in the table below. For MODS data with 2x2 binning, only LS data are supported.

    Grism/Dichroic

    Mask

    G670L

    G400L

    G670L_dual

    G400L_dual

    MOS

    LS5X60X0.3

    LS5X60X0.6

    LS5X60X0.8

    LS5X60X1.0

    LS5X60X1.2

    LS5X60X2.4

    LS60X1.0

    LS60X5.0

  • For LUCI data any configuration with the grism/mask/camera listed in the table below.

    Grism

    Mask

    Camera

    150Ks

    200H+K

    210zJHK

    MOS

    LS

    N1.8

3.1. Create a new Workspace and a new Project

Before starting with a new data reduction in SIPGI, the user must group the raw files into a directory (e.g. /home/user/raw_data, preferentially on internal disk). Ingestion of raw files in SIPGI from external hard disk takes considerably longer time.

Launch SIPGI and before creating a new Workspace and a new Projects, the user has to set some general preferences. Click on the SIPGI->Preferences menu in the Starting and Setting Area and set the first (see First Alternative Fits Viewer), and the second alternative FITS viewer (see Secondary Alternative Fits Viewer) as well as the log level desired (see the log level note).

Open the workspace creation panel selecting the Workspace->New menu in the Starting and Setting Area; in the panel insert the Workspace name (e.g. MODS_reduction) and select the Instrument. If a Workspace for the same instrument already exists, the user can copy the setting parameters from this Workspace using the Copy setup from entry.

Click on the Create button and the new Workspace is created.

In the Workspace just created, open the project creation panel selecting Project->New menu in the Starting and Setting Area. In this new panel define the Project name (e.g. BLZ_reduction) and insert an existing and empty Project directory (e.g. /home/user/BLZ_reduction), that is the directory where all the SIPGI imported files (see below) and pipeline reduction products will be stored. This directory must exist before the creation of a new Project (i.e. SIPGI will not create a directory with the name the user gives to the Project), and it must not be the raw files directory.

3.2. Files import

To be processed by SIPGI, raw FITS files must be ingested by the built-in data organizer (see Data organization). During the importing procedure, SIPGI creates a copy of raw files in the Project directory, and - among other things - rotates and flip the data and appends to them all the necessary auxiliary system files (for more details see The importing procedure).

Important

During the import procedure, SIPGI creates a copy of raw files. Before starting the ingestion, the user must check to have enough space on his/her disk for this copy.

3.2.1. Add mask definition files for MODS MOS data

MODS MOS frames do not contain information about mask geometry. This information is fundamental to locate the 2D spectra on the frames and it is required by the importing procedure. If data to import are MODS MOS files, the mask file must be added to the auxiliary files before the data import starts. Open the Custom Auxiliary Files Manager clicking on the Custom Auxiliary Files button at the bottom of the The Data Manager Area. Click on Add Auxiliary File button. Upload the mask file in the Auxiliary File panel and fill in all the entries. Please, take care to insert the Mask name. This field must match the MASKNAME keyword contained in the FITS header of the raw files to import (see Fig. 3.1).

_images/mask_file.png

Fig. 3.1 In figure is reported an example of the Auxiliary File panel for the upload of a mask file. Red box indicates the Mask name. This entry is mandatory and it must be filled with the same MASKNAME keyword in FITS header of the raw files.

Note

The Add Auxiliary File button allows the user to add all his/her customized auxiliary files (e.g. customized stellar template, customized Spectro-photometric standard, customized Lines catalogs, etc…). Once a new file is uploaded, it is ingested in SIPGI that can use it for reduction purposes. The Edit File, Rename File and Delete File buttons in the Custom Auxiliary Files Manager window allows user to edit the specifications of the selected customized auxiliary file, to rename this file or to delete it from SIPGI. In addition to this, the View/Hide System Files allows the user to see all the auxiliary system files. The FileType drop-down menu at the top of the Custom Auxiliary Files Manager window permits to select the type of auxiliary files to show in the central panel.

3.2.2. Raw frames import

Note

SIPGI imports only not compressed FITS files.

Clicking on the Import button at the bottom of the SIPGI window, the ImportRawFiles window appears. The Select directory button allows user to select the directory with raw files (e.g. /home/user/raw_data) and import all of the FITS files it contains; the Select Files button allows user to select specific files to import.

Clicking on the Start Import button, SIPGI starts the ingestion of the raw files processing them with the built-in data organizer (for more details see the The importing procedure).

Caution

The ingestion of raw data can take a while depending on the number of files to import. Look at the log to follow the import execution. Please, do not stop the import before the process is finished.

The import procedure creates a number of Datasets and Reduction Units listed in the main panel of the data manager area.

Raw files import is an incremental procedure; new raw data can be imported as they become available. For example, if data were acquired in different nights, the user can ingest and reduce the data of the first nights and then add other files as they arrive. The exposure sequence number provided by SIPGI (see the renames the file paragraph) to these new imported files follows the sequence of files already ingested.

Note

Data of the same target observed in different periods could have different OBJECT keyword (e.g. due to a different target name in the Observing Blocks). The import procedure will split these data in different Datasets. To unify the data in the same Dataset and reduce them together, click on the Datasets with the right button of the mouse, and rename them with the same name.

SIPGI will not digest and categorize raw files with incorrect entries for the keywords listed in Appendix B. The files not correctly imported will be stored in the Unclassified panel in the Data Manager Area.

3.3. Bad Pixel Image

The first step in the reduction process is to create the bad pixel image. The Create Bad Pixel Image recipe is able to automatically detect 2 kinds of bad pixels: dead pixels and hot pixels. Slitless flats are used to detect dead pixels and dark frames are used to detect hot pixels. If just one type of file is provided, the bad pixel image will store only the relative bad pixels.

The created bad pixel image will be available in the Master Frame Panel of the current Reduction Unit.

3.3.1. MODS Bad Pixel Image

Select the Reduction Unit with slitless flats. Check the parameter values of the Create Bad Pixels Image recipe going into Parameter Files->Create Bad Pixel Image menu in the Starting and Setting Area. To run the recipe, click on the Run button. A new window appears. Select the slitless flats to be used in the Reduction Unit and then click on the Add files button in the recipe pop-up window.

Caution

Take care to use correct input files, e.g. MODS1 slitless flats for the bad pixel map of MODS1 data. The drop-down filtering menu on top of Frames Panel of the Reduction Unit can help in the selection of proper inputs.

Click the Run Recipe button. A new pop-up window will appear with a suggestion for bad pixel image name. A personal name can be provided. It is good practice to give a name that preserves the filter/dichroic name (e.g G400L, G670L, G400L_Dual, G670L_Dual) and the LBT arm (e.g. MODS1, MODS2). This because, e.g., the bad pixel map for MODS1 and MODS2 will be stored in the same place (i.e. the Generic Master Frames and Calibration Files of the Project, see Append Bad Pixels Image to data) and a clear name can help in distinguishing them. The pop-up window allows also to overwrite an existing bad pixel map with the same name.

Check the result viewing it with one of the two FITS viewer selected and, if it is not satisfactory, modify the recipe parameters and re-launch the recipe.

The user can save the customized recipe setting parameters using the Save button in the parameters panel. If the user is satisfied with the parameters stored, he/she can run the recipe directly from the Create Bad Pixels Image button in the Reduction tab on the left.

3.3.2. LUCI Bad Pixel Image

To create bad pixel image for LUCI data, the procedure is the same as MODS data with the possibility to also load dark frames to identify hot pixels. Once slitless flat frames have been uploaded in the DEAD PIXELS FRAMES list panel, dark frames can be uploaded in the HOT PIXELS FRAMES list selecting them from the appropriate Reduction Unit.

Important

Once the bad pixel map is created, it must be promoted from the Master Frame Panel of the Reduction Unit in which is created to the Generic Master Frames and Calibration Files of the Project. This operation will make this data product available to other Reduction Units of the Project. To do this, click on the bad pixel map with the right mouse button and select Share as auxiliary from the context menu. This will move the bad pixel map from the Master Frame Panel to the Generic Master Frames and Calibration Files. If the user wants to delete/rename a file in the Generic Master Frames and Calibration Files panel, she/he must click on the Custom Auxiliary Files button, must select the file and perform the desired operation (see this note for more details.).

3.3.3. Append Bad Pixels Image to data

Once the user has a satisfactory bad pixels image, he/she must append it to data. This allows the reduction recipes to have information on the pixels to be corrected for each frame.

Check the parameter values of the Append Bad Pixels Map recipe going into Parameter Files menu. Select all the data to polish, the bad pixel map itself and click the Append Bad Pixels Image button in the Reduction Tab.

The bad pixel map must be appended to all the frames that user wants to clean up.

3.4. Create Master Files

The Master calibration frames are: Master Bias, Master Dark, Master Flat and Master Lamp. Together with the bad pixel maps and sensitivity functions, the master files are those files that are applied to all the scientific frames.

As for the bad pixel map, after creating the Master frames they appear in the Master Frame Panel of the Reduction Unit in which are created. In order to use them, the user must promote these file to the Generic Master Frames and Calibration Files (see the note).

3.4.1. Create Master Bias (MODS only)

Master Bias is used to subtract bias level from MODS science frames.

The creation of a Master Bias is not mandatory in SIPGI. Actually standard practice for MODS data is to subtract the bias level using the prescan/overscan region. If the user is satisfied with this choice, he/she can skip the creation of the Master Bias.

If the user wants to subtract bias level using bias frames, he/she must create a Master Bias. Check the parameter values of the Create Master Bias recipe going into Parameter Files. After this, select the bias frames and click the Create Master Bias button.

Caution

Frames used to create the Master Bias must have the same ARM and READMODE of the science frames (e.g. 8kx3k for MODS data, see MODS manual).

A pop-up window appears asking for the Master Bias name and the possibility to overwrite an existing file with the same name. It is good practice to give a name that preserves the the LBT arm. Run button launches the recipe.

3.4.2. Create Master Dark (LUCI only)

Master Dark is used to remove the dark current level plus bias level from LUCI science frames.

To create a Master Dark, check the parameter values of the Create Master Dark recipe going into Parameter Files menu. If the CLEAN_BAD_PIX keyword is activated, append the appropriate bad pixel map to dark frames (see Append Bad Pixels Image to data).

The user has to select the dark frames and click the Create Master Dark button.

Caution

Take care to use dark frames with the same ARM, DIT and READMODE of science frames to reduce.

A pop-up window appears asking for the Master Dark name and the possibility to overwrite an existing file with the same name. It is good practice to give a name that preserves the READMODE (e.g MER or LIR) and the LBT arm. Run button launches the recipe.

3.4.3. Pixel to Pixel Variation Image

This image is used to correct for the pixel-to-pixel variation in raw frames. The recipes combines together input slitless flats and removes from this combined image the large scales fluctuations. The resulting image only contains the pixel-to-pixel variation (see Create Pix2pix Variation Image for more details). This step is not mandatory.

The first step to create a Pixel to Pixel Variation Map is to check the parameter values of the Px2Px Variation Image recipe going into Parameter Files menu. If the CLEAN_BAD_PIX keyword is activated, append the appropriate bad pixel map to the slitless frames (see Append Bad Pixels Image to data). After this, select slitless flats and click the button Create Px2Px Variation Image. A pop-up window appears asking for the Pixel to Pixel Variation Image name and the possibility to overwrite an existing file with the same name. It is good practice to give a name that preserves LBT arm. Run button launches the recipe.

3.4.4. Adjust First Guesses

The Reduction recipes in SIPGI rely on a set of models which describe the spectra location on the CCD (OPT Model), the spectra curvature (CRV Model) and the inverse disperse solution (IDS Model). The coefficients of these models are stored in the PAF files. These models provided with SIPGI and appended to raw files are estimated on the basis of instrument characteristics, but may not be exactly representative of the current data because of, e.g. changes in the optical distortions on a night base or masks not perfectly aligned in the FPU. For this reason, they must be checked and if needed, re-adjusted. This interactive task allows to control the Paf file stored in the input through-slit flat on the input lamp frame.

The first step in this contest is to verify whether calibration files (e.g. through-slit flats and lamps) are aligned with scientific frames (see the Attention note in the Adjust First Guesses recipe). This can be easily done opening a flat, a lamp and a scientific frame with ds9 and selecting in it Frame->Frame Parameters->Tile->Rows.

3.4.4.1. Through-slit flat, lamp and scientific frames aligned

Select a Reduction Unit with the through-slit flat and click on the flats one desires to use for the Master Flat creation. Then click the Adjust First Guess button in the Reduction Tab. A new panel opens, showing on the left the list of flat fields selected. Browse in the Reduction Unit containing the lamp frames with the same grism/filter/dichroic/arm of the flats and select one of them. Click on the Add lamp files in the Adjust First Guess panel.

The Adjust Grism Table button permits to see and edit the Grism tables. This task allows the user to check and adjust the Grism Table. As example, in LUCI data with central wavelength different from the nominal one, the user can update the central wavelength (and extraction limits accordingly). Optionally, new sky lines can be added to the Grism Table with the the Add Skyline button.

Once the Grism Table is set, choose one through-slit flat and one lamp frame in the panel clicking on them. Then start the adjust procedure selecting the Adjustment type and clicking the Run button. It is strongly recommended to start with Shift only adjustment type, and then continue with Rotate and Complete Adjust type.

Note

Standard observations provide user with different lamp frames. Each lamp has their own emission lines. In order to have a better sampling of the wavelength solution it could be useful to merge lamps together. To do this, before running the Adjust First Guess task, select all the different lamps, the suitable Lines catalog from the Generic Master Frames and Calibration Files and then click the Merge Lamps button in the Reduction tab. A pop up window appears asking for the name of the merged files. Click Run. The resulting sum file is shown in the lamp Reduction Unit. The user can provide this file as input to the Adjust First Guess task.

3.4.4.1.1. Shift first guesses

This step corrects the OPT model offset adjusting the X and/or Y shifts of the first guesses with respect to data in use. Select Shift Only as Adjustment type, and push the Run button.

The selected lamp frame is displayed in a new ds9 window, with some green regions superimposed that show the first guesses of the spectral tracing and of the wavelength calibration following the Paf file stored in the input flat.

The green vertical lines indicate where the left edge of the spectrum is supposed to be, and the horizontal lines show where the arc lamp lines (those in the Lines catalog appended to the lamp frame and within the extraction limits) should be. Clicking on one of the blue rectangles, and moving it, all such regions will move accordingly.

In this phase the goal is to move green lines (using the blue rectangle) in order to match just the left edge of the central slit and the position of the most central lines. The center of the green horizontal lines should be positioned roughly on the center of the corresponding frame lines. It could happen that edges of other lateral spectra and/or other arc lines are still displaced but this will be fixed in the next steps. When satisfied, click Recompute on the question panel. The recipe recomputes the OPT Model using these refined input constraints and will update the green region on the image. The user must check that the new models have stored the changed suggested. If this is not the case, the user can iterate the procedure till he/she is satisfied with the newly computed first guesses, and then exit the loop by clicking on Done.

Attention

Save the first guesses clicking on the Save button on the Adjust First Guess panel. This will update the Paf file in the selected flat file.

3.4.4.1.2. Rotate

Distortions produce curved spectra, and the CRV Model is the model which describes this curvature. This step adjusts the first coefficient of the curvature model (the linear term). Choose Rotate as Adjustment type, and push Run.

The lamp image is displayed in ds9; the green vertical line shows the position of the edge of the spectrum according to the updated Paf coefficients. Click on the top end or on the bottom end of the green line to rotate it such that it follows the edge of the spectrum. If necessary, the user can move the vertical line near the edge of the spectrum to facilitate the tracking.

When satisfied, click Recompute on the question panel. A new CRV Model is computed, and the result is shown on the same image. It is possible to iterate the procedure, or exit the loop by clicking on Done when the newly computed first guesses are considered satisfactory.

Attention

Do not forget to push the Save button to store changes into the Paf file in the input flat file.

3.4.4.1.3. Complete adjustment

The complete adjustment is strictly related to the lambda calibration of the spectra and it corrects the IDS Model. Choose Complete Adjustment as Adjustment type and push Run.

The lamp image is displayed on ds9 again; the green lines show where the arc lamp lines should be according to the IDS model stored into the flat Paf file.

By clicking on one green line, you can select it and move it so that it falls exactly on the corresponding arc line. The scope of the game in this phase is to have all the green lines as near as possible to the real corresponding arc lines. All the shifts in the x directions will be ignored. It is possible to select more than one region by multiple selection facilities provided by ds9 and move them solidly. A red region with ALL lines is also shown, to help the identification of very close lines (the user can use it as a ruler and move such red region around, to help the eye in the most difficult cases). When satisfied, click Recompute on the question panel. A new set of first guesses is computed, and the result shown on the same image. It is possible to iterate the procedure, or exit the loop by clicking on Done when the newly computed first guesses are considered satisfactory.

In this step, the user can activate the entry View lines label which will show for each displayed line its wavelength.

Important

In the Complete Adjustment:

  • the user can delete green lines to exclude them from the fit;

  • do not mismatch lines;

  • push Recompute only when all slits of the whole mask are in the proper position. The recipe computes a global model and it will fit all the slits; a partially adjusted mask produces a no-sense fit and all the work will be lost;

  • In case some green lines fall out of the frame (too blue or too red) during the adjustment, they can be deleted and would be excluded from the fit;

  • in LS data, the curvature of emission lines can be so important that the straight green lines can intercept two close emission lines. This can lead to non accurate wavelength calibration. In this cases the user can split the line in Split slit part (see Fig. 3.2).

_images/split_complete.png

Fig. 3.2 The figure shows the behavior of the Complete Adjustment if Split slit =5.

Once the user is fully satisfied with the first guesses, he/she can save the first guesses pushing on the Save button.

Note

In the last step it is important to save the new Paf files in all of the through-slit flat of the list. To do this, the user can choose All frames in the menu Save results into.

3.4.4.2. Lamp frames not aligned with through-slit flat and scientific frames

In case that through-slit flat and science frames are aligned with each other but not with the lamp frames, the user can use science frames as lamp. In this case the spectral tracing is performed directly on the science frame bypassing the issue of the shift, and the wavelength calibration is performed on sky lines.

To do this, the user must convert a science frame into a lamp frame using the Science to Lamp button in the Reduction Tab (see Science to Lamp). Select a raw science frame and a suitable sky-line catalog from the Generic Master Frames and Calibration Files and click on the Science to Lamp button. This creates a new file (hereafter science-lamp frame) with the same name of the science frame plus the prefix lp. This new frame will appears in the same Reduction Unit of science frames.

Once the science-lamp is created, the user can use this file as lamp in the adjust procedure following the same steps described in the previous section.

3.4.4.3. Through-slit flat and lamp frames not aligned with scientific frames

If neither through-slit flats nor lamp frames are aligned with science frames, the user must use science frames both as flat and as lamp. As in the previous case, the user must use a science frame as lamp in order to define suitable first guesses for the spectral tracing. However if through-slit flats are shifted with respect to science frames, it makes no sense to use them as inputs in the Create Master Flat recipe. To overcome this limit, the user can convert science frames in flat frames using the Science to Flat button in the Reduction Tab (see Science to Flat). Select a raw science frame and click on the Science to Flat button. This creates a new file with the same name of the science frame plus the prefix ff (hereafter science-flat frame). This new frame will appear in the same Reduction Unit of science frames.

Once the science-lamp and the science-flat are created, the user can use these files in the adjust procedure following the same steps described in previous section. Clearly, in this case, Master Flat will not include the pixel-to-pixel variation map, unless a Pix2Pix Variation Image is provided (see later).

3.4.5. Master Flat

The creation of a Master Flat is mandatory in SIPGI. It contains two information: the location of 2D spectra on the CCD and the pixel-to-pixel variation.

The pixel-to-pixel variation is computed by the Create Master Flat using the input through-slit flats or using the Pix2Pix Variation Image, if provided.

If the user is reducing MODS data, it is strongly recommended to use a dedicated Pix2Pix Variation Image in order to take into account the different gains between quadrants. This applies also with not-aligned science and through-slit flat frames.

To create a Master Flat, check the parameter values of the Create Master Flat recipe going into Parameter Files menu. If the CLEAN_BAD_PIX keyword is activated, append the appropriate bad pixel map to the through-slit flat frames (see Append Bad Pixels Image to data). Select the through-slit flat frames (or science-flat frames) to combine, the optional Pix2Pix Variation Image and the optional Master Bias if MASTER subtracting method has been selected (only for MODS reduction). Click on the Create Master Flat button. A pop-up window will appear asking for the Master Flat name and the possibility to overwrite an existing file with the same name. It is good practice to give a name that preserves the the LBT arm. Run button will launch the recipe.

A Master Flat is created and listed in the Master Frame Panel of the Reduction Unit. The user must promote it to the Generic Master Frames and Calibration Files.

As stated in Create Master Flat, the Master Flat takes into account for the pixel-to-pixel variation, but not for large scales variations.

3.4.6. Utilities to check Master files

3.4.6.1. Show Spectra Location

SIPGI offers the possibility to check the Master Flat with the utility Show Spectra Location in the Analysis Tab. For more details on its functioning see the Show Spectra Location description.

Select one of the through-slit flats used to create the Master Flat and the Master Flat itself. Click on the Show Spectra Location button. A ds9 window appears showing the through-slit flat with the fitted tracing highlighted (see Show Spectra Location for more details).

A Master Flat is considered satisfactory when the tracing follows the spectra edges. This condition is necessary for a proper extraction of 1D spectra.

Note

Check that the extraction limits (indicated with bold black “lines” in the bluest and reddest part of the spectrum) are within the tracing limits (white vertical “lines”, see Show Spectra Location for more details).

The utility Show Spectra Location can be used also on science frames. This allows user to directly check the spectral tracing on science raw frames.

3.4.6.2. Show Lambda Calibration

Another utility SIPGI provides to check the Master Flat (and Master Lamp) is the Show Lambda Calibration in the Analysis tab (see Show Lambda Calibration for more details). The Show Lambda Calibration utility allows to check the IDS model stored in the input Master Flat.

Select one of the lamp or science-lamp frame and the Master Flat. Click on the Show Lambda Calibration button. A ds9 window will appear showing the input frame.

A satisfactory Master Flat has the first guesses of the IDS model (indicated by thin solid horizontal lines) that provides good prediction of the lamp lines position.

3.4.7. Master Lamp

The creation of a Master Lamp is mandatory in SIPGI. The Master Lamp is the frame which stores in the EXR table extension the best wavelength solution.

The Create Master Lamp recipe starts from the first guesses of the wavelength solution stored in the EXR extension of input Masters Flat and refines the models on real data. The best-fitted solution is stored into the EXR table of the Master Lamp.

The recipe works both on real lamp frames and on science-lamp frames (see Lamp frames not aligned with through-slit flat and scientific frames).

To create a Master Lamp, check the parameter values of the Create Master Lamp recipe going into Parameter Files menu. If the CLEAN_BAD_PIX keyword is activated, append the appropriate bad pixel map to input frames (see Append Bad Pixels Image to data).

Select the desired input frame(s) (lamp frames or science-lamp frames), select the appropriate Master Flat and the optional Master Bias if MASTER subtracting method has been selected (only for MODS reduction). Click on the Create Master Lamp button. A pop-up window will appear asking for the Master Lamp name and the possibility to overwrite an existing file with the same name. It is good practice to give a name that preserves the the LBT arm. Run button will launch the recipe.

A Master Lamp is created and listed in the Master Frame Panel of the Reduction Unit. The user must promote it to the Generic Master Frames and Calibration Files.

3.4.8. Utilities to check the wavelength calibration

Once a Master Lamp is created, the user can visually check the result using the Show Lambda Calibration utility.

Select one of the lamp or science-lamp frame and the Master Lamp. Click on the Show Lambda Calibration button. A ds9 window will appear showing the input frame. If no input frame is provided, the utility displays the solution directly on the master frame image.

In this check the user must control that all the thin horizontal “lines” are placed on the relative emission lines and that they are fairly continuous. Since the wavelength solutions are computed slice by slice (see Create Master Lamp for more details), a one-pixel shift between the points of each thin “line” is allowed due to interization issue. Fluctuations bigger than one pixel indicate a poor calibration in that spectral region.

Another useful check is to look at the black bold lines which delimit the extraction range. The Create Master Lamp recipe derives the wavelength solution using all the input lines. This solution is then extrapolated up/down to the red/blue extraction limits. The Show Lambda Calibration utility will show also the expected position of the extraction limits. If the extraction limits “lines” are blurred, it means that the best-fit wavelength solution is poorly constrained in the region between the bluest/reddest calibration line and the relative extraction limit.

3.4.8.1. Check Lambda Calibration

Once the visual inspection is considered satisfactory, the user can quantify the quality of the wavelength calibration using the utility Check Lambda Calibration in the Analysis tab (see Check Lambda Calibration description for more details on the utility and its output).

Click the Master Lamp and than click the Check Lambda Calibration button. Look at the log to see the output. In case the result is not satisfactory, re-perform the Create Master Lamp and/or the Adjust First Guess steps.

3.4.8.2. Plot Lambda Calibration

Select a Master Lamp and than click the Plot Lambda Calibration button (see the Plot Lambda Calibration utility for more details). A new panel will appear. It could be necessary to expand the panel to visualize the plots.

The graphical device has three tabs. In the One slit tab, the top panel shows the offset between the real wavelength of the line and that associated by the calibration for all the lines in the Lines catalog. Blue points indicate good points (see Create Master Lamp), red points are those rejected by the fitting procedure (bad lines, see Create Master Lamp), while magenta triangles are lost lines (see Create Master Lamp). The latter points are not included in the fit. The View only good points button displays only the good points in the plot.

The displayed points refers to the Slit and Column indicated in the Browse panel. The default Column is the central one. The user can change slit and position within the slit using the Previous and Next buttons or scrolling the cursor on the relative bars. Below the top plot a summary of the quality of the fit is reported: the number of good lines, the mean value of the offsets and the rms of the distribution, considering both all of the points and only good points.

The utility offers the opportunity to exclude good points from the fit and repeat the fitting procedure. A point can be excluded by clicking on it with the right button of the mouse. Its color/shape will change from blue point to red diamond. Click the Refit button on the right of the panel. A new fit will be performed, and the mean and rms values will be updated. The excluded point is shown as black triangle. If the new fit is considered worse than the original one, the Restore the original fit button restores the plot and numbers to the original status.

In the One Line tab, the user can focus his/her attention on a specific line using the menu at the bottom of the panel. Once a line is selected, the deviations measured in all of the columns of the slit for that line is displayed in the top plot as a function of the slit.

Finally, in the Summary tab, the distribution of the rms for all slits (estimated for the whole slit at the central position), and a summary table are reported. In this table all the columns can be sorted by clicking on the top.

3.5. Preliminary Reduction

When the Bad pixel Image, the Master Flat and optionally the Master Dark or Master Bias have been created “pre-reduction” of the science frames can start. In this first step the user can clean the frames removing the detector signature and artifacts as bad pixels and cosmic rays. The recipe produces new frames, with a suffix used to know the operation performed by the preliminary reduction (see Preliminary Reduction).

To preliminary reduce data, check the parameter values of the Preliminary Reduction recipe going into Parameter Files menu. If the CLEAN_BAD_PIX keyword is activated, append the appropriate bad pixel map to science raw frames. Select the raw frames to polish, the Master Flat (optionally the Master Bias or Master Dark) and then click the Preliminary Reduction button.

Caution

Take care to select the Master Flat with the same filter/dichroic/arm as the raw science.

At the end of the process, the pre-reduced polished files (e.g. BF, or BFC, or DF, or DFC, see Preliminary Reduction for more details) appear in the Reduction Unit.

3.6. Reduce Observations

This recipe calibrates in wavelength the 2D spectra, optionally subtracts the sky and calibrates in flux, and extracts the 1D spectra of the detected objects (for more details see Reduce Observations).

It requires in input the pre-reduced frames and a Master Lamp. If flux calibration is desired, a sensitivity function must be provided as input. In MODS reduction, if user wants to correct spectra for atmospheric extinction an extinction table must be provided as input.

To reduce observations, check the parameter values of the Reduce Observations recipe going into Parameter Files menu.

Select the input pre-reduced frames and the Master Lamp (and optionally the sensitivity function and the extinction table) and click on the Reduce Observations button in the Reduction tab. A new pop-up window appears. The usage of this window is related - to some extent - to the reduction strategy adopted, particularly to the sky subtraction method.

  • No sky subtraction or sky subtraction with SKY-METHOD In this case, the order of the input files is not relevant, as well as the presence or not of dithering. The pop-up window directly summarizes the reduction conditions, and the user can click on the Run Recipe button. The recipe will execute all the steps requested by the user.

  • sky subtraction with ABBA-METHOD In the ABBA-METHOD the sky subtraction is performed subtracting from each frame the subsequent dithered one. In this case, the pop-up window appears with two panels: Science frames and Sky frames. The recipe subtracts from each frame of the Science frames panel the corresponding frame in the Sky frame panel. The couple of frames must be dithered. To easily recognize dithered frames, the user can click on the Get From Header button in the OFFSETS area. The recipe computes the offsets of input files from their header and updates the Offset column in the Science frames panel. With the help of these offsets, the user can upload the Sky frames files in the right order and then run the recipe. The user can modify the order of the input files in both panels dragging and dropping them. Clearly, despite the name, this method can be used also for observations with dithering pattern more complex then ABBA (e.g. ABAB, ABCDABCD, etc…).

  • sky subtraction with DAVIES-METHOD Similarly to ABBA-METHOD, the DAVIES-METHOD subtracts from each frame the subsequent dithered one. In this case the pop-up window shows two panels: Science frames and Sky frames. If the user passes only Science frames, the recipe subtracts from each frame of the list its subsequent one. From the last frame it subtracts the previous one. Once files have been uploaded in the Science frames list, and offsets optionally computed, the user can modify the order of the input files dragging and dropping them. When the list is ordered, click the Run Recipe button. As alternative, the user can provides also Sky frames. In this case the recipe acts as in the ABBA METHOD.

In each slits, it is possible to add other files clicking the Add Files button, and remove a file by clicking on it and then on the Canc button. Finally the user can save the list using the Save List button and can upload files directly from a list, using the Load List button. This option could be particularly useful for the ABBA- and DAVIES-METHOD: if the user wants to test different reduction strategies he/she can upload the list skipping all the organizational steps.

At the end of the process, the reduced files replace the pre-reduced ones and an R is added to the name (e.g. BFR, or BFCR, or DFR, or DFCR). The primary extension of the reduced file is the pre_reduced_frame.fits. This allows the user to run multiple time the Reduce Observations recipe directly on the reduced frames, without repeating the Preliminary Reduction since the recipe reads the pre-reduced frame from their primary extension.

According to the choices made by the user different extensions will be appended to the reduced file. The 2D extracted spectra are stored in the EXR2D extension of the reduced file, while the 1D spectra in the EXR1D extension. For more details on all the extensions, see the description provided in this section.

The user may desire to compare results obtained with different reduction strategies (e.g. different sky subtraction methods). To this aim, SIPGI offers the possibility to change the name to a single file or to a block of files using the context menu (see Context menus). The user can select, e.g. the block of reduced files, open the context menu and provide the prefix of the file names. SIPGI renames the selected files using this prefix and enumerates them accordingly to the input frames.

3.6.1. Utilities to check the reduced observations

Once frames are reduced, the user can look at the results and analyze the output using different utilities.

3.6.1.1. Show Reduced Spectra utility.

SIPGI offers the possibility to check the reduced spectra with the Show Reduced Spectra utility.

Click on a reduced frame and then click the Show Reduced Spectra button in the Analysis Tab. A SPECTRAL VIEWER window as in Fig. 3.3 will open.

_images/show_spectra.png

Fig. 3.3 The Show Reduced Spectra layout

In panel A the EXR2D spectra of a selected slit is shown (the selection can be done in panel E, by default slit 1 is shown). The two solid lines indicate the extraction regions of the spectra shown in panel B. Panels C and D show the relative sky and noise spectrum, respectively.

In panel E, the Object tab shows the name, the slit number and the number of the object/detection shown in panel B (see Show Window Table). The object name can be modified using the Set Object Name button.

The Browse box allows the user to browse between different slits with the scrolling Slit menu. For a given slit, the user can visualize the multiple 1D extracted spectra, if available, with the Previous and Next button in the same box.

The View tab at the bottom of panel E provides the spectrum smoothing tool and the wavelength range selector tool. The Auto button shows the EXR2D and 1D spectra in their entirety, i.e. it shows the whole extraction range. The user can change these values according to his/her need. The user can zoom in the 1D (and accordingly in the 2D) spectrum by clicking the left mouse button and dragging it or using the mouse wheel. The Zoom Out button at the end of the panel restores the original scale.

The Smooth button allows the user to smooth the 1D spectra with a Gaussian kernel with sigma (in pixel) equal to the provided odd number. The Reset button restores the spectra to their original shape. To save the smoothed version click the Save button at the bottom of the SPECTRAL VIEWER. This will overwrite the original 1D spectra.

The 2D Image tab allows the user to set the EXR2D display cut levels (i.e. Low and High) and to modify the 2D image color-map (and revert it). The Show object limits flag allows to show or not the extraction limits of the 1D spectra on the EXR2D image (green lines).

Panel E also provides some tools for a quick analysis of the data. The Mouse tab permits to perform the following operations with the mouse buttons:

  • STATS: this option allows to perform quick statistic for a portion of the spectrum. Select the area of interest in the 1D spectrum by clicking the right button of the mouse and dragging it. In the Stats tab at the top of Panel E the selected wavelength range, the median of all of the points in the selected wavelength range, the standard deviation (std) of these points and a coarse signal-to-noise ratio (S/N) are displayed. The S/N is computed as the ratio between the median value and the std.

  • EDIT: this option allows to edit the spectrum. There are two ways to edit spectrum with mouse: i) replace a spectrum region with a straight line. Click the right button of the mouse and drag the mouse to select the editing range: a straight line is drawn from the leftmost selected point up to the rightmost selected point. ii) replace a punctual spectrum value. Click with the middle button of the mouse a point in the 1D spectrum panel. The task will connect the selected point to spectrum. All edits performed to the 1D spectrum can be undone pressing “u” key on the keyboard (up to the complete restoration of the original spectrum), provided you did not move to a different spectrum. In case something goes really wrong it is also possible to restore the original data and restart the editing from scratch clicking on the button “Undo Edits”.

  • FIT: this option allows to fit spectral lines. Spectral lines can be fitted using two methods: i) Gaussian. A Gaussian is fitted in the selected wavelength range (identified clicking and dragging the mouse) and the results are shown in the Fit tab at the top of panel E. ii) Peak. The peak of the selected line is computed finding the barycenter of the selected points. The results are shown in the Fit tab. To clear the fit result from the plot, press “c” key.

In the Redshift box, the user can quickly check the source redshift. Putting the expected redshift, and flagging the Lines entry, the SPECTRAL VIEWER shows on the 1D spectrum the position of the main absorption and emission lines according with the provided redshift.

Ext. Profile button shows the extraction profile (see Plot Extraction Profile); Window Table button shows the WIN table (see Show Window Table). Export buttons export the shown 1D, 2D, noise e sky spectrum. Clicking the button, a window appear to provide the name and location of the saved files. If the output file has a FITS extension, the data are saved in FITS file, otherwise in ASCII. Export All export all the data of the reduced file.

3.6.1.2. Manage Detections utility

The Manage Detections utility allows the user to modify/add/delete the objects detections.

Click on a reduced frame and then click the Manage Detections button in the Analysis Tab. A new window appears, with on the top the EXR2D spectra of the first slit. The user can browse between the slits with the two Previous and Next buttons at the bottom of the panel.

In the left panel is shown the extraction profile of the slit. Passing with the mouse on the plot, a blue line appears both on the profile and on the corresponding point in the EXR2D spectra. Clicking the right mouse button and dragging it, the user can select a new region of the 2D spectra to extract. This region is highlighted with a yellow area. Releasing the right mouse button, the utility automatically adds the detection in the table on the right.

In the right table is shown a summary of all the objects that have been detected in that slit. In the table, the columns Slit, Obj, Obj Start, and Obj End indicate the number of the slit which is shown, the number of the detection, the value of pixels at which the object begins and ends. Clicking with the left button of the mouse on a detection, the detection limits are shown on the extraction profile on the left. Clicking with the right button of the mouse on a detection in the table, the user can delete the selected detection clicking on Delete objects.

Once the user is satisfied with all the changes, she/he must commit the changes clicking the Commit Changes button, and only after that he/she can move to the next slit without losing all the work. Once the user has finished all the changes in all the slits, she/he must apply the changes clicking the Apply Changes button. At this point the utility extracts all the new detections, removes the deleted ones, and updated the input file accordingly. By default, the extraction is performed with the sum. If the input spectra is in ADU, the user can activate the Horne extraction clicking the Use Horne Extraction flag before clicking the Apply Changes button.

3.6.1.3. Plot Extraction Profile utility

Click on a reduced frame and then click the Plot Extraction Profile button in the Analysis Tab. It shows the Extraction Profile (see Plot Extraction Profile). The Previous and Next buttons allow to browse between slits/objects.

3.6.1.4. Show Window Table utility

Click on a reduced frame and then click the Show Window Table button in the Analysis tab. A new window will open showing the WIN table of the selected frame (see Show Window Table).

3.6.2. Molecfit

SIPGI offers the opportunity to export reduced data outside the SIPGI environment to treat them with Molecfit.

Select the files to be exported and then click on the SIPGI To Molecfit button in the Reduction Tab. A new windows appears with the list of the selected files. The user must provide the recipe with the slit/object number of the detection she/he wants to export filling in the two relative columns in the panel. To identify these numbers the user can use the Show Reduced Spectra utility.

Click the Run button and provide the directory in which to save the files.

Once the transmission function has been calculated with Molecfit, the user can apply it to data in SIPGI. This allows the user to continue to work in SIPGI with corrected frames. To do this, select the files to be corrected, click the Apply Molecfit Tra button in the Reduction Tab, and a new window will appear with two panels. In the left panel, the list of the files to be corrected is shown. The user must load in the right panel the transmission functions derived with Molecfit. If just one function is provided, it will be applied to all the files in the left panel. The Run button executes the recipe.

SIPGI offers the possibility to revert this operation with the Revert Molecfit Tra button. Select the files to revert to their original form and click the Revert Molecfit Tra button.

3.7. Sensitivity function

To account for the instrument response and to covert spectra in physical unit, it is necessary to create a sensitivity function.

3.7.1. MODS

In MODS observations the recipe requires the wavelength-calibrated spectra of the standard star (e.g. BFCR frames) and its Spectro-photometric standard. The recipe compares 1D reduced spectra of the observed standard star and its Spectro-photometric standard obtaining a raw sensitivity function. If the Spectro-photometric standard does not cover the whole wavelength range, the recipe accepts in input a template from the provided Pickle’s library to extend the coverage (see create sensitivity).

Before starting the computation of the sensitivity function it is useful compare the observed data with the Spectro-photometric standard. To do this, click the standard BFCR frame and the Spectro-photometric standard and then click the Show Reduced Spectra button in the Analysis tab. The Spectro-photometric standard is displayed on top of the 1D spectra (in panel B of Fig. 3.3). This visualization helps both into understanding the extension of the Spectro-photometric standard (and hence if it is necessary to provide a Pickle template in input) and also its quality.

To obtain the sensitivity function for MODS observations, check the parameter values of the Create Sensitivity recipe going into Parameter Files menu.

Select the wavelength-calibrated frames of the standard stars, its Spectro-photometric standard (optionally, a Pickle template of the same spectral type/class of the standard star from the Generic Master Frames and Calibration Files) and click the Create Sensitivity button in the Reduction Tab. A new panel opens with the list of input frames. The recipe identifies the standard star as the detection with the highest flux in each frame. It could happen that the detection with the highest flux is not the standard star. In this case the user must provide the recipe with the slit/object number of the standard star detection filling in the two relative columns in the panel. To identify these numbers the user can use the Show Reduced Spectra utility. Click the Run button and the sensitivity function will be created.

3.7.2. LUCI

In LUCI observations, the recipe requires the wavelength-calibrated frames of the telluric. Matching the information stored in the telluric FITS header and those in the Hipparcos catalog, the recipe identifies the spectral type of the observed telluric and automatically selects the relative Stellar template. If the header keywords are not correctly filled, the user must also provide in input a Stellar template of the same spectral type of the observed telluric. The recipe re-scales the Stellar template to the nominal magnitude of the star in a given filter and computes the sensitivity function.

To obtain the sensitivity function for LUCI observations, check the parameter values of the Create Sensitivity recipe going into Parameter Files.

Select the wavelength-calibrated frames of the telluric. If the telluric is not in the Hipparcos catalog, provide a template of the same spectra type/class of the telluric. Click the Create Sensitivity button in the Reduction Tab and then follow the instruction for MODS Sensitivity function.

Important

For MOS observation, see the MOS LUCI sensitivity function paragraph.

3.7.3. Utility to check the sensitivity function: Plot Sensitivity

Once the sensitivity function is derived, the user can look at the result using the Plot Sensitivity utility. Select the sensitivity function and then click the Plot Sensitivity button in the Analysis tab. A new window with two panels will appear.

In the bottom panel, the red crosses show the raw sensitivity function, i.e. the function obtained by dividing the median wavelength calibrated observed spectrum by its reference model. The blue line is the smoothed (final) sensitivity function.

In the top panel:

  • for LUCI observations, the magenta solid line is the Stellar template scaled to the telluric magnitude used to derive the sensitivity. Black crosses are the flux-calibrated median spectrum of input telluric spectra.

  • for MODS observations, the magenta solid line is the Spectro-photometric standard used to derive the sensitivity. Black crosses are the flux-calibrated median spectrum of input standard spectra.

In each panel, the user can zoom the functions clicking the left mouse button and dragging it or using the mouse wheel.

The utility allows to also check the sensitivity function on the single telluric/standard input spectra.

The user has to re-reduce the standard/telluric frames applying the flux calibration. Once these files have been obtained, select the sensitivity function and then click the Plot Sensitivity button in the Analysis tab.

In the bottom part of the window the Plotted Frame menu allows the user to select the spectrum shown in the upper panel; the median used to compute the sensitivity itself (default value) or one of the flux-calibrated star (if selected as input).

The user can also modify the final sensitivity function by clicking the Edit mode flag. This opens many possibilities: the user can i) smooth the sensitivity with a mean filter with dimension Smooth level pixel; ii) edit the sensitivity pixel by pixel by clicking with the central button of the mouse; iii) replace the sensitivity with a straight line in the region highlighted by clicking the mouse right button and dragging it.

Once the user is satisfied with the changes, he/she must verify the new sensitivity on data clicking the Reapply Sensitivity button. This will upgrade the flux-calibrated spectrum in the top panel according with the new sensitivity. If the user is satisfied with that, he/she can save the modified version with the Save button, otherwise the Undo All button resets all of the changes.

3.8. Combine Observations

The combine observation recipe takes in input the DFCR, BFCR frames (flux calibrated or not) and stack them together to increase the signal-to-noise ratio. Then, for each slit, it determines the extraction profiles and extracts the 1D spectra of the detected objects.

The output file is a brand new file, which will contain several extensions.

To combine frames, check the parameter values of the Combine Observations recipe going into Parameter Files.

Select all of the frames to combine and click the Combine Observations button in the Reduction tab. This opens a new panel with the input frames listed plus a column with the offsets.

The recipe offers different options to compute the offsets: Get From Headers, Compute From WIN and Compute from EXR2D (see Combine observations for more details on the methods). Clicking the Get From Headers button, the recipe is directly launched. On the contrary, the Compute From WIN and Compute from EXR2D buttons opens the relative Parameter panels of the recipes. The user can set there their customized parameters and then launch the recipe clicking the Compute button.

The user can also edit the Offset column and insert offsets values by hand.

As for the Reduce Observations, the recipe offers the opportunity to save and upload a list of files with the Save List and Load List button.

The combined frame can be visualized and checked with the Show Reduced Spectra.