Level 1A: Interferograms

The SHOW Level 1A data product stores SHOW interferograms that have been corrected for

1. Dark Current
2. Bad pixel areas
3. Non Uniformity Correction
4. Flat-field correction
5. Sub-windowing to detector useful area

Accessing Data

The data are stored in directory hierarchy that organizes data by time and science/group name. To load Level 1A data using the showapi you will need to know the group-name and UTC time range of the data you need, note the current version only has 1 minute resolution. The showapi uses the concept of a collection object to store all of the images and then unitilize indexing and iteration to access individual elements. In essence the collection object looks like an array of Level 1A data records,. An example is given below:

from showapi.level1.showlevel1a import SHOWLevel1ACollection

groupname = '2017_05_01_WHT_ARMA_BLK_3A_300ms_vect'                     # specify a SHOW science/group name
startutc  = '2017-05-01 16:09'                                          # Specify start time to 1 minute accuracy (higher accuracy is not yet supported
endutc    = '2017-05-01 16:12'                                          # Specify end time to 1 minute accuracy ( higher accuracy is not yet supported)

with SHOWLevel1ACollection('er2_2017',groupname ) as l1acoll:           # use the "with" statement to ensure the SHOWLevel1ACollection resources are closed at the end of the 'with' section
    l1acoll.load( startutc, endutc )                                    # load the records between the start and end time (note it includes all the records from the end minuite)
    n = len(l1acoll)                                                    # Get the number of records
    for i in range(n):                                                  # and for each record
        l1a = l1acoll[i]                                                # get the individual record
        print("Indexed entry:", l1a.time)                               # and do something with it

Note the following:

• The startutc and endutc parameters can be strings or datetime.dattime object. The parameters are optional to method SHOWLevel1ACollection.load(). If omitted the method will load all records for the group stored in the directory hierarchy. This latter option may take a long time.
• The with statement is highly recommended as it guarantees that all the internal netcdf caching is properly released at the end of the with block. If you choose not to use the with statement please ensure you call method SHOWLevel1ACollection.close() on each L1A collection instance.

An alternative version is given below that has the same functional code as the above example but uses a more modern iterating style using the iterator support built into the objects:

from showapi.level1.showlevel1a import SHOWLevel1ACollection

groupname = '2017_05_01_WHT_ARMA_BLK_3A_300ms_vect'                     # specify a SHOW science/group name
startutc  = '2017-05-01 16:09'
endutc    = '2017-05-01 16:12'

with SHOWLevel1ACollection('er2_2017',groupname ) as l1acoll:           # use the "with" statement to ensure the SHOWLevel1ACollection resources are closed at the end of the 'with' section
    l1acoll.load( startutc, endutc )                                    # load the records between the start and end time (note it includes all the records from the end minuite)
    for l1a in l1acoll:
        print("Iterated entry:", l1a.time)

Level 1A structure

class SHOWL1AEntry

The Level 1A files are stored in netcdf files as calibrated interferograms and are derived from a sub-window of the original detector area.

time

The UTC time of the interferogram expressed as numpy.datetime64 in micro-seconds resolution. This value can be converted to regular python datetime using:

utc = l1a.time.astype(datetime.datetime)

exposure_time

The exposure time of the interferogram in milliseconds. Scalar float

image

The 2-D interferogram image in units of digital number per second. This is a 2-D array of shape H,M where H is the number of height rows and M is the number of interferogram rows. The image is stored in the netcdf file with lossy compression that guarantees the accuracy of the second digit after the decimal point, i.e. accurate to 0.01. This compression error is generally small compared to the size of the error value.

error

A 2-D interferogram image in units of digital number per second. This is the same size and shape as the image attribute. This field is stored in the netcdf file with lossy compression that guarantees the accuracy of the second digit after the decimal point, i.e. accurate to 0.01. This compression error is generally small compared to the size of the error value.

version

A 3-element integer tuple of the form major version, minor version, build number.

heightrow

A 1-D array that maps each height (1st) row of the interferogram image back to the original detector row. All indices are 0 based.

pixelcolumn

A 1-D array that maps each interferogram (2nd ) column of the interferogram image back to the original detector column. All indices are 0 based.

sensor_names

A 1-D array that provides a human-readable name for each element in the temperatures field.

temperatures

A 1-D array of temperatures derived from the SHOW housekeeping telemetry. The names of each element can be found in the sensor_names field.

Level 1A Directory Structure

The SHOW Level 1A data product is stored in a directory structure that stores data in netcdf files that contain all the data for one group in one minute. The default base directory of the hierarchy is specified in the site specific yaml file. The hierarchy sorts data into folders by date and group. An example of the heierarchy is given below:

Level 1A basedirectory
     +---- 20170501
     +---- 20170502
     +---- 20170503
              +---- ARMA_BLK_2p6A_vect
              +---- ARMB_BLK_2p6A_vect
                         +---- l1a_20170501-1805_v000.nc
                         +---- l1a_20170501-1806_v000.nc
                         +---- l1a_20170501-1807_v000.nc
                         +---- l1a_20170501-1808_v000.nc
                         +---- l1a_20170501-1809_v000.nc
                         +---- l1a_20170501-1810_v000.nc
                         +---- l1a_20170501-1811_v000.nc
                         +---- l1a_20170501-1812_v000.nc
                         +---- l1a_20170501-1813_v000.nc
                         +---- l1a_20170501-1814_v000.nc
                         +---- l1a_20170501-1815_v000.nc
                         +---- l1a_20170501-1816_v000.nc
                         +---- l1a_20170501-1817_v000.nc
     +---- 20170504
     +---- yyyymmdd
              +---- <groupname>
                         +---- l1a_yyyymmdd-HHMM_<version>.nc

The first directory level sorts data records by date. The folder names are encoded as year, month day. The second directory level sorts data by SHOW science/group name. The group name is the same as that entered at the SHOW GSE during operations and provides an easy mechanism to sort data. A range of netcdf files are written into each group folder. These contain the data for that group in one minute increments. The filename is encoded with the prefix l1a followed by year, month, day, hour, minute and a version string.