.. _showapi.level1a: Level 1A: Interferograms ------------------------ The SHOW Level 1A data product stores SHOW interferograms that have been corrected for 1. Dark Current 2. Bad pixel areas 3. Non Uniformity Correction 4. Flat-field correction 5. Sub-windowing to detector useful area Accessing Data ^^^^^^^^^^^^^^ The data are stored in directory hierarchy that organizes data by time and science/group name. To load Level 1A data using the `showapi` you will need to know the group-name and UTC time range of the data you need, note the current version only has 1 minute resolution. The `showapi` uses the concept of a collection object to store all of the images and then unitilize indexing and iteration to access individual elements. In essence the collection object looks like an array of Level 1A data records,. An example is given below:: from showapi.level1.showlevel1a import SHOWLevel1ACollection groupname = '2017_05_01_WHT_ARMA_BLK_3A_300ms_vect' # specify a SHOW science/group name startutc = '2017-05-01 16:09' # Specify start time to 1 minute accuracy (higher accuracy is not yet supported endutc = '2017-05-01 16:12' # Specify end time to 1 minute accuracy ( higher accuracy is not yet supported) with SHOWLevel1ACollection('er2_2017',groupname ) as l1acoll: # use the "with" statement to ensure the SHOWLevel1ACollection resources are closed at the end of the 'with' section l1acoll.load( startutc, endutc ) # load the records between the start and end time (note it includes all the records from the end minuite) n = len(l1acoll) # Get the number of records for i in range(n): # and for each record l1a = l1acoll[i] # get the individual record print("Indexed entry:", l1a.time) # and do something with it Note the following: - The startutc and endutc parameters can be strings or datetime.dattime object. The parameters are optional to method SHOWLevel1ACollection.load(). If omitted the method will load all records for the *group* stored in the directory hierarchy. This latter option may take a long time. - The **with** statement is highly recommended as it guarantees that all the internal netcdf caching is properly released at the end of the **with** block. If you choose not to use the **with** statement please ensure you call method SHOWLevel1ACollection.close() on each L1A collection instance. An alternative version is given below that has the same functional code as the above example but uses a more modern iterating style using the iterator support built into the objects:: from showapi.level1.showlevel1a import SHOWLevel1ACollection groupname = '2017_05_01_WHT_ARMA_BLK_3A_300ms_vect' # specify a SHOW science/group name startutc = '2017-05-01 16:09' endutc = '2017-05-01 16:12' with SHOWLevel1ACollection('er2_2017',groupname ) as l1acoll: # use the "with" statement to ensure the SHOWLevel1ACollection resources are closed at the end of the 'with' section l1acoll.load( startutc, endutc ) # load the records between the start and end time (note it includes all the records from the end minuite) for l1a in l1acoll: print("Iterated entry:", l1a.time) Level 1A structure ^^^^^^^^^^^^^^^^^^ .. py:class:: SHOWL1AEntry The Level 1A files are stored in netcdf files as calibrated interferograms and are derived from a sub-window of the original detector area. .. py:attribute:: time The UTC time of the interferogram expressed as `numpy.datetime64` in micro-seconds resolution. This value can be converted to regular python datetime using:: utc = l1a.time.astype(datetime.datetime) .. py:attribute:: exposure_time The exposure time of the interferogram in milliseconds. Scalar float .. py:attribute:: image The 2-D interferogram image in units of digital number per second. This is a 2-D array of shape H,M where H is the number of height rows and M is the number of interferogram rows. The image is stored in the netcdf file with lossy compression that guarantees the accuracy of the second digit after the decimal point, i.e. accurate to 0.01. This compression error is generally small compared to the size of the error value. .. py:attribute:: error A 2-D interferogram image in units of digital number per second. This is the same size and shape as the `image` attribute. This field is stored in the netcdf file with lossy compression that guarantees the accuracy of the second digit after the decimal point, i.e. accurate to 0.01. This compression error is generally small compared to the size of the error value. .. py:attribute:: version A 3-element integer tuple of the form `major version`, `minor version`, `build number`. .. py:attribute:: heightrow A 1-D array that maps each height (1st) row of the interferogram `image` back to the original detector row. All indices are 0 based. .. py:attribute:: pixelcolumn A 1-D array that maps each interferogram (2nd ) column of the interferogram `image` back to the original detector column. All indices are 0 based. .. py:attribute:: sensor_names A 1-D array that provides a human-readable name for each element in the `temperatures` field. .. py:attribute:: temperatures A 1-D array of temperatures derived from the SHOW housekeeping telemetry. The names of each element can be found in the `sensor_names` field. Level 1A Directory Structure ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The SHOW Level 1A data product is stored in a directory structure that stores data in netcdf files that contain all the data for one group in one minute. The default base directory of the hierarchy is specified in the :ref:`site specific yaml file `. The hierarchy sorts data into folders by date and group. An example of the heierarchy is given below:: Level 1A basedirectory +---- 20170501 +---- 20170502 +---- 20170503 +---- ARMA_BLK_2p6A_vect +---- ARMB_BLK_2p6A_vect +---- l1a_20170501-1805_v000.nc +---- l1a_20170501-1806_v000.nc +---- l1a_20170501-1807_v000.nc +---- l1a_20170501-1808_v000.nc +---- l1a_20170501-1809_v000.nc +---- l1a_20170501-1810_v000.nc +---- l1a_20170501-1811_v000.nc +---- l1a_20170501-1812_v000.nc +---- l1a_20170501-1813_v000.nc +---- l1a_20170501-1814_v000.nc +---- l1a_20170501-1815_v000.nc +---- l1a_20170501-1816_v000.nc +---- l1a_20170501-1817_v000.nc +---- 20170504 +---- yyyymmdd +---- +---- l1a_yyyymmdd-HHMM_.nc The first directory level sorts data records by date. The folder names are encoded as year, month day. The second directory level sorts data by SHOW science/group name. The group name is the same as that entered at the SHOW GSE during operations and provides an easy mechanism to sort data. A range of netcdf files are written into each group folder. These contain the data for that group in one minute increments. The filename is encoded with the prefix *l1a* followed by year, month, day, hour, minute and a version string.