.. _showapi.utilities: Data Processing: Command Line Utilities --------------------------------------- The showapi exposes several command line utilities used in standard data processing: - :ref:`show_raw_to_l0 ` - :ref:`show_l0_to_l1a `. - :ref:`show_l1a_to_l1b ` - showgse ---------------------------------------------------------------------- .. _show_raw_to_l0: .. program:: show_raw_to_l0 **show_raw_to_l0** *input_directory* *output_filename* Convert directories of raw images to a single netcdf Level 0 files. This is a console script that can be executed from a standard windows or Linux shell. The command writes raw images in sub-folders of directory *input_directory* to netcdf file *output_filename*. The directories of raw images can be either downloaded from the SHOW instrument or collected the SHOW GSE. A new output file is generated each time the function is executed. It is not possible to append images to an existing file. The code reads in both raw images and temperature data from the *input_directory* and linearly interpolates the temperature data to the instant of each exposure before writing to the *output_file*. All level 0 images stored in Level 0 files are rotated or transposed so they have the same physical orientation regardless of framegrabber. The *showapi* file I/O routines will return numpy arrays where the first index is along the height dimension and the 2nd index is along the interfrogram dimension. The input directories must be organized according to SHOW instrument downloads or GSE formats otherwise it will not work properly. The SHOW level 0 file generated by *show_raw_to_l0* conatins all images found in the sub-directories. Images are sorted inside the netcdf file according to *SHOW science description* and time. Each unique *science description* generates a netcdf group within the file, think of it as a folder within the file, with the same name. All records within each netcdf group are sorted into chronological order. This console script does **not** access the SHOW calibration database. -------------------------------------------------------------------------- .. _show_l0_to_l1a: .. program:: show_l0_to_l1a **show_l0_to_l1a** *input_filename* ** Convert images in a single Level 0 netcdf file to Level 1A images stored in the Level 1A directory structure. This is a console script that can be executed from a standard windows or Linux shell. The command creates SHOW level 1A interferogram images from SHOW Level 0 netcdf files. .. option:: --output_folder=foldername The base directory of the level 1A directory structure. If this option is not set then the default value is extracted from the *showapi* site-settings configuration file. .. option:: --group=groupname Only extract and process images from group *groupname* within the Level 0 netcdf file. If not specified it will extract and process images in all groups in the Level 0 netcdf file. .. option:: --verbose If this option is provided it will send all verbose output to both logfile and screen. The default setting directs all verbose output only to the log file. The code currently generates a level 1A file structure as follows:: # Level 1A Base directory from site-settings yaml file + + --- # Name of group taken from Level 0 netcdf file | |--- # Date in format year, month, day | |---- l1a_yyyymmdd-HHMM_vxx.xx.xx.nc # Level 1A files for the given date grouped into 1 minute intervals l1a_yyyymmdd-HHMM_vxx.xx.xx.nc # Each filename has the following attributes l1a_yyyymmdd-HHMM_vxx.xx.xx.nc # yyyymmdd = year, month, day l1a_yyyymmdd-HHMM_vxx.xx.xx.nc # HHMM = hour, minute l1a_yyyymmdd-HHMM_vxx.xx.xx.nc # xx.xx.xx = version as major.minor.build This directory structure allows mutliple level 0 files to add level 1A files to the same directory structure. -------------------------------------------------------------------------- .. _show_l1a_to_l1b: .. program:: show_l1a_to_l1b **show_l1a_to_l1b** *groupname* ** Convert SHOW interferograms stored in the Level 1A directory structure to Level 1B spectra stored in the Level 1B directory structure. This is a console script that can be executed from a standard windows or Linux shell. The processing will include the attitude calculation if ER2 navigation information is provided through the *iwg1* option. The Level 1B data are written to a Level 1B directory structure that is similar to the Level 1A structure. .. option:: groupname Process Level 1A images stored in this group within the Level 1A directory structure. There is currently no support to process multiple groups .. option:: --output_folder=foldername Override the default output folder used as the base of the Level 1B directory structure. By default the Level 1B output folder is taken from the site settings yaml configuration file. .. option:: --input_folder=foldername Override the default input folder used as the base of the Level 1A directory structure. By default the Level 1A input folder is taken from the site settings yaml configuration file. .. option:: --l1a_version=xx.yy.zz Override the default version of level 1A files read from the Level 1A directory structure. Must be of the form xx.yy.zz. By default the default Level 1A version is taken from the site settings yaml configuration file. .. option:: --version=xx.yy.zz Override the default version of level 1B files generated by the code. Must be of the form xx.yy.zz. By default the default Level 1B version is taken from the site settings yaml configuration file. .. option:: --startutc="yyyy-mm-dd HH:MM" Select the start time used select input Level 1A interferograms from the Level 1A directory structure. Thius option will reject all images collected before this time. The *endutc* option can be used to reject images collected after a specified time. The time comparison is always truncated to the starting minute and any seconds entries are ignored. Useful when you want to quickly process a limited time period. .. option:: --endutc="yyyy-mm-dd HH:MM" Select the end time used select input Level 1A interferograms from the Level 1A directory structure. Thius option will reject all images collected after this time. The *startutc* option can be used to reject images collected before a specified time. The time comparison is always truncated to the starting minute and any seconds entries are ignored; this has the slightly odd effect that "yyyy-mm-dd HH:MM:59.999" is considered less than or equal to "yyyy-mm-dd HH:MM" but means that the code will always process all the images a single 1 minute level 1A netcdf file. Useful when you want to quickly process a limited time period. .. option:: --iwg1=er2_attitude_filename Provides attitude information for the Level 1B data records. The *er2_attitude_filename* is the file containing ER2 IWG1 format attitude records downloaded from the `NASA ER2 website `_. The records in this file are linearly interpolated and used to fill out the appropriate attitude fields in the Level 1B data structures .. option:: --verbose If this option is provided it will send all verbose output to both logfile and screen. The default setting directs all verbose output only to the log file. The code currently generates a level 1B file structure as follows:: # Level 1B Base directory from site-settings yaml file + + --- # Name of group taken from Level 1A directory structure | |--- # Date in format year, month, day | |---- l1b_yyyymmdd-HHMM_vxx.xx.xx.nc # Level 1B files for the given date grouped into 1 minute intervals l1b_yyyymmdd-HHMM_vxx.xx.xx.nc # Each filename has the following attributes l1b_yyyymmdd-HHMM_vxx.xx.xx.nc # yyyymmdd = year, month, day l1b_yyyymmdd-HHMM_vxx.xx.xx.nc # HHMM = hour, minute l1b_yyyymmdd-HHMM_vxx.xx.xx.nc # xx.xx.xx = version as major.minor.build The Level 1A and Level 1B directory structures can co-exist with the same base directory. It is the users choice whether they write Level 1A and Level 1B to the same directory structure or two separate directories.