.. _retrieval_overview:

Retrieval Overview
******************
The basic idea behind retrievals is pretty simple and relies on the following:

1. We have an instrument that took a measurement, **y**, with the atmosphere in an unknown state.
2. We can simulate the measurement, this is called **F**, and if the assumed atmospheric state is the same as the true state **y** and **F** will also be equal.
3. If **y** and **F** don't agree our atmospheric state was assumed incorrectly and needs to be updated.

**y** and **F** are are not required to be radiances. Instead **y** and **F** are typically combinations of measurements at different altitudes or wavelengths to minimize the dependence on properties of the atmospheric state that are unknown. The measurements themselves are also not required to be radiance, and may be integrated quanties, or radiances transformed by an instrument in some other fashion.

The idea behind the retrieval software is then to simulate measurements and update the atmospheric state until **y** is as close to **F** as possible. In order to do this, we need four things:
    
    meas_inst
        This is a class capable of providing the information required to
        calculate **y**.  It can either be a simulated instrument or a container
        for data for a real instrument
    
    ret_inst
        This class provides the information to calculate **F**.  Typically
        ret_inst will support :func:`calculate_radiance()` which uses a radiative
        transfer model to get the information required for **F**
        
    species
        species is a class capable of taking in an instrument (either meas_inst,
        or ret_inst) and converting the raw data (usually radiance) to a measurement
        vector.  The species and instrument type are loosly coupled together,
        for example, the :class:`H2OShow` species is unlikely to be compatible
        with the :class:`OSIRIS` instrument.  However, it is possible to write species which
        are compatible with more than one instrument.
        
        When the retrieval starts, :func:`species.configure_instrument_and_method`
        is called, which gives the species a chance to configure the instruments
        and method for the specific retrieval task at hand.
        
        The species is also responsible for updating the atmosphere, typically
        through an :func:`update_profile` call
    
    method
        The iteration method used to push the solution in a good direction.  Typically
        the general algorithm is as follows
            - call :func:`ret_inst.calculate_radiance` to prepare the instrument 
            - call :func:`species.measurement_vector` to get **F**
            - Compare **F** and **y** to figure how much to update the profile by
              (This is usually where the key differences between methods are)
            - Tell the species how much to change the profile
            - Repeat
        
        The method is very loosely coupled to the species.  Methods like Levenberg-Marquardt
        require that the Jacobian of the measurement vector be calculated, and the species
        will need to be capable of that for Levenberg-Marquardt to work.  However, usually
        the same method can be reused for many species.
    
    The techniques these four objects use to accomplish their tasks has been
    abstracted out of the retrieval, this is to ensure that this class will be
    compatible for essentially any limb retrieval. 
    
    The current way that all retrievals are done is with the help of two classes,
    
    radtran
        A wrapper to a radiative transfer model.  Basically the radiative transfer wrapper
        just has :func:`add_line_of_sight` and :func:`calculate_radiance`.  ret_inst
        holds a copy of this class and uses it to calculate radiances required
    
    atmosphere
        A wrapper class for specifying the atmosphere.  For SASKTRAN radiative
        transfer models this is typically just a list of climatologies, opticalproperties,
        and guids.  Usually this class has a function :func:`add_to_engine` which
        adds the current atmosphere to the radiative transfer model.
        Copies of this class are held in the radtran class, and also the species class,
        which uses it to update the atmosphere.
		
		
More graphically, the creation of a measurement vector proceeds in one of two ways. If we are generating the modelled vector, we have:

.. image:: ..\figures\fc1.png
   :scale: 100 %
   :alt: F flow chart
   :align: center

Where the radiative transfer engine generates radiances which are then converted by the instrument into measurements. These measurements are then combined by the retrieval species into the measurement vector, **F**. The vector **y** is slighlty simpler as it does not require a radiative tranfer engine to generate the radiance. However, the retrieval species makes no distinction between the 'measured' and 'retrieval' instruments. It simply generates a vector given an instrument. Combining this with the other components we have:

.. image:: ..\figures\fc3.png
   :scale: 100 %
   :alt: flow chart
   :align: center

   
Where the retrieval method, for example levenberg-marquardt, is used to compare **y** and **F** to generate the step size, :math:`\delta`. The retrieval species uses :math:`\delta` to update the species profile. This allows the species to use internal parameters such as limiting the altitude range of the retrieval, and what to do above/below the retrieval limits. The updated profile is then passed to the atmosphere.