Making use of the mpop package

The mpop package is the heart of mpop: here are defined the core classes which the user will then need to build satellite composites.

Conventions about satellite names

Throughout the document, we will use the following conventions:

  • platform name is the name of an individual satellite following the OSCAR naming scheme, e.g. “NOAA-19”.
  • variant will be used to differentiate the same data (from the same
    satellite and instrument) coming in different flavours. For example, we use variant to distinguish data coming from the satellite Metop-B from direct readout (no variant), regional coverage (EARS) or global coverage (GDS).

All the satellite configuration files in PPP_CONFIG_DIR should be named <variant><platform name>.cfg, e.g. NOAA-19.cfg or GDSMetop-B.cfg.

Creating a scene object

Creating a scene object can be done calling the create_scene function of a factory, (for example mpop.satellites.GenericFactory.create_scene()).

The reader is refered to the documentation of the mpop.scene.SatelliteInstrumentScene() for a description of the input arguments.

Such a scene object is roughly a container for mpop.channel.Channel objects, which hold the actual data and information for each band.

Loading the data

Loading the data is done through the mpop.scene.SatelliteInstrumentScene.load() method. Calling it effectively loads the data from disk into memory, so it can take a while depending on the volume of data to load and the performance of the host computer. The channels listed as arguments become loaded, and cannot be reloaded: a subsequent call to the method will not reload the data from disk.

Re-projecting data

Once the data is loaded, one might need to re-project the data. The scene objects can be projected onto other areas if the pyresample software is installed, thanks to the mpop.scene.SatelliteInstrumentScene.project() method. As input, this method takes either a Definition object (see pyresample’s documentation) or string identificator for the area. In the latter case, the referenced region has to be defined in the area file. The name and location of this file is defined in the mpop.cfg configuration file, itself located in the directory pointed by the PPP_CONFIG_DIR environment variable.

For more information about the internals of the projection process, take a look at the mpop.projector module.

Geo-localisation of the data

Once the data is loaded, each channel should have an area attribute containing a pyresample area object, if the pyresample package is available. These area objects should implement the get_lonlats() method, returning the longitudes and latitudes of the channel data. For more information on this matter, the reader is then referred to the documentation of the aforementioned package.

Image composites

Methods building image composites are distributed in different modules, taking advantage of the hierarchical structure offered by OOP.

The image composites common to all visir instruments are defined in the mpop.instruments.visir module. Some instrument modules, like mpop.instruments.avhrr or mpop.instruments.seviri overload these methods to adapt better for the instrument at hand.

For instructions on how to write a new composites, see Geographic images.

Adding a new satellite: configuration file

A satellite configuration file looks like the following (here Meteosat-7, mviri instrument):

The configuration file must hold a satellite section, the list of channels for the needed instruments (here mviri-n sections), and how to read the data in mipp (mviri-level1) and how to read it in mpop (mviri-level2).

Using this template we can define new satellite and instruments.

Adding a new satellite: python code

Another way of adding satellites and instruments to mpop is to write the correponding python code.

Here are example of such code:

The mpop API

Satellite scenes

The mpop.scene module defines satellite scenes. They are defined as generic classes, to be inherited when needed.

A scene is a set of mpop.channel objects for a given time, and sometimes also for a given area.

class mpop.scene.Satellite

This is the satellite class. It contains information on the satellite.

classmethod add_method(func)

Add a method to the class.

add_method_to_instance(func)

Add a method to the instance.

fullname

Full name of the satellite, that is platform name and number (eg “metop02”).

classmethod remove_attribute(name)

Remove an attribute from the class.

sat_nr(string=False)
class mpop.scene.SatelliteInstrumentScene(time_slot=None, area_id=None, area=None, orbit=None, satellite=(None, None, None), instrument=None)

This is the satellite instrument class. It is an abstract channel container, from which all concrete satellite scenes should be derived.

The constructor accepts as optional arguments the time_slot of the scene, the area on which the scene is defined (this can be used for slicing of big datasets, or can be set automatically when loading), and orbit which is a string giving the orbit number.

add_to_history(message)

Adds a message to history info.

channel_list = []
check_channels(*channels)

Check if the channels are loaded, raise an error otherwise.

estimate_cth(cth_atm='best', time_slot=None)

Estimation of the cloud top height using the 10.8 micron channel limitations: this is the most simple approach a simple fit of the ir108 to the temperature profile

  • no correction for water vapour or any other trace gas
  • no viewing angle dependency
  • no correction for semi-transparent clouds
  • no special treatment of temperature inversions
data.estimate_cth(cth_atm=”best”)
cth_atm * using temperature profile to estimate the cloud top height
possible choices are (see estimate_cth in mpop/tools.py): “standard”, “tropics”, “midlatitude summer”, “midlatitude winter”, “subarctic summer”, “subarctic winter” this will choose the corresponding atmospheric AFGL temperature profile
  • new choice: “best” -> choose according to central (lon,lat) and time from: “tropics”, “midlatitude summer”, “midlatitude winter”, “subarctic summer”, “subarctic winter”
time_slot current observation time as (datetime.datetime() object)
time_slot option can be omitted, the function tries to use self.time_slot
get_orbital()
load(channels=None, load_again=False, area_extent=None, **kwargs)

Load instrument data into the channels. Channels is a list or a tuple containing channels we will load data into, designated by there center wavelength (float), resolution (integer) or name (string). If None, all channels are loaded.

The load_again boolean flag allows to reload the channels even they have already been loaded, to mirror changes on disk for example. This is false by default.

The area_extent keyword lets you specify which part of the data to load. Given as a 4-element sequence, it defines the area extent to load in satellite projection.

The other keyword arguments are passed as is to the reader plugin. Check the corresponding documentation for more details.

loaded_channels()

Return the set of loaded_channels.

parallax_corr(fill='False', estimate_cth=False, cth_atm='best', replace=False)

perform the CTH parallax corretion for all loaded channels

project(dest_area, channels=None, precompute=False, mode=None, radius=None, nprocs=1)

Make a copy of the current snapshot projected onto the dest_area. Available areas are defined in the region configuration file (ACPG). channels tells which channels are to be projected, and if None, all channels are projected and copied over to the return snapshot.

If precompute is set to true, the projecting data is saved on disk for reusage. mode sets the mode to project in: ‘quick’ which works between cartographic projections, and, as its denomination indicates, is quick (but lower quality), and ‘nearest’ which uses nearest neighbour for best projection. A mode set to None uses ‘quick’ when possible, ‘nearest’ otherwise.

radius defines the radius of influence for neighbour search in ‘nearest’ mode (in metres). Setting it to None, or omitting it will fallback to default values (5 times the channel resolution) or 10,000m if the resolution is not available.

Note: channels have to be loaded to be projected, otherwise an exception is raised.

save(filename, to_format='netcdf4', **options)

Saves the current scene into a file of format to_format. Supported formats are:

  • netcdf4: NetCDF4 with CF conventions.
unload(*channels)

Unloads channels from memory. mpop.scene.SatelliteInstrumentScene.load() must be called again to reload the data.

class mpop.scene.SatelliteScene(time_slot=None, area_id=None, area=None, orbit=None, satellite=(None, None, None))

This is the satellite scene class. It is a capture of the satellite (channels) data at given time_slot and area_id/area.

area

Getter for area.

get_area()

Getter for area.

set_area(area)

Setter for area.

mpop.scene.assemble_segments(segments)

Assemble the scene objects listed in segment_list and returns the resulting scene object.

Instrument channels

This module defines satellite instrument channels as a generic class, to be inherited when needed.

class mpop.channel.Channel(name=None, resolution=0, wavelength_range=[0, 0, 0], data=None, calibration_unit=None)

This is the satellite channel class. It defines satellite channels as a container for calibrated channel data.

The resolution sets the resolution of the channel, in meters. The wavelength_range is a triplet, containing the lowest-, center-, and highest-wavelength values of the channel. name is simply the given name of the channel, and data is the data it should hold.

as_image(stretched=True)

Return the channel as a mpop.imageo.geo_image.GeoImage object. The stretched argument set to False allows the data to remain untouched (as opposed to crude stretched by default to obtain the same output as show()).

check_range(min_range=1.0)

Check that the data of the channels has a definition domain broader than min_range and return the data, otherwise return zeros.

data

Getter for channel data.

get_data()

Getter for channel data.

get_reflectance(tb11, sun_zenith=None, tb13_4=None)

Get the reflectance part of an NIR channel

get_viewing_geometry(orbital, time_slot, altitude=None)

Calculates the azimuth and elevation angle as seen by the observer at the position of the current area pixel. inputs:

orbital an orbital object define by the tle file
(see pyorbital.orbital import Orbital or mpop/scene.py get_oribtal)

time_slot time object specifying the observation time altitude optinal: altitude of the observer above the earth ellipsoid

outputs:
azi azimuth viewing angle in degree (south is 0, counting clockwise) ele elevation viewing angle in degree (zenith is 90, horizon is 0)
is_loaded()

Tells if the channel contains loaded data.

parallax_corr(cth=None, time_slot=None, orbital=None, azi=None, ele=None, fill='False')

Perform the parallax correction for channel at time_slot (datetime.datetime() object), assuming the cloud top height cth and the viewing geometry given by the satellite orbital “orbital” and return the corrected channel. Authors: Ulrich Hamann (MeteoSwiss), Thomas Leppelt (DWD) Example calls:

  • calling this function (using orbital and time_slot)
    orbital = data.get_oribtal() data[“VIS006”].parallax_corr(cth=data[“CTTH”].height, time_slot=data.time_slot, orbital=orbital)
  • calling this function (using viewing geometry)
    orbital = data.get_oribtal() (azi, ele) = get_viewing_geometry(self, orbital, time_slot) data[“VIS006”].parallax_corr(cth=data[“CTTH”].height, azi=azi, ele=ele)
Optional input:
cth The parameter cth is the cloud top height
(or the altitude of the object that should be shifted). cth must have the same size and projection as the channel
orbital an orbital object define by the tle file
(see pyorbital.orbital import Orbital or mpop/scene.py get_oribtal)
azi azimuth viewing angle in degree (south is 0, counting clockwise)
e.g. as given by self.get_viewing_geometry
ele elevation viewing angle in degree (zenith is 90, horizon is 0)
e.g. as given by self.get_viewing_geometry
fill specifies the interpolation method to fill the gaps

(basically areas behind the cloud that can’t be observed by the satellite instrument) “False” (default): no interpolation, gaps are np.nan values and mask is set accordingly “nearest”: fill gaps with nearest neighbour “bilinear”: use scipy.interpolate.griddata with linear interpolation

to fill the gaps
output:
parallax corrected channel
the content of the channel will be parallax corrected. The name of the new channel will be original_chan.name+’_PC’, eg. “IR_108_PC”. This name is also stored to the info dictionary of the originating channel.
project(coverage_instance)

Make a projected copy of the current channel using the given coverage_instance.

See also the mpop.projector module.

set_data(data)

Setter for channel data.

shape

Shape of the channel.

show()

Display the channel as an image.

sunzen_corr(time_slot, lonlats=None, limit=80.0, mode='cos', sunmask=False)

Perform Sun zenith angle correction for the channel at time_slot (datetime.datetime() object) and return the corrected channel. The parameter limit can be used to set the maximum zenith angle for which the correction is calculated. For larger angles, the correction is the same as at the limit (default: 80.0 degrees). Coordinate values can be given as a 2-tuple or a two-element list lonlats of numpy arrays; if None, the coordinates will be read from the channel data. Parameter mode is a placeholder for other possible illumination corrections. The name of the new channel will be original_chan.name+’_SZC’, eg. “VIS006_SZC”. This name is also stored to the info dictionary of the originating channel.

viewzen_corr(view_zen_angle_data)

Apply atmospheric correction on a copy of this channel data using the given satellite zenith angle data of the same shape. Returns a new channel containing the corrected data. The name of the new channel will be original_chan.name+’_VZC’, eg. “IR108_VZC”. This name is also stored to the info dictionary of the originating channel.

vinc_vect(phi, lembda, alpha, s, f=None, a=None, degree=True)

Vincenty’s Direct formular

Returns the lat and long of projected point and reverse azimuth given a reference point and a distance and azimuth to project. lats, longs and azimuths are passed in radians.

Keyword arguments:

phi Latitude in degree/radians lembda Longitude in degree/radians alpha Geodetic azimuth in degree/radians s Ellipsoidal distance in meters f WGS84 parameter a WGS84 parameter degree Boolean if in/out values are in degree or radians.

Default is in degree
Returns:
(phiout, lembdaout, alphaout ) as a tuple
class mpop.channel.GenericChannel(name=None)

This is an abstract channel class. It can be a super class for calibrated channels data or more elaborate channels such as cloudtype or CTTH.

area

Getter for area.

exception mpop.channel.GeolocationIncompleteError

Exception to try catch cases where the original data have not been read or expanded properly so that each pixel has a geo-location

exception mpop.channel.NotLoadedError

Exception to be raised when attempting to use a non-loaded channel.

The VisIr instrument class

This module defines the generic VISIR instrument class.

class mpop.instruments.visir.VisirCompositer(scene)

Compositer for Visual-IR instruments

airmass(fill_value=(0, 0, 0))

Make an airmass RGB image composite.

Channels Temp Gamma
WV6.2 - WV7.3 -25 to 0 K gamma 1
IR9.7 - IR10.8 -40 to 5 K gamma 1
WV6.2 243 to 208 K gamma 1
ash(fill_value=(0, 0, 0))

Make a Ash RGB image composite.

Channels Temp Gamma
IR12.0 - IR10.8 -4 to 2 K gamma 1
IR10.8 - IR8.7 -4 to 5 K gamma 1
IR10.8 243 to 303 K gamma 1
channel_image(channel, fill_value=0)

Make a black and white image of the channel.

Linear stretch without clipping is applied by default.

cloudtop(stretch=(0.005, 0.005), gamma=None, fill_value=(0, 0, 0))

Make a Cloudtop RGB image composite.

Channels Gamma
IR3.9 (inverted) gamma 1
IR10.8 (inverted) gamma 1
IR12.0 (inverted) gamma 1

Linear stretch with 0.5 % clipping at both ends.

convection(fill_value=(0, 0, 0))

Make a Severe Convection RGB image composite.

Channels Span Gamma
WV6.2 - WV7.3 -30 to 0 K gamma 1
IR3.9 - IR10.8 0 to 55 K gamma 1
IR1.6 - VIS0.6 -70 to 20 % gamma 1
dust(fill_value=(0, 0, 0))

Make a Dust RGB image composite.

Channels Temp Gamma
IR12.0 - IR10.8 -4 to 2 K gamma 1
IR10.8 - IR8.7 0 to 15 K gamma 2.5
IR10.8 261 to 289 K gamma 1
fog(fill_value=(0, 0, 0))

Make a Fog RGB image composite.

Channels Temp Gamma
IR12.0 - IR10.8 -4 to 2 K gamma 1
IR10.8 - IR8.7 0 to 6 K gamma 2.0
IR10.8 243 to 283 K gamma 1
green_snow(fill_value=(0, 0, 0))

Make a Green Snow RGB image composite.

Channels Gamma
IR1.6 gamma 1.6
VIS0.6 gamma 1.6
IR10.8 (inverted) gamma 1.6

Linear stretch without clipping.

ir108()

Make a black and white image of the IR 10.8um channel.

Channel is inverted. Temperature range from -70 °C (white) to +57.5 °C (black) is shown.

natural(stretch=None, gamma=1.8, fill_value=(0, 0, 0))

Make a Natural Colors RGB image composite.

Channels Range (reflectance) Gamma (default)
IR1.6 0 - 90 gamma 1.8
VIS0.8 0 - 90 gamma 1.8
VIS0.6 0 - 90 gamma 1.8
night_fog(fill_value=(0, 0, 0))

Make a Night Fog RGB image composite.

Channels Temp Gamma
IR12.0 - IR10.8 -4 to 2 K gamma 1
IR10.8 - IR3.9 0 to 6 K gamma 2.0
IR10.8 243 to 293 K gamma 1
night_overview(stretch='histogram', gamma=None)

Make an overview RGB image composite using IR channels.

Channels Gamma
IR3.9 (inverted) gamma 1
IR10.8 (inverted) gamma 1
IR12.0 (inverted) gamma 1

Histogram equalization is applied for each channel.

overview(stretch='crude', gamma=1.6, fill_value=(0, 0, 0))

Make an overview RGB image composite.

Channels Gamma (default)
VIS0.6 gamma 1.6
VIS0.8 gamma 1.6
IR10.8 (inverted) gamma 1.6

Linear stretch without clipping is applied.

overview_sun(stretch='linear', gamma=1.6, fill_value=(0, 0, 0))

Make an overview RGB image composite normalising with cosine to the sun zenith angle.

red_snow(fill_value=(0, 0, 0))

Make a Red Snow RGB image composite.

Channels Gamma
VIS0.6 gamma 1.6
IR1.6 gamma 1.6
IR10.8 (inverted) gamma 1.6

Linear stretch without clipping.

vis06()

Make a black and white image of the VIS 0.635um channel.

Linear stretch without clipping is applied.

wv_high()

Make a black and white image of the IR 6.7um channel.

Channel inverted and a linear stretch is applied with 0.5 % clipping at both ends.

wv_low()

Make a black and white image of the IR 7.3um channel.

Channel data inverted and a linear stretch is applied with 0.5 % clipping at both ends.

Projection facility

Satellite class loader

mpop.satellites is the module englobes all satellite specific modules. In itself, it hold the mighty mpop.satellites.get_satellite_class() method.

class mpop.satellites.GenericFactory

Factory for generic satellite scenes.

static create_scene(satname, satnumber, instrument, time_slot, orbit, area=None, variant='')

Create a compound satellite scene.

class mpop.satellites.GeostationaryFactory

Factory for geostationary satellite scenes.

static create_scene(satname, satnumber, instrument, time_slot, area=None, variant='')

Create a compound satellite scene.

class mpop.satellites.PolarFactory

Factory for polar satellite scenes.

static create_scene(satname, satnumber, instrument, time_slot, orbit=None, area=None, variant='')

Create a compound satellite scene.

mpop.satellites.build_instrument_compositer(instrument_name)

Automatically generate an instrument compositer class from its instrument_name. The class is then filled with custom composites if there are any (see get_custom_composites())

mpop.satellites.build_sat_instr_compositer()

Build a compositer class for the given satellite (defined by the three strings satellite, number, and variant) and instrument on the fly, using data from a corresponding config file. They inherit from the corresponding instrument class, which is also created on the fly is no predefined module (containing a compositer) for this instrument is available (see build_instrument_compositer()).

mpop.satellites.get_custom_composites(name)

Get the home made methods for building composites for a given satellite or instrument name.

mpop.satellites.get_sat_instr_compositer()

Get the compositer class for a given satellite, defined by the three strings satellite, number, and variant, and instrument. The class is then filled with custom composites if there are any (see get_custom_composites()). If no class is found, an attempt is made to build the class from a corresponding configuration file, see build_sat_instr_compositer().

Miscellaneous tools

Helper functions for eg. performing Sun zenith angle correction.

mpop.tools.estimate_cth(IR_108, cth_atm='standard')

Estimation of the cloud top height using the 10.8 micron channel limitations: this is the most simple approach

a simple fit of the ir108 to the temperature profile * no correction for water vapour or any other trace gas * no viewing angle dependency * no correction for semi-transparent clouds
optional input:
cth_atm * “standard”, “tropics”, “midlatitude summer”, “midlatitude winter”, “subarctic summer”, “subarctic winter”
Matching the 10.8 micron temperature with atmosphere profile (s) AFGL atmospheric constituent profile. U.S. standard atmosphere 1976. (AFGL-TR-86-0110) (t) AFGL atmospheric constituent profile. tropical. (AFGL-TR-86-0110) (mw) AFGL atmospheric constituent profile. midlatitude summer. (AFGL-TR-86-0110) (ms) AFGL atmospheric constituent profile. midlatitude winter. (AFGL-TR-86-0110) (ss) AFGL atmospheric constituent profile. subarctic summer. (AFGL-TR-86-0110) (sw) AFGL atmospheric constituent profile. subarctic winter. (AFGL-TR-86-0110) Ulrich Hamann (MeteoSwiss)
  • “tropopause” Assuming a fixed tropopause height and a fixed temperature gradient Richard Mueller (DWD)
output:
parallax corrected channel
the content of the channel will be parallax corrected. The name of the new channel will be original_chan.name+’_PC’, eg. “IR_108_PC”. This name is also stored to the info dictionary of the originating channel.
Versions: 05.07.2016 initial version
Ulrich Hamann (MeteoSwiss), Richard Mueller (DWD)
mpop.tools.sunzen_corr_cos(data, cos_zen, limit=80.0)

Perform Sun zenith angle correction to the given data using cosine of the zenith angle (cos_zen). The correction is limited to limit degrees (default: 80.0 degrees). For larger zenith angles, the correction is the same as at the limit. Both data and cos_zen are given as 2-dimensional Numpy arrays or Numpy MaskedArrays, and they should have equal shapes.

mpop.tools.viewzen_corr(data, view_zen)

Apply atmospheric correction on the given data using the specified satellite zenith angles (view_zen). Both input data are given as 2-dimensional Numpy (masked) arrays, and they should have equal shapes. The data array will be changed in place and has to be copied before.