Making use of the mpop package

The mpop package is the heart of mpop: here are defined the core classes which the user will then need to build satellite composites.

Conventions about satellite names

Throughout the document, we will use the following conventions:

  • platform name is the name of an individual satellite following the OSCAR naming scheme, e.g. “NOAA-19”.

  • variant will be used to differentiate the same data (from the same

    satellite and instrument) coming in different flavours. For example, we use variant to distinguish data coming from the satellite Metop-B from direct readout (no variant), regional coverage (EARS) or global coverage (GDS).

All the satellite configuration files in PPP_CONFIG_DIR should be named <variant><platform name>.cfg, e.g. NOAA-19.cfg or GDSMetop-B.cfg.

Creating a scene object

Creating a scene object can be done calling the create_scene function of a factory, (for example mpop.satellites.GenericFactory.create_scene()).

The reader is refered to the documentation of the mpop.scene.SatelliteInstrumentScene() for a description of the input arguments.

Such a scene object is roughly a container for mpop.channel.Channel objects, which hold the actual data and information for each band.

Loading the data

Loading the data is done through the mpop.scene.SatelliteInstrumentScene.load() method. Calling it effectively loads the data from disk into memory, so it can take a while depending on the volume of data to load and the performance of the host computer. The channels listed as arguments become loaded, and cannot be reloaded: a subsequent call to the method will not reload the data from disk.

Re-projecting data

Once the data is loaded, one might need to re-project the data. The scene objects can be projected onto other areas if the pyresample software is installed, thanks to the mpop.scene.SatelliteInstrumentScene.project() method. As input, this method takes either a Definition object (see pyresample’s documentation) or string identificator for the area. In the latter case, the referenced region has to be defined in the area file. The name and location of this file is defined in the mpop.cfg configuration file, itself located in the directory pointed by the PPP_CONFIG_DIR environment variable.

For more information about the internals of the projection process, take a look at the mpop.projector module.

Geo-localisation of the data

Once the data is loaded, each channel should have an area attribute containing a pyresample area object, if the pyresample package is available. These area objects should implement the get_lonlats() method, returning the longitudes and latitudes of the channel data. For more information on this matter, the reader is then referred to the documentation of the aforementioned package.

Image composites

Methods building image composites are distributed in different modules, taking advantage of the hierarchical structure offered by OOP.

The image composites common to all visir instruments are defined in the mpop.instruments.visir module. Some instrument modules, like mpop.instruments.avhrr or mpop.instruments.seviri overload these methods to adapt better for the instrument at hand.

For instructions on how to write a new composites, see Geographic images.

Adding a new satellite: configuration file

A satellite configuration file looks like the following (here Meteosat-7, mviri instrument):

The configuration file must hold a satellite section, the list of channels for the needed instruments (here mviri-n sections), and how to read the data in mipp (mviri-level1) and how to read it in mpop (mviri-level2).

Using this template we can define new satellite and instruments.

Adding a new satellite: python code

Another way of adding satellites and instruments to mpop is to write the correponding python code.

Here are example of such code:

The mpop API

Satellite scenes

The mpop.scene module defines satellite scenes. They are defined as generic classes, to be inherited when needed.

A scene is a set of mpop.channel objects for a given time, and sometimes also for a given area.

class mpop.scene.Satellite((satname, number, variant)=(None, None, None))

This is the satellite class. It contains information on the satellite.

classmethod add_method(func)

Add a method to the class.

add_method_to_instance(func)

Add a method to the instance.

fullname

Full name of the satellite, that is platform name and number (eg “metop02”).

classmethod remove_attribute(name)

Remove an attribute from the class.

class mpop.scene.SatelliteInstrumentScene(time_slot=None, area_id=None, area=None, orbit=None, satellite=(None, None, None), instrument=None)

This is the satellite instrument class. It is an abstract channel container, from which all concrete satellite scenes should be derived.

The constructor accepts as optional arguments the time_slot of the scene, the area on which the scene is defined (this can be used for slicing of big datasets, or can be set automatically when loading), and orbit which is a string giving the orbit number.

add_to_history(message)

Adds a message to history info.

channel_list = []
check_channels(*channels)

Check if the channels are loaded, raise an error otherwise.

load(channels=None, load_again=False, area_extent=None, **kwargs)

Load instrument data into the channels. Channels is a list or a tuple containing channels we will load data into, designated by there center wavelength (float), resolution (integer) or name (string). If None, all channels are loaded.

The load_again boolean flag allows to reload the channels even they have already been loaded, to mirror changes on disk for example. This is false by default.

The area_extent keyword lets you specify which part of the data to load. Given as a 4-element sequence, it defines the area extent to load in satellite projection.

The other keyword arguments are passed as is to the reader plugin. Check the corresponding documentation for more details.

loaded_channels()

Return the set of loaded_channels.

project(dest_area, channels=None, precompute=False, mode=None, radius=None, nprocs=1)

Make a copy of the current snapshot projected onto the dest_area. Available areas are defined in the region configuration file (ACPG). channels tells which channels are to be projected, and if None, all channels are projected and copied over to the return snapshot.

If precompute is set to true, the projecting data is saved on disk for reusage. mode sets the mode to project in: ‘quick’ which works between cartographic projections, and, as its denomination indicates, is quick (but lower quality), and ‘nearest’ which uses nearest neighbour for best projection. A mode set to None uses ‘quick’ when possible, ‘nearest’ otherwise.

radius defines the radius of influence for neighbour search in ‘nearest’ mode (in metres). Setting it to None, or omitting it will fallback to default values (5 times the channel resolution) or 10,000m if the resolution is not available.

Note: channels have to be loaded to be projected, otherwise an exception is raised.

save(filename, to_format='netcdf4', **options)

Saves the current scene into a file of format to_format. Supported formats are:

  • netcdf4: NetCDF4 with CF conventions.
unload(*channels)

Unloads channels from memory. mpop.scene.SatelliteInstrumentScene.load() must be called again to reload the data.

class mpop.scene.SatelliteScene(time_slot=None, area_id=None, area=None, orbit=None, satellite=(None, None, None))

This is the satellite scene class. It is a capture of the satellite (channels) data at given time_slot and area_id/area.

area

Getter for area.

get_area()

Getter for area.

set_area(area)

Setter for area.

mpop.scene.assemble_segments(segments)

Assemble the scene objects listed in segment_list and returns the resulting scene object.

Instrument channels

This module defines satellite instrument channels as a generic class, to be inherited when needed.

class mpop.channel.Channel(name=None, resolution=0, wavelength_range=[0, 0, 0], data=None, calibration_unit=None)

This is the satellite channel class. It defines satellite channels as a container for calibrated channel data.

The resolution sets the resolution of the channel, in meters. The wavelength_range is a triplet, containing the lowest-, center-, and highest-wavelength values of the channel. name is simply the given name of the channel, and data is the data it should hold.

as_image(stretched=True)

Return the channel as a mpop.imageo.geo_image.GeoImage object. The stretched argument set to False allows the data to remain untouched (as opposed to crude stretched by default to obtain the same output as show()).

check_range(min_range=1.0)

Check that the data of the channels has a definition domain broader than min_range and return the data, otherwise return zeros.

data

Getter for channel data.

get_data()

Getter for channel data.

get_reflectance(tb11, sun_zenith=None, tb13_4=None)

Get the reflectance part of an NIR channel

is_loaded()

Tells if the channel contains loaded data.

project(coverage_instance)

Make a projected copy of the current channel using the given coverage_instance.

See also the mpop.projector module.

set_data(data)

Setter for channel data.

shape

Shape of the channel.

show()

Display the channel as an image.

sunzen_corr(time_slot, lonlats=None, limit=80.0, mode='cos', sunmask=False)

Perform Sun zenith angle correction for the channel at time_slot (datetime.datetime() object) and return the corrected channel. The parameter limit can be used to set the maximum zenith angle for which the correction is calculated. For larger angles, the correction is the same as at the limit (default: 80.0 degrees). Coordinate values can be given as a 2-tuple or a two-element list lonlats of numpy arrays; if None, the coordinates will be read from the channel data. Parameter mode is a placeholder for other possible illumination corrections. The name of the new channel will be original_chan.name+’_SZC’, eg. “VIS006_SZC”. This name is also stored to the info dictionary of the originating channel.

class mpop.channel.GenericChannel(name=None)

This is an abstract channel class. It can be a super class for calibrated channels data or more elaborate channels such as cloudtype or CTTH.

area

Getter for area.

exception mpop.channel.GeolocationIncompleteError

Exception to try catch cases where the original data have not been read or expanded properly so that each pixel has a geo-location

exception mpop.channel.NotLoadedError

Exception to be raised when attempting to use a non-loaded channel.

The VisIr instrument class

This module defines the generic VISIR instrument class.

class mpop.instruments.visir.VisirCompositer(scene)

Compositer for Visual-IR instruments

airmass()

Make an airmass RGB image composite.

Channels Temp Gamma
WV6.2 - WV7.3 -25 to 0 K gamma 1
IR9.7 - IR10.8 -40 to 5 K gamma 1
WV6.2 243 to 208 K gamma 1
ash()

Make a Ash RGB image composite.

Channels Temp Gamma
IR12.0 - IR10.8 -4 to 2 K gamma 1
IR10.8 - IR8.7 -4 to 5 K gamma 1
IR10.8 243 to 303 K gamma 1
channel_image(channel, fill_value=0)

Make a black and white image of the channel.

Linear stretch without clipping is applied by default.

cloudtop(stretch=(0.005, 0.005), gamma=None)

Make a Cloudtop RGB image composite.

Channels Gamma
IR3.9 (inverted) gamma 1
IR10.8 (inverted) gamma 1
IR12.0 (inverted) gamma 1

Linear stretch with 0.5 % clipping at both ends.

convection()

Make a Severe Convection RGB image composite.

Channels Span Gamma
WV6.2 - WV7.3 -30 to 0 K gamma 1
IR3.9 - IR10.8 0 to 55 K gamma 1
IR1.6 - VIS0.6 -70 to 20 % gamma 1
dust()

Make a Dust RGB image composite.

Channels Temp Gamma
IR12.0 - IR10.8 -4 to 2 K gamma 1
IR10.8 - IR8.7 0 to 15 K gamma 2.5
IR10.8 261 to 289 K gamma 1
fog()

Make a Fog RGB image composite.

Channels Temp Gamma
IR12.0 - IR10.8 -4 to 2 K gamma 1
IR10.8 - IR8.7 0 to 6 K gamma 2.0
IR10.8 243 to 283 K gamma 1
green_snow()

Make a Green Snow RGB image composite.

Channels Gamma
IR1.6 gamma 1.6
VIS0.6 gamma 1.6
IR10.8 (inverted) gamma 1.6

Linear stretch without clipping.

ir108()

Make a black and white image of the IR 10.8um channel.

Channel is inverted. Temperature range from -70 °C (white) to +57.5 °C (black) is shown.

natural(stretch=None, gamma=1.8)

Make a Natural Colors RGB image composite.

Channels Range (reflectance) Gamma (default)
IR1.6 0 - 90 gamma 1.8
VIS0.8 0 - 90 gamma 1.8
VIS0.6 0 - 90 gamma 1.8
night_fog()

Make a Night Fog RGB image composite.

Channels Temp Gamma
IR12.0 - IR10.8 -4 to 2 K gamma 1
IR10.8 - IR3.9 0 to 6 K gamma 2.0
IR10.8 243 to 293 K gamma 1
night_overview(stretch='histogram', gamma=None)

Make an overview RGB image composite using IR channels.

Channels Gamma
IR3.9 (inverted) gamma 1
IR10.8 (inverted) gamma 1
IR12.0 (inverted) gamma 1

Histogram equalization is applied for each channel.

overview(stretch='crude', gamma=1.6)

Make an overview RGB image composite.

Channels Gamma (default)
VIS0.6 gamma 1.6
VIS0.8 gamma 1.6
IR10.8 (inverted) gamma 1.6

Linear stretch without clipping is applied.

overview_sun(stretch='linear', gamma=1.6)

Make an overview RGB image composite normalising with cosine to the sun zenith angle.

red_snow()

Make a Red Snow RGB image composite.

Channels Gamma
VIS0.6 gamma 1.6
IR1.6 gamma 1.6
IR10.8 (inverted) gamma 1.6

Linear stretch without clipping.

vis06()

Make a black and white image of the VIS 0.635um channel.

Linear stretch without clipping is applied.

wv_high()

Make a black and white image of the IR 6.7um channel.

Channel inverted and a linear stretch is applied with 0.5 % clipping at both ends.

wv_low()

Make a black and white image of the IR 7.3um channel.

Channel data inverted and a linear stretch is applied with 0.5 % clipping at both ends.

Projection facility

This module handles coverage objects. Such objects are used to transform area projected data by changing either the area or the projection or both. A typical usage is to transform one large area in satellite projection to an area of interest in polar projection for example.

class mpop.projector.Projector(in_area, out_area, in_latlons=None, mode=None, radius=10000, nprocs=1)

This class define projector objects. They contain the mapping information necessary for projection purposes. For efficiency reasons, generated projectors can be saved to disk for later reuse. Use the save() method for this.

To define a projector object, on has to specify in_area and out_area, and can also input the in_lonlats or the mode (‘quick’ which works only if both in- and out-areas are AreaDefinitions, or ‘nearest’). radius defines the radius of influence for nearest neighbour search in ‘nearest’ mode.

project_array(data)

Project an array data along the given Projector object.

save(resave=False)

Save the precomputation to disk, and overwrite existing file in case resave is true.

mpop.projector.get_area_def(area_name)

Get the definition of area_name from file. The file is defined to use is to be placed in the $PPP_CONFIG_DIR directory, and its name is defined in mpop’s configuration file.

mpop.projector.get_area_file()

Satellite class loader

mpop.satellites is the module englobes all satellite specific modules. In itself, it hold the mighty mpop.satellites.get_satellite_class() method.

class mpop.satellites.GenericFactory

Factory for generic satellite scenes.

static create_scene(satname, satnumber, instrument, time_slot, orbit, area=None, variant='')

Create a compound satellite scene.

class mpop.satellites.GeostationaryFactory

Factory for geostationary satellite scenes.

static create_scene(satname, satnumber, instrument, time_slot, area=None, variant='')

Create a compound satellite scene.

class mpop.satellites.PolarFactory

Factory for polar satellite scenes.

static create_scene(satname, satnumber, instrument, time_slot, orbit=None, area=None, variant='')

Create a compound satellite scene.

mpop.satellites.build_instrument_compositer(instrument_name)

Automatically generate an instrument compositer class from its instrument_name. The class is then filled with custom composites if there are any (see get_custom_composites())

mpop.satellites.build_sat_instr_compositer((satellite, number, variant), instrument)

Build a compositer class for the given satellite (defined by the three strings satellite, number, and variant) and instrument on the fly, using data from a corresponding config file. They inherit from the corresponding instrument class, which is also created on the fly is no predefined module (containing a compositer) for this instrument is available (see build_instrument_compositer()).

mpop.satellites.get_custom_composites(name)

Get the home made methods for building composites for a given satellite or instrument name.

mpop.satellites.get_sat_instr_compositer((satellite, number, variant), instrument)

Get the compositer class for a given satellite, defined by the three strings satellite, number, and variant, and instrument. The class is then filled with custom composites if there are any (see get_custom_composites()). If no class is found, an attempt is made to build the class from a corresponding configuration file, see build_sat_instr_compositer().

Miscellaneous tools

Helper functions for eg. performing Sun zenith angle correction.

mpop.tools.sunzen_corr_cos(data, cos_zen, limit=80.0)

Perform Sun zenith angle correction to the given data using cosine of the zenith angle (cos_zen). The correction is limited to limit degrees (default: 80.0 degrees). For larger zenith angles, the correction is the same as at the limit. Both data and cos_zen are given as 2-dimensional Numpy arrays or Numpy MaskedArrays, and they should have equal shapes.