Making use of the mpop
package¶
The mpop
package is the heart of mpop: here are defined the core classes
which the user will then need to build satellite composites.
Conventions about satellite names¶
Throughout the document, we will use the following conventions:
- platform name is the name of an individual satellite following the OSCAR naming scheme, e.g. “NOAA-19”.
- variant will be used to differentiate the same data (from the same
- satellite and instrument) coming in different flavours. For example, we use variant to distinguish data coming from the satellite Metop-B from direct readout (no variant), regional coverage (EARS) or global coverage (GDS).
All the satellite configuration files in PPP_CONFIG_DIR should be named <variant><platform name>.cfg, e.g. NOAA-19.cfg or GDSMetop-B.cfg.
Creating a scene object¶
Creating a scene object can be done calling the create_scene function of a
factory, (for example mpop.satellites.GenericFactory.create_scene()
).
The reader is refered to the documentation of the
mpop.scene.SatelliteInstrumentScene()
for a description of the input
arguments.
Such a scene object is roughly a container for mpop.channel.Channel
objects, which hold the actual data and information for each band.
Loading the data¶
Loading the data is done through the
mpop.scene.SatelliteInstrumentScene.load()
method. Calling it effectively
loads the data from disk into memory, so it can take a while depending on the
volume of data to load and the performance of the host computer. The channels
listed as arguments become loaded, and cannot be reloaded: a subsequent call to
the method will not reload the data from disk.
Re-projecting data¶
Once the data is loaded, one might need to re-project the data. The scene
objects can be projected onto other areas if the pyresample software is
installed, thanks to the mpop.scene.SatelliteInstrumentScene.project()
method. As input, this method takes either a Definition object (see
pyresample’s documentation) or string identificator for the area. In the latter
case, the referenced region has to be defined in the area file. The name and
location of this file is defined in the mpop.cfg configuration file, itself
located in the directory pointed by the PPP_CONFIG_DIR environment variable.
For more information about the internals of the projection process, take a look
at the mpop.projector
module.
Geo-localisation of the data¶
Once the data is loaded, each channel should have an area attribute
containing a pyresample area object, if the pyresample package is
available. These area objects should implement the get_lonlats()
method,
returning the longitudes and latitudes of the channel data. For more
information on this matter, the reader is then referred to the documentation of
the aforementioned package.
Image composites¶
Methods building image composites are distributed in different modules, taking advantage of the hierarchical structure offered by OOP.
The image composites common to all visir instruments are defined in the
mpop.instruments.visir
module. Some instrument modules, like
mpop.instruments.avhrr
or mpop.instruments.seviri
overload these
methods to adapt better for the instrument at hand.
For instructions on how to write a new composites, see Geographic images.
Adding a new satellite: configuration file¶
A satellite configuration file looks like the following (here Meteosat-7, mviri instrument):
The configuration file must hold a satellite section, the list of channels for the needed instruments (here mviri-n sections), and how to read the data in mipp (mviri-level1) and how to read it in mpop (mviri-level2).
Using this template we can define new satellite and instruments.
Adding a new satellite: python code¶
Another way of adding satellites and instruments to mpop is to write the correponding python code.
Here are example of such code:
The mpop
API¶
Satellite scenes¶
The mpop.scene
module defines satellite scenes. They are defined as
generic classes, to be inherited when needed.
A scene is a set of mpop.channel
objects for a given time, and sometimes
also for a given area.
-
class
mpop.scene.
Satellite
¶ This is the satellite class. It contains information on the satellite.
-
classmethod
add_method
(func)¶ Add a method to the class.
-
add_method_to_instance
(func)¶ Add a method to the instance.
-
fullname
¶ Full name of the satellite, that is platform name and number (eg “metop02”).
-
classmethod
remove_attribute
(name)¶ Remove an attribute from the class.
-
sat_nr
(string=False)¶
-
classmethod
-
class
mpop.scene.
SatelliteInstrumentScene
(time_slot=None, area_id=None, area=None, orbit=None, satellite=(None, None, None), instrument=None)¶ This is the satellite instrument class. It is an abstract channel container, from which all concrete satellite scenes should be derived.
The constructor accepts as optional arguments the time_slot of the scene, the area on which the scene is defined (this can be used for slicing of big datasets, or can be set automatically when loading), and orbit which is a string giving the orbit number.
-
add_to_history
(message)¶ Adds a message to history info.
-
channel_list
= []¶
-
check_channels
(*channels)¶ Check if the channels are loaded, raise an error otherwise.
-
estimate_cth
(cth_atm='best', time_slot=None)¶ Estimation of the cloud top height using the 10.8 micron channel limitations: this is the most simple approach a simple fit of the ir108 to the temperature profile
- no correction for water vapour or any other trace gas
- no viewing angle dependency
- no correction for semi-transparent clouds
- no special treatment of temperature inversions
data.estimate_cth(cth_atm=”best”)- cth_atm * using temperature profile to estimate the cloud top height
- possible choices are (see estimate_cth in mpop/tools.py): “standard”, “tropics”, “midlatitude summer”, “midlatitude winter”, “subarctic summer”, “subarctic winter” this will choose the corresponding atmospheric AFGL temperature profile
- new choice: “best” -> choose according to central (lon,lat) and time from: “tropics”, “midlatitude summer”, “midlatitude winter”, “subarctic summer”, “subarctic winter”
- time_slot current observation time as (datetime.datetime() object)
- time_slot option can be omitted, the function tries to use self.time_slot
-
get_orbital
()¶
-
load
(channels=None, load_again=False, area_extent=None, **kwargs)¶ Load instrument data into the channels. Channels is a list or a tuple containing channels we will load data into, designated by there center wavelength (float), resolution (integer) or name (string). If None, all channels are loaded.
The load_again boolean flag allows to reload the channels even they have already been loaded, to mirror changes on disk for example. This is false by default.
The area_extent keyword lets you specify which part of the data to load. Given as a 4-element sequence, it defines the area extent to load in satellite projection.
The other keyword arguments are passed as is to the reader plugin. Check the corresponding documentation for more details.
-
loaded_channels
()¶ Return the set of loaded_channels.
-
parallax_corr
(fill='False', estimate_cth=False, cth_atm='best', replace=False)¶ perform the CTH parallax corretion for all loaded channels
-
project
(dest_area, channels=None, precompute=False, mode=None, radius=None, nprocs=1)¶ Make a copy of the current snapshot projected onto the dest_area. Available areas are defined in the region configuration file (ACPG). channels tells which channels are to be projected, and if None, all channels are projected and copied over to the return snapshot.
If precompute is set to true, the projecting data is saved on disk for reusage. mode sets the mode to project in: ‘quick’ which works between cartographic projections, and, as its denomination indicates, is quick (but lower quality), and ‘nearest’ which uses nearest neighbour for best projection. A mode set to None uses ‘quick’ when possible, ‘nearest’ otherwise.
radius defines the radius of influence for neighbour search in ‘nearest’ mode (in metres). Setting it to None, or omitting it will fallback to default values (5 times the channel resolution) or 10,000m if the resolution is not available.
Note: channels have to be loaded to be projected, otherwise an exception is raised.
-
save
(filename, to_format='netcdf4', **options)¶ Saves the current scene into a file of format to_format. Supported formats are:
- netcdf4: NetCDF4 with CF conventions.
-
unload
(*channels)¶ Unloads channels from memory.
mpop.scene.SatelliteInstrumentScene.load()
must be called again to reload the data.
-
-
class
mpop.scene.
SatelliteScene
(time_slot=None, area_id=None, area=None, orbit=None, satellite=(None, None, None))¶ This is the satellite scene class. It is a capture of the satellite (channels) data at given time_slot and area_id/area.
-
area
¶ Getter for area.
-
get_area
()¶ Getter for area.
-
set_area
(area)¶ Setter for area.
-
-
mpop.scene.
assemble_segments
(segments)¶ Assemble the scene objects listed in segment_list and returns the resulting scene object.
Instrument channels¶
This module defines satellite instrument channels as a generic class, to be inherited when needed.
-
class
mpop.channel.
Channel
(name=None, resolution=0, wavelength_range=[0, 0, 0], data=None, calibration_unit=None)¶ This is the satellite channel class. It defines satellite channels as a container for calibrated channel data.
The resolution sets the resolution of the channel, in meters. The wavelength_range is a triplet, containing the lowest-, center-, and highest-wavelength values of the channel. name is simply the given name of the channel, and data is the data it should hold.
-
as_image
(stretched=True)¶ Return the channel as a
mpop.imageo.geo_image.GeoImage
object. The stretched argument set to False allows the data to remain untouched (as opposed to crude stretched by default to obtain the same output asshow()
).
-
check_range
(min_range=1.0)¶ Check that the data of the channels has a definition domain broader than min_range and return the data, otherwise return zeros.
-
data
¶ Getter for channel data.
-
get_data
()¶ Getter for channel data.
-
get_reflectance
(tb11, sun_zenith=None, tb13_4=None)¶ Get the reflectance part of an NIR channel
-
get_viewing_geometry
(orbital, time_slot, altitude=None)¶ Calculates the azimuth and elevation angle as seen by the observer at the position of the current area pixel. inputs:
- orbital an orbital object define by the tle file
- (see pyorbital.orbital import Orbital or mpop/scene.py get_oribtal)
time_slot time object specifying the observation time altitude optinal: altitude of the observer above the earth ellipsoid
- outputs:
- azi azimuth viewing angle in degree (south is 0, counting clockwise) ele elevation viewing angle in degree (zenith is 90, horizon is 0)
-
is_loaded
()¶ Tells if the channel contains loaded data.
-
parallax_corr
(cth=None, time_slot=None, orbital=None, azi=None, ele=None, fill='False')¶ Perform the parallax correction for channel at time_slot (datetime.datetime() object), assuming the cloud top height cth and the viewing geometry given by the satellite orbital “orbital” and return the corrected channel. Authors: Ulrich Hamann (MeteoSwiss), Thomas Leppelt (DWD) Example calls:
- calling this function (using orbital and time_slot)
- orbital = data.get_oribtal() data[“VIS006”].parallax_corr(cth=data[“CTTH”].height, time_slot=data.time_slot, orbital=orbital)
- calling this function (using viewing geometry)
- orbital = data.get_oribtal() (azi, ele) = get_viewing_geometry(self, orbital, time_slot) data[“VIS006”].parallax_corr(cth=data[“CTTH”].height, azi=azi, ele=ele)
- Optional input:
- cth The parameter cth is the cloud top height
- (or the altitude of the object that should be shifted). cth must have the same size and projection as the channel
- orbital an orbital object define by the tle file
- (see pyorbital.orbital import Orbital or mpop/scene.py get_oribtal)
- azi azimuth viewing angle in degree (south is 0, counting clockwise)
- e.g. as given by self.get_viewing_geometry
- ele elevation viewing angle in degree (zenith is 90, horizon is 0)
- e.g. as given by self.get_viewing_geometry
- fill specifies the interpolation method to fill the gaps
(basically areas behind the cloud that can’t be observed by the satellite instrument) “False” (default): no interpolation, gaps are np.nan values and mask is set accordingly “nearest”: fill gaps with nearest neighbour “bilinear”: use scipy.interpolate.griddata with linear interpolation
to fill the gaps
- output:
- parallax corrected channel
- the content of the channel will be parallax corrected. The name of the new channel will be original_chan.name+’_PC’, eg. “IR_108_PC”. This name is also stored to the info dictionary of the originating channel.
-
project
(coverage_instance)¶ Make a projected copy of the current channel using the given coverage_instance.
See also the
mpop.projector
module.
-
set_data
(data)¶ Setter for channel data.
-
shape
¶ Shape of the channel.
-
show
()¶ Display the channel as an image.
-
sunzen_corr
(time_slot, lonlats=None, limit=80.0, mode='cos', sunmask=False)¶ Perform Sun zenith angle correction for the channel at time_slot (datetime.datetime() object) and return the corrected channel. The parameter limit can be used to set the maximum zenith angle for which the correction is calculated. For larger angles, the correction is the same as at the limit (default: 80.0 degrees). Coordinate values can be given as a 2-tuple or a two-element list lonlats of numpy arrays; if None, the coordinates will be read from the channel data. Parameter mode is a placeholder for other possible illumination corrections. The name of the new channel will be original_chan.name+’_SZC’, eg. “VIS006_SZC”. This name is also stored to the info dictionary of the originating channel.
-
viewzen_corr
(view_zen_angle_data)¶ Apply atmospheric correction on a copy of this channel data using the given satellite zenith angle data of the same shape. Returns a new channel containing the corrected data. The name of the new channel will be original_chan.name+’_VZC’, eg. “IR108_VZC”. This name is also stored to the info dictionary of the originating channel.
-
vinc_vect
(phi, lembda, alpha, s, f=None, a=None, degree=True)¶ Vincenty’s Direct formular
Returns the lat and long of projected point and reverse azimuth given a reference point and a distance and azimuth to project. lats, longs and azimuths are passed in radians.
- Keyword arguments:
phi Latitude in degree/radians lembda Longitude in degree/radians alpha Geodetic azimuth in degree/radians s Ellipsoidal distance in meters f WGS84 parameter a WGS84 parameter degree Boolean if in/out values are in degree or radians.
Default is in degree- Returns:
- (phiout, lembdaout, alphaout ) as a tuple
-
-
class
mpop.channel.
GenericChannel
(name=None)¶ This is an abstract channel class. It can be a super class for calibrated channels data or more elaborate channels such as cloudtype or CTTH.
-
area
¶ Getter for area.
-
-
exception
mpop.channel.
GeolocationIncompleteError
¶ Exception to try catch cases where the original data have not been read or expanded properly so that each pixel has a geo-location
-
exception
mpop.channel.
NotLoadedError
¶ Exception to be raised when attempting to use a non-loaded channel.
The VisIr instrument class¶
This module defines the generic VISIR instrument class.
-
class
mpop.instruments.visir.
VisirCompositer
(scene)¶ Compositer for Visual-IR instruments
-
airmass
(fill_value=(0, 0, 0))¶ Make an airmass RGB image composite.
Channels Temp Gamma WV6.2 - WV7.3 -25 to 0 K gamma 1 IR9.7 - IR10.8 -40 to 5 K gamma 1 WV6.2 243 to 208 K gamma 1
-
ash
(fill_value=(0, 0, 0))¶ Make a Ash RGB image composite.
Channels Temp Gamma IR12.0 - IR10.8 -4 to 2 K gamma 1 IR10.8 - IR8.7 -4 to 5 K gamma 1 IR10.8 243 to 303 K gamma 1
-
channel_image
(channel, fill_value=0)¶ Make a black and white image of the channel.
Linear stretch without clipping is applied by default.
-
cloudtop
(stretch=(0.005, 0.005), gamma=None, fill_value=(0, 0, 0))¶ Make a Cloudtop RGB image composite.
Channels Gamma IR3.9 (inverted) gamma 1 IR10.8 (inverted) gamma 1 IR12.0 (inverted) gamma 1 Linear stretch with 0.5 % clipping at both ends.
-
convection
(fill_value=(0, 0, 0))¶ Make a Severe Convection RGB image composite.
Channels Span Gamma WV6.2 - WV7.3 -30 to 0 K gamma 1 IR3.9 - IR10.8 0 to 55 K gamma 1 IR1.6 - VIS0.6 -70 to 20 % gamma 1
-
dust
(fill_value=(0, 0, 0))¶ Make a Dust RGB image composite.
Channels Temp Gamma IR12.0 - IR10.8 -4 to 2 K gamma 1 IR10.8 - IR8.7 0 to 15 K gamma 2.5 IR10.8 261 to 289 K gamma 1
-
fog
(fill_value=(0, 0, 0))¶ Make a Fog RGB image composite.
Channels Temp Gamma IR12.0 - IR10.8 -4 to 2 K gamma 1 IR10.8 - IR8.7 0 to 6 K gamma 2.0 IR10.8 243 to 283 K gamma 1
-
green_snow
(fill_value=(0, 0, 0))¶ Make a Green Snow RGB image composite.
Channels Gamma IR1.6 gamma 1.6 VIS0.6 gamma 1.6 IR10.8 (inverted) gamma 1.6 Linear stretch without clipping.
-
ir108
()¶ Make a black and white image of the IR 10.8um channel.
Channel is inverted. Temperature range from -70 °C (white) to +57.5 °C (black) is shown.
-
natural
(stretch=None, gamma=1.8, fill_value=(0, 0, 0))¶ Make a Natural Colors RGB image composite.
Channels Range (reflectance) Gamma (default) IR1.6 0 - 90 gamma 1.8 VIS0.8 0 - 90 gamma 1.8 VIS0.6 0 - 90 gamma 1.8
-
night_fog
(fill_value=(0, 0, 0))¶ Make a Night Fog RGB image composite.
Channels Temp Gamma IR12.0 - IR10.8 -4 to 2 K gamma 1 IR10.8 - IR3.9 0 to 6 K gamma 2.0 IR10.8 243 to 293 K gamma 1
-
night_overview
(stretch='histogram', gamma=None)¶ Make an overview RGB image composite using IR channels.
Channels Gamma IR3.9 (inverted) gamma 1 IR10.8 (inverted) gamma 1 IR12.0 (inverted) gamma 1 Histogram equalization is applied for each channel.
-
overview
(stretch='crude', gamma=1.6, fill_value=(0, 0, 0))¶ Make an overview RGB image composite.
Channels Gamma (default) VIS0.6 gamma 1.6 VIS0.8 gamma 1.6 IR10.8 (inverted) gamma 1.6 Linear stretch without clipping is applied.
-
overview_sun
(stretch='linear', gamma=1.6, fill_value=(0, 0, 0))¶ Make an overview RGB image composite normalising with cosine to the sun zenith angle.
-
red_snow
(fill_value=(0, 0, 0))¶ Make a Red Snow RGB image composite.
Channels Gamma VIS0.6 gamma 1.6 IR1.6 gamma 1.6 IR10.8 (inverted) gamma 1.6 Linear stretch without clipping.
-
vis06
()¶ Make a black and white image of the VIS 0.635um channel.
Linear stretch without clipping is applied.
-
wv_high
()¶ Make a black and white image of the IR 6.7um channel.
Channel inverted and a linear stretch is applied with 0.5 % clipping at both ends.
-
wv_low
()¶ Make a black and white image of the IR 7.3um channel.
Channel data inverted and a linear stretch is applied with 0.5 % clipping at both ends.
-
Projection facility¶
Satellite class loader¶
mpop.satellites is the module englobes all satellite specific modules. In
itself, it hold the mighty mpop.satellites.get_satellite_class()
method.
-
class
mpop.satellites.
GenericFactory
¶ Factory for generic satellite scenes.
-
static
create_scene
(satname, satnumber, instrument, time_slot, orbit, area=None, variant='')¶ Create a compound satellite scene.
-
static
-
class
mpop.satellites.
GeostationaryFactory
¶ Factory for geostationary satellite scenes.
-
static
create_scene
(satname, satnumber, instrument, time_slot, area=None, variant='')¶ Create a compound satellite scene.
-
static
-
class
mpop.satellites.
PolarFactory
¶ Factory for polar satellite scenes.
-
static
create_scene
(satname, satnumber, instrument, time_slot, orbit=None, area=None, variant='')¶ Create a compound satellite scene.
-
static
-
mpop.satellites.
build_instrument_compositer
(instrument_name)¶ Automatically generate an instrument compositer class from its instrument_name. The class is then filled with custom composites if there are any (see
get_custom_composites()
)
-
mpop.satellites.
build_sat_instr_compositer
()¶ Build a compositer class for the given satellite (defined by the three strings satellite, number, and variant) and instrument on the fly, using data from a corresponding config file. They inherit from the corresponding instrument class, which is also created on the fly is no predefined module (containing a compositer) for this instrument is available (see
build_instrument_compositer()
).
-
mpop.satellites.
get_custom_composites
(name)¶ Get the home made methods for building composites for a given satellite or instrument name.
-
mpop.satellites.
get_sat_instr_compositer
()¶ Get the compositer class for a given satellite, defined by the three strings satellite, number, and variant, and instrument. The class is then filled with custom composites if there are any (see
get_custom_composites()
). If no class is found, an attempt is made to build the class from a corresponding configuration file, seebuild_sat_instr_compositer()
.
Miscellaneous tools¶
Helper functions for eg. performing Sun zenith angle correction.
-
mpop.tools.
estimate_cth
(IR_108, cth_atm='standard')¶ Estimation of the cloud top height using the 10.8 micron channel limitations: this is the most simple approach
a simple fit of the ir108 to the temperature profile * no correction for water vapour or any other trace gas * no viewing angle dependency * no correction for semi-transparent clouds- optional input:
- cth_atm * “standard”, “tropics”, “midlatitude summer”, “midlatitude winter”, “subarctic summer”, “subarctic winter”
- Matching the 10.8 micron temperature with atmosphere profile (s) AFGL atmospheric constituent profile. U.S. standard atmosphere 1976. (AFGL-TR-86-0110) (t) AFGL atmospheric constituent profile. tropical. (AFGL-TR-86-0110) (mw) AFGL atmospheric constituent profile. midlatitude summer. (AFGL-TR-86-0110) (ms) AFGL atmospheric constituent profile. midlatitude winter. (AFGL-TR-86-0110) (ss) AFGL atmospheric constituent profile. subarctic summer. (AFGL-TR-86-0110) (sw) AFGL atmospheric constituent profile. subarctic winter. (AFGL-TR-86-0110) Ulrich Hamann (MeteoSwiss)
- “tropopause” Assuming a fixed tropopause height and a fixed temperature gradient Richard Mueller (DWD)
- output:
- parallax corrected channel
- the content of the channel will be parallax corrected. The name of the new channel will be original_chan.name+’_PC’, eg. “IR_108_PC”. This name is also stored to the info dictionary of the originating channel.
- Versions: 05.07.2016 initial version
- Ulrich Hamann (MeteoSwiss), Richard Mueller (DWD)
-
mpop.tools.
sunzen_corr_cos
(data, cos_zen, limit=80.0)¶ Perform Sun zenith angle correction to the given data using cosine of the zenith angle (cos_zen). The correction is limited to limit degrees (default: 80.0 degrees). For larger zenith angles, the correction is the same as at the limit. Both data and cos_zen are given as 2-dimensional Numpy arrays or Numpy MaskedArrays, and they should have equal shapes.
-
mpop.tools.
viewzen_corr
(data, view_zen)¶ Apply atmospheric correction on the given data using the specified satellite zenith angles (view_zen). Both input data are given as 2-dimensional Numpy (masked) arrays, and they should have equal shapes. The data array will be changed in place and has to be copied before.