Met Office MOGREPS data¶
The Met Office has made data from its world leading MOGPREPS weather model available for research purposes. In this page, we'll go over the basics of how to download the data, how you can look at it, and what it means.
Data from two models is available:
- MOGREPS-UK: a high resolution weather forecast covering the UK, and
- MOGREPS-G: a global weather forecast.
Amazon Web Services are kindly hosting this data for free as part of their Open Data Initiative.
How is the data structured?¶
Our weather forecast models predict many weather parameters, such as air temperature, humidity or rainfall. These are predicted for every point on a regular grid, that is, as a function of latitude, longitude, and (when appropriate) altitude.
With the advent of our latest supercomputer, we now regularly run "ensemble forecasts". This means that instead of starting one model run now to predict tomorrow, we start many, purturbing the intial parameters to give several possible futures (or realisations). The spread of these different realisations gives an indication of the uncertainty of the forecast, and can be very powerful information.
Each forecast point also has several measures of time. Firstly, time indicates the real world time that the data is forecast for. Secondly, reference_time is the time that the forecast was started from i.e. the 0th time point. Finally, forecast_period is the difference between the two i.e. the number of hours that have elapsed since the start of the forecast.
Downloading data¶
Here, we'll show you a simple way to download files using Python. The data is stored in AWS S3 buckets hosted in the eu-west-2 region, named mogreps-uk and mogreps-g. If that doesn't mean anything to you, don't worry - have a look at the Python functions below.
make_data_object_name takes infomation about the data you are looking for and constructs it into a data object name (you can think of this as a file name). We'll go into what things like forecast_period and realization are in a minute. download data then saves this object to the same directory as your notebook.
import urllib.request
def make_data_object_name(dataset_name, year, month, day, hour, realization, forecast_period):
template_string = "prods_op_{}_{:02d}{:02d}{:02d}_{:02d}_{:02d}_{:03d}.nc"
return template_string.format(dataset_name, year, month, day, hour, realization, forecast_period)
def download_data_object(dataset_name, data_object_name):
url = "https://s3.eu-west-2.amazonaws.com/" + dataset_name + "/" + data_object_name
urllib.request.urlretrieve(url, data_object_name) # save in this directory with same name
Note that you can also use Python modules such as Boto to list, filter and retrieve these files.
We'll use these functions to download one data object.
obj_name = make_data_object_name('mogreps-uk', 2013, 1, 1, 3, 0, 3)
download_data_object('mogreps-uk', obj_name)
and here is is.
!ls -ltrh {obj_name}
Loading Data¶
This data is in NetCDF format, which is a type of the widely used HDF format. You can use any compatible tools, but we recommend the Met Office's Iris Python module. First of all, let's make sure we've got Iris instaled using Conda.
!conda install -y -c conda-forge iris
Now let's load the file we just downloaded
import iris
mydata = iris.load("./prods_op_mogreps-uk_20130101_03_00_003.nc")
print(mydata)
You can see here that the object we have loaded from our file contains many different meteorological parameters. iris.load returns a 'CubeList' object, which is essentially a list with a few helper methods.
Exploring Data¶
We can find our more information about a particular parameter...
air_temps = mydata.extract('air_temperature')
print(air_temps)
So there are four different forecasts for 'air_temperature'. I'll show how to dig into them, but essentially there are three forecasts for ground level, and one for particular pressure levels in the atmosphere.
The three ground level forecasts are:
- minimum forecasted value over an hour period
- maximum forecasted value over an hour period
- forecast for an exact snapshot in time
Lets grab the snapshot forecast
surface_temp = air_temps[2]
print(surface_temp)
So here's the metadata for our chosen cube. We can see a lot of information here.
- The cube shape is (4 x 548 x 421)
- There are three numeric dimension coordinates (time, grid_latitude, grid_longitude)
- There are two coordinates that are not linked to a dimension (forecast_reference_time, height)
We can drill into the cube in various ways:
print(surface_temp.coord('time'))
And here's the underlying numpy array:
surface_temp.data
You can slice a cube by index to extract particular pieces of data:
# The first forecast in the dataset
print(surface_temp[0])
# All forecasts for a particular point:
print(surface_temp[:, 100, 300])
Plotting Data¶
Iris also has built-in functions to make plotting the cube easier:
%matplotlib inline
import matplotlib.pyplot as plt
import iris.quickplot as qplt
# Set the size of the plot
plt.figure(figsize=(20, 20))
# Select the first forecast
time_slice = surface_temp[0]
# iris quickplot sets up our axes and labels so we don't have to
qplt.pcolormesh(time_slice, cmap='viridis')
Finally...¶
We hope you found this page a useful introduction. Please have a look at the data page, where you can find other notebooks covering more advance uses. In particular the Met Office Informatics Lab are working on distributed cloud computing using Iris and Dask.
You can find the Iris reference documentation, a user guide and examples at the SciTools website. There's also more information about the Met Office models at on the Mogreps page.
This data is free to use for research purposes only, and not for profit making applications. Any derived products should acknowledge the use of Met Office data. No ongoing level of support is implied and data may be changed or removed without notice.