3.5 Climate Model Data on Google Cloud

Note

The code for downloading climate model data in this section was adapted from Section 6.2 of Anderson and Smith [2021].

The ESGF offers the complete archive of CMIP5, CMIP6, and CORDEX climate model output data, but unfortunately, it can be prone to errors that can make downloading the data a bit cumbersome. If you’re running into difficulties accessing the data using the methods described in Section 3.4, it’s worth a try to see if the data you want is available on the Google Cloud Services (GCS) archive of CMIP data, which you can navigate by following this link. Accessing the data on GCS is similar to using OPeNDAP, in that you can do subsetting on their end, and transfer only the data you want to your own computer for analysis.

The downside of the GCS archive is that it isn’t as complete as the ESGF archive. For many models, there is only data at monthly temporal frequency, which isn’t so useful for downscaling. Another minor downside is that you’ll need to learn a new file format. While netCDF is the standard for climate model output data, GCS uses a cloud-optimized data format called zarr. As the focus of scientific computing shifts more and more towards the cloud, zarr has become increasingly popular as the format for climate data hosted on remote servers. Fortunately, xarray supports the zarr format, and once you’ve opened a file with xr.open_zarr, you can work with it like you would any other xr.Dataset or xr.DataArray.

3.5.1 Searching For Data

Similar to the ESGF, we can search the GCS climate data catalog in Python to find the files we want. We’ll do this using Pandas to access the CSV file that contains the data catalog, and then do filtering to isolate the access URLs for the desired files.

import xarray as xr
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import cartopy.crs as ccrs
import gcsfs
import zarr

# url for the CSV file that contains the data catalog
url_catalog = 'https://storage.googleapis.com/cmip6/cmip6-zarr-consolidated-stores.csv'

# open the data catalog with pandas, and take a peek at how it's formatted
df_catalog = pd.read_csv(url_catalog)
print(df_catalog.columns)
df_catalog.head()

Index(['activity_id', 'institution_id', 'source_id', 'experiment_id',
       'member_id', 'table_id', 'variable_id', 'grid_label', 'zstore',
       'dcpp_init_year', 'version'],
      dtype='object')

	activity_id	institution_id	source_id	experiment_id	member_id	table_id	variable_id	grid_label	zstore	dcpp_init_year	version
0	HighResMIP	CMCC	CMCC-CM2-HR4	highresSST-present	r1i1p1f1	Amon	ps	gn	gs://cmip6/CMIP6/HighResMIP/CMCC/CMCC-CM2-HR4/...	NaN	20170706
1	HighResMIP	CMCC	CMCC-CM2-HR4	highresSST-present	r1i1p1f1	Amon	rsds	gn	gs://cmip6/CMIP6/HighResMIP/CMCC/CMCC-CM2-HR4/...	NaN	20170706
2	HighResMIP	CMCC	CMCC-CM2-HR4	highresSST-present	r1i1p1f1	Amon	rlus	gn	gs://cmip6/CMIP6/HighResMIP/CMCC/CMCC-CM2-HR4/...	NaN	20170706
3	HighResMIP	CMCC	CMCC-CM2-HR4	highresSST-present	r1i1p1f1	Amon	rlds	gn	gs://cmip6/CMIP6/HighResMIP/CMCC/CMCC-CM2-HR4/...	NaN	20170706
4	HighResMIP	CMCC	CMCC-CM2-HR4	highresSST-present	r1i1p1f1	Amon	psl	gn	gs://cmip6/CMIP6/HighResMIP/CMCC/CMCC-CM2-HR4/...	NaN	20170706

As you can see, the GCS data catalog is formatted in a very similar way to the ESGF search interface. Each entry in the catalog specifies the:

• Project name (activity_id)

• Modeling centre (institution_id)

• Model name (source_id)

• Experiment type (experiment_id, i.e. historical, SSP5-8.5, etc.)

• ID of the ensemble member (member_id)

• Time frequency (table_id)

• Variable name (variale_id)

• Access URL for the Zarr data set(zstore)

Plus some other entries we won’t worry about for now. You can use the Pandas routine df.query to search for particular values of each column, and return a new DataFrame that contains the entries that meet your search criteria.

Recall from Section 3.4 that the model variable names can be a bit cryptic. The full table of long variable names, matching them to the short names (which you include in the search) can be found here.

For example, let’s search for daily precipitation data from the model MPI-ESM1-2-LR for both the historical experiment, and the SSP2-4.5 future scenario.

# prepare the search criteria as a string
search_string = "table_id == 'day' & source_id == 'MPI-ESM1-2-LR' & variable_id == 'pr'" 
# continue on the next line
search_string += " & experiment_id == ['historical', 'ssp245']"
df_search = df_catalog.query(search_string)
df_search

	activity_id	institution_id	source_id	experiment_id	member_id	table_id	variable_id	grid_label	zstore	dcpp_init_year	version
218364	ScenarioMIP	MPI-M	MPI-ESM1-2-LR	ssp245	r10i1p1f1	day	pr	gn	gs://cmip6/CMIP6/ScenarioMIP/MPI-M/MPI-ESM1-2-...	NaN	20190710
218552	ScenarioMIP	MPI-M	MPI-ESM1-2-LR	ssp245	r3i1p1f1	day	pr	gn	gs://cmip6/CMIP6/ScenarioMIP/MPI-M/MPI-ESM1-2-...	NaN	20190710
219088	ScenarioMIP	MPI-M	MPI-ESM1-2-LR	ssp245	r1i1p1f1	day	pr	gn	gs://cmip6/CMIP6/ScenarioMIP/MPI-M/MPI-ESM1-2-...	NaN	20190710
219697	ScenarioMIP	MPI-M	MPI-ESM1-2-LR	ssp245	r4i1p1f1	day	pr	gn	gs://cmip6/CMIP6/ScenarioMIP/MPI-M/MPI-ESM1-2-...	NaN	20190710
219930	ScenarioMIP	MPI-M	MPI-ESM1-2-LR	ssp245	r2i1p1f1	day	pr	gn	gs://cmip6/CMIP6/ScenarioMIP/MPI-M/MPI-ESM1-2-...	NaN	20190710
220123	ScenarioMIP	MPI-M	MPI-ESM1-2-LR	ssp245	r5i1p1f1	day	pr	gn	gs://cmip6/CMIP6/ScenarioMIP/MPI-M/MPI-ESM1-2-...	NaN	20190710
220331	ScenarioMIP	MPI-M	MPI-ESM1-2-LR	ssp245	r6i1p1f1	day	pr	gn	gs://cmip6/CMIP6/ScenarioMIP/MPI-M/MPI-ESM1-2-...	NaN	20190710
220638	ScenarioMIP	MPI-M	MPI-ESM1-2-LR	ssp245	r7i1p1f1	day	pr	gn	gs://cmip6/CMIP6/ScenarioMIP/MPI-M/MPI-ESM1-2-...	NaN	20190710
221400	ScenarioMIP	MPI-M	MPI-ESM1-2-LR	ssp245	r9i1p1f1	day	pr	gn	gs://cmip6/CMIP6/ScenarioMIP/MPI-M/MPI-ESM1-2-...	NaN	20190710
221478	ScenarioMIP	MPI-M	MPI-ESM1-2-LR	ssp245	r8i1p1f1	day	pr	gn	gs://cmip6/CMIP6/ScenarioMIP/MPI-M/MPI-ESM1-2-...	NaN	20190710
222076	CMIP	MPI-M	MPI-ESM1-2-LR	historical	r1i1p1f1	day	pr	gn	gs://cmip6/CMIP6/CMIP/MPI-M/MPI-ESM1-2-LR/hist...	NaN	20190710
224126	CMIP	MPI-M	MPI-ESM1-2-LR	historical	r10i1p1f1	day	pr	gn	gs://cmip6/CMIP6/CMIP/MPI-M/MPI-ESM1-2-LR/hist...	NaN	20190710
232949	CMIP	MPI-M	MPI-ESM1-2-LR	historical	r4i1p1f1	day	pr	gn	gs://cmip6/CMIP6/CMIP/MPI-M/MPI-ESM1-2-LR/hist...	NaN	20190710
234219	CMIP	MPI-M	MPI-ESM1-2-LR	historical	r3i1p1f1	day	pr	gn	gs://cmip6/CMIP6/CMIP/MPI-M/MPI-ESM1-2-LR/hist...	NaN	20190710
235140	CMIP	MPI-M	MPI-ESM1-2-LR	historical	r2i1p1f1	day	pr	gn	gs://cmip6/CMIP6/CMIP/MPI-M/MPI-ESM1-2-LR/hist...	NaN	20190710
235281	CMIP	MPI-M	MPI-ESM1-2-LR	historical	r5i1p1f1	day	pr	gn	gs://cmip6/CMIP6/CMIP/MPI-M/MPI-ESM1-2-LR/hist...	NaN	20190710
235391	CMIP	MPI-M	MPI-ESM1-2-LR	historical	r8i1p1f1	day	pr	gn	gs://cmip6/CMIP6/CMIP/MPI-M/MPI-ESM1-2-LR/hist...	NaN	20190710
235410	CMIP	MPI-M	MPI-ESM1-2-LR	historical	r6i1p1f1	day	pr	gn	gs://cmip6/CMIP6/CMIP/MPI-M/MPI-ESM1-2-LR/hist...	NaN	20190710
236223	CMIP	MPI-M	MPI-ESM1-2-LR	historical	r9i1p1f1	day	pr	gn	gs://cmip6/CMIP6/CMIP/MPI-M/MPI-ESM1-2-LR/hist...	NaN	20190710
236461	CMIP	MPI-M	MPI-ESM1-2-LR	historical	r7i1p1f1	day	pr	gn	gs://cmip6/CMIP6/CMIP/MPI-M/MPI-ESM1-2-LR/hist...	NaN	20190710

3.5.2 Accessing the Data

Excellent, we now have a DataFrame that has the descriptions and access URLs for the datasets we want. In order to access the data, we first need to authenticate ourselves using the gcsfs package. Since we’re only accessing data available to the public, we can authenticate as an anonymous user, no account is needed here.

# authenticate access to Google Cloud
gcs = gcsfs.GCSFileSystem(token='anon')

# Get the path to a specific zarr store (the first one from the dataframe above)
zstore_url = df_search.zstore.values[0]
print(zstore_url)

# use the gcfs package to turn the URL into an interface to the data set
mapper = gcs.get_mapper(zstore_url)

# now open the zarr store using xarray. 
# The argument "consolidated" means to include the whole zarr data store for this file
ds = xr.open_zarr(mapper, consolidated = True)
ds

gs://cmip6/CMIP6/ScenarioMIP/MPI-M/MPI-ESM1-2-LR/ssp245/r10i1p1f1/day/pr/gn/v20190710/

<xarray.Dataset> Size: 2GB
Dimensions:    (lat: 96, bnds: 2, lon: 192, time: 31411)
Coordinates:
  * lat        (lat) float64 768B -88.57 -86.72 -84.86 ... 84.86 86.72 88.57
    lat_bnds   (lat, bnds) float64 2kB dask.array<chunksize=(96, 2), meta=np.ndarray>
  * lon        (lon) float64 2kB 0.0 1.875 3.75 5.625 ... 354.4 356.2 358.1
    lon_bnds   (lon, bnds) float64 3kB dask.array<chunksize=(192, 2), meta=np.ndarray>
  * time       (time) datetime64[ns] 251kB 2015-01-01T12:00:00 ... 2100-12-31...
    time_bnds  (time, bnds) datetime64[ns] 503kB dask.array<chunksize=(15706, 2), meta=np.ndarray>
Dimensions without coordinates: bnds
Data variables:
    pr         (time, lat, lon) float32 2GB dask.array<chunksize=(980, 96, 192), meta=np.ndarray>
Attributes: (12/50)
    Conventions:            CF-1.7 CMIP-6.2
    activity_id:            ScenarioMIP
    branch_method:          standard
    branch_time_in_child:   60265.0
    branch_time_in_parent:  60265.0
    cmor_version:           3.5.0
    ...                     ...
    title:                  MPI-ESM1-2-LR output prepared for CMIP6
    tracking_id:            hdl:21.14100/4b1502fc-585a-4005-8d9d-49ad10284480...
    variable_id:            pr
    variant_label:          r10i1p1f1
    netcdf_tracking_ids:    hdl:21.14100/4b1502fc-585a-4005-8d9d-49ad10284480...
    version_id:             v20190710

xarray.Dataset

• Dimensions:
  • lat: 96
  • bnds: 2
  • lon: 192
  • time: 31411
• Coordinates: (6)
  • lat
    (lat)
    float64
    -88.57 -86.72 ... 86.72 88.57

    axis :

    Y

    bounds :

    lat_bnds

    long_name :

    Latitude

    standard_name :

    latitude

    units :

    degrees_north

    array([-88.572169, -86.722531, -84.86197 , -82.998942, -81.134977, -79.270559,
           -77.405888, -75.541061, -73.676132, -71.811132, -69.946081, -68.080991,
           -66.215872, -64.35073 , -62.485571, -60.620396, -58.755209, -56.890013,
           -55.024808, -53.159595, -51.294377, -49.429154, -47.563926, -45.698694,
           -43.833459, -41.96822 , -40.102979, -38.237736, -36.372491, -34.507243,
           -32.641994, -30.776744, -28.911492, -27.046239, -25.180986, -23.315731,
           -21.450475, -19.585219, -17.719962, -15.854704, -13.989446, -12.124187,
           -10.258928,  -8.393669,  -6.528409,  -4.66315 ,  -2.79789 ,  -0.93263 ,
             0.93263 ,   2.79789 ,   4.66315 ,   6.528409,   8.393669,  10.258928,
            12.124187,  13.989446,  15.854704,  17.719962,  19.585219,  21.450475,
            23.315731,  25.180986,  27.046239,  28.911492,  30.776744,  32.641994,
            34.507243,  36.372491,  38.237736,  40.102979,  41.96822 ,  43.833459,
            45.698694,  47.563926,  49.429154,  51.294377,  53.159595,  55.024808,
            56.890013,  58.755209,  60.620396,  62.485571,  64.35073 ,  66.215872,
            68.080991,  69.946081,  71.811132,  73.676132,  75.541061,  77.405888,
            79.270559,  81.134977,  82.998942,  84.86197 ,  86.722531,  88.572169])

  • lat_bnds
    (lat, bnds)
    float64
    dask.array<chunksize=(96, 2), meta=np.ndarray>

    Array Chunk Bytes 1.50 kiB 1.50 kiB Shape (96, 2) (96, 2) Dask graph 1 chunks in 2 graph layers Data type float64 numpy.ndarray

  • lon
    (lon)
    float64
    0.0 1.875 3.75 ... 356.2 358.1

    axis :

    X

    bounds :

    lon_bnds

    long_name :

    Longitude

    standard_name :

    longitude

    units :

    degrees_east

    array([  0.   ,   1.875,   3.75 ,   5.625,   7.5  ,   9.375,  11.25 ,  13.125,
            15.   ,  16.875,  18.75 ,  20.625,  22.5  ,  24.375,  26.25 ,  28.125,
            30.   ,  31.875,  33.75 ,  35.625,  37.5  ,  39.375,  41.25 ,  43.125,
            45.   ,  46.875,  48.75 ,  50.625,  52.5  ,  54.375,  56.25 ,  58.125,
            60.   ,  61.875,  63.75 ,  65.625,  67.5  ,  69.375,  71.25 ,  73.125,
            75.   ,  76.875,  78.75 ,  80.625,  82.5  ,  84.375,  86.25 ,  88.125,
            90.   ,  91.875,  93.75 ,  95.625,  97.5  ,  99.375, 101.25 , 103.125,
           105.   , 106.875, 108.75 , 110.625, 112.5  , 114.375, 116.25 , 118.125,
           120.   , 121.875, 123.75 , 125.625, 127.5  , 129.375, 131.25 , 133.125,
           135.   , 136.875, 138.75 , 140.625, 142.5  , 144.375, 146.25 , 148.125,
           150.   , 151.875, 153.75 , 155.625, 157.5  , 159.375, 161.25 , 163.125,
           165.   , 166.875, 168.75 , 170.625, 172.5  , 174.375, 176.25 , 178.125,
           180.   , 181.875, 183.75 , 185.625, 187.5  , 189.375, 191.25 , 193.125,
           195.   , 196.875, 198.75 , 200.625, 202.5  , 204.375, 206.25 , 208.125,
           210.   , 211.875, 213.75 , 215.625, 217.5  , 219.375, 221.25 , 223.125,
           225.   , 226.875, 228.75 , 230.625, 232.5  , 234.375, 236.25 , 238.125,
           240.   , 241.875, 243.75 , 245.625, 247.5  , 249.375, 251.25 , 253.125,
           255.   , 256.875, 258.75 , 260.625, 262.5  , 264.375, 266.25 , 268.125,
           270.   , 271.875, 273.75 , 275.625, 277.5  , 279.375, 281.25 , 283.125,
           285.   , 286.875, 288.75 , 290.625, 292.5  , 294.375, 296.25 , 298.125,
           300.   , 301.875, 303.75 , 305.625, 307.5  , 309.375, 311.25 , 313.125,
           315.   , 316.875, 318.75 , 320.625, 322.5  , 324.375, 326.25 , 328.125,
           330.   , 331.875, 333.75 , 335.625, 337.5  , 339.375, 341.25 , 343.125,
           345.   , 346.875, 348.75 , 350.625, 352.5  , 354.375, 356.25 , 358.125])

  • lon_bnds
    (lon, bnds)
    float64
    dask.array<chunksize=(192, 2), meta=np.ndarray>

    Array Chunk Bytes 3.00 kiB 3.00 kiB Shape (192, 2) (192, 2) Dask graph 1 chunks in 2 graph layers Data type float64 numpy.ndarray

  • time
    (time)
    datetime64[ns]
    2015-01-01T12:00:00 ... 2100-12-...

    axis :

    T

    bounds :

    time_bnds

    long_name :

    time

    standard_name :

    time

    array(['2015-01-01T12:00:00.000000000', '2015-01-02T12:00:00.000000000',
           '2015-01-03T12:00:00.000000000', ..., '2100-12-29T12:00:00.000000000',
           '2100-12-30T12:00:00.000000000', '2100-12-31T12:00:00.000000000'],
          dtype='datetime64[ns]')

  • time_bnds
    (time, bnds)
    datetime64[ns]
    dask.array<chunksize=(15706, 2), meta=np.ndarray>

    Array Chunk Bytes 490.80 kiB 245.41 kiB Shape (31411, 2) (15706, 2) Dask graph 2 chunks in 2 graph layers Data type datetime64[ns] numpy.ndarray

• Data variables: (1)
  • pr
    (time, lat, lon)
    float32
    dask.array<chunksize=(980, 96, 192), meta=np.ndarray>

    cell_measures :

    area: areacella

    cell_methods :

    area: time: mean

    comment :

    includes both liquid and solid phases

    history :

    2019-11-15T00:33:03Z altered by CMOR: replaced missing value flag (-9e+33) and corresponding data with standard missing value (1e+20). 2019-11-15T00:33:03Z altered by CMOR: Inverted axis: lat.

    long_name :

    Precipitation

    original_name :

    pr

    standard_name :

    precipitation_flux

    units :

    kg m-2 s-1

    Array Chunk Bytes 2.16 GiB 68.91 MiB Shape (31411, 96, 192) (980, 96, 192) Dask graph 33 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (3)
  • lat
    PandasIndex

    PandasIndex(Index([ -88.57216851400727,  -86.72253095466814,  -84.86197029204237,
            -82.99894164283755,  -81.13497683767741,  -79.27055903485967,
             -77.4058880820788,  -75.54106145287895,  -73.67613231320912,
            -71.81113211427447,  -69.94608064698343,  -68.08099098565125,
             -66.2158721139987,  -64.35073040887207,  -62.48557052203639,
            -60.62039592682648,  -58.75520926937993,  -56.89001260135711,
            -55.02480753831166,  -53.15959537001968,  -51.29437713895115,
            -49.42915369712305,  -47.56392574797867, -45.698693877701785,
            -43.83345857895126,  -41.96822026907538,   -40.1029793042494,
            -38.23773599056483,  -36.37249059281224,  -34.50724334150103,
            -32.64199443851768,  -30.77674406172325,  -28.91149236871774,
            -27.04623949994481, -25.180985581270594,  -23.31573072614093,
           -21.450475037398185,  -19.58521860882233, -17.719961526447428,
           -15.854703869694873, -13.989445712356673, -12.124187123455766,
           -10.258928168006376,  -8.393668907692383, -6.5284094014799905,
            -4.663149706177884,  -2.797889876956741,  -0.932629967837991,
             0.932629967837991,   2.797889876956741,   4.663149706177884,
            6.5284094014799905,   8.393668907692383,  10.258928168006376,
            12.124187123455766,  13.989445712356673,  15.854703869694873,
            17.719961526447428,   19.58521860882233,  21.450475037398185,
             23.31573072614093,  25.180985581270594,   27.04623949994481,
             28.91149236871774,   30.77674406172325,   32.64199443851768,
             34.50724334150103,   36.37249059281224,   38.23773599056483,
              40.1029793042494,   41.96822026907538,   43.83345857895126,
            45.698693877701785,   47.56392574797867,   49.42915369712305,
             51.29437713895115,   53.15959537001968,   55.02480753831166,
             56.89001260135711,   58.75520926937993,   60.62039592682648,
             62.48557052203639,   64.35073040887207,    66.2158721139987,
             68.08099098565125,   69.94608064698343,   71.81113211427447,
             73.67613231320912,   75.54106145287895,    77.4058880820788,
             79.27055903485967,   81.13497683767741,   82.99894164283755,
             84.86197029204237,   86.72253095466814,   88.57216851400727],
          dtype='float64', name='lat'))

  • lon
    PandasIndex

    PandasIndex(Index([    0.0,   1.875,    3.75,   5.625,     7.5,   9.375,   11.25,  13.125,
              15.0,  16.875,
           ...
            341.25, 343.125,   345.0, 346.875,  348.75, 350.625,   352.5, 354.375,
            356.25, 358.125],
          dtype='float64', name='lon', length=192))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2015-01-01 12:00:00', '2015-01-02 12:00:00',
                   '2015-01-03 12:00:00', '2015-01-04 12:00:00',
                   '2015-01-05 12:00:00', '2015-01-06 12:00:00',
                   '2015-01-07 12:00:00', '2015-01-08 12:00:00',
                   '2015-01-09 12:00:00', '2015-01-10 12:00:00',
                   ...
                   '2100-12-22 12:00:00', '2100-12-23 12:00:00',
                   '2100-12-24 12:00:00', '2100-12-25 12:00:00',
                   '2100-12-26 12:00:00', '2100-12-27 12:00:00',
                   '2100-12-28 12:00:00', '2100-12-29 12:00:00',
                   '2100-12-30 12:00:00', '2100-12-31 12:00:00'],
                  dtype='datetime64[ns]', name='time', length=31411, freq=None))

• Attributes: (50)

  Conventions :

  CF-1.7 CMIP-6.2

  activity_id :

  ScenarioMIP

  branch_method :

  standard

  branch_time_in_child :

  60265.0

  branch_time_in_parent :

  60265.0

  cmor_version :

  3.5.0

  contact :

  cmip6-mpi-esm@dkrz.de

  creation_date :

  2019-11-15T00:33:03Z

  data_specs_version :

  01.00.30

  experiment :

  update of RCP4.5 based on SSP2

  experiment_id :

  ssp245

  external_variables :

  areacella

  forcing_index :

  1

  frequency :

  day

  further_info_url :

  https://furtherinfo.es-doc.org/CMIP6.MPI-M.MPI-ESM1-2-LR.ssp245.none.r10i1p1f1

  grid :

  gn

  grid_label :

  gn

  history :

  2019-11-15T00:33:03Z ; CMOR rewrote data to be consistent with CMIP6, CF-1.7 CMIP-6.2 and CF standards.

  initialization_index :

  1

  institution :

  Max Planck Institute for Meteorology, Hamburg 20146, Germany

  institution_id :

  MPI-M

  license :

  CMIP6 model data produced by MPI-M is licensed under a Creative Commons Attribution ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law.

  mip_era :

  CMIP6

  nominal_resolution :

  250 km

  parent_activity_id :

  CMIP

  parent_experiment_id :

  historical

  parent_mip_era :

  CMIP6

  parent_source_id :

  MPI-ESM1-2-LR

  parent_time_units :

  days since 1850-1-1 00:00:00

  parent_variant_label :

  r10i1p1f1

  physics_index :

  1

  product :

  model-output

  project_id :

  CMIP6

  realization_index :

  10

  realm :

  atmos

  references :

  MPI-ESM: Mauritsen, T. et al. (2019), Developments in the MPI‐M Earth System Model version 1.2 (MPI‐ESM1.2) and Its Response to Increasing CO2, J. Adv. Model. Earth Syst.,11, 998-1038, doi:10.1029/2018MS001400, Mueller, W.A. et al. (2018): A high‐resolution version of the Max Planck Institute Earth System Model MPI‐ESM1.2‐HR. J. Adv. Model. EarthSyst.,10,1383–1413, doi:10.1029/2017MS001217

  source :

  MPI-ESM1.2-LR (2017): aerosol: none, prescribed MACv2-SP atmos: ECHAM6.3 (spectral T63; 192 x 96 longitude/latitude; 47 levels; top level 0.01 hPa) atmosChem: none land: JSBACH3.20 landIce: none/prescribed ocean: MPIOM1.63 (bipolar GR1.5, approximately 1.5deg; 256 x 220 longitude/latitude; 40 levels; top grid cell 0-12 m) ocnBgchem: HAMOCC6 seaIce: unnamed (thermodynamic (Semtner zero-layer) dynamic (Hibler 79) sea ice model)

  source_id :

  MPI-ESM1-2-LR

  source_type :

  AOGCM

  status :

  2020-09-17;created; by gcs.cmip6.ldeo@gmail.com

  sub_experiment :

  none

  sub_experiment_id :

  none

  table_id :

  day

  table_info :

  Creation Date:(09 May 2019) MD5:e6ef8ececc8f338646ebfb3aeed36bfc

  title :

  MPI-ESM1-2-LR output prepared for CMIP6

  tracking_id :

  hdl:21.14100/4b1502fc-585a-4005-8d9d-49ad10284480 hdl:21.14100/fe5b6db3-fce9-429d-b71e-5c01f8a2d5c1 hdl:21.14100/288b3146-c327-456d-9905-e97c6ab86e49 hdl:21.14100/8d2aa7ff-4a14-43ff-8805-6835cee2298b hdl:21.14100/366003ef-722c-4bdf-8f4c-e46999ce9e95

  variable_id :

  pr

  variant_label :

  r10i1p1f1

  netcdf_tracking_ids :

  hdl:21.14100/4b1502fc-585a-4005-8d9d-49ad10284480 hdl:21.14100/fe5b6db3-fce9-429d-b71e-5c01f8a2d5c1 hdl:21.14100/288b3146-c327-456d-9905-e97c6ab86e49 hdl:21.14100/8d2aa7ff-4a14-43ff-8805-6835cee2298b hdl:21.14100/366003ef-722c-4bdf-8f4c-e46999ce9e95

  version_id :

  v20190710

Here we accessed the data for the SSP2-4.5 scenario and the ensemble member named r10i1p1f1. Let’s select a spatial region and calculate a long-term average for the period 2071-2100, just to demonstrate that the functionality is the same as before.

# select a region near western Europe
lats = [40, 60]
lons = [-10, 10]

# convert longitude values in the dataset from the (0, 360) convention
# to the (-180, 180) convention
ds = ds.assign_coords(lon = (((ds.lon + 180) % 360) - 180))
ds = ds.sortby('lon')

# subset the data
ds_subset = ds.sel(lat = slice(*lats), lon = slice(*lons),
                   time = ds.time.dt.year.isin(range(2071, 2101)))
ds_subset

<xarray.Dataset> Size: 6MB
Dimensions:    (lat: 11, bnds: 2, lon: 11, time: 10957)
Coordinates:
  * lat        (lat) float64 88B 40.1 41.97 43.83 45.7 ... 55.02 56.89 58.76
    lat_bnds   (lat, bnds) float64 176B dask.array<chunksize=(11, 2), meta=np.ndarray>
    lon_bnds   (lon, bnds) float64 176B dask.array<chunksize=(11, 2), meta=np.ndarray>
  * time       (time) datetime64[ns] 88kB 2071-01-01T12:00:00 ... 2100-12-31T...
    time_bnds  (time, bnds) datetime64[ns] 175kB dask.array<chunksize=(10957, 2), meta=np.ndarray>
  * lon        (lon) float64 88B -9.375 -7.5 -5.625 -3.75 ... 5.625 7.5 9.375
Dimensions without coordinates: bnds
Data variables:
    pr         (time, lat, lon) float32 5MB dask.array<chunksize=(126, 11, 11), meta=np.ndarray>
Attributes: (12/50)
    Conventions:            CF-1.7 CMIP-6.2
    activity_id:            ScenarioMIP
    branch_method:          standard
    branch_time_in_child:   60265.0
    branch_time_in_parent:  60265.0
    cmor_version:           3.5.0
    ...                     ...
    title:                  MPI-ESM1-2-LR output prepared for CMIP6
    tracking_id:            hdl:21.14100/4b1502fc-585a-4005-8d9d-49ad10284480...
    variable_id:            pr
    variant_label:          r10i1p1f1
    netcdf_tracking_ids:    hdl:21.14100/4b1502fc-585a-4005-8d9d-49ad10284480...
    version_id:             v20190710

xarray.Dataset

• Dimensions:
  • lat: 11
  • bnds: 2
  • lon: 11
  • time: 10957
• Coordinates: (6)
  • lat
    (lat)
    float64
    40.1 41.97 43.83 ... 56.89 58.76

    axis :

    Y

    bounds :

    lat_bnds

    long_name :

    Latitude

    standard_name :

    latitude

    units :

    degrees_north

    array([40.102979, 41.96822 , 43.833459, 45.698694, 47.563926, 49.429154,
           51.294377, 53.159595, 55.024808, 56.890013, 58.755209])

  • lat_bnds
    (lat, bnds)
    float64
    dask.array<chunksize=(11, 2), meta=np.ndarray>

    Array Chunk Bytes 176 B 176 B Shape (11, 2) (11, 2) Dask graph 1 chunks in 3 graph layers Data type float64 numpy.ndarray

  • lon_bnds
    (lon, bnds)
    float64
    dask.array<chunksize=(11, 2), meta=np.ndarray>

    Array Chunk Bytes 176 B 176 B Shape (11, 2) (11, 2) Dask graph 1 chunks in 4 graph layers Data type float64 numpy.ndarray

  • time
    (time)
    datetime64[ns]
    2071-01-01T12:00:00 ... 2100-12-...

    axis :

    T

    bounds :

    time_bnds

    long_name :

    time

    standard_name :

    time

    array(['2071-01-01T12:00:00.000000000', '2071-01-02T12:00:00.000000000',
           '2071-01-03T12:00:00.000000000', ..., '2100-12-29T12:00:00.000000000',
           '2100-12-30T12:00:00.000000000', '2100-12-31T12:00:00.000000000'],
          dtype='datetime64[ns]')

  • time_bnds
    (time, bnds)
    datetime64[ns]
    dask.array<chunksize=(10957, 2), meta=np.ndarray>

    Array Chunk Bytes 171.20 kiB 171.20 kiB Shape (10957, 2) (10957, 2) Dask graph 1 chunks in 3 graph layers Data type datetime64[ns] numpy.ndarray

  • lon
    (lon)
    float64
    -9.375 -7.5 -5.625 ... 7.5 9.375

    array([-9.375, -7.5  , -5.625, -3.75 , -1.875,  0.   ,  1.875,  3.75 ,  5.625,
            7.5  ,  9.375])

• Data variables: (1)
  • pr
    (time, lat, lon)
    float32
    dask.array<chunksize=(126, 11, 11), meta=np.ndarray>

    cell_measures :

    area: areacella

    cell_methods :

    area: time: mean

    comment :

    includes both liquid and solid phases

    history :

    2019-11-15T00:33:03Z altered by CMOR: replaced missing value flag (-9e+33) and corresponding data with standard missing value (1e+20). 2019-11-15T00:33:03Z altered by CMOR: Inverted axis: lat.

    long_name :

    Precipitation

    original_name :

    pr

    standard_name :

    precipitation_flux

    units :

    kg m-2 s-1

    Array Chunk Bytes 5.06 MiB 463.20 kiB Shape (10957, 11, 11) (980, 11, 11) Dask graph 13 chunks in 4 graph layers Data type float32 numpy.ndarray

• Indexes: (3)
  • lat
    PandasIndex

    PandasIndex(Index([  40.1029793042494,  41.96822026907538,  43.83345857895126,
           45.698693877701785,  47.56392574797867,  49.42915369712305,
            51.29437713895115,  53.15959537001968,  55.02480753831166,
            56.89001260135711,  58.75520926937993],
          dtype='float64', name='lat'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2071-01-01 12:00:00', '2071-01-02 12:00:00',
                   '2071-01-03 12:00:00', '2071-01-04 12:00:00',
                   '2071-01-05 12:00:00', '2071-01-06 12:00:00',
                   '2071-01-07 12:00:00', '2071-01-08 12:00:00',
                   '2071-01-09 12:00:00', '2071-01-10 12:00:00',
                   ...
                   '2100-12-22 12:00:00', '2100-12-23 12:00:00',
                   '2100-12-24 12:00:00', '2100-12-25 12:00:00',
                   '2100-12-26 12:00:00', '2100-12-27 12:00:00',
                   '2100-12-28 12:00:00', '2100-12-29 12:00:00',
                   '2100-12-30 12:00:00', '2100-12-31 12:00:00'],
                  dtype='datetime64[ns]', name='time', length=10957, freq=None))

  • lon
    PandasIndex

    PandasIndex(Index([-9.375, -7.5, -5.625, -3.75, -1.875, 0.0, 1.875, 3.75, 5.625, 7.5,
           9.375],
          dtype='float64', name='lon'))

• Attributes: (50)

  Conventions :

  CF-1.7 CMIP-6.2

  activity_id :

  ScenarioMIP

  branch_method :

  standard

  branch_time_in_child :

  60265.0

  branch_time_in_parent :

  60265.0

  cmor_version :

  3.5.0

  contact :

  cmip6-mpi-esm@dkrz.de

  creation_date :

  2019-11-15T00:33:03Z

  data_specs_version :

  01.00.30

  experiment :

  update of RCP4.5 based on SSP2

  experiment_id :

  ssp245

  external_variables :

  areacella

  forcing_index :

  1

  frequency :

  day

  further_info_url :

  https://furtherinfo.es-doc.org/CMIP6.MPI-M.MPI-ESM1-2-LR.ssp245.none.r10i1p1f1

  grid :

  gn

  grid_label :

  gn

  history :

  2019-11-15T00:33:03Z ; CMOR rewrote data to be consistent with CMIP6, CF-1.7 CMIP-6.2 and CF standards.

  initialization_index :

  1

  institution :

  Max Planck Institute for Meteorology, Hamburg 20146, Germany

  institution_id :

  MPI-M

  license :

  CMIP6 model data produced by MPI-M is licensed under a Creative Commons Attribution ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law.

  mip_era :

  CMIP6

  nominal_resolution :

  250 km

  parent_activity_id :

  CMIP

  parent_experiment_id :

  historical

  parent_mip_era :

  CMIP6

  parent_source_id :

  MPI-ESM1-2-LR

  parent_time_units :

  days since 1850-1-1 00:00:00

  parent_variant_label :

  r10i1p1f1

  physics_index :

  1

  product :

  model-output

  project_id :

  CMIP6

  realization_index :

  10

  realm :

  atmos

  references :

  MPI-ESM: Mauritsen, T. et al. (2019), Developments in the MPI‐M Earth System Model version 1.2 (MPI‐ESM1.2) and Its Response to Increasing CO2, J. Adv. Model. Earth Syst.,11, 998-1038, doi:10.1029/2018MS001400, Mueller, W.A. et al. (2018): A high‐resolution version of the Max Planck Institute Earth System Model MPI‐ESM1.2‐HR. J. Adv. Model. EarthSyst.,10,1383–1413, doi:10.1029/2017MS001217

  source :

  MPI-ESM1.2-LR (2017): aerosol: none, prescribed MACv2-SP atmos: ECHAM6.3 (spectral T63; 192 x 96 longitude/latitude; 47 levels; top level 0.01 hPa) atmosChem: none land: JSBACH3.20 landIce: none/prescribed ocean: MPIOM1.63 (bipolar GR1.5, approximately 1.5deg; 256 x 220 longitude/latitude; 40 levels; top grid cell 0-12 m) ocnBgchem: HAMOCC6 seaIce: unnamed (thermodynamic (Semtner zero-layer) dynamic (Hibler 79) sea ice model)

  source_id :

  MPI-ESM1-2-LR

  source_type :

  AOGCM

  status :

  2020-09-17;created; by gcs.cmip6.ldeo@gmail.com

  sub_experiment :

  none

  sub_experiment_id :

  none

  table_id :

  day

  table_info :

  Creation Date:(09 May 2019) MD5:e6ef8ececc8f338646ebfb3aeed36bfc

  title :

  MPI-ESM1-2-LR output prepared for CMIP6

  tracking_id :

  hdl:21.14100/4b1502fc-585a-4005-8d9d-49ad10284480 hdl:21.14100/fe5b6db3-fce9-429d-b71e-5c01f8a2d5c1 hdl:21.14100/288b3146-c327-456d-9905-e97c6ab86e49 hdl:21.14100/8d2aa7ff-4a14-43ff-8805-6835cee2298b hdl:21.14100/366003ef-722c-4bdf-8f4c-e46999ce9e95

  variable_id :

  pr

  variant_label :

  r10i1p1f1

  netcdf_tracking_ids :

  hdl:21.14100/4b1502fc-585a-4005-8d9d-49ad10284480 hdl:21.14100/fe5b6db3-fce9-429d-b71e-5c01f8a2d5c1 hdl:21.14100/288b3146-c327-456d-9905-e97c6ab86e49 hdl:21.14100/8d2aa7ff-4a14-43ff-8805-6835cee2298b hdl:21.14100/366003ef-722c-4bdf-8f4c-e46999ce9e95

  version_id :

  v20190710

# calculate the long-term average precip, and trigger computation
pr_ltm = ds_subset.pr.mean('time').compute() * 86400 # also convert units to mm/day

p = pr_ltm.plot.contourf(subplot_kws = dict(projection = ccrs.PlateCarree(), 
                                            transform = ccrs.PlateCarree()),
                         levels = 15, cbar_kwargs = dict(label = "mm/day"))
p.axes.coastlines()
p.axes.set_title("MPI-ESM1-2-LR Average Daily Precipitation\n SSP2-4.5 2071-2100")
plt.show()

3.5.3 Saving the Data

To save data from Google Cloud to your local machine, you can use either the xarray to_netcdf method discussed in the previous section or save the data in zarr format, just like it is on the cloud. Just like to_netcdf, there is a to_zarr method for xarray Dataset or DataArray objects (documentation here). The Zarr format doesn’t save the data to a single file like netCDF, instead it creates a directory that contains multiple files, each corresponding to different data variables, coordinates, metadata, etc. For most users, there isn’t really a benefit of using Zarr versus using netCDF, so it’s probably worth keeping it simple and sticking with netCDF as the format of your data files.