USGS dataretrieval Python Package `get_qwdata()` Examples

This notebook provides examples of using the Python dataretrieval package to retrieve water quality sample data for United States Geological Survey (USGS) monitoring sites. The dataretrieval package provides a collection of functions to get data from the USGS National Water Information System (NWIS) and other online sources of hydrology and water quality data, including the United States Environmental Protection Agency (USEPA).

Install the Package

Use the following code to install the package if it doesn’t exist already within your Jupyter Python environment.

[1]:

!pip install dataretrieval

Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: dataretrieval in /home/runner/.local/lib/python3.12/site-packages (0.1.dev1+g4b3a3e8)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from dataretrieval) (2.31.0)
Requirement already satisfied: pandas==2.* in /home/runner/.local/lib/python3.12/site-packages (from dataretrieval) (2.2.3)
Requirement already satisfied: numpy>=1.26.0 in /home/runner/.local/lib/python3.12/site-packages (from pandas==2.*->dataretrieval) (2.2.5)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/lib/python3/dist-packages (from pandas==2.*->dataretrieval) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/lib/python3/dist-packages (from pandas==2.*->dataretrieval) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /home/runner/.local/lib/python3.12/site-packages (from pandas==2.*->dataretrieval) (2025.2)

Load the package so you can use it along with other packages used in this notebook.

[2]:

from dataretrieval import nwis
from IPython.display import display

Basic Usage

The dataretrieval package has several functions that allow you to retrieve data from different web services. This examples uses the get_qwdata() function to retrieve water quality sample data for USGS monitoring sites from NWIS. The following arguments are supported:

Arguments (Additional arguments, if supplied, will be used as query parameters)

sites (string or list of strings): A list of USGS site identifiers for which to retrieve data. If the qwdata parameter site_no is supplied, it will overwrite the sites parameter.
parameterCd (string or list of strings): A list of USGS parameter codes for which to retrieve data.
start (string): The beginning date for a period for which to retrieve data. If the qwdata parameter begin_date is supplied, it will overwrite the start parameter.
end (string): The ending date for a period for which to retrieve data. If the qwdata parameter end_date is supplied, it will overwrite the end parameter.
datetime_index (boolean): If True, create a datetime index
wide_format (boolean): If True, return data in wide format with multiple samples per row and one row per time.

Example 1: Get all water quality sample data for a single monitoring site

[3]:

siteID = '10109000'
wq_data = nwis.get_qwdata(sites=siteID)
print('Retrieved data for ' + str(len(wq_data[0])) + ' samples.')

/home/runner/.local/lib/python3.12/site-packages/dataretrieval/nwis.py:193: UserWarning: WARNING: Starting in March 2024, the NWIS qw data endpoint is retiring and no longer receives updates. For more information, refer to https://waterdata.usgs.gov.nwis/qwdata and https://doi-usgs.github.io/dataRetrieval/articles/Status.html or email CompTools@usgs.gov.
  warnings.warn(

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3805, in Index.get_loc(self, key)
   3804 try:
-> 3805     return self._engine.get_loc(casted_key)
   3806 except KeyError as err:

File index.pyx:167, in pandas._libs.index.IndexEngine.get_loc()

File index.pyx:196, in pandas._libs.index.IndexEngine.get_loc()

File pandas/_libs/hashtable_class_helper.pxi:7081, in pandas._libs.hashtable.PyObjectHashTable.get_item()

File pandas/_libs/hashtable_class_helper.pxi:7089, in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'sample_start_time_datum_cd'

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
Cell In[3], line 2
      1 siteID = '10109000'
----> 2 wq_data = nwis.get_qwdata(sites=siteID)
      3 print('Retrieved data for ' + str(len(wq_data[0])) + ' samples.')

File ~/.local/lib/python3.12/site-packages/dataretrieval/nwis.py:254, in get_qwdata(sites, start, end, multi_index, wide_format, datetime_index, ssl_check, **kwargs)
    251 df = _read_rdb(response.text)
    253 if datetime_index is True:
--> 254     df = format_datetime(df, "sample_dt", "sample_tm", "sample_start_time_datum_cd")
    256 return format_response(df, **kwargs), NWIS_Metadata(response, **kwargs)

File ~/.local/lib/python3.12/site-packages/dataretrieval/utils.py:79, in format_datetime(df, date_field, time_field, tz_field)
     56 """Creates a datetime field from separate date, time, and
     57 time zone fields.
     58
   (...)     76
     77 """
     78 # create a datetime index from the columns in qwdata response
---> 79 df[tz_field] = df[tz_field].map(tz)
     81 df["datetime"] = pd.to_datetime(
     82     df[date_field] + " " + df[time_field] + " " + df[tz_field],
     83     format="ISO8601",
     84     utc=True,
     85 )
     87 # if there are any incomplete dates, warn the user

File ~/.local/lib/python3.12/site-packages/pandas/core/frame.py:4102, in DataFrame.__getitem__(self, key)
   4100 if self.columns.nlevels > 1:
   4101     return self._getitem_multilevel(key)
-> 4102 indexer = self.columns.get_loc(key)
   4103 if is_integer(indexer):
   4104     indexer = [indexer]

File ~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3812, in Index.get_loc(self, key)
   3807     if isinstance(casted_key, slice) or (
   3808         isinstance(casted_key, abc.Iterable)
   3809         and any(isinstance(x, slice) for x in casted_key)
   3810     ):
   3811         raise InvalidIndexError(key)
-> 3812     raise KeyError(key) from err
   3813 except TypeError:
   3814     # If we have a listlike key, _check_indexing_error will raise
   3815     #  InvalidIndexError. Otherwise we fall through and re-raise
   3816     #  the TypeError.
   3817     self._check_indexing_error(key)

KeyError: 'sample_start_time_datum_cd'

Interpreting the Result

The result of calling the get_qwdata() function is an object that contains a Pandas data frame object and an associated metadata object. The Pandas data frame contains the water quality sample data for the requested site, and or observed variables and time frame.

Once you’ve got the data frame, there’s several useful things you can do to explore the data.

Display the data frame as a table. The default data frame for this function is a wide, cross-tabulated table, with columns for each observed variable and a row for each sample date (wide_format=True).

[4]:

display(wq_data[0])

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[4], line 1
----> 1 display(wq_data[0])

NameError: name 'wq_data' is not defined

Show the data types of the columns in the resulting data frame.

[5]:

print(wq_data[0].dtypes)

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[5], line 1
----> 1 print(wq_data[0].dtypes)

NameError: name 'wq_data' is not defined

The other part of the result returned from the get_qwdata() function is a metadata object that contains information about the query that was executed to return the data. For example, you can access the URL that was assembled to retrieve the requested data from the USGS web service. The USGS web service responses contain a descriptive header that defines and can be helpful in interpreting the contents of the response.

[6]:

print('The query URL used to retrieve the data from NWIS was: ' + wq_data[1].url)

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[6], line 1
----> 1 print('The query URL used to retrieve the data from NWIS was: ' + wq_data[1].url)

NameError: name 'wq_data' is not defined

Additional Examples

Example 2: Get water quality sample data for multiple sites for a single parameter

[7]:

site_ids = ['04024430', '04024000']
parameter_code = '00065'
wq_multi_site = nwis.get_qwdata(sites=site_ids, parameterCd=parameter_code)
print('Retrieved data for ' + str(len(wq_multi_site[0])) + ' samples.')
display(wq_multi_site[0])

/home/runner/.local/lib/python3.12/site-packages/dataretrieval/nwis.py:193: UserWarning: WARNING: Starting in March 2024, the NWIS qw data endpoint is retiring and no longer receives updates. For more information, refer to https://waterdata.usgs.gov.nwis/qwdata and https://doi-usgs.github.io/dataRetrieval/articles/Status.html or email CompTools@usgs.gov.
  warnings.warn(

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3805, in Index.get_loc(self, key)
   3804 try:
-> 3805     return self._engine.get_loc(casted_key)
   3806 except KeyError as err:

File index.pyx:167, in pandas._libs.index.IndexEngine.get_loc()

File index.pyx:196, in pandas._libs.index.IndexEngine.get_loc()

File pandas/_libs/hashtable_class_helper.pxi:7081, in pandas._libs.hashtable.PyObjectHashTable.get_item()

File pandas/_libs/hashtable_class_helper.pxi:7089, in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'sample_start_time_datum_cd'

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
Cell In[7], line 3
      1 site_ids = ['04024430', '04024000']
      2 parameter_code = '00065'
----> 3 wq_multi_site = nwis.get_qwdata(sites=site_ids, parameterCd=parameter_code)
      4 print('Retrieved data for ' + str(len(wq_multi_site[0])) + ' samples.')
      5 display(wq_multi_site[0])

File ~/.local/lib/python3.12/site-packages/dataretrieval/nwis.py:254, in get_qwdata(sites, start, end, multi_index, wide_format, datetime_index, ssl_check, **kwargs)
    251 df = _read_rdb(response.text)
    253 if datetime_index is True:
--> 254     df = format_datetime(df, "sample_dt", "sample_tm", "sample_start_time_datum_cd")
    256 return format_response(df, **kwargs), NWIS_Metadata(response, **kwargs)

File ~/.local/lib/python3.12/site-packages/dataretrieval/utils.py:79, in format_datetime(df, date_field, time_field, tz_field)
     56 """Creates a datetime field from separate date, time, and
     57 time zone fields.
     58
   (...)     76
     77 """
     78 # create a datetime index from the columns in qwdata response
---> 79 df[tz_field] = df[tz_field].map(tz)
     81 df["datetime"] = pd.to_datetime(
     82     df[date_field] + " " + df[time_field] + " " + df[tz_field],
     83     format="ISO8601",
     84     utc=True,
     85 )
     87 # if there are any incomplete dates, warn the user

File ~/.local/lib/python3.12/site-packages/pandas/core/frame.py:4102, in DataFrame.__getitem__(self, key)
   4100 if self.columns.nlevels > 1:
   4101     return self._getitem_multilevel(key)
-> 4102 indexer = self.columns.get_loc(key)
   4103 if is_integer(indexer):
   4104     indexer = [indexer]

File ~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3812, in Index.get_loc(self, key)
   3807     if isinstance(casted_key, slice) or (
   3808         isinstance(casted_key, abc.Iterable)
   3809         and any(isinstance(x, slice) for x in casted_key)
   3810     ):
   3811         raise InvalidIndexError(key)
-> 3812     raise KeyError(key) from err
   3813 except TypeError:
   3814     # If we have a listlike key, _check_indexing_error will raise
   3815     #  InvalidIndexError. Otherwise we fall through and re-raise
   3816     #  the TypeError.
   3817     self._check_indexing_error(key)

KeyError: 'sample_start_time_datum_cd'

The following example is the same as the previous example but with multi index turned off (multi_index=False)

[8]:

site_ids = ['04024430', '04024000']
parameter_code = '00065'
wq_multi_site = nwis.get_qwdata(sites=site_ids, parameterCd=parameter_code, multi_index=False)
print('Retrieved data for ' + str(len(wq_multi_site[0])) + ' samples.')
display(wq_multi_site[0])

/home/runner/.local/lib/python3.12/site-packages/dataretrieval/nwis.py:193: UserWarning: WARNING: Starting in March 2024, the NWIS qw data endpoint is retiring and no longer receives updates. For more information, refer to https://waterdata.usgs.gov.nwis/qwdata and https://doi-usgs.github.io/dataRetrieval/articles/Status.html or email CompTools@usgs.gov.
  warnings.warn(

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3805, in Index.get_loc(self, key)
   3804 try:
-> 3805     return self._engine.get_loc(casted_key)
   3806 except KeyError as err:

File index.pyx:167, in pandas._libs.index.IndexEngine.get_loc()

File index.pyx:196, in pandas._libs.index.IndexEngine.get_loc()

File pandas/_libs/hashtable_class_helper.pxi:7081, in pandas._libs.hashtable.PyObjectHashTable.get_item()

File pandas/_libs/hashtable_class_helper.pxi:7089, in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'sample_start_time_datum_cd'

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
Cell In[8], line 3
      1 site_ids = ['04024430', '04024000']
      2 parameter_code = '00065'
----> 3 wq_multi_site = nwis.get_qwdata(sites=site_ids, parameterCd=parameter_code, multi_index=False)
      4 print('Retrieved data for ' + str(len(wq_multi_site[0])) + ' samples.')
      5 display(wq_multi_site[0])

File ~/.local/lib/python3.12/site-packages/dataretrieval/nwis.py:254, in get_qwdata(sites, start, end, multi_index, wide_format, datetime_index, ssl_check, **kwargs)
    251 df = _read_rdb(response.text)
    253 if datetime_index is True:
--> 254     df = format_datetime(df, "sample_dt", "sample_tm", "sample_start_time_datum_cd")
    256 return format_response(df, **kwargs), NWIS_Metadata(response, **kwargs)

File ~/.local/lib/python3.12/site-packages/dataretrieval/utils.py:79, in format_datetime(df, date_field, time_field, tz_field)
     56 """Creates a datetime field from separate date, time, and
     57 time zone fields.
     58
   (...)     76
     77 """
     78 # create a datetime index from the columns in qwdata response
---> 79 df[tz_field] = df[tz_field].map(tz)
     81 df["datetime"] = pd.to_datetime(
     82     df[date_field] + " " + df[time_field] + " " + df[tz_field],
     83     format="ISO8601",
     84     utc=True,
     85 )
     87 # if there are any incomplete dates, warn the user

File ~/.local/lib/python3.12/site-packages/pandas/core/frame.py:4102, in DataFrame.__getitem__(self, key)
   4100 if self.columns.nlevels > 1:
   4101     return self._getitem_multilevel(key)
-> 4102 indexer = self.columns.get_loc(key)
   4103 if is_integer(indexer):
   4104     indexer = [indexer]

File ~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3812, in Index.get_loc(self, key)
   3807     if isinstance(casted_key, slice) or (
   3808         isinstance(casted_key, abc.Iterable)
   3809         and any(isinstance(x, slice) for x in casted_key)
   3810     ):
   3811         raise InvalidIndexError(key)
-> 3812     raise KeyError(key) from err
   3813 except TypeError:
   3814     # If we have a listlike key, _check_indexing_error will raise
   3815     #  InvalidIndexError. Otherwise we fall through and re-raise
   3816     #  the TypeError.
   3817     self._check_indexing_error(key)

KeyError: 'sample_start_time_datum_cd'

Example 3: Retrieve water quality sample data for multiple sites, including a list of parameters, within a time period defined by start and end dates

[9]:

site_ids = ['04024430', '04024000']
parameterCd = ['34247', '30234', '32104', '34220']
startDate = '2012-01-01'
endDate = ''
wq_data2 = nwis.get_qwdata(sites=site_ids, parameterCd=parameterCd,
                           start=startDate, end=endDate)
print('Retrieved data for ' + str(len(wq_multi_site[0])) + ' samples.')
display(wq_data2[0])

/home/runner/.local/lib/python3.12/site-packages/dataretrieval/nwis.py:193: UserWarning: WARNING: Starting in March 2024, the NWIS qw data endpoint is retiring and no longer receives updates. For more information, refer to https://waterdata.usgs.gov.nwis/qwdata and https://doi-usgs.github.io/dataRetrieval/articles/Status.html or email CompTools@usgs.gov.
  warnings.warn(

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3805, in Index.get_loc(self, key)
   3804 try:
-> 3805     return self._engine.get_loc(casted_key)
   3806 except KeyError as err:

File index.pyx:167, in pandas._libs.index.IndexEngine.get_loc()

File index.pyx:196, in pandas._libs.index.IndexEngine.get_loc()

File pandas/_libs/hashtable_class_helper.pxi:7081, in pandas._libs.hashtable.PyObjectHashTable.get_item()

File pandas/_libs/hashtable_class_helper.pxi:7089, in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'sample_start_time_datum_cd'

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
Cell In[9], line 5
      3 startDate = '2012-01-01'
      4 endDate = ''
----> 5 wq_data2 = nwis.get_qwdata(sites=site_ids, parameterCd=parameterCd,
      6                            start=startDate, end=endDate)
      7 print('Retrieved data for ' + str(len(wq_multi_site[0])) + ' samples.')
      8 display(wq_data2[0])

File ~/.local/lib/python3.12/site-packages/dataretrieval/nwis.py:254, in get_qwdata(sites, start, end, multi_index, wide_format, datetime_index, ssl_check, **kwargs)
    251 df = _read_rdb(response.text)
    253 if datetime_index is True:
--> 254     df = format_datetime(df, "sample_dt", "sample_tm", "sample_start_time_datum_cd")
    256 return format_response(df, **kwargs), NWIS_Metadata(response, **kwargs)

File ~/.local/lib/python3.12/site-packages/dataretrieval/utils.py:79, in format_datetime(df, date_field, time_field, tz_field)
     56 """Creates a datetime field from separate date, time, and
     57 time zone fields.
     58
   (...)     76
     77 """
     78 # create a datetime index from the columns in qwdata response
---> 79 df[tz_field] = df[tz_field].map(tz)
     81 df["datetime"] = pd.to_datetime(
     82     df[date_field] + " " + df[time_field] + " " + df[tz_field],
     83     format="ISO8601",
     84     utc=True,
     85 )
     87 # if there are any incomplete dates, warn the user

File ~/.local/lib/python3.12/site-packages/pandas/core/frame.py:4102, in DataFrame.__getitem__(self, key)
   4100 if self.columns.nlevels > 1:
   4101     return self._getitem_multilevel(key)
-> 4102 indexer = self.columns.get_loc(key)
   4103 if is_integer(indexer):
   4104     indexer = [indexer]

File ~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3812, in Index.get_loc(self, key)
   3807     if isinstance(casted_key, slice) or (
   3808         isinstance(casted_key, abc.Iterable)
   3809         and any(isinstance(x, slice) for x in casted_key)
   3810     ):
   3811         raise InvalidIndexError(key)
-> 3812     raise KeyError(key) from err
   3813 except TypeError:
   3814     # If we have a listlike key, _check_indexing_error will raise
   3815     #  InvalidIndexError. Otherwise we fall through and re-raise
   3816     #  the TypeError.
   3817     self._check_indexing_error(key)

KeyError: 'sample_start_time_datum_cd'