Python Equivalents to R Vignette Examples
[1]:
from dataretrieval import nwis, waterdata, wqp
The dataretrieval Python package was created as an equivalent to the R dataRetrieval package.
The following shows Python equivalents for the methods outlined in the R dataRetrieval vignette, with the equivalent R code in comments.
[2]:
"""
{r getSite, echo=TRUE, eval=FALSE}
siteNumbers <- c("01491000","01645000")
siteINFO <- readNWISsite(siteNumbers)
"""
siteNumbers = ["USGS-01491000", "USGS-01645000"]
siteINFO, md = waterdata.get_monitoring_locations(
monitoring_location_id=siteNumbers, skip_geometry=True
)
Retrieving: monitoring-locations · 1 page · 2 rows
No API key detected — register for higher rate limits at https://api.waterdata.usgs.gov/signup/
[3]:
"""
# Continuing from the previous example:
# This pulls out just the daily, mean data:
dailyDataAvailable <- whatNWISdata(siteNumbers,
service="dv", statCd="00003")
"""
dailyDataAvailable, md = waterdata.get_time_series_metadata(
monitoring_location_id=siteNumbers, statistic_id="00003", skip_geometry=True
)
Retrieving: time-series-metadata · 1 page · 11 rows
[4]:
"""
# Choptank River near Greensboro, MD:
siteNumber <- "01491000"
parameterCd <- "00060" # Discharge
startDate <- "2009-10-01"
endDate <- "2012-09-30"
discharge <- readNWISdv(siteNumber, parameterCd, startDate, endDate)
"""
# Choptank River near Greensboro, MD:
siteNumber = "USGS-01491000"
parameterCd = "00060" # Discharge
discharge, md = waterdata.get_daily(
monitoring_location_id=siteNumber,
parameter_code=parameterCd,
time="2009-10-01/2012-09-30",
)
Retrieving: daily · 1 page · 1,096 rows
[5]:
"""
siteNumber <- "01491000"
parameterCd <- c("00010","00060") # Temperature and discharge
statCd <- c("00001","00003") # Maximum and mean
startDate <- "2012-01-01"
endDate <- "2012-05-01"
temperatureAndFlow <- readNWISdv(siteNumber, parameterCd, startDate, endDate, statCd=statCd)
"""
siteNumber = "USGS-01491000"
parameterCd = ["00010", "00060"] # Temperature and discharge
statisticId = ["00001", "00003"] # Maximum and mean
temperatureAndFlow, md = waterdata.get_daily(
monitoring_location_id=siteNumber,
parameter_code=parameterCd,
statistic_id=statisticId,
time="2012-01-01/2012-05-01",
)
Retrieving: daily · 1 page · 364 rows
[6]:
"""
parameterCd <- "00060" # Discharge
startDate <- "2012-05-12"
endDate <- "2012-05-13"
dischargeUnit <- readNWISuv(siteNumber, parameterCd, startDate, endDate)
"""
siteNumber = "USGS-01491000"
parameterCd = "00060" # Discharge
dischargeUnit, md = waterdata.get_continuous(
monitoring_location_id=siteNumber,
parameter_code=parameterCd,
time="2012-05-12/2012-05-13",
)
Retrieving: continuous · 1 page · 97 rows
[7]:
"""
# Dissolved Nitrate parameter codes:
parameterCd <- c("00618","71851")
startDate <- "1985-10-01"
endDate <- "2012-09-30"
dfLong <- read_USGS_samples(monitoringLocationIdentifier=sprintf("USGS-%s", siteNumber),
usgsPCode=parameterCd, activityStartDateLower=startDate, activityStartDateUpper=endDate)
"""
siteNumber = "USGS-01491000"
parameterCd = ["00618", "71851"]
dfLong, md = waterdata.get_samples(
monitoringLocationIdentifier=siteNumber,
usgsPCode=parameterCd,
activityStartDateLower="1985-10-01",
activityStartDateUpper="2012-09-30",
)
/tmp/ipykernel_9214/2092726829.py:12: DeprecationWarning: The 'activityStartDateLower' argument is deprecated and will be removed in a future release; use 'activity_start_date_lower' instead.
dfLong, md = waterdata.get_samples(
/tmp/ipykernel_9214/2092726829.py:12: DeprecationWarning: The 'activityStartDateUpper' argument is deprecated and will be removed in a future release; use 'activity_start_date_upper' instead.
dfLong, md = waterdata.get_samples(
/tmp/ipykernel_9214/2092726829.py:12: DeprecationWarning: The 'usgsPCode' argument is deprecated and will be removed in a future release; use 'usgs_pcode' instead.
dfLong, md = waterdata.get_samples(
/tmp/ipykernel_9214/2092726829.py:12: DeprecationWarning: The 'monitoringLocationIdentifier' argument is deprecated and will be removed in a future release; use 'monitoring_location_id' instead.
dfLong, md = waterdata.get_samples(
[8]:
"""
siteNumber <- '01594440'
peakData <- readNWISpeak(siteNumber)
"""
peakData, md = waterdata.get_peaks(
monitoring_location_id="USGS-01594440", parameter_code="00060"
)
---------------------------------------------------------------------------
RateLimited Traceback (most recent call last)
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/dataretrieval/ogc/chunking.py:655, in ChunkedCall._run.<locals>.track(index, args)
654 """One sub-request (with retry) + result-store + progress tick."""
--> 655 result = await _retry(lambda: fetch_gated(args), self.retry_policy)
656 self._chunks[index] = result
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/dataretrieval/ogc/retry.py:338, in _retry(afn, policy)
337 try:
--> 338 return await afn()
339 except Exception as exc: # noqa: BLE001 — re-raised unless retryable
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/dataretrieval/ogc/chunking.py:649, in ChunkedCall._run.<locals>.fetch_gated(args)
648 async with semaphore:
--> 649 return await self.fetch(args)
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/dataretrieval/ogc/engine.py:942, in _fetch_once(args)
941 req = _construct_api_requests(**args)
--> 942 return await _walk_pages(geopd=GEOPANDAS, req=req)
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/dataretrieval/ogc/engine.py:816, in _walk_pages(geopd, req, client)
814 return await sess.request(method, cursor, headers=headers, content=content)
--> 816 return await _paginate(
817 req,
818 parse_response=functools.partial(_ogc_parse_response, geopd=geopd),
819 follow_up=follow_up,
820 client=client,
821 )
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/dataretrieval/ogc/engine.py:695, in _paginate(initial_req, parse_response, follow_up, client, raise_for_status)
694 resp = await sess.send(initial_req)
--> 695 raise_for_status(resp)
696 initial_response = resp
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/dataretrieval/ogc/errors.py:127, in _raise_for_non_200(resp)
126 return
--> 127 raise error_for_status(
128 status,
129 _error_body(resp),
130 retry_after=_parse_retry_after(resp.headers.get("Retry-After")),
131 )
RateLimited: 429: Too many requests made. Please obtain an API token or try again later.
The above exception was the direct cause of the following exception:
QuotaExhausted Traceback (most recent call last)
Cell In[8], line 5
1 """
2 siteNumber <- '01594440'
3 peakData <- readNWISpeak(siteNumber)
4 """
----> 5 peakData, md = waterdata.get_peaks(
6 monitoring_location_id="USGS-01594440", parameter_code="00060"
7 )
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/dataretrieval/waterdata/api.py:2252, in get_peaks(monitoring_location_id, parameter_code, time_series_id, unit_of_measure, time, last_modified, water_year, year, month, day, peak_since, properties, skip_geometry, bbox, limit, filter, filter_lang, convert_type, max_rows, **queryables)
2248 service = "peaks"
2250 args = _get_args(locals(), exclude={"max_rows"})
-> 2252 return get_ogc_data(args, service, max_rows=max_rows)
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/dataretrieval/waterdata/utils.py:251, in get_ogc_data(args, service, output_id, max_rows)
249 if output_id is None:
250 output_id = _OUTPUT_ID_BY_SERVICE[service]
--> 251 return engine.get_ogc_data(
252 args,
253 service,
254 output_id,
255 max_rows=max_rows,
256 base_url=OGC_API_URL,
257 extra_id_cols=_EXTRA_ID_COLS,
258 dialect=WATERDATA_DIALECT,
259 )
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/dataretrieval/ogc/engine.py:920, in get_ogc_data(args, service, output_id, max_rows, base_url, extra_id_cols, dialect)
918 with _progress.progress_context(service=service), _row_cap(max_rows):
919 with _ogc_base_url(base_url), _dialect(dialect):
--> 920 return _fetch_once(args, finalize=finalize)
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/dataretrieval/ogc/chunking.py:771, in multi_value_chunked.<locals>.decorator.<locals>.wrapper(args, finalize)
767 retry_policy = RetryPolicy.from_env()
768 # The concurrency cap is resolved inside ``resume()`` from
769 # ``API_USGS_CONCURRENT``; ``1`` is a sequential gather,
770 # ``total <= 1`` a one-element gather — no special branch.
--> 771 return ChunkedCall(plan, fetch, retry_policy, finalize).resume()
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/dataretrieval/ogc/chunking.py:545, in ChunkedCall.resume(self)
504 """
505 Drive the chunked call to completion and return the combined result.
506
(...) 536 condition to clear and call ``exc.call.resume()`` again.
537 """
538 # Drive inside the snapshot taken at construction (see ``__init__``).
539 # ``start_blocking_portal`` copies the *calling* context into its
540 # worker thread, and running here means that calling context is the
(...) 543 # even when this is a resume fired long after the original ``with``
544 # blocks exited.
--> 545 return self._ctx.run(self._resume_in_context)
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/dataretrieval/ogc/chunking.py:555, in ChunkedCall._resume_in_context(self)
549 concurrency = _read_concurrency_env()
550 with start_blocking_portal() as portal:
551 # ``portal.call`` returns ``Any`` because ``functools.partial``
552 # erases ``_run``'s return type; restore the declared tuple.
553 return cast(
554 "tuple[pd.DataFrame, Any]",
--> 555 portal.call(functools.partial(self._run, concurrency)),
556 )
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/anyio/from_thread.py:338, in BlockingPortal.call(self, func, *args)
323 def call(
324 self,
325 func: Callable[[Unpack[PosArgsT]], Awaitable[T_Retval] | T_Retval],
326 *args: Unpack[PosArgsT],
327 ) -> T_Retval:
328 """
329 Call the given function in the event loop thread.
330
(...) 336
337 """
--> 338 return cast(T_Retval, self.start_task_soon(func, *args).result())
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/concurrent/futures/_base.py:460, in Future.result(self, timeout)
458 raise CancelledError()
459 elif self._state == FINISHED:
--> 460 return self.__get_result()
461 else:
462 raise TimeoutError()
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/concurrent/futures/_base.py:402, in Future.__get_result(self)
400 if self._exception is not None:
401 try:
--> 402 raise self._exception
403 finally:
404 # Break a reference cycle with the exception in self._exception
405 self = None
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/anyio/from_thread.py:263, in BlockingPortal._call_func(self, func, args, kwargs, future)
261 with CancelScope() as scope:
262 future.add_done_callback(callback)
--> 263 retval = await retval_or_awaitable
264 else:
265 retval = retval_or_awaitable
File /opt/hostedtoolcache/Python/3.13.14/x64/lib/python3.13/site-packages/dataretrieval/ogc/chunking.py:695, in ChunkedCall._run(self, max_concurrent)
693 if first_transient is not None:
694 interrupted, exc = first_transient
--> 695 raise interrupted from exc
697 return self.finalize(*self._combine_raw())
QuotaExhausted: HTTP 429 after 0/1 sub-requests; catch QuotaExhausted (or ChunkInterrupted) to access .partial_frame or .call.resume() once the rate-limit window has rolled over. Cause: RateLimited: 429: Too many requests made. Please obtain an API token or try again later.
[9]:
"""
ratingData <- readNWISrating(siteNumber, "base")
attr(ratingData, "RATING")
"""
# get_ratings returns a dict keyed by "<id>.<file_type>.rdb"
ratings_data = waterdata.get_ratings(
monitoring_location_id="USGS-01594440", file_type="base"
)
list(ratings_data.keys())
Failed to download / parse USGS-01594440.base.rdb: 429: Too many requests made. Please obtain an API token or try again later.
[9]:
[]
[10]:
"""
discharge_stats <- readNWISstat(siteNumbers=c("02319394"),
parameterCd=c("00060"),
statReportType="annual")
"""
discharge_stats, md = waterdata.get_stats_date_range(
monitoring_location_id="USGS-02319394",
parameter_code="00060",
computation_type="arithmetic_mean",
)
Retrieving: observationIntervals · 1 page · 344 rows
[11]:
# R: readNWISdata(service="dv", stateCd="WI", parameterCd="00060",
# drainAreaMin="50", statCd="00003")
#
# The Water Data API serves daily values per monitoring location. To assemble a
# state-wide set, first find the locations (optionally filtering by drainage
# area) with waterdata.get_monitoring_locations(state_name="Wisconsin", ...),
# then pass their ids to waterdata.get_daily(parameter_code="00060",
# statistic_id="00003").
[12]:
# '''
# sitesNJ <- whatWQPsites(statecode="US:34",
# characteristicName="Chloride")
# '''
# sitesNJ, md = wqp.what_sites(statecode="US:34", characteristicName="Chloride")
[13]:
# '''
# dataPH <- readWQPdata(statecode="US:55",
# characteristicName="pH")
# '''
# dataPH, md = wqp.what_sites(statecode="US:55", characteristicName="pH")
[14]:
# '''
# type <- "Stream"
# sites <- whatWQPdata(countycode="US:55:025",siteType=type)
# '''
# streamType = "Stream"
# sites, md = wqp.get_results(countycode="US:55:025", siteType=streamType)
[15]:
"""site <- whatWQPsamples(siteid="USGS-01594440")"""
site, md = wqp.what_sites(siteid="USGS-01594440")
[16]:
"""
type <- "Stream"
sites <- whatWQPmetrics(countycode="US:55:025",siteType=type)
"""
streamType = "Stream"
sites, md = wqp.what_sites(countycode="US:55:025", siteType=streamType)
Embedded Metadata
Most waterdata and wqp service methods return a tuple of the requested data (a pandas DataFrame) and a metadata object.
md is an object with the following attributes:
Metadata
url # the URL used to query the service
query_time # how long the query took
header # the response headers
Note: USGS water use data has no Water Data API equivalent yet. The legacy nwis.get_water_use() service has been decommissioned and now raises a “defunct” error, so there is currently no runnable way to retrieve water-use data through dataretrieval.