hydrolink package

Submodules

hydrolink.nhd_hr module

Allows for HydroLinking of data to the National Hydrography Dataset High Resolution (NHDHR).

Module that HydroLinks data to the National Hydrography Dataset High Resolution (NHDHR). Classes and methods are designed to handle one feature at a time. This module is designed to handle errors by returning a message to objects to help facilitate HydroLinking of multiple points in a single program. Currently only HydroLinks point data (not lines or polygons) to flowlines or waterbodies. The terms “HydroLinking” and “addressing” are used synonymously throughout this code and both refer to making a relationship between a feature and a stream network, similar to addresses assigned to road networks.

Author

Name: Daniel Wieferich Contact: dwieferich@usgs.gov

class hydrolink.nhd_hr.HighResPoint(source_identifier, input_lat, input_lon, input_crs=4269, water_name=None, buffer_m=1000)

Bases: object

Class specific for HydroLinking point data to the NHDHR.

build_nhd_query(query=['hem_flowline', 'hem_waterbody'])

Build queries to return required data for HydroLink process.

Parameters:

• query (list, default ['hem_flowline', 'hem_waterbody']) –

  Specifies MapServer instance to use for HydroLink. Uses user specified information from object. Supported queries include

  • 'hem_flowline': Default query that returns data for flowline features within a buffer of

  a given location. This query uses the following Hydro Event Management (HEM) MapServer layer https://hydromaintenance.nationalmap.gov/arcgis/rest/services/HEM/NHDHigh/MapServer/1. - 'hem_waterbody': Query that returns data for the waterbody feature that intersects a given location. This query uses the following Hydro Event Management (HEM) MapServer layer https://hydromaintenance.nationalmap.gov/arcgis/rest/services/HEM/NHDHigh/MapServer/2. - 'hem_waterbody_flowline': Query that returns data for flowline features within a waterbody. This query uses the following Hydro Event Management (HEM) MapServer layer https://hydromaintenance.nationalmap.gov/arcgis/rest/services/HEM/NHDHigh/MapServer/1.

• Note(s) –
• ---------- –
• queries for NHDHR are currently those supported by Hydro Event Management (HEM) In this MapServer (Recommended) –
• flowlines are in one layer and this layer provides a recent version of NHDHR data. (all) –
• National Map Mapservers (including HighResPlus) can also be used but these services split flowlines (The) –
• in-network and non-network and will require multiple service calls (not handled in current code) (into) –

error_handling()

Handle errors throughout HydroLink.

hydrolink_flowlines()

Evaluate flowlines in self.flowlines_json to understand certainty for HydroLink selection.

Evaluate flowlines in self.flowlines_json to understand certainty for HydroLink selection. Evaluations include calculating distance to each flowline, calculating distance to closest confluence, calculate name similarity for water names. Handles instances where no flowlines are available and failed evaluations.

Parameters:

• self.flowlines_json (dictionary) – JSON returned from request of flowline_query. JSON contains data about flowlines.
• self.water_name (str) – User supplied name of the waterbody that a point occurs on. Optional
• self.input_point (shapely point) – Input location for hydrolinking. For formatting see shapely.geometry Point method

Returns:

• self.closest_confluence_meters (float) – Distance from input point to closest confluence in meters, see utils.closest_confluence
• self.flowline_data (dictionary) – Contains information about a flowline

hydrolink_method(method='name_match', hydro_type='flowline', outfile_name='nhdhr_hydrolink_output.csv', similarity_cutoff=0.6)

Build HydroLinking pipeline based on specified method and hydro_type.

Builds commonly used HydroLink pipelines for users. These pipelines write data to the outfile specified in “outfile_name”.

Parameters:

• method ({'name_match', 'closest'}, default 'name_match') –

  Method for HydroLinking data. Supported methods are

  • 'name_match': This default method HydroLinks data to the closest NHD feature with a name similarity

  that meets the specified similarity_cutoff. If no flowlines meet similarity cutoff the method HydroLinks data to the closest NHD feature. - 'closest': This method HydroLinks data to the closest NHD feature.

• hydro_type ({'waterbody', 'flowline'}, default 'flowline') –

  Type of features to HydroLink. Feature types as defined by NHDHR.

  • 'flowline': This default feature type specifies NHD feature type of flowline.

  Flowline features represent water types such as streams, rivers, canals/ditches. Waterbodies also have line representations as flowline type. - 'waterbody': This feature type specifies NHD feature type of waterbody. Waterbody features represent water types such as lakes, ponds, estuaries, reservoirs, marshes, swamps.

• outfile_name (str) – Name and directory of csv output file. default is ‘nhdhr_hydrolink_output.csv’.
• similarity_cutoff (float) – Values between 0 and 1.0, range of similarity between 0 representing no match to 1.0 being perfect match.

is_in_waterbody()

Check to see if point location falls within waterbody feature.

Check to see if point location falls within waterbody feature. If it does it collects HydroLink data for the waterbody and also resets query of flowlines (self.flowline_query) to only return flowlines that intersect with the waterbody.

Parameters:self.waterbody_query (str) – Query built in build_nhd_query Returns:

• self.hydrolink_waterbody (dictionary) – Attributes of HydroLink (address) to waterbody
• self.build_nhd_query (list) – Reset self.build_nhd_query to query only flowlines within waterbody

query_flowlines()

Query flowlines using query built in build_nhd_query.

Query flowlines using query built in build_nhd_query. Handles failed requests and instances where no flowlines are returned.

Parameters:self.flowline_query (str) – Query built in build_nhd_query Returns:self.flowlines_json – JSON returned from request of flowline_query. JSON contains data about flowlines. Return type:dictionary

select_closest_flowline(similarity_cutoff=0.6)

Select closest flowline.

Selects closest flowline from flowlines_data, including all evaluation information for the flowline. Although name similarity is not considered for selection it is used to document if a name matched flowline is available. Requires output from hydrolink_flowlines.

select_closest_flowline_w_name_match(similarity_cutoff=0.6)

Select closest flowline with matching water name.

HydroLink data to the closest NHD feature with a name similarity that meets the specified similarity_cutoff. If no flowlines meet similarity cutoff the method HydroLinks data to the closest NHD feature. Requires output from hydrolink_flowlines.

write_hydrolink(outfile_name='nhdhr_hydrolink_output.csv')

Write HydroLink data output to CSV.

hydrolink.nhd_mr module

Allows for HydroLinking of data to the National Hydrography Dataset Plus Version 2.1 (NHDPlusV2.1).

Module that HydroLinks data to the National Hydrography Dataset Plus Version 2.1 (NHDPlusV2.1). Classes and methods are designed to handle one feature at a time. This module is designed to handle errors by returning a message to objects to help facilitate HydroLinking of multiple points in a single program. Currently only HydroLinks point data (not lines or polygons) to flowlines or waterbodies. The terms “HydroLink” and “addressing” are used synonymously throughout this code and both refer to making a relationship between a feature and a stream network, similar to addresses assigned to road networks.

Author

Name: Daniel Wieferich Contact: dwieferich@usgs.gov

class hydrolink.nhd_mr.MedResPoint(source_identifier, input_lat, input_lon, input_crs=4269, water_name=None, buffer_m=1000)

Bases: object

Class specific for HydroLinking point data to the NHDPlusV2.1.

build_nhd_query(query=['network_flow', 'waterbody'])

Build queries to return required data for HydroLink process.

Parameters:

• query (list, default ['network_flow', 'waterbody']) –

  Specifies MapServer instance to use for HydroLink. Uses user specified information from object. Supported queries include

  • 'network_flow': Default query that returns data for flowline features within a buffer of

  a given location. This query uses the WatersGeo MapServer layer from EPA. - 'nonnetwork_flow': Default query that returns data for flowline features within a buffer of a given location. This query uses the WatersGeo MapServer layer from EPA. - 'waterbody': Query that returns data for the waterbody feature that intersects a given location. This query uses the WatersGeo MapServer layer from EPA. - 'waterbody_flowline': Query that returns data for flowline features within a waterbody. This query uses the WatersGeo MapServer layer from EPA.

• Note(s) –
• ---------- –
• nonnetwork streams are only used if no streams are returned with network flowlines. (Currently) –
• would be best to query both network and nonnetwork and combine before hydrolinking but that (It) –
• the requests and would increase processing time and service loads... (doubles) –

error_handling()

Handle errors throughout HydroLink.

hydrolink_flowlines()

Evaluate flowlines in self.flowlines_json to understand certainty for HydroLink selection.

Evaluate flowlines in self.flowlines_json to understand certainty for HydroLink selection. Evaluations include calculating distance to each flowline, calculating distance to closest confluence, calculate name similarity for water names. Handles instances where no flowlines are available and failed evaluations.

Parameters:

• self.flowlines_json (dictionary) – JSON returned from request of flowline_query. JSON contains data about flowlines.
• self.water_name (str) – User supplied name of the waterbody that a point occurs on. Optional
• self.input_point (shapely point) – Input location for hydrolinking. For formatting see shapely.geometry Point method

Returns:

• self.closest_confluence_meters (float) – Distance from input point to closest confluence in meters, see utils.closest_confluence
• self.flowline_data (dictionary) – Contains information about a flowline

hydrolink_method(method='name_match', hydro_type='flowline', outfile_name='nhdplusv2_hydrolink_output.csv', similarity_cutoff=0.6)

Build HydroLinking pipeline based on specified method and hydro_type.

Builds commonly used HydroLink pipelines for users. These pipelines write data to the outfile specified in “outfile_name”.

Parameters:

• method ({'name_match', 'closest'}, default 'name_match') –

  Method for HydroLinking data. Supported methods are

  • 'name_match': This default method HydroLinks data to the closest NHD feature with a name similarity

  that meets the specified similarity_cutoff. If no flowlines meet similarity cutoff the method HydroLinks data to the closest NHD feature. - 'closest': This method HydroLinks data to the closest NHD feature.

• hydro_type ({'waterbody', 'flowline'}, default 'flowline') –

  Type of features to HydroLink. Feature types as defined by NHDPlusV2.1.

  • 'flowline': This default feature type specifies NHD feature type of flowline.

  Flowline features represent water types such as streams, rivers, canals/ditches. Waterbodies also have line representations as flowline type. - 'waterbody': This feature type specifies NHD feature type of waterbody. Waterbody features represent water types such as lakes, ponds, estuaries, reservoirs, marshes, swamps.

• outfile_name (str) – Name and directory of csv output file. default is ‘nhdplusv2_hydrolink_output.csv’.
• similarity_cutoff (float) – Values between 0 and 1.0, range of similarity between 0 representing no match to 1.0 being perfect match.

is_in_waterbody()

Check to see if point location falls within waterbody feature.

Check to see if point location falls within waterbody feature. If it does it collects HydroLink data for the waterbody and also resets query of flowlines (self.flowline_query) to only return flowlines that intersect with the waterbody.

Parameters:self.waterbody_query (str) – Query built in build_nhd_query Returns:

• self.hydrolink_waterbody (dictionary) – Attributes of HydroLink (address) to waterbody
• self.build_nhd_query (list) – Reset self.build_nhd_query to query only flowlines within waterbody

query_flowlines()

Query flowlines using query built in build_nhd_query.

Query flowlines using query built in build_nhd_query. Handles failed requests and instances where no flowlines are returned.

Parameters:self.flowline_query (str) – Query built in build_nhd_query Returns:self.flowlines_json – JSON returned from request of flowline_query. JSON contains data about flowlines. Return type:dictionary

select_closest_flowline(similarity_cutoff=0.6)

Select closest flowline.

Selects closest flowline from flowlines_data, including all evaluation information for the flowline. Although name similarity is not considered for selection it is used to document if a name matched flowline is available. Requires output from hydrolink_flowlines.

select_closest_flowline_w_name_match(similarity_cutoff=0.6)

Select closest flowline with matching water name.

HydroLink data to the closest NHD feature with a name similarity that meets the specified similarity_cutoff. If no flowlines meet similarity cutoff the method HydroLinks data to the closest NHD feature. Requires output from hydrolink_flowlines.

write_hydrolink(outfile_name='nhdplusv2_hydrolink_output.csv')

Write HydroLink data output to CSV.

hydrolink.utils module

Utility functions used across hydrolink modules.

Author

Daniel Wieferich: dwieferich@usgs.gov

hydrolink.utils.build_distance_line(point_1, point_2, crs='epsg:4269')

Build line from point1 to point2 an measure distance in meters.

Parameters:

• point_1 (shapely point) – Location from which distance calculations are made
• point_2 (shapely point) – Location to which distance calculations are made
• crs (str, default = 'epsg:4269') – The value can be anything accepted by pyproj.CRS.from_user_input(), such as an authority string (eg “EPSG:4326”) or a WKT string.

Returns:

line_length_meters – Distance in meters

Return type:

float

hydrolink.utils.build_flowline_details(flowline_data, input_point, nhd_version='nhdhr', source_water_name='')

Brings together functions to get hydrolink details for a flowline.

Parameters:

• flowline_data (dictionary) – Contains information about a flowline, built in nhd_hr.hydrolink_flowlines or nhd_mr.hydrolink_flowlines
• input_point (shapely point) – Input location for hydrolinking. For formatting see shapely.geometry Point method
• nhd_version ({'nhdhr', 'nhdplusv2'}, default 'nhdhr') –

  Version of National Hydrography Dataset. Supported versions include

  • 'nhdhr': National Hydrography Dataset High Resolution
  • 'nhdplusv2': National Hydrography Dataset Plus Version 2.1 Medium Resolution
• source_water_name (str, optional, default '') – If available name of water body (e.g. name of lake, river, estuary…)

Returns:

• flowline_attributes (dictionary) – Hydrolink calculated attributes updated to input flowline_data
• terminal_node_points (list) – List of shapely points expressed in wkt representing terminal nodes of flowline Example [POINT (-70.63598606746581 41.7689812018329)’, ‘POINT (-70.63435706746833 41.77051200183053)’]
• flowline_geo (list) – List of shapely points representing shapely linestring of the flowline

hydrolink.utils.clean_water_name(name)

Quick and dirty approach to clean up unstandardized water names.

Replaces common abbreviations, and deals with unnneeded spaces. This needs improvement but need to be careful not to replace unwanted strings. This step is implemented with the assumption that GNIS_NAME never contains abbreviations… something to verify. If you have a better way to do this let me know!!!!

Parameters:name (str) – name of water feature as defined by user Returns:water_name_cleaned – resulting waterbody name with common abbreviations (hopefully) spelled out Return type:str

hydrolink.utils.closest_confluence(terminal_node_points, input_point, flowline_geo)

Calculate distance from input point coordinates to closest confluence.

This function accepts a list of terminal nodes. A confluence is considered where a terminal node is shared in three or more flowlines. The smaller this number the less certain a hydrolink will be.

Parameters:

• terminal_node_points (list) – complete list of terminal node points from a subset of flowlines including duplicate values
• input_point (shapely point) – location from which distance calculations are made. For formatting see shapely.geometry Point method
• flowline_geo (list) – List of shapely points representing shapely linestring of the flowline, see build_flowline_details

Returns:

closest_confluence_meters – distance from input point to closest confluence in meters

Return type:

float

hydrolink.utils.crs_to_nad83(input_point, crs)

Reproject point data to NAD83, aka crs 4269.

Parameters:

• input_point (shapely point) – See shapely.geometry Point method
• input_crs (int) – EPSG defined coordinate reference system, default is 4269 which is NAD83

Returns:

• lon_nad83 (float) – longitude in crs 4269 (NAD83)
• lat_nad83 (float) – latitude in crs 4269 (NAD83)

hydrolink.utils.df_for_selection(flowlines_data)

Organizes flowline data into pandas dataframe for ease in selection of information.

Parameters:flowlines_data (dictionary) – data for all flowlines Returns:df Return type:pandas dataframe

hydrolink.utils.gnis_name_similarity(gnis_name, source_water_name)

Similarity comparison of two names using difflib.

Measures similarity between two names using difflib Returned dictionary has a measure of similarity called ‘flowline name similarity’

Parameters:

• gnis_name (str) – name of water feature as defined by USGS Geographic Names Information System
• source_water_name (str) – name of water feature as defined by user

Returns:

name_similarity – dictionary capturing measures of similarity ‘flowline name similarity’ is a float and values range from 0 (no match) to 1 (exact match) ‘flowline name similarity message’ is a text representation of the similarity measure

Return type:

dictionary

hydrolink.utils.nhd_flowline_measure(flowline_geo, node_measures, flowline_snap_point)

Measure along flowline where the flowline_snap_point is located (the address).

Parameters:

• flowline_geo (list) – List of shapely points representing shapely linestring of the flowline, see build_flowline_details
• node_measures (list) – nhd measures within flowline_geo (m values)
• flowline_snap_point (shapely point) – shapely point that marks snap location (closest location on line to a coordinate), see point_to_line_meters

Returns:

Measure along NHD flowline where 0 is most downstream node and 100 is most upstream node within the reachcode Note a reachcode can span multiple flowlines

Return type:

nhd_measure

hydrolink.utils.point_to_line_meters(flowline_geo, input_point)

Calculate distance in meters from input point coordinates to closest point along a line.

Parameters:

• input_point (shapely point) – Location from which distance calculations are made. For formatting see shapely.geometry Point method
• flowline_geo (list) – List of shapely points representing shapely linestring of the flowline, see build_flowline_details

Returns:

flowline_snap_point – Location along flowline that is closest in distance to input_point.

Return type:

shapely point

Module contents

Top-level package for HydroLink.