Practical Implementation Advice
- WQX is a valuable data flow for programs, and it is stable and ready to implement. Partners not already flowing WQX data via the Exchange Network are encouraged to implement the flow through an EN node or node client.
- Institutions that do not already have a data management system for water quality data may want to focus on developing such a system before focusing on flowing data via the Exchange Network. There are various approaches that Partners can implement.
- Plugins are available for both OpenNode2 and EN-Node. Dataflow plugins can help with the mapping process and make data flow implementation more efficient.
WQX Data Flow Options
The current options for flowing WQX data are presented here. Exchange Network flow options are shown in green. There are no non-EN options.
Summary of Current Practice
Starting in 2009, WQX replaced the use of the distributed database STORET for reporting water quality data. WQX is now the only method for reporting water quality data. Submission of water quality data can be a completely automated node-to-node flow. Partners who do not have a node or are not ready for a completely automated flow, can use WQX Web, which is an EN web client. Partners may also use their EN local clients to submit XML data.
WQX Flow Status and Milestones
WQX is stable and ready to implement. The main focus of the Office of Water over the near term will be developing specifications for data access services to encourage sharing of water quality data among the user community. Institutional responsibilities and target completion dates for each activity are presented here. EPA’s general criteria for assessing the “readiness” of National System Flows follow.
|Solutions for all partners||Done|
|Access to transaction status||Done|
|Accessible and stable flow documentation||Done|
|Specifications for Data Access Services||Attention Required||Document and make available via the EN standard specifications for data access services based on existing Network or public-facing services that meet the needs of the user community.||OW, with stakeholder input and NOB support||Q2 2013|
|Clear path to eliminate alternatives||Done|
Node: A Partner’s point of presence on the EN consisting of a server (hardware and software) enabled with web services that allow Partners to automatically provide and receive information via the Network and to publish data for use by other Partners.
Node Client: A stand-alone application (i.e., software code) that lets Partners share data, request data, and receive results from an EN request. Clients differ from nodes in that they cannot respond to queries from other nodes and so cannot publish data. Clients also need more manual (rather than automated) steps, for example, to extract data and generate and review reports before submission.
EN Services Center: A website that allows Partners to easily send, get, and download information from other Partners. The Services Center will serve as a replacement for manual submissions of information through CDX Web. It is an appropriate solution for those Partners who do not require or are not yet ready for the automation and data publishing capabilities of an EN Node. The EN Services Center is available at https://enservices.epa.gov.
CDX: EPA’s Central Data Exchange. It serves as EPA’s centralized electronic report receiving system. It receives data from Partners and directs the data to EPA’s program-specific National Systems (e.g., AQS, WQX, etc.).
CDX Node: CDX Node is EPA’s node on the Network, allowing EPA to receive, send, and provide information via the Network. CDX Node can also publish EPA data for use by other Partners.
CDX Web (non-EN) Application: A legacy CDX application that receives data (flat file or XML format) via standard web browsers. CDX Web applications are not consistent with EN protocols (e.g., they have a separate authentication and authorization service from the EN) and typically involve more manual steps than a node-to-node exchange of data.
Data Access Services: Using web services to make data available to Partners by querying nodes and returning environmental data in the form of XML documents. Published data can be accessed using a node or clients. Published data can be used in a number of ways, such as populating Web pages, synchronizing data between sites, viewing data in a Web service client, or building new sources of data into an integrated application.
Direct User: A Partner entering data directly into a National Data System through a system-specific interface (manual entry).
EPA National Data System: Program-specific data systems at EPA that can receive and publish data..
Local Data System: A Partner’s database or series of databases in which environmental data are stored, managed, and manipulated.
XML: eXtensible Markup Language is a flexible language for creating common information formats and sharing both the format and content of data over the Internet and elsewhere. The electronic language that expresses and transports data standards and transaction sets. XML uses an extensible set of tags to describe the meaning of data.
National System Flow “Ready-to-Use” Criteria
A focus of Governance has been developing the National System Flows to help Partners take advantage of the Network’s business value. Governance has identified six criteria for each flow to meet to make these flows “ready to use,” they are:
- Automation-ready flows. Support fully automated node-to-node flows.
- Solutions for all Partners. Provide appropriately scaled solutions for Partners of all sizes, needs, and capabilities. Some Partners such as Tribes and local clean air authorities may not need a fully functional node, therefore, other solutions should be available.
- Access to transaction status. Support a fully automated process for reporting transaction status, processing results, and QA results from receipt by CDX through final processing in the National System.
- Accessible and stable flow documentation. Develop and make accessible stable documentation that describes all flow requirements. This includes a complete Flow Configuration Document (FCD) that is in compliance with EN procedures for version management.
- Specifications for Data Access Services. Provide a national standard set of query/solicit services defined in the FCD whether or not data are currently published. Implement a publishing interface where published data are critical to partner business processes (such as NPDES permit information for NetDMR).
- Clear path to eliminate alternatives. Have a clear path to eliminate legacy system alternatives to data exchanges, including transition support for Partners.
The STORET Team
U.S. Environmental Protection Agency
storet [at] epa.gov