Skip to content

Navigation Menu

Sign in
Appearance settings
Sign up
Appearance settings

New data source: PLOS Water via the PLOS API (pure R, no scraping) #14

Copy link
Copy link

Description

@larnsce
Issue body actions

Background

PLOS Water is a natural third source for this package. Unlike iwaponline.com, PLOS requires no scraping and no browser automation:

  • The public search API (https://api.plos.org/search, Solr syntax, JSON responses) returns DOI, title, volume, issue, publication date, authors, article type, and subject terms. Verified July 2026: journal:"PLOS Water" AND doc_type:full returns 436 articles, of which 333 are of type Research Article, from volume 1 (2022) onward.
  • Each article's full JATS XML is served at https://journals.plos.org/water/article/file?id=<DOI>&type=manuscript. The XML contains a <custom-meta id="data-availability"> block with the DAS text, <supplementary-material> elements, <corresp> author information, and affiliations.

DAS is mandatory at PLOS, so has_das will be close to always TRUE, which is itself an interesting contrast with washdev.

Proposal

New script data-raw/ploswater.R in pure R:

  • Page through the search API with httr2 (rows=100, filter doc_type:full) to collect all DOIs and article-level metadata. Mind the API's fair-use limits (stay under roughly 300 requests per hour).
  • For each DOI, download the JATS XML and parse with xml2: DAS text, supplementary material count, file types and links, first and correspondence author name, affiliation, country, email, and ORCID.
  • Consider whether to include all 436 articles with an article_type column, or only the 333 research articles. Opinion pieces and editorials often have no DAS and no data.
  • Write data-raw/ploswater.csv as the raw artifact, mirroring the washdev pattern.

The old rplos package is archived on CRAN, so call the API directly.

Notes

  • Keywords: some PLOS Water articles have no author keywords in the XML <kwd> elements; the API's subject categories can serve as a fallback. Document whichever is used.
  • paperid for this source should be the DOI (character), unlike the integer ids of washdev. See the harmonization issue for schema implications.
Reactions are currently unavailable

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

Morty Proxy This is a proxified and sanitized view of the page, visit original site.