Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Should pvlib have a get_solaranywhere function for fetching weather data? #1310

kandersolar started this conversation in Ideas
Discussion options

SolarAnywhere is a commercial weather data provider popular in the industry system modeling community. I only have a little experience with it, but my understanding is that they provide dataset downloads through both a point-and-click website and an API. I am aware of a few industry analysts that have rolled their own python wrappers for the SolarAnywhere weather API for fetching weather data in bulk, so I speculate a function in pvlib.iotools would get some use.

However the API does require authentication to use. Maybe CPR could give us credentials suitable for testing purposes? Maybe CPR would be interested in contributing the wrapper function itself? cc @cpr-chas

Note that their API docs already provide example python snippets, so you could argue that we're not bringing much value by duplicating it in pvlib: https://developers.cleanpower.com/irradiance-and-weather-data/irradiance-and-weather-requests/

Note also that I'm talking specifically about fetching weather data, not their recent pvlib-based power offering.

You must be logged in to vote

Replies: 2 comments · 14 replies

Comment options

Also, not sure if you heard, but SolarAnywhere Public just launched. You raise a good question about the proliferation of code. Usually less is more, and the zen of python says there should be one - and only one - way to do things. If CPR already has a client, why should we reproduce it so now there's two? What does ours add? Maybe consistent output?

You must be logged in to vote
9 replies
@kandersolar
Comment options

kandersolar Sep 27, 2021
Maintainer Author

To me, ease of integration w/ pvlib is the compelling point in favor. Because these providers have paid employees that could do this, and I myself have no use for commercial data, I am unlikely to add functions along these lines. But I wouldn't be opposed if others contribute them. Someone would need to coordinate with the provider for authentication and such for the tests.

Just another note: commercial weather data access is not the only potential pvlib feature for proprietary/non-public data or formats. For example PVsyst PAN files (#1002 (comment)). This is a case where some kind of steering committee and better knowledge of our users could be helpful.

@williamhobbs
Comment options

I support this idea, and not just for SolarAnywhere. Adding @KWagnerCPR.

@KWagnerCPR
Comment options

@AdamRJensen SolarAnywhere recently enabled the use of JSON. This page of our documentation contains a python sample that should return data in JSON rather than XML: [https://developers.cleanpower.com/irradiance-and-weather-data/irradiance-and-weather-requests/].

We are happy to collaborate on this function. API access is now self-serve, so anyone who registers for a SolarAnywhere Public account is able to generate their own API keys to use in accessing the SolarAnywhere Public dataset. This page of our support center provides instructions on that process: [https://www.solaranywhere.com/support/using-solaranywhere/api/]

@AdamRJensen
Comment options

@AdamRJensen SolarAnywhere recently enabled the use of JSON. This page of our documentation contains a python sample that should return data in JSON rather than XML: [https://developers.cleanpower.com/irradiance-and-weather-data/irradiance-and-weather-requests/].

@KWagnerCPR good to hear about the json option. I'll draft up a pull request in the next few days!

@williamhobbs
Comment options

Would a read_solaranywhere function for reading .csv files in the standard SolarAnywhere format also be appropriate to add?

Comment options

I believe the SATGY format is similar to TMY3 format except it only has about 7 columns and the header has some extra info after the last field, so hoping to repurpose or reuse some of the existing read_tmy3() code maybe?

You must be logged in to vote
5 replies
@williamhobbs
Comment options

The SolarAnywhere and TMY3 formats are fairly similar.

Note that SolarAnywhere can be downloaded in "TMY3" format or "SolarAnywhere" format.

image

The "TMY3" appears to deviate from the "official" TMY3 format: I think SolarAnywhere in TMY3 format uses hours 00 through 23, while official TMY3 uses 01:00 through 24:00.

SolarAnywhere files can also start and end on any day of the year, and can span multiple years. I think they always include leap day, if that matters.

Sample files for SolarAnywhere v3.4 are here, NREL/SAM#491 (comment), although as of May 2022 version 3.6 is available (https://www.solaranywhere.com/support/historical-data/release-notes/).

Finally, https://github.com/NREL/SAM/blob/develop/deploy/runtime/macros/Solar%20Resource%20File%20Converter.lk might be a useful reference.

@KWagnerCPR
Comment options

@williamhobbs Thanks for providing all this info! One small correction is that our TMY3 files do start at 01:00 and end at 24:00 (00:00 of the next day).

SolarAnywhere users specify the start and end date of the data they'd like to download. When you select the TMY3 output format option, the data will be automatically broken down into individual year files. If you specify a start or end time part of the way through a year, then you'll get a partial year file for that year. When you select the SolarAnywhere output format option, you get one file containing the full time series for the requested period.

We'd recommend using the SolarAnywhere output format if possible. This format allows our users to include the snow depth, precipitation, clear sky irradiance and particulate matter fields. The TMY3 format does not have existing columns for these fields so users can't include them in TMY3 output format downloads. I've attached a file containing the full SolarAnywhere V3.6 time series in SolarAnywhere output format for one of our public sites in case this is helpful.

El Paso, United States SolarAnywhere Time Series 19980101 to 20220102 Lat_31_775 Lon_-106_485 SA format.csv

@williamhobbs
Comment options

@KWagnerCPR I was referring to the use of "00:00" vs "24:00" in CPR's "TMY3" format vs NREL's TMY3 format. Depending on how times are handled, I think that difference can matter.

Example of CPR's TMY3-formatted file:
image

Example of TMY3 file included with pvlib:
image

@KWagnerCPR
Comment options

@williamhobbs I see! Thank you for clarifying.

@AdamRJensen
Comment options

@williamhobbs please see #1494 which proposes a change to the pvlib read_tmy3 function to handle this difference in the TMY3 format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
💡
Ideas
Labels
6 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.