Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

srml.py limitations #2351

chzapo started this conversation in General
Jan 9, 2025 · 3 comments · 2 replies
Discussion options

I was looking over the source code and wanted to check my understanding of the variable map and its usage in srml.py.

The website link provided at
https://solardata.uoregon.edu/DataElementNumbers.html

Provides a more extensive list of column header translation than the variable map in pvlib-python. It appears pvlib-python only translates and ingests the columns that have built-in support within pvlib-python.

The comment in the code:
map_variables: bool, default: True When true, renames columns of the DataFrame to pvlib variable names where applicable. See variable :const: VARIABLE_MAP.

seems to corroborate this.

Is this correct?

The reason I was looking at this piece of code was I was hoping to dump a human readable .csv from the srml coded data file. I could write a piece of code to do this myself, but I was made aware that pvlib-python had a similar function. Is the above module the only place this exists or is there somewhere else in pvlib-python that has such a function (with 1-1 correspondence from number to string for the column headers) exists?

If this does not exist...would there be a desire for such a function (for srml) or functions for the other iotools in order to do this type of work (1-1 translation to human readable .csv files from the coded forms)?

Thanks for any feedback on this topic.

(Feels like there are probably 10 personal variations floating around that do this for srml...but maybe not).

You must be logged in to vote

Replies: 3 comments · 2 replies

Comment options

Is this correct?

Yes.

would there be a desire for such a function (for srml)

The challenge is keeping the VARIABLE_MAP constant up to date with changes in the data source, which are nearly always a surprise to pvlib maintainers.

There are quantities that are useful for pvlib that aren't in the current map, e.g., barometric pressure, rainfall, precipitable water. PR welcome here.

I also don't see any harm in adding mapping for other variables that pvlib doesn't currently use, e.g., tilted irradiance. If we can agree on how to name those quantities.

You must be logged in to vote
2 replies
@chzapo
Comment options

Okay I can work on this.

One thing I was thinking as well was maybe breaking out the variable map dictionary into a separate module and calling the module from the primary library module. This doubles the number of files, but separates the data (variable map) piece from the actual code. The advantage here is that we don't have to update the primary library module every time we need to update the variable map. A second advantage is as the variable map dictionary grows then we don't have to have data taking up more and more of the primary library module real estate (thus making reading it unwieldy). I can see two ways of doing this, one just having a global dictionary in the secondary module and calling it via the a module import in the primary library module or two putting the dictionary in a function in the variable map module and calling the function (with its return being the dictionary) after a module import.

Given my lack of experience with programming guidelines in pvlib-python I am not sure which fits project structure best. Which of these options (or if there is another option) would be the preferable route for doing this? Or do we just want to leave the variable map in the primary library module.

Thanks.

@echedey-ls
Comment options

That's amazing @chzapo !!
I recommend keeping it in the current module for the following reasons:

  • There are other VARIABLE_MAP's in pvlib, each one for different API services.
  • Documentation and references are all kept in the same place, to ease maintenance.

Just a heads up, a lot of things go on in each PR and at first it may feel a little overwhelming. So if you need any help feel free to reach out. A good starting point is this documentation page.

Comment options

It looks like pull requests can only be submitted after there is a change in the code base. So I think I will hold off on that till I get this all working (since it is only like 60 entries in a dictionary). What I do want to get a comment on is my naming scheme. I am going to post my proposed structure under issues. I also just found the Nomenclature page in yours scheme which is very helpful. I am going to try follow that where ever possible. Hopefully issues is the right place to put that and get comments.

You must be logged in to vote
0 replies
Comment options

It appears pvlib-python only translates and ingests the columns that have built-in support within pvlib-python.

@chzapo pvlib ingests all the available columns for each station but only renames the columns which are present in the variable maps. Note, that the available columns differ significantly from station to station, as some stations only measure GHI/DHI/DNI and others have tilted irradiance, etc.

You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
4 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.