Travis config to cache pdb dirs across builds#781
Travis config to cache pdb dirs across builds#781josemduarte wants to merge 3 commits intobiojava:masterbiojava/biojava:masterfrom josemduarte:travis-pdb-cachingjosemduarte/biojava:travis-pdb-cachingCopy head branch name to clipboard
Conversation
|
This is not an ideal solution. We often first discover changes to databases when Travis fails. However, the frequent network failures are an even bigger problem. Let's try this and see if it cuts down on the CI failures. |
|
Indeed I'm not convinced that this is the best solution. But as we are having so many failures I was eager to add some mitigation. I've found out that one of the frequently failing tests is |
|
It looks like FTP isn't supported: https://docs.travis-ci.com/user/common-build-problems/#FTP-SMTP-other-protocol-does-not-work It's weird that this used to work though. |
Travis blocks FTP so we're avoiding it.
eb8552a to
bef2fe2
Compare
|
I added a commit to your branch switching from FTP to HTTP for Sifts. The only place BioJava still references ftp is in BlastClustReader (ftp://resources.rcsb.org/sequence/clusters/). Maybe @josemduarte or @MrColeC could enable http or https for that server? On the other hand, it doesn't appear to be tested (👎) so hopefully travis will be OK. |
|
@sbliven It is not advertised, but has existed for ~2 years. You can get the sequence clusters here via HTTPS or HTTP https://cdn.rcsb.org/resources/sequence/clusters/bc-95.out You cannot get a directory listing, but if you know the file names you can pull them. |
|
Thank you @sbliven and @MrColeC ! Actually did not realise that EBI were providing the data over http too. Since most of the problems came from those ftp resources, I'd propose for now to close this Pull Request and create a new one with only the ftp->http change. If problems with building become bad again we can revisit adding the pdb dirs caching. |
Travis still fails at downloading external resources quite often (especially ftp resources). This change would cache some of the resources across builds.
I haven't tested this yet. The build of this PR will do the test.
It is still possible to clear the cache when needed, see https://docs.travis-ci.com/user/caching/#Clearing-Caches