I think we should have nightly/weekly test runs against master and canary. This would help us know earlier when something breaks. nodejs/node#17685 was an awesome start, but as @bnoordhuis said in the PR it won't catch semantic changes.

I'm willing to setup the infrastructure for it. I just need to know if we can have this on Travis (by using https://docs.travis-ci.com/user/cron-jobs/) or if we should introduce this on Node.js CI.

/cc @joyeecheung