Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
This repository was archived by the owner on Nov 6, 2023. It is now read-only.

Run node utils/trivialize-rules/trivialize-targets.js#18068

Merged
Hainish merged 13 commits into
EFForg:masterEFForg/https-everywhere:masterfrom
cschanaj:patch-17855Copy head branch name to clipboard
Jun 7, 2019
Merged

Run node utils/trivialize-rules/trivialize-targets.js#18068
Hainish merged 13 commits into
EFForg:masterEFForg/https-everywhere:masterfrom
cschanaj:patch-17855Copy head branch name to clipboard

Conversation

@cschanaj

@cschanaj cschanaj commented Jun 7, 2019

Copy link
Copy Markdown
Collaborator

This is a follow-up PR for #17855

There is around 500+ targets which are gone from DNS, however I am not going to handle the fetch errors in the PR.

Besides, it surprises me that the node URL object accepts hostname with leading dots. This make the script from #18068 a little buggy. Since it is not possible to whitelist rulesets from schematic errors, Travis failed. The problematic rulesets are fixed in eef2855

$ node
> new URL('https://.example.com/')
URL {
  href: 'https://.example.com/',
  origin: 'https://.example.com',
  protocol: 'https:',
  username: '',
  password: '',
  host: '.example.com',
  hostname: '.example.com',
  port: '',
  pathname: '/',
  search: '',
  searchParams: URLSearchParams {},
  hash: '' }
$ node --expose-gc utils/memusage.js

[Before]
Initial usage: 42 MB
Maximum usage: 77.8 MB

[After]
Initial usage: 42.3 MB
Maximum usage: 77.6 MB

@zoracon zoracon added the top-10k label Jun 7, 2019
@cschanaj cschanaj changed the title Run node utils/trivialize-rules/trivialize-targets.js WIP: Run node utils/trivialize-rules/trivialize-targets.js Jun 7, 2019
@cschanaj cschanaj changed the title WIP: Run node utils/trivialize-rules/trivialize-targets.js Run node utils/trivialize-rules/trivialize-targets.js Jun 7, 2019
@cschanaj

cschanaj commented Jun 7, 2019

Copy link
Copy Markdown
Collaborator Author

cc @Hainish @J0WI @Bisaloo @pipboy96 this is ready for review. note that this might cause a merge conflict with #18066 . thanks.

@Hainish

Hainish commented Jun 7, 2019

Copy link
Copy Markdown
Member

I know it's our normal policy to trivialize rules wherever possible, but have we done any memory analysis on this? I'm a bit worried about balooning the all_rules object...

@pipboy96

pipboy96 commented Jun 7, 2019

Copy link
Copy Markdown
Contributor

@Hainish I believe complex regexes themselves use up lots of memory.

@Hainish

Hainish commented Jun 7, 2019

Copy link
Copy Markdown
Member

I missed the memory analysis above! Thanks for this, merging

@Hainish Hainish merged commit 286dc23 into EFForg:master Jun 7, 2019
@cschanaj cschanaj deleted the patch-17855 branch June 7, 2019 22:26
@Bisaloo Bisaloo mentioned this pull request Jul 10, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.