Skip to content

Commit

Permalink
Add HNTrie-based filter classes to store origin-only filters
Browse files Browse the repository at this point in the history
Related issue:
- uBlockOrigin/uBlock-issues#528 (comment)

Following STrie-related work in above issue, I noticed that a large
number of filters in EasyList were filters which only had to match
against the document origin. For instance, among just the top 10
most populous buckets, there were four such buckets with over
hundreds of entries each:

- bits: 72, token: "http", 146 entries
- bits: 72, token: "https", 139 entries
- bits: 88, token: "http", 122 entries
- bits: 88, token: "https", 118 entries

These filters in these buckets have to be matched against all
the network requests.

In order to leverage HNTrie for these filters[1], they are now handled
in a special way so as to ensure they all end up in a single HNTrie
(per bucket), which means that instead of scanning hundreds of entries
per URL, there is now a single scan per bucket per URL for these
apply-everywhere filters.

Now, any filter which fulfill ALL the following condition will be
processed in a special manner internally:

- Is of the form `|https://` or `|http://` or `*`; and
- Does have a `domain=` option; and
- Does not have a negated domain in its `domain=` option; and
- Does not have `csp=` option; and
- Does not have a `redirect=` option

If a filter does not fulfill ALL the conditions above, no change
in behavior.

A filter which matches ALL of the above will be processed in a special
manner:

- The `domain=` option will be decomposed so as to create as many
  distinct filter as there is distinct value in the `domain=` option
- This also apply to the `badfilter` version of the filter, which
  means it now become possible to `badfilter` only one of the
  distinct filter without having to `badfilter` all of them.
- The logger will always report these special filters with only a
  single hostname in the `domain=` option.

***

[1] HNTrie is currently WASM-ed on Firefox.
  • Loading branch information
gorhill committed Apr 19, 2019
1 parent fd9df4b commit 3f3a154
Show file tree
Hide file tree
Showing 4 changed files with 590 additions and 440 deletions.
4 changes: 2 additions & 2 deletions src/js/background.js
Original file line number Diff line number Diff line change
Expand Up @@ -137,8 +137,8 @@ const µBlock = (function() { // jshint ignore:line

// Read-only
systemSettings: {
compiledMagic: 8, // Increase when compiled format changes
selfieMagic: 9 // Increase when selfie format changes
compiledMagic: 10, // Increase when compiled format changes
selfieMagic: 10 // Increase when selfie format changes
},

restoreBackupSettings: {
Expand Down
Loading

0 comments on commit 3f3a154

Please sign in to comment.