Skip to content

Commit

Permalink
Add new filter option queryprune=
Browse files Browse the repository at this point in the history
Related issue:
- uBlockOrigin/uBlock-issues#760

The purpose of this new network filter option is to remove
query parameters form the URL of network requests.

The name `queryprune` has been picked over `querystrip`
since the purpose of the option is to remove some
parameters from the URL rather than all parameters.

`queryprune` is a modifier option (like `csp`) in that it
does not cause a network request to be blocked but rather
modified before being emitted.

`queryprune` must be assigned a value, which value will
determine which parameters from a query string will be
removed. The syntax for the value is that of regular
expression *except* for the following rules:

- do not wrap the regex directive between `/`
- do not use regex special values `^` and `$`
- do not use literal comma character in the value,
  though you can use hex-encoded version, `\x2c`
- to match the start of a query parameter, prepend `|`
- to match the end of a query parameter, append `|`

`queryprune` regex-like values will be tested against each
key-value parameter pair as `[key]=[value]` string. This
way you can prune according to either the key, the value,
or both.

This commit introduces the concept of modifier filter
options, which as of now are:

- `csp=`
- `queryprune=`

They both work in similar way when used with `important`
option or when used in exception filters. Modifier
options can apply to any network requests, hence the
logger reports the type of the network requests, and no
longer use the modifier as the type, i.e. `csp` filters
are no longer reported as requests of type `csp`.

Though modifier options can apply to any network requests,
for the time being the `csp=` modifier option still apply
only to top or embedded (frame) documents, just as before.
In some future we may want to apply `csp=` directives to
network requests of type script, to control the behavior
of service workers for example.

A new built-in filter expression has been added to the
logger: "modified", which allow to see all the network
requests which were modified before being emitted. The
translation work for this new option will be available
in a future commit.
  • Loading branch information
gorhill committed Oct 31, 2020
1 parent ba2ef92 commit 1e2eb03
Show file tree
Hide file tree
Showing 14 changed files with 951 additions and 482 deletions.
2 changes: 1 addition & 1 deletion platform/chromium/vapi-background.js
Original file line number Diff line number Diff line change
Expand Up @@ -1145,7 +1145,7 @@ vAPI.warSecret = (( ) => {
url.lastIndexOf(`?secret=${secret}`) !== -1
);
if ( pos === -1 ) {
return { redirectUrl: root };
return { cancel: true };
}
secrets.splice(pos, 1);
};
Expand Down
4 changes: 3 additions & 1 deletion src/js/about.js
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,9 @@
vAPI.messaging.send('dashboard', {
what: 'sfneBenchmark',
}).then(result => {
document.getElementById('sfneBenchmarkResult').textContent = result;
document.getElementById('sfneBenchmarkResult').prepend(
document.createTextNode(result.trim() + '\n')
);
button.removeAttribute('disabled');
});
});
Expand Down
4 changes: 2 additions & 2 deletions src/js/background.js
Original file line number Diff line number Diff line change
Expand Up @@ -139,8 +139,8 @@ const µBlock = (( ) => { // jshint ignore:line

// Read-only
systemSettings: {
compiledMagic: 29, // Increase when compiled format changes
selfieMagic: 29, // Increase when selfie format changes
compiledMagic: 30, // Increase when compiled format changes
selfieMagic: 30, // Increase when selfie format changes
},

// https://github.com/uBlockOrigin/uBlock-issues/issues/759#issuecomment-546654501
Expand Down
2 changes: 1 addition & 1 deletion src/js/codemirror/ubo-static-filtering.js
Original file line number Diff line number Diff line change
Expand Up @@ -420,7 +420,7 @@ const initHints = function() {
if ( assignPos !== -1 ) { seedRight = seedRight.slice(0, assignPos); }
const isException = parser.isException();
const hints = [];
for ( let [ text, bits ] of parser.netOptionTokens ) {
for ( let [ text, bits ] of parser.netOptionTokenDescriptors ) {
if ( isNegated && (bits & parser.OPTCanNegate) === 0 ) { continue; }
if ( isException ) {
if ( (bits & parser.OPTBlockOnly) !== 0 ) { continue; }
Expand Down
Loading

6 comments on commit 1e2eb03

@uBlock-user
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we can't use , a filter of this type ||youtube.com^$queryprune=fbclid|gclid,1p will not work ?

@gorhill
Copy link
Owner Author

@gorhill gorhill commented on 1e2eb03 Oct 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do not use literal comma character in the value

In the context, "in the value" means "in the queryprune value".

@uBlock-user
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With ||youtube.com^$queryprune=fbclid|gclid,1p query doesn't get pruned, but with ||youtube.com^$queryprune=fbclid,1p does, however with this one reverse-lookup is failing fyi

@gorhill
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi

I have no idea how to reproduce. I always need repro steps when being told to check something.

@uBlock-user
Copy link
Contributor

@uBlock-user uBlock-user commented on 1e2eb03 Oct 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With ||youtube.com^$queryprune=fbclid|gclid,1p query doesn't get pruned,

Nevermind that, seems query is getting pruned, but the filter appears differently (||youtube.com^$queryprune=fbclid|gclid,1p appears as ||youtube.com^$3p,queryprune=fbclid|gclid in the logger) and reverse-lookup fails --

STR --

  • Add ||youtube.com^$queryprune=fbclid|gclid,1p and open the logger

  • Browse to https://www.youtube.com/watch?v=0K61rP6t0L8&fbclid=uBO&gclid=uBO

  • Switch to the logger, type fbclid in the filtersearch widget and click on the filter to see this --

Capture

@gorhill
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regression from some refactoring, not specific to queryprune, this happens with any requests using 1p, 3p, important. I will pusblish a new release to replace the current one.

Please sign in to comment.