Skip to content

Commit

Permalink
Add support for cname type and denyallow option
Browse files Browse the repository at this point in the history
This concerns the static network filtering engine.

Related issue:
- uBlockOrigin/uBlock-issues#943

* * *

New static network filter type: `cname`

By default, network requests which are result of
resolving a canonical name are subject to filtering.
This filtering can be bypassed by creating exception
filters using the `cname` option. For example:

    @@*$cname

The filter above tells the network filtering engine
to except network requests which fulfill all the
following conditions:

- network request is blocked
- network request is that of an unaliased hostname

Filter list authors are discouraged from using
exception filters of `cname` type, unless there no
other practical solution such that maintenance
burden become the greater issue. Of course, such
exception filters should be as narrow as possible,
i.e. apply to specific domain, etc.

* * *

New static network filter option: `denyallow`

The purpose of `denyallow` is bring
default-deny/allow-exceptionally ability into static
network filtering arsenal. Example of usage:

    *$3p,script, \
        denyallow=x.com|y.com \
        domain=a.com|b.com

The above filter tells the network filtering engine that
when the context is `a.com` or `b.com`, block all
3rd-party scripts except those from `x.com` and `y.com`.

Essentially, the new `denyallow` option makes it easier
to implement default-deny/allow-exceptionally in static
filter lists, whereas before this had to be done with
unwieldy regular expressions[1], or through the mix of
broadly blocking filters along with exception filters[2].

[1] https://hg.adblockplus.org/ruadlist/rev/f362910bc9a0

[2] Typically filters which pattern are of the
    form `|http*://`
  • Loading branch information
gorhill committed Mar 15, 2020
1 parent 84d4111 commit c3bc2c7
Show file tree
Hide file tree
Showing 6 changed files with 172 additions and 41 deletions.
4 changes: 2 additions & 2 deletions src/js/background.js
Original file line number Diff line number Diff line change
Expand Up @@ -130,8 +130,8 @@ const µBlock = (( ) => { // jshint ignore:line

// Read-only
systemSettings: {
compiledMagic: 23, // Increase when compiled format changes
selfieMagic: 25, // Increase when selfie format changes
compiledMagic: 26, // Increase when compiled format changes
selfieMagic: 26, // Increase when selfie format changes
},

// https://github.com/uBlockOrigin/uBlock-issues/issues/759#issuecomment-546654501
Expand Down
6 changes: 2 additions & 4 deletions src/js/filtering-context.js
Original file line number Diff line number Diff line change
Expand Up @@ -81,11 +81,9 @@
this.setDocOriginFromURL(details.documentUrl);
} else {
const pageStore = µBlock.pageStoreFromTabId(this.tabId);
const docStore = pageStore && pageStore.frames.get(this.docId);
const docStore = pageStore && pageStore.getFrame(this.docId);
if ( docStore ) {
this.docOrigin = undefined;
this.docHostname = docStore.pageHostname;
this.docDomain = docStore.pageDomain;
this.setDocOriginFromURL(docStore.rawURL);
} else {
this.setDocOrigin(this.tabOrigin);
}
Expand Down
8 changes: 4 additions & 4 deletions src/js/messaging.js
Original file line number Diff line number Diff line change
Expand Up @@ -532,8 +532,8 @@ const retrieveContentScriptParameters = function(senderDetails, request) {
// `generichide` must be evaluated in the frame context.
if ( noCosmeticFiltering === false ) {
const genericHide =
µb.staticNetFilteringEngine.matchStringElementHide(
'generic',
µb.staticNetFilteringEngine.matchStringReverse(
'generichide',
request.url
);
response.noGenericCosmeticFiltering = genericHide === 2;
Expand All @@ -559,8 +559,8 @@ const retrieveContentScriptParameters = function(senderDetails, request) {
// Add support for `specifichide`.
if ( noCosmeticFiltering === false ) {
const specificHide =
µb.staticNetFilteringEngine.matchStringElementHide(
'specific',
µb.staticNetFilteringEngine.matchStringReverse(
'specifichide',
request.url
);
response.noSpecificCosmeticFiltering = specificHide === 2;
Expand Down
77 changes: 64 additions & 13 deletions src/js/pagestore.js
Original file line number Diff line number Diff line change
Expand Up @@ -151,10 +151,7 @@ NetFilteringResultCache.prototype.shelfLife = 15000;

/******************************************************************************/

// Frame stores are used solely to associate a URL with a frame id. The
// name `pageHostname` is used because of historical reasons. A more
// appropriate name is `frameHostname` -- something to do in a future
// refactoring.
// Frame stores are used solely to associate a URL with a frame id.

// To mitigate memory churning
const frameStoreJunkyard = [];
Expand All @@ -166,15 +163,19 @@ const FrameStore = class {
}

init(frameURL) {
const µburi = µb.URI;
this.pageHostname = µburi.hostnameFromURI(frameURL);
this.pageDomain =
µburi.domainFromHostname(this.pageHostname) || this.pageHostname;
this.exceptCname = undefined;
this.rawURL = frameURL;
if ( frameURL !== undefined ) {
this.hostname = vAPI.hostnameFromURI(frameURL);
this.domain =
vAPI.domainFromHostname(this.hostname) || this.hostname;
}
return this;
}

dispose() {
this.pageHostname = this.pageDomain = '';
this.exceptCname = undefined;
this.rawURL = this.hostname = this.domain = '';
if ( frameStoreJunkyard.length < frameStoreJunkyardMax ) {
frameStoreJunkyard.push(this);
}
Expand Down Expand Up @@ -239,7 +240,6 @@ const PageStore = class {
this.rawURL = tabContext.rawURL;
this.hostnameToCountMap = new Map();
this.contentLastModified = 0;
this.frames = new Map();
this.logData = undefined;
this.perLoadBlockedRequestCount = 0;
this.perLoadAllowedRequestCount = 0;
Expand All @@ -250,6 +250,9 @@ const PageStore = class {
this.internalRedirectionCount = 0;
this.extraData.clear();

this.frames = new Map();
this.setFrame(0, tabContext.rawURL);

// The current filtering context is cloned because:
// - We may be called with or without the current context having been
// initialized.
Expand Down Expand Up @@ -303,6 +306,7 @@ const PageStore = class {
// As part of https://github.com/chrisaljoudi/uBlock/issues/405
// URL changed, force a re-evaluation of filtering switch
this.rawURL = tabContext.rawURL;
this.setFrame(0, this.rawURL);
return this;
}

Expand Down Expand Up @@ -541,9 +545,22 @@ const PageStore = class {

// Static filtering has lowest precedence.
if ( result === 0 || result === 3 ) {
result = µb.staticNetFilteringEngine.matchString(fctxt);
if ( result !== 0 && µb.logger.enabled ) {
fctxt.filter = µb.staticNetFilteringEngine.toLogData();
const snfe = µb.staticNetFilteringEngine;
result = snfe.matchString(fctxt);
if ( result !== 0 ) {
if ( µb.logger.enabled ) {
fctxt.filter = snfe.toLogData();
}
// https://github.com/uBlockOrigin/uBlock-issues/issues/943
// Blanket-except blocked aliased canonical hostnames?
if (
result === 1 &&
fctxt.aliasURL !== undefined &&
snfe.isBlockImportant() === false &&
this.shouldExceptCname(fctxt)
) {
return 2;
}
}
}

Expand Down Expand Up @@ -646,6 +663,40 @@ const PageStore = class {
return 1;
}

shouldExceptCname(fctxt) {
let exceptCname;
let frameStore;
if ( fctxt.docId !== undefined ) {
frameStore = this.getFrame(fctxt.docId);
if ( frameStore instanceof Object ) {
exceptCname = frameStore.exceptCname;
}
}
if ( exceptCname === undefined ) {
const result = µb.staticNetFilteringEngine.matchStringReverse(
'cname',
frameStore instanceof Object
? frameStore.rawURL
: fctxt.getDocOrigin()
);
if ( result === 2 ) {
exceptCname = µb.logger.enabled
? µb.staticNetFilteringEngine.toLogData()
: true;
} else {
exceptCname = false;
}
if ( frameStore instanceof Object ) {
frameStore.exceptCname = exceptCname;
}
}
if ( exceptCname === false ) { return false; }
if ( exceptCname instanceof Object ) {
fctxt.setFilter(exceptCname);
}
return true;
}

getBlockedResources(request, response) {
const normalURL = µb.normalizePageURL(this.tabId, request.frameURL);
const resources = request.resources;
Expand Down
8 changes: 4 additions & 4 deletions src/js/reverselookup.js
Original file line number Diff line number Diff line change
Expand Up @@ -172,13 +172,13 @@ const fromCosmeticFilter = async function(details) {
domain: µBlock.URI.domainFromHostname(hostname),
hostname: hostname,
ignoreGeneric:
µBlock.staticNetFilteringEngine.matchStringElementHide(
'generic',
µBlock.staticNetFilteringEngine.matchStringReverse(
'generichide',
details.url
) === 2,
ignoreSpecific:
µBlock.staticNetFilteringEngine.matchStringElementHide(
'specific',
µBlock.staticNetFilteringEngine.matchStringReverse(
'specifichide',
details.url
) === 2,
rawFilter: details.rawFilter
Expand Down
Loading

5 comments on commit c3bc2c7

@uBlock-user
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

 *$3p,script, \
        denyallow=x.com|y.com \
        domain=a.com|b.com

What does \ signify in this format ?

@jspenguin2017
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing , after y.com?

@gwarser
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does \ signify in this format ?

Line continuation: 703c525b01aa

@uBlock-user
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we do it like this -- *$3p,script,denyallow=x.com|y.com,domain=a.com|b.com ?

@gwarser
Copy link
Contributor

@gwarser gwarser commented on c3bc2c7 Mar 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we do it like this -- *$3p,script,denyallow=x.com|y.com,domain=a.com|b.com ?

Nothing is preventing you to do this. It's meant to help formatting very long lines. In this case added as example - "it's possible". Line continuation is common in programing.

Please sign in to comment.