Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

removeparam rule not being enforced #140

Open
dportvine opened this issue Jun 21, 2024 · 7 comments
Open

removeparam rule not being enforced #140

dportvine opened this issue Jun 21, 2024 · 7 comments

Comments

@dportvine
Copy link

Google Chrome
uBlock Origin Lite 2024.6.17.766

Does this rule not work due to MV3 limitations?

*$removeparam=xtor ~> AdGuard URL Tracking Protection

https://www.bbc.com/ukrainian/articles/c7223ygzmvmo?xtor=AL-73-%5Bpartner%5D-%5Bukr%5D-%5Bheadline%5D-%5Bukrainian%5D-%5Bbizdev%5D-%5Bisapi%5D
@gorhill
Copy link
Member

gorhill commented Jun 21, 2024

Weird, it doesn't work in latest release, but it works when I compile a local version of uBOL.


Now it works in the latest release, not sure what is going on.

@dportvine
Copy link
Author

Still doesn't work for me. 🤷🏻‍♂️

@dportvine
Copy link
Author

Very weird... Re-enabling the basic or regional subscription helps me.

Video
Video_2024-06-21_231213.mp4

@garry-ut99
Copy link

garry-ut99 commented Jun 22, 2024

In Firefox, for me it always works when "Ads, trackers, miners, and more" are disabled, and never works when enabled, I've found out that removing this rule from removeparam/default.json fixes the issue for me:

SPOILER 1
{
    "action": {
        "redirect": {
            "transform": {
                "queryTransform": {
                    "removeParams": [
                        "__hsfp",
                        "__hssc",
                        "__hstc",
                        "_gl",
                        "_hsenc",
                        "_kx",
                        "_openstat",
                        "adjust_adgroup",
                        "adjust_campaign",
                        "adjust_creative",
                        "adjust_tracker",
                        "adjust_tracker_limit",
                        "at_campaign",
                        "at_campaign_type",
                        "at_creation",
                        "at_emailtype",
                        "at_link",
                        "at_link_id",
                        "at_link_origin",
                        "at_link_type",
                        "at_medium",
                        "at_ptr_name",
                        "at_recipient_id",
                        "at_recipient_list",
                        "at_send_date",
                        "bbeml",
                        "bsft_aaid",
                        "bsft_clkid",
                        "bsft_eid",
                        "bsft_ek",
                        "bsft_mid",
                        "bsft_uid",
                        "cx_click",
                        "cx_recsOrder",
                        "cx_recsWidget",
                        "dclid",
                        "et_rid",
                        "fb_action_ids",
                        "fb_action_types",
                        "fb_comment_id",
                        "fb_ref",
                        "fb_source",
                        "fbclid",
                        "gbraid",
                        "gclid",
                        "gclsrc",
                        "gps_adid",
                        "guce_referrer",
                        "guce_referrer_sig",
                        "hsCtaTracking",
                        "hsa_acc",
                        "hsa_ad",
                        "hsa_cam",
                        "hsa_grp",
                        "hsa_kw",
                        "hsa_la",
                        "hsa_mt",
                        "hsa_net",
                        "hsa_ol",
                        "hsa_src",
                        "hsa_tgt",
                        "hsa_ver",
                        "ir_adid",
                        "ir_campaignid",
                        "ir_partnerid",
                        "irclickid",
                        "mc_eid",
                        "ml_subscriber",
                        "ml_subscriber_hash",
                        "msclkid",
                        "mtm_campaign",
                        "mtm_cid",
                        "mtm_content",
                        "mtm_group",
                        "mtm_keyword",
                        "mtm_medium",
                        "mtm_placement",
                        "mtm_source",
                        "oft_c",
                        "oft_ck",
                        "oft_d",
                        "oft_id",
                        "oft_ids",
                        "oft_k",
                        "oft_lk",
                        "oft_sk",
                        "oly_anon_id",
                        "pk_campaign",
                        "pk_cid",
                        "pk_medium",
                        "pk_source",
                        "rb_clickid",
                        "s_cid",
                        "ss_email_id",
                        "twclid",
                        "unicorn_click_id",
                        "vero_conv",
                        "vero_id",
                        "vgo_ee",
                        "wbraid",
                        "wickedid",
                        "yclid",
                        "ymclid",
                        "ysclid"
                    ]
                }
            }
        },
        "type": "redirect"
    },
    "condition": {
        "resourceTypes": [
            "main_frame",
            "sub_frame",
            "xmlhttprequest"
        ]
    },
    "id": 4
}

By the way, it looks similar to rule from adguard-spyware-url.json which contains *$removeparam=xtor:

SPOILER 2
{
    "action": {
        "redirect": {
            "transform": {
                "queryTransform": {
                    "removeParams": [
                        "Tcsack",
                        "__hsfp",
                        "__hssc",
                        "__hstc",
                        "__io_lv",
                        "_branch_match_id",
                        "_branch_referrer",
                        "_clde",
                        "_cldee",
                        "_ga",
                        "_gl",
                        "_hsenc",
                        "_hsmi",
                        "_io_session_id",
                        "_kx",
                        "_ope",
                        "_openstat",
                        "_sgm_action",
                        "_sgm_campaign",
                        "_sgm_source",
                        "_zucks_suid",
                        "action_object_map",
                        "action_ref_map",
                        "action_type_map",
                        "adfrom",
                        "adjust_adgroup",
                        "adjust_campaign",
                        "adjust_creative",
                        "adjust_referrer",
                        "adjust_tracker",
                        "adjust_tracker_limit",
                        "admitad_uid",
                        "af_ad",
                        "af_adset",
                        "af_click_lookback",
                        "af_force_deeplink",
                        "af_xp",
                        "aiad_clid",
                        "asgtbndr",
                        "at_campaign",
                        "at_campaign_type",
                        "at_creation",
                        "at_emailtype",
                        "at_link",
                        "at_link_id",
                        "at_link_origin",
                        "at_link_type",
                        "at_medium",
                        "at_ptr_name",
                        "at_recipient_id",
                        "at_recipient_list",
                        "at_send_date",
                        "awc",
                        "bsft_aaid",
                        "bsft_clkid",
                        "bsft_eid",
                        "bsft_ek",
                        "bsft_mid",
                        "bsft_uid",
                        "btag",
                        "cjdata",
                        "cjevent",
                        "cm_cr",
                        "cm_me",
                        "cmpid",
                        "cstrackid",
                        "cuid",
                        "cx_click",
                        "cx_recsOrder",
                        "cx_recsWidget",
                        "dclid",
                        "dpg_campaign",
                        "dpg_content",
                        "dpg_medium",
                        "dpg_source",
                        "ebisAdID",
                        "erid",
                        "external_click_id",
                        "famad_xuid",
                        "fb_action_ids",
                        "fb_action_types",
                        "fb_comment_id",
                        "fb_ref",
                        "fb_source",
                        "fbclid",
                        "gad_source",
                        "gbraid",
                        "gci",
                        "gclid",
                        "gclsrc",
                        "gps_adid",
                        "guccounter",
                        "guce_referrer",
                        "guce_referrer_sig",
                        "hsCtaTracking",
                        "hsa_acc",
                        "hsa_ad",
                        "hsa_cam",
                        "hsa_grp",
                        "hsa_kw",
                        "hsa_la",
                        "hsa_mt",
                        "hsa_net",
                        "hsa_ol",
                        "hsa_src",
                        "hsa_tgt",
                        "hsa_ver",
                        "iclid",
                        "ir_adid",
                        "ir_campaignid",
                        "ir_partnerid",
                        "irclickid",
                        "irgwc",
                        "is_retargeting",
                        "itm_campaign",
                        "itm_content",
                        "itm_medium",
                        "itm_source",
                        "itm_term",
                        "jmtyClId",
                        "maf",
                        "mc_eid",
                        "mindbox-click-id",
                        "mindbox-message-key",
                        "mkt_tok",
                        "ml_subscriber",
                        "ml_subscriber_hash",
                        "msclkid",
                        "mt_link_id",
                        "mtm_campaign",
                        "mtm_cid",
                        "mtm_content",
                        "mtm_group",
                        "mtm_keyword",
                        "mtm_medium",
                        "mtm_placement",
                        "mtm_source",
                        "nx_source",
                        "oly_anon_id",
                        "oly_enc_id",
                        "oprtrack",
                        "pk_campaign",
                        "pk_medium",
                        "pk_source",
                        "pk_vid",
                        "rb_clickid",
                        "rtkcid",
                        "s_cid",
                        "sms_click",
                        "sms_source",
                        "sms_uph",
                        "spot_im_redirect_source",
                        "srsltid",
                        "sscid",
                        "tduid",
                        "ttclid",
                        "twclid",
                        "unicorn_click_id",
                        "user_email_address",
                        "usqp",
                        "utm_ad",
                        "utm_adgroup",
                        "utm_affiliate",
                        "utm_brand",
                        "utm_campaign",
                        "utm_campaignid",
                        "utm_channel",
                        "utm_cid",
                        "utm_content",
                        "utm_creative",
                        "utm_emcid",
                        "utm_emmid",
                        "utm_id",
                        "utm_id_",
                        "utm_keyword",
                        "utm_medium",
                        "utm_name",
                        "utm_place",
                        "utm_product",
                        "utm_pubreferrer",
                        "utm_reader",
                        "utm_referrer",
                        "utm_serial",
                        "utm_servlet",
                        "utm_session",
                        "utm_siteid",
                        "utm_social",
                        "utm_social-type",
                        "utm_source",
                        "utm_source_platform",
                        "utm_supplier",
                        "utm_swu",
                        "utm_term",
                        "utm_umguk",
                        "utm_userid",
                        "utm_viz_id",
                        "vc_lpp",
                        "vero_conv",
                        "vero_id",
                        "vsm_cid",
                        "vsm_pid",
                        "vsm_type",
                        "wbraid",
                        "wickedid",
                        "wt_mc",
                        "xtor",
                        "yclid",
                        "ymid",
                        "ysclid"
                    ]
                }
            }
        },
        "type": "redirect"
    },
    "condition": {
        "resourceTypes": [
            "main_frame",
            "sub_frame",
            "xmlhttprequest"
        ]
    },
    "id": 97
}

(if removing the offending rule still doesn't work, remove all rules from removeparam/default.json and it will fix the issue, since then you can do bisection on your own, to find which one causes the issue, I'm saying it because either I'm tired or I saw some strange random behaviour and I though for a while that it was some other rule which was interferring, in might turn out to be some nasty random issue)

@gorhill gorhill changed the title bbc.com removeparam rule no tbeing enforced Jun 25, 2024
@gorhill gorhill changed the title removeparam rule no tbeing enforced removeparam rule not being enforced Jun 25, 2024
@gorhill
Copy link
Member

gorhill commented Jun 25, 2024

@garry-ut99 Thanks for the investigation, I confirm there is an issue with how the DNR rule works.

Simple repro steps is to take two generic removeparam rules from each default and adguard-spyware-url.json ruleset, and pick a parameter in each rule not found in the other, and see whether both parameters are removed. Example (as of writing):

vgo_ee is found in a generic removeparam rule found in default.json ruleset, xtor is found in a generic removeparam rule in adguard-spyware-url.json. When navigating to URL below, only one of the parameter is removed.

https://example.com/?vgo_ee=1&xtor=1

Probably related webextensions issue: w3c/webextensions#468

@subham8907
Copy link

Hello,

I'm working on an MV3 extension project to remove nasty tracking parameters from URLs. I've encountered an issue with DNR (Declarative Net Request) limitations:

DNR claims:

  • 30,000 static rules work
  • 50 rulesets can work at the same time

But from my research, there's a hidden limit:

  • If you're developing an extension to remove parameters, you should know that if 1 rule contains 30 or 40 parameters, you can't add 30 similar rules
  • There's a limit to that

Real example:

  • uBlock Origin Lite's adguard-spyware-url.json has 499 rules in the JSON file
  • But not all rules are enforced because of rule complexity

Solution:
You have to divide those 499 rules into multiple JSON files if you want to enforce all 499 rules together.

@subham8907
Copy link

So in short before you reached 30000 limit you exaused 50 ruleset limit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants