"Legitimate URL Shortener" General

[DandelionSprout]

This is now the place to share new parameters that you want to see be filtered by "➗ Actually Legitimate URL Shortener Tool" and by "⚙️ Dandelion Sprout's Annoyances List".

There is currently no set time for when new reports are looked into, due to e.g. vacations, other computer work, and hobbies, but it's currently less than 2 weeks.

This also is the place to report issues related to the ClearURLs for uBo list. Note that new params for that list must either be reported upstream to https://github.com/ClearURLs/Rules or added to "Legit URL"

READ BEFORE POSTING

Important notes:

• Legitimate URL Shortener can not bypass tracker redirects
• Legitimate URL Shortener can not remove anything after the # in urls
• Legitimate URL Shortener can only remove query parameters (only what is after the ? in URLs)

For example, for the url https://example.com/content/badtracker/?utm_source=secret&tracker=true#utm_source=othersecret, Legitimate URL Shortener could shorten it down to https://example.com/content/badtracker/#utm_source=othersecret. However, Legitimate URL Shortener can not clean https://example.com/content/ref=somewhere because /ref=somewhere is not a query parameter. For more information, read the documentation on removeparam.

• Please make sure that Legit URL is installed and up-to-date, and your content/ad blocker is enabled on the affected website before reporting an issue
• ClearURLs for uBo is known to have broad allowlisting rules which can completely nullify Legitimate URL Shortener (i.e. preventing removal of ?ref=thing from Amazon). Please test with it disabled before reporting unblocked parameters.
• Parameters added without reloading the page (such as on Bilibili or Netflix) can not be removed as uBlock Origin/AdGuard never sees them added & thus can not remove them. At best, we can prevent the page from using this trick, but in many cases that is not possible without causing breakage.
  To quote the uBo documentation (added emphasis):

To remove query parameters from the URL of network requests [...]
Cosmetically added params cannot be removed via removeparam

This method does not result in a new network request.

Additional rules:

• Added 11th of July 2024: If a hidden, secret, private or offline source for entries is hinted to, talked about or linked to, you agree to that we will have dirt on you. Publicly available sources are not affected by this.

[DandelionSprout]

@wout-codes I'm moving the discussion and future parameter comments to here, to make it more clear that there's no need to panic if I don't immediately reply to your comments.

As for 88d1cd0#commitcomment-47741008, there is the theorethical possibility that you could submit entries to AdGuard Annoyances Filter or even uBlock Filters. The AdGuard team seemed in my eyes to imply back in November that they'd include "Legitimate URL Shortener" in AdGuard, but I never heard back from them about it after that.

[wout-codes]

@DandelionSprout
Link: https://www.bing.com/search?q=hello&form=QBLH&sp=-1&pq=hello&sc=8-5&qs=n&sk=&cvid=49B83A335B1C4884B71B2FAD4A8027A9

Parameter cvid is not needed.

[SeriousHoax]

@yokoffing It's happening for me for all Bing searches. Here I am attaching a GIF of what happens on my system.

GIF:

BTW, the extension "ClearURLs" also has this issue. Check this:
https://gitlab.com/KevinRoebert/ClearUrls/-/issues/391

[yokoffing]

I see what you're saying now. Thanks for providing the GIF.

I am reproducing this issue: Search suggestions on bing.com do not appear.

[SeriousHoax]

It should be fixed hopefully because search suggestion is one of the fundamental features of a search engine IMO.

[DandelionSprout]

I have now removed that entry. Thanks for the tip.

[yokoffing]

I'd like to keep an eye on this. ClearURLs seem to be leaning to towards creating a static value for cvid:

At the moment I'm still thinking if it's better to use static values or randomized values. Static values have the advantage that the anonymity group grows with every ClearURLs user, similar to TOR. However, static values at least allow the conclusion that the user is a ClearURLs user, this could be used to exclude all ClearURLs users from certain pages.

Randomized values have the disadvantage that they are possibly unique and can therefore be used for tracking, so there would be nothing to be gained.

So I think I will implement static values and design workarounds for possible problems.

[kowith337]

Multi-layers wrapped Facebook external URL

• c[0]
• h

EV from kowith337/PersonalFilterListCollection#19 (comment) : image

Google

• rlz in chrome
• pcampaignid (Play store: Track app campaign promote)

Instagram

• igshid

Steam

• curator_clanid (Click game banners in steam curators)
• ser (Found in hyperlinks inside e-mail newsletters)
• snr (Everywhre, even in AJAX/XHR)

[DandelionSprout]

This is now covered by the list. Thanks for the tip.

[karosserie]

This doesn't work:
$removeparam=curator_clanid,domain=~store.steampowered.*

Example:
https://store.steampowered.com/app/660160/Field_of_Glory_II/?curator_clanid=4777282

Any reason why we're not applying this filter on Steam?

[Nojuuu]

AdguardTeam/AdguardFilters#128512

[iam-py-test]

$removeparam=curator_clanid,domain=~store.steampowered.*

This removes curator_clanid from everywhere but store.steampowered.com. You need:

$removeparam=curator_clanid,domain=store.steampowered.*

Or drop the domain restriction and just remove it from everywhere (but will break both steam and other websites):

$removeparam=curator_clanid

Though, as @Nojuuu shared, this will cause problems
Edit: I have removed that rule in d3fd48c

[karosserie]

The only breakage I can see is the removal of the curator review, but you must open the game page from the curator page(curator_clanid) to see that review to begin with.

Understandable why it might not be wanted in the list.

[wout-codes]

@DandelionSprout
Link: https://www.bing.com/search?q=hello&sp=-1&pq=hello&sk=&sc=8-5&qs=n
sp not needed.

Link: https://safe-offers.net/bitcoinrevolution/lp.php?intgrtn_clickID=pA1gWYeKrzdZb0jxPLDRJ8EnyZLkNam95yv4q3VkoMl6Q2OnG&intgrtn_custom1=wrkuk28bb4ug07u5ihuv5aak&country=ZA&intgrtn_redirectReturningLead=auto&intgrtn_language=gb

Few tracking parameters.

[DandelionSprout]

I've now added your Bing fix.

safe-offers.net seems to be a Bitcoin scam site, and will instead be added to Dandelion Sprout's Anti-Malware List.

[wout-codes]

Yes.

[wout-codes]

@DandelionSprout
Link: https://www.bing.com/search?q=hello&pq=hello&sc=8-5&sk=&qs=n

Add: sc, sk, qs.

[wout-codes]

@DandelionSprout
Link: `https://www.bing.com/search?PC=U531&q=hello&FORM=ANNTA1
If searched from Microsoft Edge Browser, there are still some parameters.

PC, FORM not needed.

[DandelionSprout]

This is now covered by the list. Thanks for the tip.

[yokoffing]

Gorhill of uBlock Origin recommended this filter list to me since uBlock Origin supports $removeparam.

Are the various utm parameters purposefully left out? I didn't see $removeparam=/^utm_/ to strip utm, utm_source, etc.

In the meantime, I've added *$doc,frame,xhr,removeparam=/^utm_/ to my personal filter list.

[DandelionSprout]

I've used /[a-z]tm_.*/ instead of /^utm_/, believing the former to work better. I am not personally aware that an initial ^ is needed, whereas I feel the .* as an infinite-length wildcard is. Additionally, it also covers itm_ and stm_.

[yokoffing]

Ahhh. Excuse my ignorance on the subject matter. I wasn't aware that /[a-z]tm_.*/ encompasses utm, much less itm_ and stm_.

utm is a pretty well-known tracking parameter, so I started to get the feeling that I had some misunderstanding on the matter. (Surprise! lol)

Thanks for clarifying, Dandelion! Excellent work as usual.

[squeak9]

https://www.tradera.com/item/111111111111/1111111111/fakelisting?transactionalEmail=wishlist-reminder&utm_source=wishlist-reminder&utm_medium=email&utm_campaign=transactional_wishlist-reminder&utm_nooverride=1
It removes most of it but not all of it.

I am also new to this list, I used to use this list https://github.com/kowith337/PersonalFilterListCollection/blob/master/filterlist/other/SurvivedTrackingLinkWarning.txt
But your list should cover all this stuff right? It will remove and not warn obviously but maybe there is some thing there you haven't seen?

[DandelionSprout]

That list seems to attempt to block the links with the trackers entirely, instead of removing the trackers; but I do see considerable potential in borrowing from it. I'll take care of tradera.com as well.

[gorhill]

I've used /[a-z]tm_.*/ instead of /^utm_/, believing the former to work better.

/[a-z]tm_.*/ is untokenizable, and as a result the content blocker will have to scan every URLs just to find out whether there are utm_... parameters to remove. Additionally, the trailing .* accomplish nothing useful, beside asking to regex engine to match more stuff. There is always between 0 and infinite number of characters after any match, so it's a pointless pattern match, and pointless pattern matches are best avoided.

/^utm_/is tokenizable, and hence only URLs with instances of utm_ in them will be processed. The performance benefits of tokenizable filters is such that it's worth to have have two separate filters for /^itm_/ and /^stm_/.

[DandelionSprout]

Took me some hours to think about what "tokenizable" could mean in this context, but I do understand it now.

My use of .* was because I had no knowledge at all that RegEx = no start-end boundaries by default; in sharp contrast to non-RegEx, where $removeparam=utm_ would not block utm_source that I'm aware of.

Although it is tempting for me to meet at the halfway point with ?*tm_$removeparam=/^[a-z]tm_/, I'll implement your suggestion instead for now, provided I don't stumble across additional ones like (hypothetically) ptm_.

[gwarser]

I have few notes for my own filters in "My filters", they may not bee 100% accurate (I don't have much time to investigate), but may be helpful for you:

$removeparam=ref,domain=~github.com|~behind-the-scene #caused issue on github
! ref_$removeparam=/^ref_/ #issues on one of the many rarbg mirrors
! $removeparam=cid #rarbg
! origin=$removeparam=/^origin=(?=.*\W)/ # allegro.pl notification counter was broken - using ^origin=(?=.*\W)/ fixed this, still - gmail.com contacts list do not load, allegro uses words/tokens, gmail uses addresses
! $removeparam=usegapi #gmail contacts stop working

[DandelionSprout]

Of those five, only ref affected me directly and had to be fixed, which I've done now.

[squeak9]

Spotify links could be shorter, example:

https://open.spotify.com/playlist/37i9dQZF1DWXRvPx3nttRN?si=asdf1234567890&utm_source=copy-link

[yokoffing]

Are there some places that extensions can't touch? For instance, when I'm on Reddit and click Share > Copy Link to share a post, there's always a utm attached to it. Example: https://www.reddit.com/r/NatureIsFuckingLit/comments/m8stpe/picture_taken_by_my_mom_we_pic_mushrooms_every/?utm_source=share&utm_medium=web2x&context=3

No matter if I run ClearURLs, Neat URLS, or Dandelion's filter list in uBlock Origin, clicking Share > Copy Link always gives me a link with utm_* attached. Problematic if I'm just copy+pasting a link to share with a friends.

Now if I take that same link and copy+paste it in my URL bar and press enter while using Dandelion's list, I get: https://www.reddit.com/r/NatureIsFuckingLit/comments/m8stpe/picture_taken_by_my_mom_we_pic_mushrooms_every/?context=3 and Dandelion's list in uBlock Origin removes the tracking parameter.

But no solution I've found removes the tracking parameter upon copy, and looks like what you're describing except with Spotify instead of Reddit.

[gorhill]

uBO will remove query parameters only from outgoing network requests, it does not modify the DOM to rewrite the href attribute of all a tags in the DOM. The removeparam option applies to network filters, it's not a DOM-based utility.

[yokoffing]

wsj.com urls can be made shorter by removing ?mod=
Original: https://www.wsj.com/articles/the-christian-baker-who-said-no-11616452331?mod=trending_now_opn_pos1
Proposed: https://www.wsj.com/articles/the-christian-baker-who-said-no-11616452331

nyt.com urls appeared fine but there was one that would not shorten:
Original: https://www.nytimes.com/2021/03/22/parenting/covid-children-isolation-mental-health.html?surface=home-discovery-vi-prg&fellback=false&req_id=137890943&algo=identity&variant=no-exp&imp_id=960654917
Proposed: https://www.nytimes.com/2021/03/22/parenting/covid-children-isolation-mental-health.html

[liamengland1]

Some might not want this, for example, links to WSJ on techmeme have the param ?mod=djemalertNEWS which bypasses the paywall.

[DandelionSprout]

Both of these points are now taken care of by the list. Thanks for the tip.

[yokoffing]

@llacb47 whoops. I use Bypass Paywalls Clean. So we should just leave this alone then, @DandelionSprout?

[DandelionSprout]

I've made it so that $removeparam=mod only applies if the mod value starts with trending.

[yokoffing]

TCGPlayer: everything after ?xid= can be removed.

Example 1 (includes my original search query as indicated by &q= )
Original: https://shop.tcgplayer.com/yugioh/legendary-duelists-sisters-of-the-rose/harpie-lady-elegance?xid=pi59304a70-e073-4a6a-96f1-4ae63bc02dea&q=harpie
Proposed: https://shop.tcgplayer.com/yugioh/legendary-duelists-sisters-of-the-rose/harpie-lady-elegance

Example 2
Original: https://shop.tcgplayer.com/yugioh/legendary-duelists-sisters-of-the-rose/lunalight-blue-cat?xid=piac3287f7-0eee-4e0a-a8c1-bf49f478e4dc&productLineName=yugioh&setName=legendary-duelists-sisters-of-the-rose
Proposed: https://shop.tcgplayer.com/yugioh/legendary-duelists-sisters-of-the-rose/lunalight-blue-cat

[DandelionSprout]

On most sites, blocking q would be a bad idea, because most stores' search criteria determines the placeholder text in the search bar on store pages (e.g. on Amazon). However, that does not seem to be the case on tcgplayer, making it superfluous and indeed removable.

This is now covered by the list. Thanks for the tip.

[yokoffing]

@DandelionSprout, we need a fix for TCGplayer. I cannot fetch search results from https://www.tcgplayer.com/search/all/product.

With the URL Shortener list disabled: The appropriate results show.

With the URL Shortener list enabled: The products from https://www.tcgplayer.com/search/all/product refresh on the page and it doesn't find search results.

Can we still remove ?xid but retain &q somehow?

[DandelionSprout]

Thanks for the headsup. This has been fixed in version 15June2021v2-Beta.

[yokoffing]

Would it be possible for us to handle redirect tracking?

Specifically:

• YouTube redirect links
• Google link tracking
• Bing search result redirects (when "Open links in new tabs" is active):
  • Original: https://www.bing.com/newtabredir?url=https%3A%2F%2Fwww.cnn.com%2Fstyle%2Farticle%2Flil-nas-x-mschf-satan-nike-shoes%2Findex.html
  • Proposed: https://www.cnn.com/style/article/lil-nas-x-mschf-satan-nike-shoes/index.html

[DandelionSprout]

Sadly not. However, https://github.com/DandelionSprout/adfilt/tree/master/Dandelion%20Sprout-s%20Redirector%20Assistant%20List offers it on a very small scale.

[yokoffing]

utm_source, utm_medium, and utm_content are not being removed from the Firefox addons page.

https://addons.mozilla.org/en-US/firefox/addon/to-google-translate/?utm_source=addons.mozilla.org&utm_medium=referral&utm_content=search

[SeriousHoax]

Did you make the change that is required for extensions to work on Firefox addons page? By default, extensions can't do anything on the addons page as well as some other Firefox related sites.
On my system the above link you posted is being cleared by ClearURLs.

[yokoffing]

Did you make the change that is required for extensions to work on Firefox addons page?

Yes.

[liamengland1]

uBO does not have the permission to run on browser protected pages afaik. So it will not work.

[kowith337]

It's may only work when you...

• set pref (and maybe you should lockPref(); it?)
• open in new tab instead of click straight.
• refresh page once after clicked.

[rusty-snake]

uBlockOrigin/uBlock-issues#1704 / uBlockOrigin/uBlock-issues#1767

[rd-su]

When I use this list, it blocks a lot of images in Facebook.

Every post of this link have it's image blocked:
https://www.facebook.com/permalink.php?story_fbid=3578168542306148&id=321420801314288

With ClearURLs works fine...

[DandelionSprout]

This has now been fixed by the list. Thanks for the tip.

[yokoffing]

Academia.edu | ?email_work_card

Original: https://www.academia.edu/9513828/Open_Theism_and_the_Millennial_in_the_Pew_Evangelical_Theology_and_Marketing_in_the_Age_of_the_World_Wide_Web?email_work_card=view-paper

Proposed: https://www.academia.edu/9513828/Open_Theism_and_the_Millennial_in_the_Pew_Evangelical_Theology_and_Marketing_in_the_Age_of_the_World_Wide_Web

[mixon606]

! https://www.roblox.com/games/15992589987/Shrek-Swamp-Tycoon?gameSetTypeId=100000000&homePageSessionInfo=c5781995-9d83-4b60-8128-de4a892bbeb6&isAd=false&numberOfLoadedTiles=8&page=homePage&placeId=15992589987&position=4&sortPos=2&universeId=5529067985
||www.roblox.com^$removeparam=gameSetTypeId
||www.roblox.com^$removeparam=homePageSessionInfo
||www.roblox.com^$removeparam=isAd
||www.roblox.com^$removeparam=nativeAdData
||www.roblox.com^$removeparam=numberOfLoadedTiles
||www.roblox.com^$removeparam=page
||www.roblox.com^$removeparam=placeId
||www.roblox.com^$removeparam=position
||www.roblox.com^$removeparam=sortPos
||www.roblox.com^$removeparam=universeId
! https://www.roblox.com/games/920587237/Adopt-Me?discoverPageSessionInfo=5d7be81e-56e3-40ee-bdc6-c602bd8664a2&gameSetTargetId=199&gameSetTypeId=23&isAd=false&nativeAdData=&numberOfLoadedTiles=50&page=gamesPage&placeId=920587237&position=0&sortPos=0&universeId=383310974
||www.roblox.com^$removeparam=discoverPageSessionInfo
||www.roblox.com^$removeparam=gameSetTargetId
! https://www.roblox.com/games/refer?PlaceId=1136851104&PageType=GroupDetail&LocalTimestamp=%7BlocalTimestamp%7D
||www.roblox.com^$removeparam=PlaceId
||www.roblox.com^$removeparam=PageType
||www.roblox.com^$removeparam=LocalTimestamp
! https://www.roblox.com/games/refer?PlaceId=189707&Position=0&PageType=ItemDetail&RelatedAssetType=Badge&RelatedAssetID=66918611
||www.roblox.com^$removeparam=Position
||www.roblox.com^$removeparam=RelatedAssetType
||www.roblox.com^$removeparam=RelatedAssetID
! https://www.roblox.com/games/189707/Natural-Disaster-Survival?gameSearchSessionInfo=a6c93764-445e-42a2-a83e-621b241f4c98&isAd=false&nativeAdData=&numberOfLoadedTiles=80&page=searchPage&placeId=189707&position=0&universeId=65241
||www.roblox.com^$removeparam=gameSearchSessionInfo
! https://www.roblox.com/discover#/sortName/Top_Trending_V2?discoverPageSessionInfo=7b1db972-f625-45a6-acdb-2aaf503c078c&gameSetTargetId=199&gameSetTypeId=23&page=gamesPage&position=0&treatmentType=Carousel
||www.roblox.com^$removeparam=treatmentType
! fixes store tab in games
@@||www.roblox.com/*/getgamepassesinnerpartial^$removeparam=placeId
! fixes game links in groups, badges, gamepasses and maybe some other places
@@||www.roblox.com/*/refer^$removeparam=PlaceId

[iam-py-test]

Sorry for the delay.
Would you mind sharing how you found these, and why you chose to remove these parameters.
Thanks

[mixon606]

From gameSetTypeId to universeId can be found in links to games in https://www.roblox.com/discover#/, game categories (https://www.roblox.com/discover#/sortName/MostEngaging), search (https://www.roblox.com/discover/?Keyword=natural%20disaster%20survival) and https://www.roblox.com/home

discoverPageSessionInfo and gameSetTargetId can be found in links to games in https://www.roblox.com/discover#/

PlaceId, PageType and LocalTimestamp can be found in links to games in groups, badges, and gamepasses. examples: https://www.roblox.com/groups/8505490/Bloxburg-Development#!/about, https://www.roblox.com/game-pass/1057151/Excellent-Employee, https://www.roblox.com/badges/66918518/Survived-a-Disaster

gameSearchSessionInfo can be found in links to games in search. example: https://www.roblox.com/discover/?Keyword=natural%20disaster%20survival

treatmentType can be found in links to game categories found in https://www.roblox.com/discover#/

Removing them didn't seem to break anything other than than placeId and PlaceId which I provided fixes for (although PlaceId could be removed completely since its only ever used in the links that need that fix anyway)

[iam-py-test]

Sorry for the delays.
I am concerned some of these parameters, such as page, could be used elsewhere due to their generic names. Moreover, I have not added the placeid parameters.
That said, I have added some which appear unlikely to cause issues.
Thank you for reporting this. I have fixed it in a7e17ed.

[hurgir]

original link 1:
https://www.temu.com/goods.html?_bg_fs=1&goods_id=601099515313495&_x_sessn_id=uhnu2qvuvo&refer_page_name=bgt_order_receipt&refer_page_id=10471_1718277961666_tdvyd6mvrs

unnecessary parameters 1:
_bg_fs
_x_sessn_id
refer_page_name
refer_page_id

original link 2:
https://www.temu.com/goods.html?goods_id=601099514889442&sku_id=17592205444096&refer_share_id=okaxyflouvlz7jbeaxvmgj8zrej9117t&refer_share_channel=copy_link&_oak_page_source=417&_oak_region=203&refer_share_suin=RSSBA4XLGIME5JTNPRQGA3WFXGHT5GU6AMGYXG7DHM3GFJTAAMKEG6ISXWATDUKPI2DL55EF5I

unnecessary parameters 2:
sku_id
refer_share_id
refer_share_channel
_oak_page_source
_oak_region
refer_share_suin

original link 3:
https://www.temu.com//bronze-ay-y%C4%B1ld%C4%B1z-kuvars-cep-saati-vintage-moda-kolye-zinciri-saat-eid-al-adha-ramadan-kareem-kad%C4%B1nlar-ve-erkekler-i%C3%A7in-hediyeler-g-601099549677058.html?_oak_mp_inf=EIKsiKmm1ogBGhZnb29kc181M2NyOWNfcmVjb21tZW5kIPSUk4uBMg%3D%3D&top_gallery_url=https%3A%2F%2Fimg.kwcdn.com%2Fproduct%2Fopen%2F2024-03-04%2F1709566868743-1316b2d12adc4bbfbf57d61359ba7969-goods.jpeg&spec_gallery_id=4106700595&refer_source=10016&freesia_scene=11&_oak_freesia_scene=11&_oak_rec_ext_1=MTAwNDg&_oak_gallery_order=690148383%2C961207598%2C859358083%2C779219786%2C40271338&_x_channel_scene=spike&_x_channel_src=1

unnecessary parameters 3:
_oak_mp_inf
top_gallery_url
spec_gallery_id
refer_source
freesia_scene
_oak_freesia_scene
_oak_rec_ext_1
_oak_gallery_order
_x_channel_scene
_x_channel_src

original link 4:
https://www.temu.com/goods.html?share_img=https%3A%2F%2Fimg.kwcdn.com%2Fproduct%2Ffancy%2Fec18f382-a617-492f-bdcb-a51277fd8d35.jpg&from_share=1&goods_id=601099581551934

unnecessary parameters 4:
share_img
from_share

[iam-py-test]

How did you discover these parameters?
Have you tested these when logged into an account?
Thanks

[hurgir]

@iam-py-test

How did you discover these parameters? Have you tested these when logged into an account? Thanks
These links are to a forum where users share deals and shopping."https://forum.donanimhaber.com/temu-indirim-firsat-ve-kampanyalar-ana-konu--158899451-3#158899857"
Yes I logged in and experimented and added blocking rules to my personal lists and used them for about 3 days and found that removing the parameters I listed did not cause any problems.

[yokoffing]

rule

$removeparam=entrypoint,domain=bing.com

original

https://www.bing.com/shop?FORM=Z9FD1&entrypoint=msn

revised

https://www.bing.com/shop?FORM=Z9FD1

IIRC, we leave the FORM because it caused unintended behavior, with rewards or being logged in. Something like that lol.

[iam-py-test]

Thank you for reporting this. I have fixed it in c7edb72.

IIRC, we leave the FORM because it caused unintended behavior, with rewards or being logged in. Something like that lol.

Agreed.

[partingscientist]

Original:

https://www.nytimes.com/athletic/1471318/2019/12/28/suarez-higuain-ba-sahin-dudek-the-inside-story-of-the-arsenal-signings-that-never-happened/?source=user_shared_articleSuarez,Higuain,Ba,Sahin,Dudek%E2%80%A6theinsidestoryoftheArsenalsigningsthatneverhappened

Cleansed:

https://www.nytimes.com/athletic/1471318/2019/12/28/suarez-higuain-ba-sahin-dudek-the-inside-story-of-the-arsenal-signings-that-never-happened

Also, the domain theathletic.com now redirects to nytimes.com/athletic; every entry for theathletic.com should also have nytimes.com be added to them.

[iam-py-test]

Thank you for reporting this. I have fixed it in 873db1b.

Also, the domain theathletic.com now redirects to nytimes.com/athletic; every entry for theathletic.com should also have nytimes.com be added to them.

Good idea; done.

[stdedos]

Hello there!

I am facing a weird issue:

I have the following rules:

$removeparam=/^trk_.+/
$removeparam=trk
$removeparam=/^trk($|_.+)/

However, it seems that https://github.com/fluxcd/flux?trk=article-ssr-frontend-pulse_little-text-block (re)loads just fine (i.e., without removing that trk)

... which seems to come from

adfilt/ClearURLs for uBo/clear_urls_uboified.txt

Line 745 in 3b5b32d

@@||github.com^$removeparam

Which seems to come from https://github.com/ClearURLs/Rules/blob/0d1c6e00d2b91df1d70c84067c16c320e7976a3f/data.min.json#L329.

Would you be ... somehow considering fixing the generation script? Or is there somehow a way for "My Filters" to get precedence?
(... without having to consult to !important-equivalent definitions?)

[iam-py-test]

If the intent of the ClearURLs rule was to completely allowlist GitHub, than it was for a reason, and removing that rule will cause issues.
The ClearURLs for uBo list is intended to be a translation of the ClearURLs rules, with limited modifications.
That said, if this is not how the original rule intended, the script can and should be corrected.
Looking at the rule you linked to, it appears this is intended behavior, and as such, there is little I can do.

Or is there somehow a way for "My Filters" to get precedence?

You can disable the filter (@@||github.com^$removeparam,badfilter) or use $important (as you mention), but otherwise rules in My Filters are treated with the same level of precedence as rules elsewhere.
Thank you.

[stdedos]

Looking at the rule you linked to, it appears this is intended behavior, and as such, there is little I can do.

Eeeeh ... idk if that's a fair statement.

Of course I (most probably) could be reading the rule wrong, but what I read is:

• For every matching urlPattern,
• block rules,
• block referralMarketing too (?)
• except if exceptions

The way I feel this is applied in uBlock (and again, I might be wrong again) is:

• For every matching urlPattern,
• block rules,
• block referralMarketing too (?)
• ignore every $removeparam for exceptions

which tbh is a bit too much? 😕

Not that I "manipulate" (AFAIGuess) any other of the sites there - but your filter list is effectively blocking using any kind of $removeparam for those ~30 "generified exceptions"

[iam-py-test]

Sorry for being dismissive. Your explanation makes sense for me. My reasoning was erroneous.
This is a valid issue, but I am unsure how to correct it. I will look into it more later this week.
Thank you

[stdedos]

I didn't see any dismissal - don't worry.

Of course, at the end of the day, things are best-effort.
... Worst case, and especially if these exceptions grow too much, maybe I'll just disable the filter and that's it.

I just wanted you to be aware.

[Johnnycyan]

On Ebay an item link looks like this:

https://www.ebay.com/itm/395169634934?itmmeta=01J0JS6F5MF9D9S2CX95RSRWKG&itmprp=enc%3AAQAJAAABIEnIphS0WSrbQBXtwuZ0IDcrSmkxv2vC3bdqzcLYut9AWXLrOZsX1vw3IxHFpbCYjWYAT9%2FwVNLEFqwOau55nCa5QqmZLeQ9x1IbmQHQX54jXH4s%2F0WaQUyDUMu%2Boo44ad4j7q23ZoPax%2FHHcYhkNS5m0wtT7tWVdZPlbxDYZ7pbNvUg%2BLv3cKxUL3RKeWg144DzRMgJGg4qvWr0x52z9vFI%2BC4dCrmjUKV7Wm%2FXP1V3FjW4UbxONZr2hPRGr520hjoEX3T2w%2FveQg545GytTXeyCotuvUSE8JI6ks0m8NO4cc628%2FYEkaG79W9mcKgTDACFIQJJ8fatVcFEEbfWrdUWFoOGEQM5CYRb7gcqI6SuU%2FCbfXbI1hesdd80Pak6Cg%3D%3D%7Ctkp%3ABFBM9PKZ2YRk

Which is absurd and you only need

https://www.ebay.com/itm/395169634934

So the params itmmeta and itmprp are not needed

||ebay.*/itm/*$removeparam=itmmeta
||ebay.*/itm/*$removeparam=itmprp

[DandelionSprout]

I presume that since parameters that start with itm_ are very common across the internet, that eBay felt it'd make sense on their part to simply remove the underscore.

As such, I think I know how to fix it, but it could take me some hours.

[alfredonodo]

Hi,
I'm not sure if this is the best place to ask this. Anyway, I have a problem with brave (any version: stable, beta and nightly) with ClearURLs for uBo unofficial. If I import these filtering rules on brave, I can't open reddit as it continuously reloads the page while everything works fine with firefox (any version) and ublock origin. What could be the problem?
Thanks

[stephenhawk8054]

It's caused by ||reddit.com^$removeparam=rdt. You can test disabling it by

!#if env_chromium
||reddit.com^$removeparam=rdt,badfilter
!#endif

[iam-py-test]

Thanks @stephenhawk8054
@alfredonodo Thank you for reporting this. I have fixed it in bb05df3.

[partingscientist]

Original:

https://sport.sky.it/calciomercato/2024/07/04/calafiori-arsenal-news-calciomercato?social=link_skysport_sharebutton_null&share_id=dc6d1c70-47d0-4779-a458-41bb2a932295

Cleansed:

https://sport.sky.it/calciomercato/2024/07/04/calafiori-arsenal-news-calciomercato

[iam-py-test]

Thank you for reporting this. I have fixed it in 8d96254.

[partingscientist]

The site producthunt.com adds a ref parameter in the form of ?ref=producthunt for every outbound links on their site.

[iam-py-test]

Thank you for reporting this. I have fixed it in da9d281.

[Yuki2718]

domain=producthunt.com makes the rule useless. But why don't provide an example? You can find here https://www.producthunt.com/products/adobe-xd-cc-13-0 (right-side Links)

[iam-py-test]

@Yuki2718 thank you for pointing that out. Fixed in 7f7934e. I'm not sure why I did that...

[szczykafka]

I noticed that "sca_upv" parameter still appears in Google search.
In the LegitimateURLShortener.txt file (Version: 05August2024v2, line 2155) there is ||google.$doc,removeparam=sca_esv
Maybe could be just changed to ||google.$doc,removeparam=/^sca_/

[iam-py-test]

Thank you for reporting this. I have fixed it in a2b29b7.
Unless there are other parameters on Google starting with sca_, I do not think a RegEx is needed, and as such kept it as two separate filters.

[szczykafka]

Version: 05August2024v2
Regarding: ||audible.

Right now I'm fixing manually for parameters:
creativeId=
pageLoadId=
ref_pageloadid=
pr_rd=

My manual fix	Possible corresponding line number	Current line command
||audible.*$removeparam=/^pf_rd/	2574	$removeparam=/^pf_rd_.*$/,domain=
||audible.*$removeparam=/^ref/	1725	$removeparam=ref,doc,domain=
||audible.*$removeparam=/^pageLoadId/	null	null
||audible.*$removeparam=/^creativeId/	null	null

[iam-py-test]

Would you mind sharing a URL/URLs containing these parameters, and preferably where they can be found (i.e. in Audible emails)?
Thank you

[szczykafka]

Audible.com directly on the website. Accessed on a MacBook.

Example urls below. I started on main page and each link is the next click.

audible.com > logo click
https://www.audible.com/?ref_pageloadid=not_applicable&ref=a_hp_t1_nav_header_logo&pf_rd_p=fab27d5b-e940-433e-9160-ab31d0fd0269&pf_rd_r=VYABW9NQ7RFWR2C2CYRX&pageLoadId=3Hi7bKuF39aeiVHs&creativeId=a5ac87d6-d218-4a97-8f3c-e1a3d7b98977

… > Wish List click
https://www.audible.com/library/wishlist?ref_pageloadid=3Hi7bKuF39aeiVHs&ref=a_hp_t1_navTop_pl1&pf_rd_p=fab27d5b-e940-433e-9160-ab31d0fd0269&pf_rd_r=W10QJV26WCTZ464B4B7K&pageLoadId=L9IeVgu1LejVxzQd&creativeId=a5ac87d6-d218-4a97-8f3c-e1a3d7b98977

… > Library click
https://www.audible.com/library/wishlist?ref_pageloadid=3Hi7bKuF39aeiVHs&ref=a_hp_t1_navTop_pl1&pf_rd_p=fab27d5b-e940-433e-9160-ab31d0fd0269&pf_rd_r=W10QJV26WCTZ464B4B7K&pageLoadId=L9IeVgu1LejVxzQd&creativeId=a5ac87d6-d218-4a97-8f3c-e1a3d7b98977

… > change to "Show 50 items per page"
https://www.audible.com/library/titles?pageSize=50&ref_pageloadid=QsPvn5RaHawdcVE1&ref=a_library_t_c6_pageSize_3&pf_rd_p=497a5d2f-f652-4240-98c1-536a807b1396&pf_rd_r=1YH0H7FN2J41ZFAWFZFX&pageLoadId=nxW8hpkCpOCUL3bR&creativeId=669567e8-e606-4382-9cad-a455ff362580

… > Page 2 click
https://www.audible.com/library/titles?pageSize=50&page=2&ref_pageloadid=nxW8hpkCpOCUL3bR&ref=a_library_t_c6_pageNum_1&pf_rd_p=497a5d2f-f652-4240-98c1-536a807b1396&pf_rd_r=KHFP9QQAW2PWNMXTW58P&pageLoadId=Sxx7FGX9u7zUHUch&creativeId=669567e8-e606-4382-9cad-a455ff362580

These solved everything for me and no problems:
||audible.*$removeparam=/^creativeId/
||audible.*$removeparam=/^pageLoadId/
||audible.*$removeparam=/^ref/
||audible.*$removeparam=/^pf_rd/

I know $removeparam=/^pf_rd_.*$/,domain= works in the example urls I provided. I can't find where it was where your file with the filters wasn't affecting it.

[iam-py-test]

The existing $removeparam=/^pf_rd_.*$/,domain= (GitHub tries to render this as an equation) works fine for me. Are you using uBlock Origin, AdGuard, or something else?

||audible.*$removeparam=/^creativeId/

It is not necessary to use ||audible.*$ - ||audible.$ works perfectly well (I think it has exactly the same effect?).
Thank you for reporting this. I have fixed it in 3e4b6e6.

[szczykafka]

uBlock Origin
I think I encountered something slightly else. Disregard. I'll report it back if I encounter it.
Thanks for fixing all this.

[szczykafka]

&fbs=
&dpr=
&ictx=
/webhp?

Example URLs:

Google search > 1 year filter > Videos
https://www.google.com/search?tbs=qdr:y&q=fuck+google&tbm=vid&fbs=AEQNm0Aa4sjWe7Rqy32pFwRj0UkWd8nbOJfsBGGB5IQQO6L3J_86uWOeqwdnV0yaSF-x2jrJh7Dt5wV71ckxEPe_0GQyc61_Jkg5ZI9z4zNW20fWd52E8UO95Fa_VxoakS9zyCiAT-lUw9SqXLpdCTRHyZ2TB397CTLvedOJGxV_D_7JRzn827oY3wzpySThiJ8JihGoKqQR2_EkH-yaVIj8nLqyaR61pA&dpr=2

… > Shopping
https://www.google.com/search?q=fuck+google&udm=28&ictx=111&dpr=2

… > Main logo
https://www.google.com/webhp?ictx=111

New google search > Images
https://www.google.com/search?q=suck+it+google&udm=2&fbs=AEQNm0A2upiO_GHeTz6R89sNEjTHXSUfB8x3gweQ77S5CBNH1n08O_h14F1-2GfxydSM86nAWk96tjJ3dXECldqUyDZS1f8CixSfZE1lHJ6l0ACTlLNiZvX28MHfh_1xdSOVtzVoUW6oFHVPXR9w55FYJlXgRZ-B17AKS-BX_VpKRTN-BxXGHGW0aMhK1tBLi1ET9zLAFtgvchy9B1SmW9eTBy2fq7A6rA&dpr=2

… > Web
https://www.google.com/search?q=suck+it+google&udm=14&fbs=AEQNm0DrkyAsj3-ytZzdGYQ--Lq3z9oZz8V7fxIGhCSQTvtb1KiYxNikuu3HMqIFp3EmtEwdg4Hb8VG0iUXtv6QhkURWRCDYHIpTU96yqxbP_GkukNCP5NGPPyDHCvzntn0w-NgDZ_O3XdgXbA-g8_2zlBGMXTBdbUD9KLry3D8PXpAoIGsINxYG3j0Ic8ov7T6-cbr2a1DmkSiv2Rk9ADNmJw3VeJuuLA&dpr=2&tbs=qdr%3Aw

[iam-py-test]

/webhp?

I am sorry, but the Actually Legitimate URL Shortener can not remove that.
As for the others, I will investigate further tomorrow.
Thank you

[szczykafka]

Amazon reference injection into the URL but within the URL, not as a parameter:

https://www.amazon.com/ROTTOGOON-Dish-Drying-Rack-Kitchenware/dp/B0CNGW1L3T/ref=pd_ci_mcx_pspc_dp_d_2_i_2

[szczykafka]

@iam-py-test

_encoding
https://www.amazon.com/Dish-Drying-Rack-Kitchen-Counter/dp/B0CD3TQHWZ/?_encoding=UTF8
Clicking from the main page.

smid
https://www.amazon.com/gp/product/B06XX61MWN/?smid=A8MA32QEVO3GI
Clicking on item in cart (right screen side).

psc
https://www.amazon.com/dp/B0D4XQVNWD?psc=1&
Clicking on an item from https://www.amazon.com/gp/css/order-history

ccs_id
https://www.amazon.com/books-used-books-textbooks/b/?node=283155&ccs_id=b48db2a6-8e35-4436-a146-643aa590c30d
Clicking on "Books" under the search bar.

[DandelionSprout]

• "(…)Dish-Drying-Rack-Kitchen-Counter/dp/B0CD3TQHWZ/?_encoding=UTF8"

I've traditionally been concerned about removing _encoding=UTF8, as there's not all that many years since Japanese tech setups in particular relied heavily on the "Shift JIS" encoding instead of UTF-8 (Unicode). Such was their reliance that many Japanese homebrew RPG games made well into the late 2010's crash on startup on non-Japanese Windows setups.

So I feared a(n im)plausible scenario where Amazon would inexplicably decide to use Shift JIS or various Latin-1252 encodings, resulting in a whole lot of empty placeholder boxes for text characters on Amazon pages.

Though since the world is now in 2024, it's anyone's guess as to whether Amazon still have such baffling backup encodings in use anymore.

[szczykafka]

I edited my previous comment to consolidate everything but I think after you replied.
Look at smid, psc, and ccs_id...

[szczykafka]

Also, for ccs_is, manual ||amazon.$removeparam=/^ccs_id/ has no effect. And the value changes upon each refresh.

[iam-py-test]

smid breaks product pages: AdguardTeam/AdguardFilters#148958
psc breaks size selection: #163 (reply in thread)
However, I have added ccs_id.
Thank you for reporting this. I have fixed it in d2a1e54.

[partingscientist]

Original:

https://store.steampowered.com/app/2751000/Prince_of_Persia_The_Lost_Crown/?ucid=INF-ID_193783&maltcode=digitalmarketing_C2W_INF_influencersite___MEMO____&addinfo=

Cleansed:

https://store.steampowered.com/app/2751000/Prince_of_Persia_The_Lost_Crown

[iam-py-test]

Thank you for reporting this. I have fixed it in 0aa7c6e.

[szczykafka]

bp=
example: https://www.youtube.com/?bp=wgUCEAE%3D

Clicked from a video on left corner main logo and "bp=wgUCEAE%3D" was added to the url. Some scripts didn't load because they supposed to only on the main page. Clicked the logo again and it was back to normal "…tube.com/".

[szczykafka]

It's because of the periodically changed logo reflecting some day/month/initiavive etc.

[iam-py-test]

Thank you for reporting this. I have fixed it in 84a9505.