How to use the url list cleaner
- Paste your web addresses into the box, one per line. A line without http:// or https:// is given https:// when that box is ticked.
- Check the list of parameter names to remove. The names shown are examples; add your own, separated by commas, and end a name with a star to match everything that starts that way, such as utm_*.
- Tick or clear the choices for sorting the remaining parameters, removing the #fragment, removing a trailing slash and removing www.
- Read the table line by line, then use Copy result to take the unique cleaned addresses, or Download CSV for the full table.
Formula
host = lower case host, default port (80 or 443) dropped
path = path with dot segments resolved, trailing slash removed if chosen
query = pairs whose name does not match your list, sorted by name (equal names keep their order)
cleaned = scheme + host + path + query + fragment (only if you keep it)
Two lines are repeats when their cleaned forms are identical. The first one is kept and the others are marked as a repeat of its line number.
Seven lines become four addresses
The line https://Example.com/shop/?utm_source=news&b=2&a=1&utm_medium=email#top becomes https://example.com/shop?a=1&b=2. The host is lowered, the two utm_ parameters are removed, a and b are sorted, and the fragment and trailing slash go.
In the example list that line and https://example.com/shop?a=1&b=2 are now the same, so the second is marked as a repeat of line 1. In all, 7 lines give 4 unique addresses with 4 parameters removed (utm_source, utm_medium, fbclid and gclid), 2 repeats removed and 1 line that cannot be used because ftp is not a web scheme.
What cleaning does and does not change
Cleaning changes how an address is written, not which page it points to, with one exception: if a site uses a parameter you removed to decide what to show, the cleaned address can lead somewhere different. Only remove names you know are for tracking. The path keeps its capital letters because many servers treat paths as case sensitive.
The tool reads each address with the same rules a browser uses, so spaces and non-ASCII characters in a path are percent-encoded in the output. That is correct, but it means a cleaned address can look different from what you typed even when nothing was removed.
Using the result
A cleaned list is easier to compare, count and de-duplicate in a spreadsheet. To take a single address apart, use the URL Parser. To build links with tracking parameters on purpose, use the UTM Link Builder. To pull addresses out of a block of text first, use the Email and URL Extractor.
Assumptions and limits
- Only http and https addresses are cleaned. Other schemes, such as ftp and mailto, are listed as not usable.
- Parameter names are matched without regard to case, after percent-decoding, and a name that appears several times is removed every time.
- Parameters you keep are written exactly as you typed them, so spacing styles like + and %20 are not changed.
- The sort is by parameter name only, and parameters with the same name keep their original order.
- Nothing is fetched, so the tool cannot tell whether an address works or where a redirect leads, and your list never leaves your browser.
Frequently asked questions
Why does the page add https:// to some of my lines?
A line such as example.org/page has no scheme, and a browser would not know whether to use http or https. With the box ticked the tool assumes https, which is the safer default. Untick it to have such lines listed as not usable instead.
Does removing a trailing slash change the page?
On most sites /docs and /docs/ lead to the same page, but a few treat them as different. If you are unsure, untick the box and compare a few addresses by opening them.
Can I keep some tracking parameters and drop others?
Yes. The list holds exactly the names that are removed, so delete the ones you want to keep, or clear the box to keep all of them.
What does Repeat of 4 mean?
After cleaning, the line has the same address as line 4, which was kept. The repeat is left out of the copied list but still shown in the table.