URL normalization is the process by which URLs are modified and standardized in a consistent manner. The goal of the normalization process is to transform a URL into a normalized or canonical URL so it is possible to determine if two syntactically different URLs may be equivalent.
For our normalization we will use normalizations that preserve semantics. You should normalize a given url using the next rules (only these rules. They are slightly different from RFC).
1. Converting the scheme and host to lower case.
HTTP://www.Example.com/ → http://www.example.com/
- 2. Capitalizing letters in escape sequences. All letters within a percent-encoding triplet (e.g., "%3B") are case-insensitive,...
4. Removing the default port. The default port (port 80 for the “http” scheme) should be removed from a URL.
http://www.example.com:80/bar.html → http://www.example.com/bar.html
5. Removing dot-segments. The segments “..” and “.” can be removed from a URL according to the algorithm
described in RFC 3986 (or a similar algorithm). ".." is a parent directory, "." is the same directory.
http://www.example.com/a/b/../c/./d.html → http://www.example.com/a/c/d.html