README
¶
🧭 uri
Resilient URL and Hostname Helpers for Go
Resilient helpers for parsing, validating, and inspecting URLs and hostnames: scheme/protocol handling, IANA top-level-domain validation, local-network detection, and SSRF-safe public-IP classification.
Most functions are forgiving by design — they take a raw string, do their best, and return an empty string or false rather than an error when the input is malformed. The Validate* functions are the exception: they return a derp error describing exactly what was wrong.
Map for the Impatient
- URLs — url.go:
ParseURL(a stricterurl.Parsethat requires an http/https scheme and a valid hostname),ValidateURL,IsValidURL. - Hostnames — hostname.go:
Hostname(strip everything down to the host),ValidateHostname(RFC 1035 lengths + IANA TLD check, with IP and local names exempted). - Scheme / protocol — scheme.go, protocol.go:
Scheme(no://) vsProtocol(with://);GuessSchemeForHostname/PrependProtocolpick http for local hosts and https for public ones. - Redirect / link safety — redirect.go:
IsSafeRedirectURLaccepts only same-site relative paths or absolute http(s) URLs, rejectingjavascript:/data:/protocol-relative targets that turn an open redirect into XSS. - Local-network detection — localhost.go, loopback.go:
IsLocalHostname,IsLocalURL,IsLoopback. - Public-IP / SSRF — public.go:
IsPublicIP,NotPublicIP,IsPublicIPAddress. - TLDs — tld.go, init.go:
IsValidTLD,ValidateTLD,RefreshTLDs; the IANA list is embedded from _iana.txt.
See Also
- benpate/derp — the error type returned by the
Validate*functions.
Pull Requests Welcome
I'm trying to make uri the best it can be, and your help is greatly appreciated. If you find a bug or have an idea for a new feature, please open an issue or submit a pull request. We're all in this together! 🧭
Documentation
¶
Overview ¶
Package uri provides helpers for parsing, validating, and inspecting URLs and hostnames, including scheme/protocol handling, local-network detection, and IANA top-level-domain validation.
Index ¶
- Constants
- func GuessProtocolForHostname(hostname string) string
- func GuessSchemeForHostname(hostname string) string
- func Host(value string) string
- func Hostname(value string) string
- func IsLocalHostname(hostname string) bool
- func IsLocalURL(url string) bool
- func IsLoopback(domain string) bool
- func IsPublicIP(ip net.IP) bool
- func IsPublicIPAddress(value string) bool
- func IsSafeRedirectURL(target string) bool
- func IsSameOrigin(first string, second string) bool
- func IsSchemeValid(scheme string) bool
- func IsValidHostname(hostname string) bool
- func IsValidIP4Address(ip string) bool
- func IsValidIP6Address(ip string) bool
- func IsValidIPAddress(ip string) bool
- func IsValidTLD(tld string) bool
- func IsValidURL(uri string) bool
- func NormalizeHost(value string) string
- func NotLocalHostname(hostname string) bool
- func NotLocalURL(url string) bool
- func NotLoopback(domain string) bool
- func NotPublicIP(ip net.IP) bool
- func NotPublicIPAddress(value string) bool
- func NotSameOrigin(first string, second string) bool
- func NotSchemeValid(scheme string) bool
- func NotValidHostname(hostname string) bool
- func NotValidIPAddress(ip string) bool
- func NotValidTLD(tld string) bool
- func NotValidURL(uri string) bool
- func ParseURL(uri string) (*url.URL, error)
- func Path(uri string) string
- func PathAndQuery(uri string) string
- func PrependProtocol(uri string) string
- func Protocol(uri string) string
- func RefreshTLDs()
- func SafeURL(raw string) string
- func Scheme(uri string) string
- func ValidateHostname(hostname string) error
- func ValidateTLD(tld string) error
- func ValidateURL(uri string) error
Constants ¶
const ProtocolHTTP = "http://"
ProtocolHTTP represents the "http://" protocol string (including the :// suffix)
const ProtocolHTTPS = "https://"
ProtocolHTTPS represents the "https://" protocol string (including the :// suffix)
const ProtocolSuffix = "://"
ProtocolSuffix represents the "://" suffix that follows the protocol name
const SchemeHTTP = "http"
SchemeHTTP represents the "http" network scheme (without the :// suffix)
const SchemeHTTPS = "https"
SchemeHTTPS represents the "https" network scheme (without the :// suffix)
Variables ¶
This section is empty.
Functions ¶
func GuessProtocolForHostname ¶
GuessProtocolForHostname returns the correct protocol for a given hostname. Hosts on the public network always use HTTPS, and hosts on the local network always use HTTP.
func GuessSchemeForHostname ¶
GuessSchemeForHostname returns the correct scheme for a given hostname. Hosts on the public network always use HTTPS, and hosts on the local network always use HTTP.
func Host ¶
Host returns the scheme and authority of a URL (e.g. "https://server.com:8080"), dropping any userinfo, path, query, and fragment. The scheme and host are lower-cased. Only http and https URLs are recognized; anything else (an unparseable value, a missing scheme or host, or another scheme) returns "".
func Hostname ¶
Hostname returns ONLY the hostname, removing the protocol, port, path, and querystring from a hostname
func IsLocalHostname ¶
IsLocalHostname returns TRUE if the hostname is a local domain — a loopback or mDNS/internal name, or any IP address that is not publicly routable.
func IsLocalURL ¶
IsLocalURL returns TRUE if the URL contains a local hostname
func IsLoopback ¶
IsLoopback returns TRUE if the provided domain is a loopback address. This matches loopback hostnames ("localhost", the RFC 6761 ".localhost" TLD, and common /etc/hosts aliases) as well as any loopback IP, which covers the whole 127.0.0.0/8 block, ::1, and aliases such as "::ffff:127.0.0.1".
func IsPublicIP ¶ added in v0.0.7
IsPublicIP reports whether ip is a globally-routable public address that is safe to connect to. It returns FALSE for loopback, private, link-local (including the 169.254.169.254 cloud-metadata endpoint), unspecified, multicast, broadcast, carrier-grade NAT, and other special-use ranges.
Use this to defend against SSRF: validate the resolved IP at connection time (e.g. in a net.Dialer Control hook) before connecting to a user-supplied host.
func IsPublicIPAddress ¶ added in v0.0.7
IsPublicIPAddress reports whether the given string is a valid IP address that is also a public, safe-to-connect address. An unparseable value returns FALSE.
func IsSafeRedirectURL ¶ added in v0.3.0
IsSafeRedirectURL reports whether target is safe to use as an HTTP redirect destination or a user-facing link href.
func IsSameOrigin ¶ added in v0.5.0
IsSameOrigin returns TRUE when both values are valid http(s) URLs that share the same web origin (scheme, host, and port). Comparison is origin-level, so an https URL and an http URL on the same host are NOT the same origin, defeating a scheme-downgrade credential leak. If either value is not a valid http(s) URL (Host returns ""), the result is FALSE — an origin that cannot be established is never treated as matching.
func IsSchemeValid ¶
IsSchemeValid returns TRUE if the scheme is http or https
func IsValidHostname ¶
IsValidHostname returns TRUE if the provided value contains a valid hostname.
func IsValidIP4Address ¶
IsValidIP4Address checks if the given string is a valid IPv4 address.
func IsValidIP6Address ¶
IsValidIP6Address checks if the given string is a valid IPv6 address.
func IsValidIPAddress ¶
IsValidIPAddress checks if the given string is a valid IP address (either IPv4 or IPv6).
func IsValidTLD ¶
IsValidTLD returns TRUE if the provided "top level domain" is found in the IANA list.
func NormalizeHost ¶ added in v0.1.1
NormalizeHost extracts the bare hostname from any URL-ish value, regardless of its shape: a bare hostname, a "host:port", a full "scheme://" URL, a value with userinfo ("user@host"), or a bracketed IPv6 literal ("[::1]:8080"). The result is lower-cased. Unlike Hostname, the result MAY contain colons, because an IPv6 address (e.g. "::1") is returned in its canonical, unbracketed form.
func NotLocalHostname ¶
NotLocalHostname returns TRUE if the hostname is NOT a local domain
func NotLocalURL ¶
NotLocalURL returns TRUE if the URL does NOT contain a local hostname
func NotLoopback ¶
NotLoopback returns TRUE if the provided domain is NOT a loopback address (otherserver.com, 192.168.0.5)
func NotPublicIP ¶ added in v0.0.7
NotPublicIP returns TRUE if ip is NOT a public, safe-to-connect address.
func NotPublicIPAddress ¶ added in v0.0.7
NotPublicIPAddress returns TRUE if the given string is NOT a public IP address.
func NotSameOrigin ¶ added in v0.5.0
NotSameOrigin is the inverse of IsSameOrigin. It returns TRUE when the two values do NOT share the same http(s) web origin (including when either value is not a valid http(s) URL).
func NotSchemeValid ¶
NotSchemeValid returns TRUE if the scheme is NOT http or https
func NotValidHostname ¶ added in v0.0.3
NotValidHostname returns TRUE if the provided value does NOT contain a valid hostname.
func NotValidIPAddress ¶ added in v0.0.3
NotValidIPAddress checks if the given string is NOT a valid IP address.
func NotValidTLD ¶
NotValidTLD returns TRUE if the provided "top level domain" is NOT found in the IANA list.
func NotValidURL ¶
NotValidURL returns TRUE if the URL is NOT valid
func ParseURL ¶
ParseURL is a drop-in replacement for url.Parse that requires a valid hostname and http/https scheme.
func Path ¶
Path returns the path portion of a URL as a safe, root-absolute, same-origin reference. The scheme and host (if any) are discarded.
func PathAndQuery ¶
PathAndQuery returns the path and query portions of a URL as a safe, root-absolute, same-origin reference. The scheme and host (if any) are discarded, so the result is suitable to place directly in an HTTP "Location" header without risking an open redirect to another host.
func PrependProtocol ¶
PrependProtocol adds the correct protocol to the beginning of a URL if needed.
func Protocol ¶
Protocol returns the protocol portion of a URL. If the URL is not valid, then an empty string is returned.
func RefreshTLDs ¶
func RefreshTLDs()
RefreshTLDs loads the most recent TLD list from the IANA website.
func SafeURL ¶ added in v0.4.0
SafeURL returns a re-serialized copy of raw when raw is safe to place in a link href or image src, and "" otherwise. It accepts exactly what IsSafeRedirectURL accepts — a true relative path, or an absolute http(s) URL (an off-site host is allowed) — and rejects "javascript:"/"data:" schemes and protocol-relative "//host" values.
Unlike IsSafeRedirectURL, which only reports safety, SafeURL NEUTRALIZES the value: re-serializing through net/url percent-encodes the characters that would otherwise break out of an HTML attribute (url.URL.String encodes ", <, >, space, and backtick), and SafeURL additionally encodes the single quote that String leaves intact. The result is therefore safe inside an attribute of any quote style even when the consumer does not escape it — e.g. a client that interpolates the value into src="..." or onclick="f('...')" without its own escaping. Returning "" lets the consumer omit the attribute rather than emit a dangerous one.
func Scheme ¶
Scheme returns the scheme portion of a URL (without the "://" suffix), or an empty string if the URL cannot be parsed.
func ValidateHostname ¶
ValidateHostname validates a hostname and returns an error describing any issues found.
func ValidateTLD ¶
ValidateTLD returns an error if the provided top-level domain is empty or is not found in the IANA list.
func ValidateURL ¶
ValidateURL checks to see that a URL properly formatted, has a valid hostname, and uses http or https as the scheme.
Types ¶
This section is empty.
Source Files