webpage_requisites_go

package module

v0.0.0-...-2580251 Latest Latest Go to latest Published: Nov 16, 2022 License: MIT Imports: 7 Imported by: 0

Details

Extract webpage requisite URLs from HTML and CSS documents. Requisites are the resources a browser would load to render the web page.

go get github.com/birneee/webpage-requisites-go

Requisites are:

src of img elements, not starting with data:
href of link elements with rel "stylesheet", "manifest" or "shortcut icon"
src of script elements
url values in stylesheets, not starting with data:

Non-requisites are:

This section is empty.

View Source

var CSS_URL_REGEXP *regexp.Regexp = regexp.MustCompile(`^url\(['"]?(.*?)['"]?\)$`)

func GetCssRequisites(css string) ([]*url.URL, error)

GetCssRequisites might return duplicates inspired by https://github.com/cornelk/goscrape

func GetHtmlRequisites(html io.Reader) ([]*url.URL, error)

GetHtmlRequisites might return duplicates

This section is empty.