pget

package module

v1.0.1 Latest Latest Go to latest Published: Oct 21, 2022 License: MIT Imports: 21 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/fideism/pget

Links

Open Source Insights

README ¶

Pget - The fastest file download client

Description

Multi-Connection Download using parallel requests.

Fast
Resumable
Cross-compiled (windows, linux, macOS)

This is an example to download linux kernel. It will be finished between 15s.

pget

Disclaimer

This program comes with no warranty. You must use this program at your own risk.

Note

Using a large number of connections to a single URL can lead to DOS attacks.
The case is increasing that if you use multiple connections to 1 URL does not increase the download speed with the spread of CDNs.
- I recommend to use multiple mirrors simultaneously for faster downloads (And the number of connections is 1 for each).

Installation

Homebrew

$ brew install pget

Go

$ go install github.com/Code-Hex/pget/cmd/pget@latest

Synopsis

This example will be used 2 connections per URL.

$ pget -p 2 MIRROR1 MIRROR2 MIRROR3

If you have created such as this file

cat list.txt
MIRROR1
MIRROR2
MIRROR3

You can do this

cat list.txt | pget -p 2

Options

  Options:
  -h,  --help                   print usage and exit
  -p,  --procs <num>            the number of connections for a single URL (default 1)
  -o,  --output <filename>      output file to <filename>
  -t,  --timeout <seconds>      timeout of checking request in seconds
  -u,  --user-agent <agent>     identify as <agent>
  -r,  --referer <referer>      identify as <referer>
  --check-update                check if there is update available
  --trace                       display detail error messages

Pget vs Wget

URL: https://mirror.internet.asn.au/pub/ubuntu/releases/21.10/ubuntu-21.10-desktop-amd64.iso

Using

time wget https://mirror.internet.asn.au/pub/ubuntu/releases/21.10/ubuntu-21.10-desktop-amd64.iso
time pget -p 6 https://mirror.internet.asn.au/pub/ubuntu/releases/21.10/ubuntu-21.10-desktop-amd64.iso

Results

wget   3.92s user 23.52s system 3% cpu 13:35.24 total
pget -p 6   10.54s user 34.52s system 25% cpu 2:56.93 total

wget 13:35.24 total, pget -p 6 2:56.93 total (6x faster)

Binary

You can download from here

Author

codehex

Documentation ¶

Index ¶

func Download(ctx context.Context, c *DownloadConfig, opts ...DownloadOption) error
type CheckConfig
type DownloadConfig
type DownloadOption
- func WithReferer(referer string) DownloadOption
- func WithUserAgent(ua string) DownloadOption
type Options
type Pget
- func New() *Pget
type Range
- func (r Range) BytesRange() string
type Target
- func Check(ctx context.Context, c *CheckConfig) (*Target, error)

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func Download ¶

func Download(ctx context.Context, c *DownloadConfig, opts ...DownloadOption) error

Download ...

Types ¶

type CheckConfig ¶

type CheckConfig struct {
	URLs    []string
	Timeout time.Duration
	Client  *http.Client
}

CheckConfig is a configuration to check download target.

type DownloadConfig ¶

type DownloadConfig struct {
	Filename      string
	Dirname       string
	ContentLength int64
	Procs         int
	URLs          []string
	Client        *http.Client
	// contains filtered or unexported fields
}

DownloadConfig ...

type DownloadOption ¶

type DownloadOption func(c *DownloadConfig)

DownloadOption ...

func WithReferer ¶

func WithReferer(referer string) DownloadOption

WithReferer ...

func WithUserAgent ¶

func WithUserAgent(ua string) DownloadOption

WithUserAgent ....

type Options ¶

type Options struct {
	Help          bool   `short:"h" long:"help"`
	NumConnection int    `short:"p" long:"procs" default:"1"`
	Output        string `short:"o" long:"output"`
	Timeout       int    `short:"t" long:"timeout" default:"10"`
	UserAgent     string `short:"u" long:"user-agent"`
	Referer       string `short:"r" long:"referer"`
	Update        bool   `long:"check-update"`
	Trace         bool   `long:"trace"`
}

Options struct for parse command line arguments

type Pget ¶

type Pget struct {
	Trace  bool
	Output string
	Procs  int
	URLs   []string
	// contains filtered or unexported fields
}

Pget structs

func New ¶

func New() *Pget

New for pget package

func (*Pget) Ready ¶

func (pget *Pget) Ready(version string, args []string) error

Ready method define the variables required to Download.

func (*Pget) Run ¶

func (pget *Pget) Run(ctx context.Context, version string, args []string) error

Run execute methods in pget package

func (*Pget) Save ¶

func (pget *Pget) Save(ctx context.Context, url, dir, filename string) error

Save 下载并保存

type Range ¶

type Range struct {
	// contains filtered or unexported fields
}

Range struct for range access

func (Range) BytesRange ¶

func (r Range) BytesRange() string

BytesRange ...

type Target ¶

type Target struct {
	Filename      string
	ContentLength int64
	URLs          []string
}

Target represensts download target.

func Check ¶

func Check(ctx context.Context, c *CheckConfig) (*Target, error)

Check checks be able to download from targets

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
cmd
pget command

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL