Nothing better than Wikipedia to explain what this tool consists of:
GNU Wget is a free software tool that allows the downloading of content from web servers in a simple way. Its name derives from World Wide Web (w), and from "get" (in English get), this means: get from the WWW.
Currently it supports downloads using the HTTP, HTTPS and FTP protocols.
Among the most outstanding features it offers wget there is the possibility of easy downloading of complex mirrors recursively, conversion of links to display HTML content locally, support for proxies ...
It is true that there are other applications that help us to perform this type of work such as httrack or even extensions for Firefox Be Scrapbook, but nothing like the simplicity of a terminal 😀
Doing the magic
I was curious about the movie: The Social Network, as the character of mark_zuckerberg use the phrase: «A bit of magic wget«, When I was about to download the photos for Facemash 😀 and it's true, wget allows you to do magic with the appropriate parameters.
Let's look at a couple of examples, let's start with the simple use of the tool.
To go down a page:
$ wget https://blog.desdelinux.net/con-el-terminal-bajar-un-sitio-web-completo-con-wget
To download the entire site recursively, including images and other types of data:
$ wget -r https://blog.desdelinux.net/
And here comes the magic. As they explain us in the article of Humans, many sites verify the identity of the browser to apply various restrictions. With Wget we can circumvent this in the following way:
wget -r -p -U Mozilla https://blog.desdelinux.net/
Or we can also pause between each page, since otherwise the site owner may realize that we are downloading the site completely with Wget.
wget --wait=20 --limit-rate=20K -r -p -U Mozilla https://blog.desdelinux.net/