Awk: Learning Shell Scripting Using the awk Terminal Command

The "awk" command in Free Operating Systems of the GNU / Linux type is a very powerful command, which helps us to have greater control when processing data from other commands or command commands. The name AWK derives from the initials of the surnames of its authors: Alfred Aho, peter Weinberger, and Brian KErnighan.

This command provides a scripting language for text processing with which we can: Define variables, use strings and arithmetic operators, use flow control and cycles, and generate formatted reports. Actually, Awk is more than just a pattern processing command, it is an entire semantic analysis language.

Introduction to the "awk" command

AWK was one of the first very popular console utilities to manage (handle / extract) data by maximizing the functionality of UNIX pipes. The language provided by this utility is currently a standard in almost all modern UNIX-type Operating Systems, so much so that it is part of the basic UNIX Specifications, so it is usually already installed in most of them by default.

Its most common syntax is: awk 'program' files and where 'program' can be: pattern {action} pattern {action}. The command reads the file input one line at a time. Each row is compared to each pattern in order; For each pattern that matches the row, the corresponding action is performed.

A simple example like:

awk -F: '$1=="root" {print}' /etc/passwd

it asks for the first field if it is root of each line in / etc / passwd and prints it considering the ":" with -F: as a field separator.

Practical examples

Most used command list

View a list based on the history of the System terminal ordered from the most frequently used to the least.

history | awk '{a[$2]++}END{for(i in a){print a[i] " " i}}' | sort -rn | head

Example:

Graph (Table) with number of connections for each host

Create an ASCII text-based graph that shows the total number of connections from the host to each IP address.

netstat -an | grep ESTABLISHED | awk '{print $5}' | awk -F: '{print $1}' | sort | uniq -c | awk '{ printf("%s\t%s\t",$2,$1) ; for (i = 0; i < $1; i++) {printf("*")}; print "" }'

Example:

List of Directories by size

Print on screen the list of directories of the current path from the largest to the smallest showing the labels corresponding to their size using the letters: g, m, and k.

du --max-depth=1 | sort -r -n | awk '{split("k m g",v); s=1; while($1>1024){$1/=1024; s++} print int($1)" "v[s]"\t"$2}'

Example:

Show traffic from a Network Interface

View inbound and outbound data consumption online in kilobytes (kb) for a specified network interface.

while [ /bin/true ]; do OLD=$NEW; NEW=`cat /proc/net/dev | grep eth0 | tr -s ' ' | cut -d' ' -f "3 11"`; echo $NEW $OLD | awk '{printf("\rin: % 9.2g\t\tout: % 9.2g", ($1-$3)/1024, ($2-$4)/1024)}'; sleep 1; done

Example:

Obtain a fibonacci number series

Create a list of X quantity numbers following a fibonacci pattern given 2 base numbers to be constructed.

seq 50| awk 'BEGIN {a=1; b=1} {print a; c=a+b; a=b; b=c}'

Example:

Graphic listing of file structure by size

Obtain a List of all the folders in the root of the operating system together with the graphic (bars) and numerical representation as a percentage of their size within it.

t=$(df|awk 'NR!=1{sum+=$2}END{print sum}');sudo du / --max-depth=1|sed '$d'|sort -rn -k1 | awk -v t=$t 'OFMT="%d" {M=64; for (a=0;a<$1;a++){if (a>c){c=a}}br=a/c;b=M*br;for(x=0;x<b;x++){printf "\033[1;31m" "|" "\033[0m"}print " "$2" "(a/t*100)"% total"}'

Example:

List a summary of file types in a Directory

Show a list of the types of files contained in a directory together with the number of files that correspond to it.

find . -type f | awk -F'.' '{print $NF}' | sort| uniq -c | sort -g

Example:

These examples of command lines using the awk command I hope you find it very useful. And they can be used to be used directly from the console or indirectly by inserting them in a script (automated task).

In the next post on Shell Scripting we will explore other commands.

If you want to learn a little more about the "grep" command visit this link "DEBIAN Wiki”And if you want to learn more about Shell Scripting in our own Blog, you can take a look at our other publications on the subject, by clicking here: Scripting in FromLinux