How to get started using awk

awk, sed, and grep are three of my favorite tools in the Linux or UNIX command line. They are all pretty powerful. Today we’ll look at how to get cracking with awk to help you ease into using it. Then we’ll look at some useful awk one liners to make things a bit more fun for you.

AWK is a programming language designed for processing text-based data, either in files or data streams. It was created at Bell Labs in the 1970s. Although it’s quite old, don’t get fooled by it’s age. It is extremely powerful and efficient at what it does. Let’s get our hands dirty now.

Before we delve into the complex working and usage of awk let’s get you started on it’s basics. We’ll create and use a dummy file for this exercise. You can use pretty much any text file, such as a log from your system. I will be using an sample output from one of my favorite system monitoring tools – Dstat. Here’s the output:

screenshot of the output of dstat
click to enlarge

This is an ideal output for awk to handle. awk is great with comma or tab separated content. You’ll see why soon. So either create some similar data or copy and paste my example the above into a dummy file called something like test.txt. Launch a terminal window on your Linux computer. Almost all flavors of Linux ship with awk. In case you have found one that does not have it for some reason please install it. On the terminal window type the following from the directory where you have stored the test.txt file –

# awk {‘print’} test.txt

The output should contain the entire contents of the text file. What’s the fun in that.

Now let’s see how you can pick a column and print just that one. Execute the following command:

# awk {‘print $1’} test.txt

Now we are asking awk to print just the first column of the text file. It will automatically figure out that the file is a tab separated one and print just the first column of the contents. You should see something like this in the output:


You can do the same for any column you like. If you want awk to print the third column change command above shown command to:

# awk {‘print $3’} test.txt

You can also have awk print multiple columns. So if you want the first, third, and seventh columns printed add them to the command separated by commas.

# awk {‘print $1, $3, $7′} test.txt

would do the trick for you:

—-total-cpu-usage—- -net/total-
usr idl read
5 93 154k
13 87 0
8 92 0
0 99 0
1 97 0
1 98 0
1 99 0
0 99 0
1 99 0
1 100 0

If you have a trickier file like the /etc/password file where the data is separated by colons rather that spaces or tabs, awk doesn’t pick that up automatically. In such cases you can feed awk with the correct separator. Use a command like this to print the second column of the file:

# awk -F’:’ {‘print $1’} /etc/passwd

This command will give you an output of the usernames of all the users on your system:


You can do the same with any other type of separators. You can also use awk to parse your log files. For example, if you want to view all the IP addresses and the related web URLs that have been accesses on your web server you can use awk to parse your web server’s access log to get this information. Use the following command:

# awk ‘$9 == 200 { print $1, $7}’ access.log /2008/10/my-5-favourite-hangouts/ /2009/02/querious-a-mysql-client-for-the-mac/ /2009/05/ /2009/01/bicycle-rental/ /wp-comments-post.php

Using parsing like this you can figure out if someone is visiting your website a lot, as they may be stealing information. You can also sort this information. Say you wanted to know how many times a particular IP address visited your website

# awk ‘$9 == 200 { print $1}’ access.log | sort | uniq -c | sort -nr


If this article helped you, I'd be grateful if you could share it on your preferred social network - it helps me a lot. If you're feeling particularly generous, you could buy me a coffee and I'd be super grateful :)

buy a coffee for

Home » Linux » How to get started using awk

9 thoughts on “How to get started using awk”

  1. James – I totally see the logic in that. I do however think that tools like sed and awk have a place for ‘quick and dirty’ work. I never learned Perl, and I’m certain it would have been time well spent – but knowing the basics of awk has helped me on a number of occasions.

  2. As a long time AWK user I know the magic one can do using this wonderful tool. However, after learning Perl some 12 years ago, I’ve not once had to return to AWK. I would think it a waste of time for a newcomer to spend time learning AWK at the cost of not learning Perl.

Leave a Comment

Your email address will not be published.