AWK Commands in Linux II

AWK commands in Linux

AWK programming has various built in commands which enables scanning every record and field inside a file. However, one must be careful to use as Linux has eyes that can look spaces too. Basically AWK built in commands has defined values like field separator, record separator etc.

AWK FS command

The FS command acts as a separator in the fields. Any character can be provided to FS which is used as the delimiter. The delimiter can be a single character or a combination of single characters to make it as a word.

For example, if we have a word “DICTATORSHIP” and the FS=”T” then the output will be

DIC A ORSHIP

T acts as a separator or better say field separator.

The value of FS can be set in the BEGIN clause or using –F command line.

Also, we have NR command which counts the records we have passed in the file. NR can be used in the BEGIN clause or even in the END clause.

Here is an example of AWK FS as well as NR commands

Awk ‘BEGIN {FS = “O”;}

{ Print $1”\t”$2”\t”$3”\t”$4”;}

END {print NR, “ records have been passed”;}’ test.txt

Another example, but this time FS is not a character, but a word.

Awk ‘BEGIN {FS = “CEO”;}

{ Print $1”\t”$2”\t”$3”\t”$4”;}

END {print NR, “ records have been passed”;}’ test.txt

The output of both the above codes.

AWK OFS COMMAND

OFS stands for Output Field Separator. OFS is similar to AWK FS command. The default value of OFS command is a space. The AWK OFS appears in between the fields.

Here is an example with AWK OFS

Awk ‘BEGIN{ OFS = “CEO” ;}

{ Print $2,$4; }

END { print NR, “ records have been passed”; }’ test.txt

Example 2 

Awk ‘BEGIN{ OFS = “<here lies OFS>” ;}

{ Print $2,$4; }

END { print NR, “ records have been passed”; }’ test.txt

Here is the output of the above code.

Note- Remember that while editing the data in VI or VIM editor a line is also considered as a record. I was getting 6 records have been passed in output.

One can easily decipher here that AWK FS command decides the separation character or word. That letter or word is treated as space an the output is thrown at stdout. However the AWK OFS command decides with which character will two fields will be separated. Default its space however we changed it in previous code.

Here is the output 

(OFS is basically a character or word with which concatenation is done)

AWK ORS Command

ORS stands for Output Record Separator. Its operation is very simple. Each record will be separated with this delimiter.

An example for AWK ORS

Awk ‘BEGIN{ ORS = “\n@\n” ;}

{ Print; }

END { print NR, “ records have been passed”; }’ test.txt

AWK NR Command

NR stands for number of records preocessed variable. It even counts a blank line as a record so one has to be careful. Well we already covered it in most other above examples in the END clause. It can be used in the coding section and will be evaluated to the records that has passed each time.

AWK NF Command

NF stands for number of fields. It counts the number of fields in outputs them at stdout. 

Example

awk '{ print "Record Number ",NR," has ",NF," fields";}' test.txt

AWK FNR Command

FNR stands for File Number of Records. It will give the number of records in each file.

Example 

awk '{print FILENAME, FNR;} test.txt test2.txt

Output for ORS and FNR along with NR Command

The difference between FNR and NR is in the below image. Enough for the geeks out there to understand.

AWK If-Else Commands  will be updated soon.

AWK commands in Linux

AWK commands in Linux

AWK programming has various built in commands which enables scanning every record and field inside a file. However, one must be careful to use as Linux has eyes that can look spaces too. Basically AWK built in commands has defined values like field separator, record separator etc.

AWK FS command

The FS command acts as a separator in the fields. Any character can be provided to FS which is used as the delimiter. The delimiter can be a single character or a combination of single characters to make it as a word.

For example, if we have a word “DICTATORSHIP” and the FS=”T” then the output will be

DIC A ORSHIP

T acts as a separator or better say field separator.

The value of FS can be set in the BEGIN clause or using –F command line.

Also, we have NR command which counts the records we have passed in the file. NR can be used in the BEGIN clause or even in the END clause.

Here is an example of AWK FS as well as NR commands

Awk ‘BEGIN {FS = “O”;}

{ Print $1”\t”$2”\t”$3”\t”$4”;}

END {print NR, “ records have been passed”;}’ test.txt

Another example, but this time FS is not a character, but a word.

Awk ‘BEGIN {FS = “CEO”;}

{ Print $1”\t”$2”\t”$3”\t”$4”;}

END {print NR, “ records have been passed”;}’ test.txt

The output of both the above codes.

AWK OFS COMMAND

OFS stands for Output Field Separator. OFS is similar to AWK FS command. The default value of OFS command is a space. The AWK OFS appears in between the fields.

Here is an example with AWK OFS

Awk ‘BEGIN{ OFS = “CEO” ;}

{ Print $2,$4; }

END { print NR, “ records have been passed”; }’ test.txt

Example 2 

Awk ‘BEGIN{ OFS = “<here lies OFS>” ;}

{ Print $2,$4; }

END { print NR, “ records have been passed”; }’ test.txt

Here is the output of the above code.

Note- Remember that while editing the data in VI or VIM editor a line is also considered as a record. I was getting 6 records have been passed in output.

One can easily decipher here that AWK FS command decides the separation character or word. That letter or word is treated as space an the output is thrown at stdout. However the AWK OFS command decides with which character will two fields will be separated. Default its space however we changed it in previous code.

Here is the output 

(OFS is basically a character or word with which concatenation is done)

AWK ORS Command

ORS stands for Output Record Separator. Its operation is very simple. Each record will be separated with this delimiter.

An example for AWK ORS

Awk ‘BEGIN{ ORS = “\n@\n” ;}

{ Print; }

END { print NR, “ records have been passed”; }’ test.txt

AWK NR Command

NR stands for number of records preocessed variable. It even counts a blank line as a record so one has to be careful. Well we already covered it in most other above examples in the END clause. It can be used in the coding section and will be evaluated to the records that has passed each time.

AWK NF Command

NF stands for number of fields. It counts the number of fields in outputs them at stdout. 

Example

awk '{ print "Record Number ",NR," has ",NF," fields";}' test.txt

AWK FNR Command

FNR stands for File Number of Records. It will give the number of records in each file.

Example 

awk '{print FILENAME, FNR;} test.txt test2.txt

Output for ORS and FNR along with NR Command

The difference between FNR and NR is in the below image. Enough for the geeks out there to understand.

AWK If-Else Commands  will be updated soon.

AWK Commands in Linux I

Awk Linux Tutorial

Awk is a programming language which involves manipulation of data is a strict structured format. Awk stands proudly representing the names of its authors who dealt a great time with it. The authors were “Aho, Weinberger, and Kernighan”. Since Awk handles data very efficiently and sequentially hence it is used for basic pattern searching in a permutation of records and data. It can even match between various files and return the result, whether they required pattern matches or not.

Awk always prefers text files and not random files because random files consists of certain binary numbers which tends to the miscalculation of the records and field.

So either Awk will read from a file or one can provide the data in the command line itself. The code on which the Awk will work can be entered directly in the command line interface or it can be stored in a file using editor software’s like VI or VIM.

One must always remember that Awk commands always work on records and fields. It’s way of coding  basically resembles to the language C.

Now we will discuss the working way of Awk command inside Linux shell:

•  Awk reads 1^st line. Since it works sequentially, thus it will always read a line at a time.
• Then it scans the required pattern. If the pattern search results true, then a specific set of instructions gets executed and increment of line takes place else if no match is found, then also it may or may not execute a particular set of commands. It completely depends on the user whether to execute any set in any one condition. Both cannot be run simultaneously. It’s similar to the if-else syntax of the language C.
• Delimiter must be placed after a statement in the action clause.

Syntax of Awk :

Awk '(pattern 1){Action performed}

(Pattern 2){Action performed}' <File on which action has to be performed>

Each record is separated within segments represented as $n.

For example, if we have a sentence

Thomas Alva Edison was a great innovator of his time.

Here $1 = Thomas $2 = Alva $3 = Edison $4 = was $5 = a $6 = great $7 = innovator $8= of $9 = his $10 = time

Phew, it took ages to type all those dollars.

One important thing here is that Awk also has a $0 too, which represent the whole record i.e. all $1 to $10 combined.

Example 1: Awk simple search

1 Robert CEO Stark

2 Pepper Secretary Stark

3 Shashi CEO thessuman

4 Ambika CEO D&Dmotors

5 Aditi Secretary D&Dmotors

The above 5 commands were given in VI editor and saved as “test”.

Now we will the following commands

Awk ‘/CEO/’ test

Awk ‘/Stark/’ test

Awk ‘/D&Dmotors/’ test

Below image is the output of all above commands

The command Awk '/{print;}/ <filename> will print all the records.

The command Awk '/{print $n,......,}/ <filename> will print the dollar value of the record.

Awk in Linux has two clauses that function as beginning and ending execution. The first one is the BEGIN clause and the second one is the END clause. The BEGIN clause gets executed only once at the beginning whereas the END clause gets executed only at the end of when all records have been finished. 

Synatx for BEGIN in Awk

BEGIN { actions }
{ commands to be executed in each line }
END  { actions }

Awk in Linux also has conditions and statement execution. Let us write a simple program to count number of CEO's and number of employees in "D&Dmotors".

A simple example for BEGIN and END
Awk 'BEGIN {ce=0;d=0}
$3 ~ /CEO/ {ce++;}
$4 ~ /D&Dmotors/ {d++;}
END { print "No. of CEOs are ",ce,"\t","No. of employees in D&D are ",d;}' test.txt

For AWK built-in commands Click here 

AWK Linux Tutorial

Awk Linux Tutorial

Awk is a programming language which involves manipulation of data is a strict structured format. Awk stands proudly representing the names of its authors who dealt a great time with it. The authors were “Aho, Weinberger, and Kernighan”. Since Awk handles data very efficiently and sequentially hence it is used for basic pattern searching in a permutation of records and data. It can even match between various files and return the result, whether they required pattern matches or not.

Awk always prefers text files and not random files because random files consists of certain binary numbers which tends to the miscalculation of the records and field.

So either Awk will read from a file or one can provide the data in the command line itself. The code on which the Awk will work can be entered directly in the command line interface or it can be stored in a file using editor software’s like VI or VIM.

One must always remember that Awk commands always work on records and fields. It’s way of coding  basically resembles to the language C.

Now we will discuss the working way of Awk command inside Linux shell:

•  Awk reads 1^st line. Since it works sequentially, thus it will always read a line at a time.
• Then it scans the required pattern. If the pattern search results true, then a specific set of instructions gets executed and increment of line takes place else if no match is found, then also it may or may not execute a particular set of commands. It completely depends on the user whether to execute any set in any one condition. Both cannot be run simultaneously. It’s similar to the if-else syntax of the language C.
• Delimiter must be placed after a statement in the action clause.

Syntax of Awk :

Awk '(pattern 1){Action performed}

(Pattern 2){Action performed}' <File on which action has to be performed>

Each record is separated within segments represented as $n.

For example, if we have a sentence

Thomas Alva Edison was a great innovator of his time.

Here $1 = Thomas $2 = Alva $3 = Edison $4 = was $5 = a $6 = great $7 = innovator $8= of $9 = his $10 = time

Phew, it took ages to type all those dollars.

One important thing here is that Awk also has a $0 too, which represent the whole record i.e. all $1 to $10 combined.

Example 1: Awk simple search

1 Robert CEO Stark

2 Pepper Secretary Stark

3 Shashi CEO thessuman

4 Ambika CEO D&Dmotors

5 Aditi Secretary D&Dmotors

The above 5 commands were given in VI editor and saved as “test”.

Now we will the following commands

Awk ‘/CEO/’ test

Awk ‘/Stark/’ test

Awk ‘/D&Dmotors/’ test

Below image is the output of all above commands

The command Awk '/{print;}/ <filename> will print all the records.

The command Awk '/{print $n,......,}/ <filename> will print the dollar value of the record.

Awk in Linux has two clauses that function as beginning and ending execution. The first one is the BEGIN clause and the second one is the END clause. The BEGIN clause gets executed only once at the beginning whereas the END clause gets executed only at the end of when all records have been finished. 

Synatx for BEGIN in Awk

BEGIN { actions }
{ commands to be executed in each line }
END  { actions }

Awk in Linux also has conditions and statement execution. Let us write a simple program to count number of CEO's and number of employees in "D&Dmotors".

A simple example for BEGIN and END
Awk 'BEGIN {ce=0;d=0}
$3 ~ /CEO/ {ce++;}
$4 ~ /D&Dmotors/ {d++;}
END { print "No. of CEOs are ",ce,"\t","No. of employees in D&D are ",d;}' test.txt

For AWK built-in commands Click here