sort & uniq command examples

• It is used to Sorts text to STDOUT.
• Original file unchanged.
• Sorts the lines in a text file.
• This data can be in a file or the output of another command.
• sort is often used with pipes as in example below. 

 Syntax:

#sort    <options>     <filename>

Common Options are :::

-b : Ignores leading spaces in each line
-d : Uses dictionary sort order. Conisders only spaces and alphanumeric characters in sorting
-f : Uses case insensitive sorting.
-M : Sorts based on months. Considers only first 3 letters as month. Eg: JAN, FEB
-n : Uses numeric sorting
-R : Sorts the input file randomly.
-r : Reverse order sorting
-k : Sorts file based on the data in the specified field positions.
-u : Suppresses duplicate lines
-t : input field separator

EXAMPLE-1

#grep   bash   /etc/passwd    |  sort 

#sort   -t  :  -k  3  -n  /etc/passwd

#ls | sort

#ls | sort  -r

#touch   {1..10}

#ls
#ls | sort -r

#ls  | sort -r  -n

 

Sort -u 

is used to remove duplicate lines from input.

example- 
# cat   report
hello
hello
network

# cat   report  |  sort -u

hello
network

# ps -ef | sort -nk2

#ps -ef | sort -rnk2          {for reverse order}

example -2-----

 Sorting lines of text

 #cat     report.txt

Unix distributed 05 server
Linux virtual 3 server
Unix distributed 05 server
Distributed processing 6 system

#sort   rerport.txt

Distributed processing 6 system
Linux virtual 3 server
Unix distributed 05 server
Unix distributed 05 server

example-3

Sorting based on the field positions. 

#sort -k2  report.txt 

Unix distributed 05 server
Unix distributed 05 server
Distributed processing 6 system
Linux virtual 3 server

((Note:)- You can also pecify more than field with k option as a comma separated list. The below command uses the second and fourth fields to sort the data.

#sort -k2,4  report.txt

 example-4
Numeric sorting

#sort -nk3 report.txt
Linux virtual 3 server
Unix distributed 05 server
Unix distributed 05 server
Distributed processing 6 system 

example-5 

Sort in reverse order

#sort -nrk3 report.txt
Distributed processing 6 system
Unix distributed 05 server
Unix distributed 05 server
Linux virtual 3 server  

example-6

Suppressing duplicates or Print only unique values 

You can produce only unique values in the output using the - u option of the sort command.
#sort -u report.txt
Distributed processing 6 system
Linux virtual 3 server
Unix distributed 05 server

Another way is piping the output of sort command to uniq command.

# sort report.txt | uniq 

 Example-7 

#cat sort.txt
Mayday|4
Janmon|1
Declast|12 

 #sort -t'|' -nrk2 sort.txt  

Declast|12
Mayday|4
Janmon|1

example-8 

Sorting on months. 

 #sort -M   sort.txt
 

Janmon|1
Mayday|4
Declast|12

Treats the first 3 characters in the string as month and then sorts in months order.



UNIQ  Command

* used to remove duplicate adjacent lines from input

#uniq  <options >    <filename>

-c	Precede each output line with a count of the number of times the line occurred in the input.
-d	Suppress the writing of lines that are not repeated in the input.
-u	Suppress the writing of lines that are repeated in the input.

[root@krnetwork~]# cut  -d:  -f7  /etc/passwd |  sort  |  uniq

/bin/bash

/bin/sync

/sbin/halt

/sbin/nologin

/sbin/shutdown

# cat   report

hello

hello

network

[root@krnetwork ~]# cat report  | uniq

hello

network

[root@krnetwork ~]# cat report  | uniq -d

hello

-d  means to output only one copy of the lines that are repeated in the input.

Note:   use  uniq command always with sort for best result.

More Uniq Command Examples:
 

First create the following example.txt file in your unix or linux operating system.

# cat example.txt 

Unix operating system
unix operating system
unix dedicated server
linux dedicated server

1. Suppress duplicate lines

The default behavior of the uniq command is to suppress the duplicate line. Note that, you have to pass sorted input to the uniq, as it compares only successive lines.

# uniq example.txt
unix operating system
unix dedicated server
linux dedicated server

If the lines in the file are not in sorted order, then use the sort command and then pipe the output to the uniq command.

# sort example.txt | uniq

2. Count of lines.

The -c option is used to find how many times each line occurs in the file. It prefixes each line with the count.

# uniq -c example.txt
      2 unix operating system
      1 unix dedicated server
      1 linux dedicated server

3. Display only duplicate lines.

You can print only the lines that occur more than once in a file using the -d option.

# uniq -d example.txt
unix operating system 

# uniq -D example.txt
unix operating system
unix operating system

The -D option prints all the duplicate lines.

4. Skip first N fields in comparison.

The -f option is used to skip the first N columns in comparison. Here the fields are delimited by the space character.

# uniq -f2 example.txt
unix operating system
unix dedicated server

In the above example the uniq command, just compares the last fields. For the first two lines, the last field contains the string "system". Uniq prints the first line and skips the second. Similarly it prints the third line and skips the fourth line.

5. Print only unique lines.

You can skip the duplicate lines and print only unique lines using the -u option

#uniq -u example.txt
unix dedicated server
linux dedicated server

Stat Command Examples

Stat command can be used either to check the status/properties of a single file or the filesystem.

[root@desktop17 ~]# touch abc
[root@desktop17 ~]# stat abc
  File: `abc'
  Size: 0             Blocks: 0          IO Block: 4096   regular empty file
Device: fd00h/64768d    Inode: 29605       Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2013-06-14 14:07:23.987939744 +0530
Modify: 2013-06-14 14:07:23.987939744 +0530
Change: 2013-06-14 14:07:23.987939744 +0530
[root@desktop17 ~]#
[root@desktop17 ~]#
[root@desktop17 ~]# stat  -f   /
  File: "/"
    ID: c840164b7db4b8fd Namelen: 255     Type: ext2/ext3
Block size: 4096       Fundamental block size: 4096
Blocks: Total: 2064238    Free: 1390370    Available: 1285513
Inodes: Total: 524288     Free: 411081


here -f  for filesystem