sed: uniq

 
 7.17 Make Duplicate Lines Unique
 ================================
 
 This is an example of the art of using the 'N', 'P' and 'D' commands,
 probably the most difficult to master.
 
      #!/usr/bin/sed -f
      h
 
      :b
      # On the last line, print and exit
      $b
      N
      /^\(.*\)\n\1$/ {
          # The two lines are identical.  Undo the effect of
          # the n command.
          g
          bb
      }
 
      # If the N command had added the last line, print and exit
      $b
 
      # The lines are different; print the first and go
      # back working on the second.
      P
      D
 
    As you can see, we maintain a 2-line window using 'P' and 'D'.  This
 technique is often used in advanced 'sed' scripts.