vim and awk cheat sheet

vim

Search and replace

Searching and replacing one occurrence from the current line in the file

:s/<search>/<replace>/g

Searching and replacing all occurrences from all lines in an open file

:%s/<search>/<replace>/g

awk

Splits text into fields per command line instructions, on data piped into awk from another command.

Awk may forget to process the first line of a file, considering it a header! Remember to add an empty line to line 1 before awk'ing anything!

Remember to redirect input to awk...

awk will generally require input piped from a source file, so before all the examples below, remember to cat the file output, for example:

cat /foo/bar/mySourceFile.txt | <awk pattern>

Remember to redirect awk output to a file...

If not, it will be displayed on screen. Perfect for checking of your pattern is correct but if you have lots of lines, you may want to redirect to another file to hold the results:

cat /foo/bar/mySourceFile.txt | <awk pattern> > /foo/bar/awkProcessedFile.txt

Remember the difference between > and >>

  • > will send the output to the defined file, and it will be overwritten if it exists - be careful if you are processing in a loop!
  • » will send the output to the end of the defined file. If the file does not exist, it will be created, and if it already exists, content will be appended to the end without overwriting existing content.

Basic split on single character separator

awk '{FS="<SINGLE CHARACTER SEPARATOR>"} {print $1 $1 $x}'

The print command between curly braces at the end of the awk command outputs the columns that are generated for each line of data that awk converts into columns, $1 is the first column, $2 the second and so on.

Split and format text with something else than spaces

awk '{FS="<SINGLE CHARACTER SEPARATOR>"} {print $1 " - " $1 "\t" $3}'

If you want to output pieces of arbitrary text, such as field separators in the print section, you can, but they must be quoted in double quotes as in the example above.

Set multiple separators

Use a regex to specify a list of characters to split on. Very useful when parsing url's where you may want to split on multiple characters, eg:

http://www.example.com;session=12345?foo=bar&foo1=bar1

You may want to split all variables out, including session, foo and foo1 and separate by semi-colons:

awk '{FS="[;?&]"} {print $2 "; " $3 "; " $4}'

Using special characters in the Field Separator argument

Given that awk can use regex, and that it is already using as command delimiters the symbols ' { } [ ] and , if you need to use these as a delimiter, you can, but you must remember to escape them with a backslash!

awk '{FS="[;?&\"\[\{\}\]\}"} {print $2 "; " $3 "; " $4}'