Transforming columns into lines

Last update on October 1, 2009 09:47 AM by deri58
Published by jak58

Transforming columns into lines







It is endemic for most Linux tools to work with lines, but not with columns (sed, awk, grep, etc..).
However, it may happens,that you have a file where the data should be read in columns.

Example


$ cat in.txt2008-09-21|2008-09-22|2008-09-23
Sunday|Monday|Tuesday


Solution

Here below is how to convert this file into a 3-lines format (date|day)
$ split -l1 in.txt tmp_
$ sed -i 's/|/\x0A/g' tmp_
$ paste -d'|' tmp_* >out.txt
$ rm -f tmp_*
$ cat out.txt
2008-09-21|Sunday
2008-09-22|Monday
2008-09-23|Tuesday


Check out the manual to understand these 2 commands:split and paste.

Limitations


This script will only work with a limited number of lines, (input : less than 676). This is due to suffix added by the split command which by default includes only 2 characters, but if necessary the -a option can be used to increase that limit.

For the sed (according to the versions used), two problems may arise:

The -i option requires a suffix (. bak, or another) in this case, be careful to change the paste command to take into account the files generated by split and not by sed (tmp_? eg) commands

If the version does not support octal notation \ 012 or hexadecimal \X0A: Press backslash, then following two combinations <Ctrl+V> <Ctrl+J> (or press enter key), then complete the order and to quit the sequence.
Best answers for « Transforming columns into lines » in :
Log in remotely with SSH (Linux) ShowLog in remotely with SSH (Linux) Issue Solution Using SSH through a proxy: Issue Log in remotely with SSH under Linux. Solution The commands below are relevant only if you have an existing account on the PC you want to connect and...
[Linux]Cutting a file into several parts Show[Linux]Cutting a file into several parts Linux is full of utility tools allowing you to manipulate files. There is a very handy tool to split a file into several parts (for example to carry data on low capacity media). The split...
[Sed] Delete one or more lines from a file Show[Sed] Delete one or more lines from a file Removing one (or several) line (s) of a file Syntax: sed '{[/]||[/]}d' sed '{[/][,][/]d' /.../=delimiters n = line number...
Worksheet - Cells ShowThe Concept of a Cell A "cell" is the intersection between a line (horizontal) and a column (vertical) on a worksheet. Thus, the name of the line combined with the name of the column gives the cell's coordinates (the term address is sometimes also...
Mini HowTo Documents ShowMini HOW-TO Documents The mini HowTo documents are a set of documentations written by different people on very specific topics concerning Linux. Below you will find a (non exhaustive) list of HowTo documents written or translated into...
UNIX system - The shell ShowIntroduction to the shell The command interpreter is the interface between the user and the operating system, hence its name "shell". The shell therefore acts as an intermediary between the operating system and the user using command lines...