Wednesday, November 25, 2009

How to split large files in Linux and *BSD

Suppose you have a large file that you need to "break down" to smaller pieces and then reconstruct. It's fairly simple, using the split command.

I will only give a brief usage example. For more information, check out man split (and info coreutils 'split invocation' on Linux).

1. Splitting

You split a larger file into smaller ones issuing a command such as:
split -b 100M your.file
The command above will split your.file in 100 megabytes volumes, called by default "xaa", "xab", "xac" and so on.

If you feel you need another "pattern" or prefix, add that string to the end of the command:
split -b 100M your.file file.part.
The above command will split your.file in volumes called "file.part.aa", "file.part.ab" etc. (instead of the default "xaa", "xab" etc.).

If you prefer the suffix (the "increment") to be digits rather than letters, use the -d flag.
split -db 1G your.file file.part.
would produce volumes of your file of exactly 1 gigabyte, called "file.part.00", "file.part.01" etc.

2. Joining

To join the volumes, it is this simple:
cat `echo file.part.* | sort` > your.file.reloaded
Both your.file and your.file.reloaded have the same MD5 sum – they are identical.


Post a Comment