Backup with Knoppix
Quick Overview: Linux fans are unlikely to learn much from this page, which is primarily aimed at Windows users who are unable to access their data for one reason or another and need to find a way of backing-up (or simply accessing) material held on their hard drives. As long as Knoppix can see the hard drive there is a good chance of being able to copy or rescue material. This may even be the case when there are a considerable numbers of bad sectors on the drive. The dd_rescue utility can come into its own in the latter case and is discussed in greater detail on another page. For those with bad drives there may be some helpful information on our page dealing with the basics of data recovery.
Seven various back-up methods
-  Simply copy the files to an internal or external drive (inc USB) or Network Share.
-  Internet Backup using E-Mail or FTP.
-  Burn the files to a CD/DVD with the K3b burning software.
-  Create a wide variety of cloned material using the dd utility.
-  Create image files using the PartImage utility.
-  Use Gparted to Copy and Paste Partitions
-  Copy/Clone a complete drive (even many damaged drives as long as visible in the BIOS setup) to another drive using dd_rescue.
- First create a free bootable Live Knoppix CD by burning an appropriate ISO. You could directly download KNOPPIX_V5.1.0CD-2006-12-30-EN.iso (Free 714448 KB) and boot it up. Both it and other versions can be downloaded or ordered from http://www.knoppix.org. Remember to burn the ISO's image (use CDCC if having problems with your own software) as opposed to simply burning a copy of the downloaded .iso file onto your CD.
- We have used Knoppix 5.1.0. Just note that other versions are likely to be similar but not identical. One can get the ISO for ones own particular language and note that you can change the default language when you go to the Knoppix site. If the wrong language happens to load then this can be changed by using the appropriate "cheat code" at the first boot prompt. The text would be knoppix lang=en) for UK English but there are many others, for example: ch|cn|de|da|en|fr|it|nl|pl|ru|sk|tr|tw|us. There are very many other "cheat codes" which can help customise or improve the way the Live Knoppix CD functions.
- If you want to start entering commands then open the Root Shell as outlined at the start of the next paragraph. Note that acting as root gives one great freedom but also understand and note that with that same freedom one can also do real harm to ones system. Notwithstanding that warning - we recommend that you only work as Root or in the Root Shell for all the following examples in this article.
A few things worth knowing (based on version 5.1.0) and a few tips
- One can "Set Password for Root" and indeed open a "Root Shell" from the
K-Menu-Button >> Knoppix. It shouldn't therefore be necessary to
issue SuperUser access by entering sudo plus a root password in any other
consoles. (Note that under Linux
password entry may not show any characters or their placeholders as the
password is typed-in; the prompt may appear to do nothing till you
finally hit the Enter key).
- The first hard drives are likely to be /dev/hda, /dev/hdb etc if you
only have IDE drives or /dev/sda etc for non-IDE drives. Entering fdisk
-l -u in the Root Shell should list all the partitions and hard drives in
the system. Such devices are
special files under the Linux file system.
- Partitions can be
devices (eg /dev/sda1) or they can be folders within the directory
tree (eg /media/sda1). The latter designation is where you can access a
volume's files and folders; the former whereby you access its data in
blocks. Don't confuse these two different designations - they are for
completely different things.
- Available volumes are normally lined-up as Icons on the Desktop. These will need mounting or unmounting appropriately for any particular task at hand. When mounted, a volume should show a green triangle on its icon or when a mouse hovers over the icon.
- Clicking on these desktop icons should both open and mount most volumes. A volume must be mounted before its files can be accessed. A volume should be unmounted before accessing it as a device.
- At start-up, all volumes are only readable by default. This may be to
inhibit any disk access in data recovery situations. To make them
writable one needs to right-click on them and choose the "Change read/write mode" option, Fig 1,
as appropriate. (It shouldn't be necessary to play around with chmod to
alter permissions; enter man
chmod at a command prompt if you don't know what we are referring
to and want to know).
- We generally add USB devices after Knoppix has loaded and wait for the new icon to appear on the desktop but if for any reason this fails then turn the computer off and restart Knoppix with the USB drive attached. Make sure you unmount USB drives before detaching them from the PC. Unmounting, programatically, is done with the umount (not the unmount) command.
- The Root Shell displays as a black console box with a few characters (the command prompt) such as root@knoppix:~#. We will abbreviate this to just # from now on in this article. This prompt is followed by a cursor (either flashing or non-flashing as a simple block of colour, when active, and an empty box when the console is not active; clicking on the console itself should make it the active item).
- Once you have entered any commands at a console's command prompt they are remembered and you can thereafter use the up and down arrows to scroll through previously entered commands.
- Partitions in each hard drive have a numerical suffix. Logicals start at number 5 (/dev/hdb5 would be the first logical) and numbers 1 through 4 relate to any primary partitions but as referenced in their corresponding parition table entries in the MBR. This explains why Linux may jump from /dev/hdb2 to /dev/hdb5, which is what would happen if there were only two primary partitions (one of which would be an extended partition) and one logical partition.
- NTFS partitions are generally writable in these later versions of Knoppix. If a file fails to copy onto an NTFS partition it is probably because its $MFT (Master File Table) file is fragmented. You will have the fewest problems writing to NTFS volumes if they were recently formatted and not over-used. There are advocates of using Diskeeper to defragment NTFS volumes but we have never yet had the need. An alternative to 'defragging' is to use an NT-based OS such as WindowsXP or a BartPE to copy all the files from an existing (but not in current use) partition to a brand new partition. The only thing likely to complicate such copying is if encryption had been used on the source volume.
 Simple copying of data between folders
- LOCAL: Just ensure that you have opened (and thus mounted) the source and destination folders and that the destination's "Make Writable" option has been enabled. Then just drag the files across. You will then be prompted to move, copy or cancel.
- NETWORK OUT: If on a LAN then you may well be able to access a shared folder on another PC by opening Konqueror and enteringsmb://ComputerName/FolderName (with no trailing backslash) into the Address Bar. Just entering smb:/ should list workgroups and from there navigate on to individual PCs and their network shares.
- NETWORK IN: To make all volumes accessible to a Windows network first "Startup Samba Server" from K-Button >> KNOPPIX >> Services. An XWindow Input box opens to "Set password for user knoppix". Enter your desired password twice. Next say Yes to the message "Export all harddrives so that they can be mounted, read and written from the remote machine?". Now when you go to Workgroup Computers on a Windows PC you should be able to log in and access the partitions on the Knoppix machine.
 Internet Backup
Assuming you have internet access then:
- Simply send your files as attachments via webmail using Mozilla's IceWeasel or other browser or fire-up Kmail and configure it with your own mail settings.
- You can FTP upload your files if you have access to a WebServer. Just hit the Home icon on the Task bar (or open Konqueror in other ways) and enter a composite URI into the AddressBar Location. The format is most commonly: ftp://username:email@example.com/www (the /www is the common default location for webpages and associated files). The ftp or other sub-domain may need to be added and the /www folder is not usually mandatory; eg: ftp://username:firstname.lastname@example.org. When the folder structure opens-up in Konqueror just copy the relevant files and folders into the desired directory.
Most ISPs will give you some free web-space and you can also set-up free
web-hosting at places like Netfirms.com
at the cost of having their banner ad at the top of each page.
If you have a dial-up connection then the biggest problem is likely to be having a modem that Knoppix can use and then configuring it. It can be done and we can only say that we have found that the cheap Intel/Harmony V92 PCI hardware modems which use an Intel 536EP Chipset work very well with the Linux distros we have used.
 Simple burning of data to a CD
- NB: You will need a free second CD/DVD burner. We always bring an external USB DVD burner around with us.
- Place a blank CD or DVD in its drive. Open K3b from the K-Menu Button >> Multimedia or by entering k3b as a run command.
- Click on the New Data CD (or DVD) Project button.
- Drag the relevant data into the application and burn.
 Move blocks of data with dd
Use of "man"
If you ever want to read a description of a utility (by reading its manual or "man" entry) use man followed by the command name. For example for the dd utility, which is at the heart of this article, it would be:-
# man dd
- The most commonly used dd "switches" are: if=InputFile, of=OutputFile, bs=BlockSize, skip=IgnoredInputBlocks, seek=IgnoredOutputBlocks and Count=SelfExplanatory.
- There is further information on dd at Wikipedia and at Chrysocome and a very useful article at LinuxQuestions Wiki but it is always worth reading through the local "man <commandname>" to view the specifics for the Linux distro that one is currently using. It seems that dd is analagous to cc (carbon copy); possibly any combination of direct, duplicate, data, disks, drives and devices. Be particularly careful when using dd to write directly to a drive rather than when you are using it to make files in a file system. This is always the case for any application that writes directly to a hard drive's sectors.
- We have concentrated on the practicalities of using dd under Knoppix but for those interested in some of the history and of some computer forensic usage there is also a page called Unix DD Command and Image Creation which some might like to read through.
View Partition Details
When making and restoring images you will probably want to know what is what and where it is and what size it has. The Linux fdisk program should help you enumerate all the relevant devices on your system. The lowercase fdisk switches, L (lists details) and U (uses lba not chs), should create a nice list of available devices and display the results with LBA geometry. Remember that under Linux using the correct case can be vital, so you would enter:-
# fdisk -l -u
You can also visualise (and for that matter manipulate) your hard drives using Gparted's GUI (it is a Partition Magic clone) by starting it up from the K-Menu-Button or by starting it from a Root Shell/Console, viz:-
Once the GUI opens, use the drop down list button at the top right of the gparted window to select the device you want to visualise/manipulate. This is just one place where you can mount/unmount a partition with a RightClick shortcut menu choice from the partition in question. Gparted can be obtained as a standalone utility and its appearance can be seen at its own website.
Some Screenshots of fdisk and dd in action.
- The concept of mounting/unmounting may seem somewhat alien to Windows afficianados but needs special attention under Linux. Mounting identifies a partition and makes its file system available for use. Under Windows the assignment of Drive Letters comes closest to mounting under Linux. On old main-frames the physical attachment of a tape drive, to make its contained files available, was also known as mounting a drive.
- Any files in a partition/drive thus cannot be accessed until it has been mounted. With Knoppix, most available partitions should have been identified on arrival at the desktop - though initially unmounted. We wont describe mounting from a command line other that to say that a folder must have been prepared in advance and that the relevant device is then mounted to that folder and with a correctly designated file system. Use man mount for fuller specifics.
- So remember to mount a partition or drive before you attempt to add or access any files on it and to unmount it (or them) if you intend it (or them) to be the source or destination that you are reading-from or writing-to using direct disk block access of the sectors on the device. Direct disk access would include such actions as formatting with a utility like gparted because that is another action that doesn't work using file i/o (file in-out or file read-write).
- Fig 3. Shows hdb5 mounted (green triangle) and with a directory
location of /media/hdb5. This should be unmounted (from the icon's
right-click shortcut menu) if an underlying device (/dev/hdb or
/dev/hdb5) is to be accessed by direct disk access.
Back-up Partition Formats and File Size Consideratons
- We mostly recommend that you make your back-ups to FAT (or Linux)
partitions because it is widely reported that Linux can have problems
writing to NTFS partitions (though we have yet to have problems with
version 5.1.0). FAT partitions are particularly useful because they can
be read by a wide range of operating systems but the maximum file size is
somewhat limited. Such FAT partitions could be on floppies or on hard
drives (internal or external) or on USB flash memory drives. Remember
that the file system is irrelevant if copying in data blocks from device
to device - it is only relevant when an image file needs to be stored
inside a partition or folder as a
file. The maximum file size for FAT partitions is usually 2GB
(thought it can occasionally be 4GB if the cluster size can be made large
- We have used a floppy as the destination for these backups and assumed that hdb is the first physical hard drive and that a floppy disk is in its drive and has been mounted in preparation for the file output. The floppy can obviously already contain an appropriate sub-folder if so desired and we have used a folder called backups in our example.
- Both /dev/hdb (as the referenced drive) and /media/fd0/backups (the path to a folder on a mounted floppy partition) would need adjusting accordingly to suit your own system. The file name and extension (or no extension) are completely up to you. We generally use mbr01.dat since it is self-explanatory to us. The default block size varies with whatever is being referenced. The term block in such usage is a generic term with no statutory size. It can refer to what dd uses, to the sector size, to a page in memory and so on.
To copy the first sector from the first pysical disk to a backups folder
on a mounted floppy you could use ...
# dd if=/dev/hdb of=/media/fd0/backups/mbr01.dat bs=512 count=1
... and the results should show as lines in the console showing records in and out, the number of bytes and the time taken.
We actually recommend backing up the whole of the first track (the first 63 sectors on the hard drive) since various boot managers and DDO can spill onto more that just the very first sector - called the MBR. For this you could use ...
# dd if=/dev/hdb of=/media/fd0/backups/mbr63.dat bs=512 count=63
Restoration of Master Boot Record (MBR)
Mount the relevant floppy holding the image file and double check that of= the correct destination drive.
# dd if=/media/fd0/backups/mbr01.dat of=/dev/hdb
... or if it was the copy of the whole of Track-0 use the other file you may have made ...
# dd if=/media/fd0/backups/mbr63.dat of=/dev/hdb
Back-up of a 2GiB (gibibyte) first primary partition on first hard drive
- You will obviously have to make adjustments to suit but the examples
here used an unmounted internal partition as source and an external usb
drive mounted as /media/sda1 for
the destination file. We have a USB2 system and simply connecting the USB
drive, after Knoppix has loaded, produces icons on our desktop. These can
then be opened in Konqueror - just as with the floppy in the above
example. On our system the green triangle may only show on mounted USB
drives when we hover over their desktop icons.
Assuming there is enough room on the destination device (in this example the first scsi/sata/usb device) and the file format can deal with the file size, you can use the simplest form of all without specifying any block size:-
# dd if=/dev/hdb1 of=/media/sda1/hdd1part1.dat
But you might want to use a fuller description specifying the blocks of data to be copied. For the record, note that 1MB would refer to a decimal megabyte (of 1000^2 bytes) but that 1M refers to a binary megabyte (or mebibyte of 1024^2). Fdisk, as shown earlier, can tell you the size of all the partitions. A block size of 1M is probably as much as most systems can handle but if you are unsure of the figures just use the default and don't specify any such parameters.
# dd if=/dev/hdb1 of=/media/sda1/hdd1part1.dat bs=1M count=2048
You can also make a compressed file by "piping" it directly into a zipped format.
# dd if=/dev/hdb1 bs=1M count=2048 | gzip > /media/sda1/hddpart1.dat.gz
NB:- The dd utility does not give you any progress indicator though when a big block transfer is in progress the HDD LED is likely to almost permanently on. Your system may be fast or slow but only trial or error will let you estimate how long such imaging will take. It is a good idea to simply image a small test partition first and time how long that amount of data transfer takes. Compression generally adds to the time taken. The command prompt freezes while dd is doing its stuff and only goes to subsequent lines after completion or with an error report. These console lines (the standard output) can be suppressed, if desired, with a suffix to the normal commands. That suffix is " >& /dev/null" and the first of the three above examples could thus be re-written as:-
# dd if=/dev/hdb1 of=/media/sda1/hdd1part1.dat >& /dev/null
Restoring a previously created partition Image File
Mount the relevant partition holding the image file and double check that of= the correct destination partition and that it has been unmounted. The following command is equivalent to restoring the example image file first used earlier. Viz:-
# dd if=/media/sda1/hdd1part1.dat of=/dev/hdb1
If you created a compressed image file then it can be unzipped and piped to the destingation. For example:-
# zcat /media/sda1/hddpart1.gz | dd of=/dev/hdb1
Using zcat is equivalent to gzip -dc or to gunzip -c both of which could probably be used in place of zcat.
- We repeat that particular care is needed when restoring. It is easy to overwrite the wrong partition and you will be given no warnings. The relevant partition must also be big enough to take the contents of the original that was imaged.
N.B. DOUBLE-CHECK THAT YOU HAVE THE CORRECT DESTINATION DRIVE SELECTED. THERE IS NO GOING BACK AND NO WARNING AFTER YOU ISSUE dd COMMANDS. Its name may be small but dd can deliver an enormous punch.
- It isn't necessary to specify a block size though this can speed up the transfer of the data. Thus bs=1k or 2k or 4k might all improve the time taken for these operations.
- The destination must be equal to or larger than the source.
- Unmount all relevant partitions before proceeding.
Complete disk-to-disk copy (sector-by sector-clone) using a device as both source and destination.
# dd if=/dev/hdb of=/dev/sda
Partition-to-partition cloning is completely analagous (here with a specific block size).
# dd if=/dev/hdb3 of=/dev/sda2 bs=4k
Device-to-Disk Zeroing (using specialist input files/devices)
DO NOT ATTEMPT OR EXPERIMENT WITH THIS UNLESS YOU DELIBERATELY WANT TO ZERO YOUR DRIVE
- We realise that this is the opposite of back-up but include it here because the basics are the same.
- You might (for example) do this to clean your disk prior to selling your PC.
- If in any doubt about which hard drive you want to zero then have only that drive attached to the PC at the time. To write zeros (that is to say hexadecimal zeros 0x00 values) be prepared for a wait and then issue such as:-
# dd if=/dev/zero of=/dev/hdb
If you want to just zero the MBR (a quick way of "wiping" a disk) then use:-
# dd if=/dev/zero of=/dev/hdb bs=512 count=1
- Make the count=63 if you want to zero the whole of Track0
- The input file /dev/zero in the examples can be replaced by /dev/random to write pseudorandom data to the drive and make data recovery an even less likely possibility.
The FAT file system normally only supports a 2GB maximum file size (4GB in some circumstances). Thus you may have to use Linux or NTFS and not FAT partitions for large image files or learn how to split up image files into parts and then on how to write these parts back on restoration. If you are in dire straights you can always copy a whole drive to another drive of the same or larger size using dd to clone directly sector-by-sector and thus by-pass any file systems.
The input file can be piped to gzip (with standard output using the -c switch) and then split into blocks of a specified size (2GB in the example below and with the lone - (minus sign) switch indicating this should be taken from standard input). The file name ends gz. and should have a trailing dot operator. The resulting files will then have the suffices .aa and .ab and .ac and so on.
# dd if=/dev/hdc2 | gzip -c | split -b 2000m - /media/sda1/dev-hdc2.img.gz.
The resultant dev-hdc2.img.gz.aa etc files can later be recombined from the files on a mounted partition using the cat utility and decompression with gzip's -d switch output to the relevant and unmounted partition.
# cat dev-hdc2.img.gz.* | gzip -dc | dd if=/dev/hdc2
An image can also be manually split-up using the skip command; both the quantity to be copied and a skip switch are needed. Just an example of how one might split-up a 4GiB partition into four 1GiB files. A count of 1048576 could be represented by 1M (being 1 mebibyte); 1000000 could be represented by 1MB (being 1 megabyte); 1024 = 1k (being one kibibyte) and 1000 = 1kB (being one kilobyte). We have found no real problem in creating these files but haven't yet tried to restore them.
# dd if=/dev/hdc2 bs=1024 count=1048576 of=/media/sda1/part2-1.img
# dd if=/dev/hdc2 bs=1024 count=1048576 skip=1048576 of=/media/sda1/part2-2.img
# dd if=/dev/hdc2 bs=1024 count=1048576 skip=2097152 of=/media/sda1/part2-3.img
# dd if=/dev/hdc2 bs=1024 count=1048576 skip=3145728 of=/media/sda1/part2-4.img
One skips blocks on the input side but seeks them on the output side. So to write these files back to disk one might use:-
# dd if=/media/sda1/part2-1.img bs=1024 count=1048576 of=/dev/hdc2
# dd if=/media/sda1/part2-2.img bs=1024 count=1048576 seek=1048576 of=/dev/hdc2
# dd if=/media/sda1/part2-3.img bs=1024 count=1048576 seek=2097152 of=/dev/hdc2
# dd if=/media/sda1/part2-4.img bs=1024 count=1048576 skip=3145728 of=/dev/hdc2
We believe the above syntax is correct but have yet to use it in a real situation.[Top]
 PartImage back-up and restore
Partimage is a console-based program that can run from a command line or
from a GUI-like
Ncurses Interface. Fire it up from the K-Menu Button >> KNOPPIX
>> Root Shell by entering partimage at the command prompt. We will
describe just one way of using it which includes some notes on using Gzip
compression and splitting the file.
Before you start ensure you have unmounted the partition for any device
you are going to use for direct block access and open (thus also mount) the
partition/directory that contains or is intended to hold any resulting image
files. If about to create image files also ensure you have enabled the drive
for Write Access. One advantage of opening the volume in Konqueror like this
is that you will be able to see the correct path to the partition and its
files in its address bar.
Note too that a destination device being restored should ideally be the
same size as the original, will not work if it is smaller and you will have
lost space if it is too big.
- Newcomers to ncurses consoles (we think that's the right word) often have difficulty navigating the various options so we will outline the keyboard presses you will need to use.
- On the first screen highlight the partition for backup or restore with the Up and Down Arrow keys and then hit Tab to move to the next option.
- Enter the full path to the volume and relevant file and assuming you
are going to use gzip compression add a .gz file extension if making an image
and add a .gz.000 file extension
if you are restoring from an existing image file. Even if there are
multiple spanned images it is only necessary to enter the one with the
.000 extension. For example you might use /mnt/sda1/hda4part.gz when creating an
image file but use /mnt/sda1/hda4part.gz.000 when
restoring from it; (any associated hda4part.gz.001, hda4part.gz.002, ...
files do not need to be manually entered - the application will take care
of them). When partimage makes just one image file it will still give it
the .000 file extension even though this wasn't specifed by you.
- When the file path has been entered hit Tab again and select Save or Restore as appropriate using the arrow keys. Now mark the option you want to use with the Space Bar and then Tab your way to NEXT (F5). One can usually keep using Tab to go forwards through options and sometimes use the Up and Down Arrows to move in either direction. We will not address transmitting the file across a network so those options can be left as they were. Check that you have entered everything correctly and hit Enter.
- In the next console that should appear we suggest using Gzip compression and Automatic splitting. As before you move through the options with the Tab key and make choices with the Space Bar. When you have checked the entries use Tab to get to Continue (F5) and hit Enter.
- If creating an image, you should now be able to watch the progress on the subsequent screens. If restoring an image you will be given a couple more options. We suggest just accepting the defaults but by all means run a simulation to get to know how things work. IT IS ESSENTIAL (EVEN IF IT IS BORING TO KEEP REPEATING IT) that the correct destination partition gets chosen.
- If presented with any OK buttons that don't work when you hit Enter. First make sure the console is active by clicking anywhere on it with the mouse and then use the Tab key till the OK becomes highlighted. Then hitting Enter should work. Hitting ESC should cancel a highlighted OK.
There are obviously other ways and variations of doing things but we hope we have given you enough to get PartImage working for you. It can also be run from the command line and you can read the on-line PartImage manual or visit its forums for additional help. We believe this to be a very fine application and though a little bit "geekish" to use is a great free way to make excellent back-up images. For ease of use and because it is a small download we personally prefer TerabyteUnlimited's ImageForDOS which should function quite OK as the trial version and is not that expensive to buy if you find you like it. Knoppix can of course be obtained for nothing. Partimage can also be run from Slax and SystemRescue CDs (both of which have smaller ISOs to download). However mounting and navigating within these other Linux Live CDs would require a greater understanding of Linux to make them useful to the more casual user.[Top]
 Using Gparted to Copy and Paste Partitions
- This utility (being run from a GUI) is probably easier and safer to use than dd but cannot be used to make image files. It can only copy partition to partition.
- Fire it up from "KNOPPIX >> Root Shell" by issuing gparted at the command prompt to open it with root privileges. Switch/Select drives from the upper right-hand drop-down list.
- Use the Toolbar Buttons (or Menu Lists) to copy and paste partitions to a big enough area of available free/unallocated space
- You can "Undo" any changes made or, when ready, use "Apply" to commit the operations.
- You should only be given options that are possible to do.
- If copying a logical partition you will need to have free space inside the same or another extended partition if you want to keep its logical designation when pasted - otherwise it will get pasted as a primary partition. Create or resize an extended partition as necessary.
- Partitions can also be formatted, moved and non-destructively resized. Please backup your important data before using any partitioning utility but especially with moving and resizing operations.
Notes on alternatives:
- The Gparted utility can be installed onto and used from many Linux distros and can also be downloaded and run on its own Standalone boot CD.
- The dd utility can be run from all sorts of platforms and this includes a Windows version; (but note the different /dev syntax in the Windows version). Thus you can also run it from Linux on a floppy (we like TomsRtbt) or using smaller ISOs such as found with Slax (will fit on a mini CD) or System Rescue Live CDs. You may also find a version to install onto a pen/flash-drive but the biggest problem with these is that they wont boot from every PC and are not always easy to create. The dd switches will vary from place to place so read the man or help files for the exact version being used.