| Deletions are marked like this. | Additions are marked like this. |
| Line 63: | Line 63: |
| root@zg-1 (~/bin) 519# ./mount_gbde.sh vol002 | root@zg-1 # ./mount_gbde.sh vol002 |
| Line 75: | Line 75: |
| root@zg-1 (~/bin) 521# gpart show | root@zg-1 # gpart show |
| Line 84: | Line 84: |
| root@zg-1 (/var/log) 564# ls -1 /dev/mirror/ | root@zg-1 # ls -1 /dev/mirror/ |
| Line 101: | Line 101: |
| root@zg-1 (/usr/local/etc) 593# bsdlabel /dev/mirror/gm0s1 | root@zg-1 # bsdlabel /dev/mirror/gm0s1 |
| Line 113: | Line 113: |
| root@zg-1 (/usr/local/etc) 594# bsdlabel /dev/mirror/gm0s2 | root@zg-1 # bsdlabel /dev/mirror/gm0s2 |
| Line 126: | Line 126: |
|
This didn't work as the ports tree was not up to date. Had to download the following distfile by hand '''scan_ffs-1.2.tar.bz2''', which I found at: http://www.filewatcher.com/m/scan_ffs-1.2.tar.bz2.5119-0.html And run it on '''gm0s2''', which took a long time (nearly 8 hours): {{{ root@zg-1 (~) 725# date |
This didn't work as the ports tree was not up to date. Had to download the following distfile by hand '''scan_ffs-1.2.tar.bz2''', which I found at: * http://www.filewatcher.com/m/scan_ffs-1.2.tar.bz2.5119-0.html Run '''scan_ffs''' on '''gm0s2''', which took a long time (nearly 8 hours): {{{ root@zg-1 # date |
| Line 133: | Line 135: |
| root@zg-1 (~) 726# scan_ffs -l /dev/ad2 ; date | root@zg-1 # scan_ffs -l /dev/ad2 ; date |
| Line 144: | Line 146: |
|
Note that vol003 was detected, but not vol002. This indicates that the data in the vol002 and vol003 file system is probably OK. It discovered that the vo |
Notes 1. vol003 was detected, but not vol002 2. The output, according to the scan_ffs man page, is in a format that can be read by the '''bsdlabel''' command This result indicates that at least some the data in vol002 and vol003 file system is probably recoverable. Searching thought my notes I found of the output from the '''bsdlabel''' command, made when I was reconfiguring the vol003 partition (to reformat it and make it a non-encrypted partition). {{{ root@zg-1 # bsdlabel mirror/gm0s1 # /dev/mirror/gm0s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 2097152 0 4.2BSD 0 0 0 b: 8388608 2097152 swap c: 1363148577 0 unused 0 0 # "raw" part, don't edit d: 20971520 10485760 4.2BSD 0 0 0 e: 20971520 31457280 4.2BSD 0 0 0 f: 52428800 52428800 4.2BSD 0 0 0 g: 1258290977 104857600 4.2BSD 0 0 0 # <-- /e/vol001 root@zg-1 # bsdlabel mirror/gm0s2 # /dev/mirror/gm0s2: 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 2543880528 0 unused 0 0 # "raw" part, don't edit d: 1268776960 0 4.2BSD 0 0 0 # <-- /e/vol002 e: 1275103568 1268776960 4.2BSD 0 0 0 # <-- /e/vol003 }}} As mentioned above, the output of the scan_ffs command can be used as input for updating (setting) the disk' '''label'''. This is explained in the following web page: * http://mikuru.tw/wordpress/archives/468 I prepared a text file with the offset and size information for vol002 and vol003, taken from the output of the bsdlabel command shown above: {{{ root@zg-1 # cat /tmp/vol-label d: 1268776960 0 4.2BSD 0 0 0 # /e/vol002 (605 GB) e: 1275103568 1268776960 4.2BSD 0 0 0 # /e/vol003 (614 GB) }}} Update the disk label: {{{ root@zg-1 # bsdlabel -R /dev/mirror/gm0s2 /tmp/vol-label }}} The command completed without error and devices files '''gm0s2d''' and '''gm0s2e''' appeared in the /dev directory * '''gm0s2d''' - vol002 encrypted file system * '''gm0s2e''' - vol003 normal file system Try and mount vol003, but before mounting run a file system check: {{{ root@zg-1 # fsck /dev/mirror/gm0s2e fsck: Could not determine filesystem type }}} This failed. But I tried mounting the file vol003 system anyways. {{{ root@zg-1 # mount /e/vol003 }}} It worked! The file system was there, but empty. Which was correct. Try mounting the encrypted file system vol002: {{{ root@zg-1 # ./mount_gbde.sh vol002 ./mount_gbde.sh: mount point: /e/vol002 ./mount_gbde.sh: device file: gm0s2d ./mount_gbde.sh: attaching: gm0s2d Enter passphrase: ./mount_gbde.sh: found file: /dev/mirror/gm0s2d.bde ./mount_gbde.sh: checking: gm0s2d.bde /dev/mirror/gm0s2d.bde: 338185 files, 269310313 used, 35520820 free (42276 frags, 4434818 blocks, 0.0% fragmentation) ./mount_gbde.sh: mounting: gm0s2d.bde ./mount_gbde.sh: /e/vol002 mounted successfully from gm0s2d.bde Filesystem Size Used Avail Capacity Mounted on /dev/mirror/gm0s2d.bde 581G 514G 62G 89% /e/vol002 }}} Worked, but it took a long time for the file system check to complete (about 20 minutes). '''Lessons learned''': 1. We should probably unmount the vol002 partition by hand before shutting down the system 1. Make sure we have a copy of the output of the '''bsdlabel''' for all disks on all server command saved in a safe place |
FreeBSD Notes
Contents
Restoring Partition Tables
Following occurred on 16-Feb-2012.
This section describes the restoration of the partition table on zg-1.softxs.ch, a FreeBSD 9.0 system located at the Datawire data center in Cham. The partition table was corrupted after the system was shutdown and restarted.
The reason for the restart was that the IP addresses of the machine were changed:
94.231.80.205 --> 94.231.88.100
94.231.80.206 --> 94.231.88.101
Events on zg-1.softxs.ch leading the problem:
- The system was shutdown, without manually un-mounting the encrypted vol002 file system
- After over 60 seconds the power was turned off. We did not have monitor connected and we were unable to see in the system was really stopped
- The system was turned on again
- The system booted but reported a file system error and went into single user mode
- Edited the /etc/fstab and commented out the entry for vol002
- Rebooted the system fron the console
- The system cam up without problems, but the vol002 and vol003 file systems were not mounted
I left the data center and when back on line remotely logged into to zg-1.softxs.ch. Discovered the following:
- vol001 could be mounted
- There were no /dev files present for the vol002 or vol003 file systems
The /etc/fstab contains the following:
cat /etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/mirror/gm0s1b none swap sw 0 0 /dev/mirror/gm0s1a / ufs rw 1 1 /dev/mirror/gm0s1e /tmp ufs rw 2 2 /dev/mirror/gm0s1f /usr ufs rw 2 2 /dev/mirror/gm0s1d /var ufs rw 2 2 /dev/mirror/gm0s1g /e/vol001 ufs rw 2 2 # Note: vol002 is gbde encrypted and must be mounted manually #/dev/mirror/gm0s2d /e/vol002 ufs rw 2 2 /dev/mirror/gm0s2e /e/vol003 ufs rw 2 2 # -- CD-ROM /dev/acd0 /cdrom cd9660 ro,noauto 0 0
Tried to mount the encrypted vol002 file system:
root@zg-1 # ./mount_gbde.sh vol002 /mount_gbde.sh: mount point: /e/vol002 ./mount_gbde.sh: device file: gm0s2d ./mount_gbde.sh: attaching: gm0s2d Enter passphrase: gbde: Attach to mirror/gm0s2d failed: Provider not found: "mirror/gm0s2d" ./mount_gbde.sh: ERROR: file not found: /dev/mirror/gm0s2d.bde
The /dev entry for the file system was missing, as was that for vol003.
root@zg-1 # gpart show => 63 3907029042 mirror/gm0 MBR (1.8T) 63 1363148577 1 freebsd [active] (650G) <--- This is vol001 1363148640 2543880528 2 freebsd [active] (1.2T) <--- Is this the vol002 & vol003?
See what's in /dev
root@zg-1 # ls -1 /dev/mirror/ gm0 gm0s1 <-- has a label that seems OK gm0s1a 0s1b gm0s1d gm0s1e gm0s1f gm0s1g gm0s2 <-- has no label. And no file systems found. This is where vol002 and vol003 should be
The gm0s1 devices are the root, swap, /var, /tmp and /usr file systems. There should also be two gm0s2 device files. But they were missing.
Check the BSD disk labels:
root@zg-1 # bsdlabel /dev/mirror/gm0s1 # /dev/mirror/gm0s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 2097152 0 4.2BSD 0 0 0 b: 8388608 2097152 swap c: 1363148577 0 unused 0 0 # "raw" part, don't edit d: 20971520 10485760 4.2BSD 0 0 0 e: 20971520 31457280 4.2BSD 0 0 0 f: 52428800 52428800 4.2BSD 0 0 0 g: 1258290977 104857600 4.2BSD 0 0 0 root@zg-1 # bsdlabel /dev/mirror/gm0s2 bsdlabel: /dev/mirror/gm0s2: no valid label found
Note that gm0s2 appear to have no partitions defined on it. This is where vol002 and vol003 should be.
Install the scan_ffs to investigate what's there and see if anything can be recovered.
cd /usr/ports/sysutils/scan_ffs make install
This didn't work as the ports tree was not up to date. Had to download the following distfile by hand scan_ffs-1.2.tar.bz2, which I found at:
Run scan_ffs on gm0s2, which took a long time (nearly 8 hours):
root@zg-1 # date Sat Feb 16 00:05:15 CET 2013 root@zg-1 # scan_ffs -l /dev/ad2 ; date X: 2097152 63 4.2BSD 2048 16384 0 # / X: 20971520 10485823 4.2BSD 2048 16384 0 # /var X: 20971520 31457343 4.2BSD 2048 16384 0 # /tmp X: 52428800 52428863 4.2BSD 2048 16384 0 # /usr X: 1258290976 104857663 4.2BSD 2048 16384 0 # /e/vol001 X: 1275100000 2631925600 4.2BSD 2048 16384 0 # /e/vol003 scan_ffs: read: Input/output error Sat Feb 16 07:52:12 CET 2013
Notes
- vol003 was detected, but not vol002
The output, according to the scan_ffs man page, is in a format that can be read by the bsdlabel command
This result indicates that at least some the data in vol002 and vol003 file system is probably recoverable.
Searching thought my notes I found of the output from the bsdlabel command, made when I was reconfiguring the vol003 partition (to reformat it and make it a non-encrypted partition).
root@zg-1 # bsdlabel mirror/gm0s1 # /dev/mirror/gm0s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 2097152 0 4.2BSD 0 0 0 b: 8388608 2097152 swap c: 1363148577 0 unused 0 0 # "raw" part, don't edit d: 20971520 10485760 4.2BSD 0 0 0 e: 20971520 31457280 4.2BSD 0 0 0 f: 52428800 52428800 4.2BSD 0 0 0 g: 1258290977 104857600 4.2BSD 0 0 0 # <-- /e/vol001 root@zg-1 # bsdlabel mirror/gm0s2 # /dev/mirror/gm0s2: 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 2543880528 0 unused 0 0 # "raw" part, don't edit d: 1268776960 0 4.2BSD 0 0 0 # <-- /e/vol002 e: 1275103568 1268776960 4.2BSD 0 0 0 # <-- /e/vol003
As mentioned above, the output of the scan_ffs command can be used as input for updating (setting) the disk' label. This is explained in the following web page:
I prepared a text file with the offset and size information for vol002 and vol003, taken from the output of the bsdlabel command shown above:
root@zg-1 # cat /tmp/vol-label d: 1268776960 0 4.2BSD 0 0 0 # /e/vol002 (605 GB) e: 1275103568 1268776960 4.2BSD 0 0 0 # /e/vol003 (614 GB)
Update the disk label:
root@zg-1 # bsdlabel -R /dev/mirror/gm0s2 /tmp/vol-label
The command completed without error and devices files gm0s2d and gm0s2e appeared in the /dev directory
gm0s2d - vol002 encrypted file system
gm0s2e - vol003 normal file system
Try and mount vol003, but before mounting run a file system check:
root@zg-1 # fsck /dev/mirror/gm0s2e fsck: Could not determine filesystem type
This failed. But I tried mounting the file vol003 system anyways.
root@zg-1 # mount /e/vol003
It worked! The file system was there, but empty. Which was correct.
Try mounting the encrypted file system vol002:
root@zg-1 # ./mount_gbde.sh vol002 ./mount_gbde.sh: mount point: /e/vol002 ./mount_gbde.sh: device file: gm0s2d ./mount_gbde.sh: attaching: gm0s2d Enter passphrase: ./mount_gbde.sh: found file: /dev/mirror/gm0s2d.bde ./mount_gbde.sh: checking: gm0s2d.bde /dev/mirror/gm0s2d.bde: 338185 files, 269310313 used, 35520820 free (42276 frags, 4434818 blocks, 0.0% fragmentation) ./mount_gbde.sh: mounting: gm0s2d.bde ./mount_gbde.sh: /e/vol002 mounted successfully from gm0s2d.bde Filesystem Size Used Avail Capacity Mounted on /dev/mirror/gm0s2d.bde 581G 514G 62G 89% /e/vol002
Worked, but it took a long time for the file system check to complete (about 20 minutes).
Lessons learned:
- We should probably unmount the vol002 partition by hand before shutting down the system
Make sure we have a copy of the output of the bsdlabel for all disks on all server command saved in a safe place