UPDATE: Feb 22, 2008
OK guys, I am using a USB stick on my Sun V240 , to contain a metadb, for the disksuite.
the reason is simple. if one drive craps out, then I can always boot, cause there is always more than 50% of the state replicas available
(of course we will test this out here !)
check this out, I got myself a nice Sony Tiny Vault 2gb (not necessary, but green is more fitting than pink, if I was a female administrator, I would have gone for the 1gb one,hehe )
Now, I plug the thing into the box, and I notice in /var/adm/messages
Feb 22 14:29:16 multi02 usba: [ID 912658 kern.info] USB 2.0 device (usb54c,2a5) operating at full speed (USB 1.x) on USB 1.10 root hub: storage@1, scsa2usb0
at bus address 2
Feb 22 14:29:16 multi02 usba: [ID 349649 kern.info] Sony torage Media 1A07110908407
Feb 22 14:29:16 multi02 genunix: [ID 936769 kern.info] scsa2usb0 is /pci@1e,600000/usb@a/storage@1
Feb 22 14:29:16 multi02 genunix: [ID 408114 kern.info] /pci@1e,600000/usb@a/storage@1 (scsa2usb0) online
Feb 22 14:29:16 multi02 scsi: [ID 193665 kern.info] sd2 at scsa2usb0: target 0 lun 0
Feb 22 14:29:16 multi02 genunix: [ID 936769 kern.info] sd2 is /pci@1e,600000/usb@a/storage@1/disk@0,0
Feb 22 14:29:16 multi02 scsi: [ID 107833 kern.warning] WARNING: /pci@1e,600000/usb@a/storage@1/disk@0,0 (sd2):
Feb 22 14:29:16 multi02 Corrupt label - bad geometry
Feb 22 14:29:16 multi02 scsi: [ID 107833 kern.notice] Label says 4155680 blocks; Drive says 3963904 blocks
Feb 22 14:29:16 multi02 genunix: [ID 408114 kern.info] /pci@1e,600000/usb@a/storage@1/disk@0,0 (sd2) online
Now, I do, svcs | grep vol, and sure enough, volfs is running, so lets shut it off
svcadm disable svc:/system/filesystem/volfs:default
Now, I execute rmformat
and I see:
(wait a bit, it takes a while to get the usb params)
bash-3.00# rmformat
Looking for devices...
1. Logical Node: /dev/rdsk/c0t0d0s2
Physical Node: /pci@1e,600000/ide@d/sd@0,0
Connected Device: TSSTcorp CD/DVDW TS-L632D SI03
Device Type: DVD Reader/Writer
2. Logical Node: /dev/rdsk/c3t0d0s2
Physical Node: /pci@1e,600000/usb@a/storage@1/disk@0,0
Connected Device: Sony Storage Media 0100
Device Type: Removable
Then, I type:
bash-3.00# fdisk /dev/rdsk/c3t0d0s2
No fdisk table exists. The default partition for the disk is:
a 100% "SOLARIS System" partition
Type "y" to accept the default partition, otherwise type "n" to edit the
partition table.
y
bash-3.00#
Now, lets see if I can actually see the partitioning
nope imposs to actually see the internal partitioning, so I decide I will just use the overlap (s2), as my actual file system
lets see if it works:
bash-3.00# newfs -v /dev/rdsk/c3t0d0s2
newfs: construct a new file system /dev/rdsk/c3t0d0s2: (y/n)? y
mkfs -F ufs /dev/rdsk/c3t0d0s2 3951990 63 255 8192 1024 32 3 3 4096 t 0 -1 8 16 n
With a block size of 8192 minimum bytes per inode is 5884
Minimum bytes per inode is 5884
With 16065 sectors per cylinder, minimum cylinders per group is 16
This requires the fragment size to be changed from 1024 to 4096
Please re-run mkfs with corrected parameters
Nope, ok, lets fix this (fragment size 4096 then):
bash-3.00# newfs -v -f 4096 /dev/rdsk/c3t0d0s2
newfs: construct a new file system /dev/rdsk/c3t0d0s2: (y/n)? y
mkfs -F ufs /dev/rdsk/c3t0d0s2 3951990 63 255 8192 4096 32 3 3 4096 t 0 -1 8 16 n
With a block size of 8192 minimum bytes per inode is 5884
Minimum bytes per inode is 5884
Unable to make filesystem fit with the given constraints
Please re-run mkfs with corrected parameters
Nope, not yet, lets see, I bet its the number of inodes, it wants to make 4096 which is way small
bash-3.00# newfs -v -f 4096 -i 20000 /dev/rdsk/c3t0d0s2
newfs: construct a new file system /dev/rdsk/c3t0d0s2: (y/n)? y
mkfs -F ufs /dev/rdsk/c3t0d0s2 3951990 63 255 8192 4096 32 3 3 20000 t 0 -1 8 16 n
Warning: insufficient space in super block for
rotational layout tables with nsect 63, ntrack 255, and nrpos 8.
Omitting tables - file system performance may be impaired.
Warning: 6 sector(s) in last cylinder unallocated
/dev/rdsk/c3t0d0s2: 3951984 sectors in 246 cylinders of 255 tracks, 63 sectors
1929.7MB in 16 cyl groups (16 c/g, 125.51MB/g, 6464 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
32, 257136, 514240, 771344, 1028448, 1285552, 1542656, 1799760, 2056864,
2313968,
2571072, 2828176, 3085280, 3342384, 3599488, 3856592
SUCCESS.
lets see if I can mount it!
bash-3.00# mkdir /crap
bash-3.00# mount /dev/dsk/c3t0d0s2 /crap
df -k
/dev/dsk/c3t0d0s2 1962788 1948 1901960 1% /crap
WHOA, super.
ok, I will now unmount, and add 2 instances of the state database replicas on that partition
umount /crap
bash-3.00# metadb
flags first blk block count
a m p luo 16 8192 /dev/dsk/c1t0d0s5
a p luo 8208 8192 /dev/dsk/c1t0d0s5
a p luo 16 8192 /dev/dsk/c1t1d0s5
a p luo 8208 8192 /dev/dsk/c1t1d0s5
a p luo 16 8192 /dev/dsk/c1t0d0s6
a p luo 8208 8192 /dev/dsk/c1t0d0s6
a p luo 16 8192 /dev/dsk/c1t1d0s6
a p luo 8208 8192 /dev/dsk/c1t1d0s6
bash-3.00# metadb -a -c 2 /dev/dsk/c3t0d0s2
bash-3.00# metadb
flags first blk block count
a m p luo 16 8192 /dev/dsk/c1t0d0s5
a p luo 8208 8192 /dev/dsk/c1t0d0s5
a p luo 16 8192 /dev/dsk/c1t1d0s5
a p luo 8208 8192 /dev/dsk/c1t1d0s5
a p luo 16 8192 /dev/dsk/c1t0d0s6
a p luo 8208 8192 /dev/dsk/c1t0d0s6
a p luo 16 8192 /dev/dsk/c1t1d0s6
a p luo 8208 8192 /dev/dsk/c1t1d0s6
a u r 16 8192 /dev/dsk/c3t0d0s2
a u r 8208 8192 /dev/dsk/c3t0d0s2
Fantastic, lets reboot the box now. for a first test:
bash-3.00# reboot
machine came back, all looking good:
bash-3.00# metadb
flags first blk block count
a m p luo 16 8192 /dev/dsk/c1t0d0s5
a p luo 8208 8192 /dev/dsk/c1t0d0s5
a p luo 16 8192 /dev/dsk/c1t1d0s5
a p luo 8208 8192 /dev/dsk/c1t1d0s5
a p luo 16 8192 /dev/dsk/c1t0d0s6
a p luo 8208 8192 /dev/dsk/c1t0d0s6
a p luo 16 8192 /dev/dsk/c1t1d0s6
a p luo 8208 8192 /dev/dsk/c1t1d0s6
a p luo r 16 8192 /dev/dsk/c3t0d0s2
a p luo r 8208 8192 /dev/dsk/c3t0d0s2
the beasty is there.
Now, lets try to kak out a drive, and boot the machine !!!!
I will take out one drive (im too chicken to try disk 0, so lets try disk1), when machine is down, and boot it
as you can see I am mirroring everythin, including the swap partition.
bash-3.00# metastat
d30: Mirror
Submirror 0: d31
State: Okay
Submirror 1: d32
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 14348160 blocks (6.8 GB)
d31: Submirror of d30
State: Okay
Size: 14348160 blocks (6.8 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t0d0s4 0 No Okay Yes
d32: Submirror of d30
State: Okay
Size: 14348160 blocks (6.8 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t1d0s4 0 No Okay Yes
d20: Mirror
Submirror 0: d21
State: Okay
Submirror 1: d22
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 7184256 blocks (3.4 GB)
d21: Submirror of d20
State: Okay
Size: 7184256 blocks (3.4 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t0d0s3 0 No Okay Yes
d22: Submirror of d20
State: Okay
Size: 7184256 blocks (3.4 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t1d0s3 0 No Okay Yes
d90: Mirror
Submirror 0: d91
State: Okay
Submirror 1: d92
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 16403712 blocks (7.8 GB)
d91: Submirror of d90
State: Okay
Size: 16403712 blocks (7.8 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t0d0s1 0 No Okay Yes
d92: Submirror of d90
State: Okay
Size: 16403712 blocks (7.8 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t1d0s1 0 No Okay Yes
d10: Mirror
Submirror 0: d11
State: Okay
Submirror 1: d12
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 8201856 blocks (3.9 GB)
d11: Submirror of d10
State: Okay
Size: 8201856 blocks (3.9 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t0d0s0 0 No Okay Yes
d12: Submirror of d10
State: Okay
Size: 8201856 blocks (3.9 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t1d0s0 0 No Okay Yes
d40: Mirror
Submirror 0: d41
State: Okay
Submirror 1: d42
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 240479232 blocks (114 GB)
d41: Submirror of d40
State: Okay
Size: 240479232 blocks (114 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t0d0s7 0 No Okay Yes
d42: Submirror of d40
State: Okay
Size: 240479232 blocks (114 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t1d0s7 0 No Okay Yes
Device Relocation Information:
Device Reloc Device ID
c1t1d0 Yes id1,sd@SFUJITSU_MAX3147NCSUN146G000723G03F5C____DN20P7603F5C
c1t0d0 Yes id1,sd@SFUJITSU_MAX3147NCSUN146G000725G03KV4____DN20P7603KV4
-------------------
ok, time for some damage !
bash-3.00# init 0
--------------------
WOOOOHOOOOOOOO HALLELUYAHH !!
Rebooting with command: boot
Boot device: /pci@1c,600000/scsi@2/disk@0,0:a File and args:
SunOS Release 5.10 Version Generic_118833-36 64-bit
Copyright 1983-2006 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
NOTICE: mddb: unable to get devid for 'sd', 0x12
NOTICE: mddb: unable to get devid for 'sd', 0x12
WARNING: md: d12: (Unavailable) needs maintenance
Hardware watchdog enabled
Hostname: multi02
WARNING: md: d32: (Unavailable) needs maintenance
WARNING: md: d92: (Unavailable) needs maintenance
WARNING: md: d22: (Unavailable) needs maintenance
WARNING: md: d42: (Unavailable) needs maintenance
checking ufs filesystems
/dev/md/rdsk/d40: is logging.
multi02 console login:
Now lets see:
bash-3.00# metadb
flags first blk block count
a m p luo 16 8192 /dev/dsk/c1t0d0s5
a p luo 8208 8192 /dev/dsk/c1t0d0s5
M p lu 16 unknown /dev/dsk/c1t1d0s5
M p lu 8208 unknown /dev/dsk/c1t1d0s5
a p luo 16 8192 /dev/dsk/c1t0d0s6
a p luo 8208 8192 /dev/dsk/c1t0d0s6
M p lu 16 unknown /dev/dsk/c1t1d0s6
M p lu 8208 unknown /dev/dsk/c1t1d0s6
a p luo r 16 8192 /dev/dsk/c3t0d0s2
a p luo r 8208 8192 /dev/dsk/c3t0d0s2
bash-3.00# metastat
d30: Mirror
Submirror 0: d31
State: Okay
Submirror 1: d32
State: Needs maintenance
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 14348160 blocks (6.8 GB)
d31: Submirror of d30
State: Okay
Size: 14348160 blocks (6.8 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t0d0s4 0 No Okay Yes
d32: Submirror of d30
State: Needs maintenance
Invoke: metareplace d30 c1t1d0s4
Size: 14348160 blocks (6.8 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t1d0s4 0 No Maintenance Yes
d20: Mirror
Submirror 0: d21
State: Okay
Submirror 1: d22
State: Needs maintenance
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 7184256 blocks (3.4 GB)
d21: Submirror of d20
State: Okay
Size: 7184256 blocks (3.4 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t0d0s3 0 No Okay Yes
d22: Submirror of d20
State: Needs maintenance
Invoke: metareplace d20 c1t1d0s3
Size: 7184256 blocks (3.4 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t1d0s3 0 No Maintenance Yes
d90: Mirror
Submirror 0: d91
State: Okay
Submirror 1: d92
State: Needs maintenance
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 16403712 blocks (7.8 GB)
d91: Submirror of d90
State: Okay
Size: 16403712 blocks (7.8 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t0d0s1 0 No Okay Yes
d92: Submirror of d90
State: Needs maintenance
Invoke: metareplace d90 c1t1d0s1
Size: 16403712 blocks (7.8 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t1d0s1 0 No Maintenance Yes
d10: Mirror
Submirror 0: d11
State: Okay
Submirror 1: d12
State: Needs maintenance
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 8201856 blocks (3.9 GB)
d11: Submirror of d10
State: Okay
Size: 8201856 blocks (3.9 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t0d0s0 0 No Okay Yes
d12: Submirror of d10
State: Needs maintenance
Invoke: metareplace d10 c1t1d0s0
Size: 8201856 blocks (3.9 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t1d0s0 0 No Maintenance Yes
d40: Mirror
Submirror 0: d41
State: Okay
Submirror 1: d42
State: Needs maintenance
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 240479232 blocks (114 GB)
d41: Submirror of d40
State: Okay
Size: 240479232 blocks (114 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t0d0s7 0 No Okay Yes
d42: Submirror of d40
State: Needs maintenance
Invoke: metareplace d40 c1t1d0s7
Size: 240479232 blocks (114 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t1d0s7 0 No Maintenance Yes
Device Relocation Information:
Device Reloc Device ID
c1t0d0 Yes id1,sd@SFUJITSU_MAX3147NCSUN146G000725G03KV4____DN20P7603KV4
----------
ok, I will now insert the drive I took out, and see what happens.....
Feb 22 16:02:57 multi02 rmclomv: DISK @ HDD1 has been inserted.
SC Alert: DISK @ HDD1 has been inserted.
OK, now
lets start with one partition:
bash-3.00# metareplace -e d40 c1t1d0s7
d40: device c1t1d0s7 is enabled
bash-3.00# metastat d40
d40: Mirror
Submirror 0: d41
State: Okay
Submirror 1: d42
State: Resyncing
Resync in progress: 0 % done
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 240479232 blocks (114 GB)
d41: Submirror of d40
State: Okay
Size: 240479232 blocks (114 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t0d0s7 0 No Okay Yes
d42: Submirror of d40
State: Resyncing
Size: 240479232 blocks (114 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c1t1d0s7 0 No Resyncing Yes
Device Relocation Information:
Device Reloc Device ID
c1t0d0 Yes id1,sd@SFUJITSU_MAX3147NCSUN146G000725G03KV4____DN20P7603KV4
c1t1d0 Yes id1,sd@SFUJITSU_MAX3147NCSUN146G000723G03F5C____DN20P7603F5C
bash-3.00# metadb
flags first blk block count
a m p luo 16 8192 /dev/dsk/c1t0d0s5
a p luo 8208 8192 /dev/dsk/c1t0d0s5
M p lu 16 unknown /dev/dsk/c1t1d0s5
M p lu 8208 unknown /dev/dsk/c1t1d0s5
a p luo 16 8192 /dev/dsk/c1t0d0s6
a p luo 8208 8192 /dev/dsk/c1t0d0s6
M p lu 16 unknown /dev/dsk/c1t1d0s6
M p lu 8208 unknown /dev/dsk/c1t1d0s6
a p luo r 16 8192 /dev/dsk/c3t0d0s2
a p luo r 8208 8192 /dev/dsk/c3t0d0s2
bash-3.00# metadb -d /dev/dsk/c1t1d0s5
bash-3.00# metadb -a -c 2 /dev/dsk/c1t1d0s5
bash-3.00# metadb -d /dev/dsk/c1t1d0s6
bash-3.00# metadb -a -c 2 /dev/dsk/c1t1d0s6
bash-3.00# metadb
flags first blk block count
a m p luo 16 8192 /dev/dsk/c1t0d0s5
a p luo 8208 8192 /dev/dsk/c1t0d0s5
a u 16 8192 /dev/dsk/c1t1d0s5
a u 8208 8192 /dev/dsk/c1t1d0s5
a p luo 16 8192 /dev/dsk/c1t0d0s6
a p luo 8208 8192 /dev/dsk/c1t0d0s6
a u 16 8192 /dev/dsk/c1t1d0s6
a u 8208 8192 /dev/dsk/c1t1d0s6
a p luo r 16 8192 /dev/dsk/c3t0d0s2
a p luo r 8208 8192 /dev/dsk/c3t0d0s2
------
I will continue to sync them all.
Maybe tomorrow after some beers I get enough courage to take out the 0 drive !!!, I will post here when I get to it.