6TB storage expansion
Contents
Introduction
56 instead of 50TB are now available for the bioinformatics cluster. The tasks required to realize this extra space are:
- rescan multipath, details: multipath WWID; 36000d3100051f800000000000000016e (on mpathd)
Tools
listing
multipath -l
will give a useful if not very understandable list of characteristics. Most important, the device: dm-6 is noted, i.e.
mpathd (36000d3100051f800000000000000016e) dm-6 COMPELNT,Compellent Vol size=50T features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=0 status=active |- 6:0:0:1 sdd 8:48 active undef running |- 7:0:1:1 sde 8:64 active undef running |- 6:0:1:1 sdg 8:96 active undef running `- 7:0:2:1 sdf 8:80 active undef running
configuration
/etc/multipath.conf
Doesn't really have much. The only actives are:
defaults {
user_friendly_names yes
}
blacklist {
wwid 3600605b006c5f11019eadc840e589ae9
wwid 3600605b006a1af8019ee9a2d1036bfa7
wwid 3600605b006a1af801b46c4773d54b7d4
}
interpreter
There is also a interpreter available to query status and other things. To fire it up, type
multipathd -k
The
show maps topology
command in this interpeter will give you the same output as the "multipath -l" command. It mayy be necessary to find out more about the tool. For example, this command
paths count
gives 4 as the answer, which probably refers to sdd, sde, sdg, sdf mentioned in the above output. Indeed the
show devices
gives these four as the output. Unfortunately, there is nothign about rescanning here. I think this tool is at a higher-level than what is required for the rescanning, which seems to be a lower level jobs (multipath's understanding of "devices" gives this away somewhat).
Indeed this tool will be useful for resizing, but unfortunately not for rescanning.
SCSI Considerations
There is a way to see the SCSI devices:
cat /proc/scsi/scsi
Output:
Attached devices: Host: scsi8 Channel: 00 Id: 08 Lun: 00 Vendor: AIC CORP Model: SAS 6G Expander Rev: 0b05 Type: Enclosure ANSI SCSI revision: 05 Host: scsi8 Channel: 00 Id: 09 Lun: 00 Vendor: AIC CORP Model: SAS 6G Expander Rev: 0b05 Type: Enclosure ANSI SCSI revision: 05 Host: scsi8 Channel: 00 Id: 10 Lun: 00 Vendor: AIC CORP Model: SAS 6G Expander Rev: 0b05 Type: Enclosure ANSI SCSI revision: 05 Host: scsi8 Channel: 00 Id: 11 Lun: 00 Vendor: AIC CORP Model: SAS 6G Expander Rev: 0b05 Type: Enclosure ANSI SCSI revision: 05 Host: scsi8 Channel: 02 Id: 00 Lun: 00 Vendor: LSI Model: MR9286-8e Rev: 3.29 Type: Direct-Access ANSI SCSI revision: 05 Host: scsi9 Channel: 02 Id: 00 Lun: 00 Vendor: LSI Model: MR9261-8i Rev: 2.13 Type: Direct-Access ANSI SCSI revision: 05 Host: scsi9 Channel: 02 Id: 01 Lun: 00 Vendor: LSI Model: MR9261-8i Rev: 2.13 Type: Direct-Access ANSI SCSI revision: 05 Host: scsi6 Channel: 00 Id: 00 Lun: 01 Vendor: COMPELNT Model: Compellent Vol Rev: 0602 Type: Direct-Access ANSI SCSI revision: 05 Host: scsi7 Channel: 00 Id: 01 Lun: 01 Vendor: COMPELNT Model: Compellent Vol Rev: 0602 Type: Direct-Access ANSI SCSI revision: 05 Host: scsi7 Channel: 00 Id: 02 Lun: 01 Vendor: COMPELNT Model: Compellent Vol Rev: 0602 Type: Direct-Access ANSI SCSI revision: 05 Host: scsi6 Channel: 00 Id: 01 Lun: 01 Vendor: COMPELNT Model: Compellent Vol Rev: 0602 Type: Direct-Access ANSI SCSI revision: 05
There is a relationship to the "multipath -l" output in terms of the vendor and model.
There is also a rescan tool for SCSI
rescan-scsi-bus.sh
Out put is a bit voluminous, but here goes:
Scanning SCSI subsystem for new devices
Scanning host 0 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 1 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 2 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 3 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 4 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 5 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 6 for all SCSI target IDs, all LUNs
sg7 changed: device 6 0 0 1 ...
from:Direct-Access : 01
to: Direct-Access Model: Compellent Vol Rev: 0602
Type: Direct-Access ANSI SCSI revision: 05
sg10 changed: device 6 0 1 1 ...
from:Direct-Access : 01
to: Direct-Access Model: Compellent Vol Rev: 0602
Type: Direct-Access ANSI SCSI revision: 05
Scanning host 7 for all SCSI target IDs, all LUNs
sg8 changed: device 7 0 1 1 ...
from:Direct-Access : 01
to: Direct-Access Model: Compellent Vol Rev: 0602
Type: Direct-Access ANSI SCSI revision: 05
sg9 changed: device 7 0 2 1 ...
from:Direct-Access : 01
to: Direct-Access Model: Compellent Vol Rev: 0602
Type: Direct-Access ANSI SCSI revision: 05
Scanning host 8 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
sg2 changed: device 8 0 10 0 ...
from:Enclosure : 00
to: Enclosure Model: SAS 6G Expander Rev: 0b05
Type: Enclosure ANSI SCSI revision: 05
sg3 changed: device 8 0 11 0 ...
from:Enclosure : 00
to: Enclosure Model: SAS 6G Expander Rev: 0b05
Type: Enclosure ANSI SCSI revision: 05
sg0 changed: device 8 0 8 0 ...
from:Enclosure : 00
to: Enclosure Model: SAS 6G Expander Rev: 0b05
Type: Enclosure ANSI SCSI revision: 05
sg1 changed: device 8 0 9 0 ...
from:Enclosure : 00
to: Enclosure Model: SAS 6G Expander Rev: 0b05
Type: Enclosure ANSI SCSI revision: 05
sg4 changed: device 8 2 0 0 ...
from:Direct-Access : 00
to: Direct-Access Model: MR9286-8e Rev: 3.29
Type: Direct-Access ANSI SCSI revision: 05
Scanning host 9 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
sg5 changed: device 9 2 0 0 ...
from:Direct-Access : 00
to: Direct-Access Model: MR9261-8i Rev: 2.13
Type: Direct-Access ANSI SCSI revision: 05
sg6 changed: device 9 2 1 0 ...
from:Direct-Access : 00
to: Direct-Access Model: MR9261-8i Rev: 2.13
Type: Direct-Access ANSI SCSI revision: 05
0 new or changed device(s) found.
0 remapped or resized device(s) found.
0 device(s) removed.
Despite some mentions of changed devices, the final summary is of no changes.
There is a pattern to the device numbered 6 however, which is always a Compellent volume.
Procedure
The rescan operation is most definitely required. Both Red Hat and Ubuntu guides give the same method of doing this
# echo 1 > /sys/block/device_name/device/rescan
However, under /sys/block/dm-6, there is no device subdirectory, and, needless to say, no rescan file. Mind, this is for SCSI.
Other sites repeat the recipe ad nauseam. There is no mention about whether this can in fact be done live.
Breakthrough
Here is the raw recipe:
# echo 1 > /sys/block/device_name/device/rescan # multipathd -k'resize map mpatha'
The realisation is that the device name is not dm-6, and there is more than one. I.e. the sd{d,e,f,g} mentioned in the "multipath -l" command are the devices. This is clear because each have a device/rescan file which is write-only. Also attempt to create such a path/rescan in dm-6 were fruitless. Therefore 1 wa echo'd to each of the four rescan files in the device subdirectory of /sys/block/sd{d,e,f,g}
