Jw297: Created page with "= Introduction = Expanding purchased storage on /storage/ is actually surprisingly easy. This is all taken from Ramon's expansion of 50Tb to 56Tb (page on the wiki). More det..."

2018-11-21T09:52:18Z

Created page with "= Introduction = Expanding purchased storage on /storage/ is actually surprisingly easy. This is all taken from Ramon's expansion of 50Tb to 56Tb (page on the wiki). More det..."

New page

= Introduction =

Expanding purchased storage on /storage/ is actually surprisingly easy. This is all taken from Ramon's expansion of 50Tb to 56Tb (page on the wiki). More details taken from this page are below, but effectively it's just

echo 1 > /sys/block/sdd/device/rescan
echo 1 > /sys/block/sde/device/rescan
echo 1 > /sys/block/sdf/device/rescan
echo 1 > /sys/block/sdg/device/rescan
multipathd -k'resize map mpathd'
pvresize /dev/mapper/mpathd
lvextend /dev/STORAGE/storage /dev/mapper/mpathd
xfs_growfs /dev/STORAGE/storage

= Tools =

== listing ==
multipath -l

will give a useful if not very understandable list of characteristics. Most important, the device: dm-6 is noted, i.e.

mpathd (36000d3100051f800000000000000016e) dm-6 COMPELNT,Compellent Vol
size=50T features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=0 status=active
|- 6:0:0:1 sdd 8:48 active undef running
|- 7:0:1:1 sde 8:64 active undef running
|- 6:0:1:1 sdg 8:96 active undef running
`- 7:0:2:1 sdf 8:80 active undef running

== configuration ==
/etc/multipath.conf
Doesn't really have much. The only actives are:
defaults {
user_friendly_names yes
}

blacklist {
wwid 3600605b006c5f11019eadc840e589ae9
wwid 3600605b006a1af8019ee9a2d1036bfa7
wwid 3600605b006a1af801b46c4773d54b7d4
}

== interpreter ==

There is also a interpreter available to query status and other things. To fire it up, type
multipathd -k

The
show maps topology

command in this interpeter will give you the same output as the "multipath -l" command. It mayy be necessary to find out more about the tool. For example, this command

paths count

gives 4 as the answer, which probably refers to sdd, sde, sdg, sdf mentioned in the above output. Indeed the
show devices

gives these four as the output. Unfortunately, there is nothign about rescanning here. I think this tool is at a higher-level than what is required for the rescanning, which seems to be a lower level jobs (multipath's understanding of "devices" gives this away somewhat).

Indeed this tool will be useful for resizing, but unfortunately not for rescanning.

==SCSI Considerations ==
There is a way to see the SCSI devices:

cat /proc/scsi/scsi

Output:
Attached devices:
Host: scsi8 Channel: 00 Id: 08 Lun: 00
Vendor: AIC CORP Model: SAS 6G Expander Rev: 0b05
Type: Enclosure ANSI SCSI revision: 05
Host: scsi8 Channel: 00 Id: 09 Lun: 00
Vendor: AIC CORP Model: SAS 6G Expander Rev: 0b05
Type: Enclosure ANSI SCSI revision: 05
Host: scsi8 Channel: 00 Id: 10 Lun: 00
Vendor: AIC CORP Model: SAS 6G Expander Rev: 0b05
Type: Enclosure ANSI SCSI revision: 05
Host: scsi8 Channel: 00 Id: 11 Lun: 00
Vendor: AIC CORP Model: SAS 6G Expander Rev: 0b05
Type: Enclosure ANSI SCSI revision: 05
Host: scsi8 Channel: 02 Id: 00 Lun: 00
Vendor: LSI Model: MR9286-8e Rev: 3.29
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi9 Channel: 02 Id: 00 Lun: 00
Vendor: LSI Model: MR9261-8i Rev: 2.13
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi9 Channel: 02 Id: 01 Lun: 00
Vendor: LSI Model: MR9261-8i Rev: 2.13
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi6 Channel: 00 Id: 00 Lun: 01
Vendor: COMPELNT Model: Compellent Vol Rev: 0602
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi7 Channel: 00 Id: 01 Lun: 01
Vendor: COMPELNT Model: Compellent Vol Rev: 0602
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi7 Channel: 00 Id: 02 Lun: 01
Vendor: COMPELNT Model: Compellent Vol Rev: 0602
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi6 Channel: 00 Id: 01 Lun: 01
Vendor: COMPELNT Model: Compellent Vol Rev: 0602
Type: Direct-Access ANSI SCSI revision: 05

There is a relationship to the "multipath -l" output in terms of the vendor and model.

There is also a rescan tool for SCSI

rescan-scsi-bus.sh

Out put is a bit voluminous, but here goes:

Scanning SCSI subsystem for new devices
Scanning host 0 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 1 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 2 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 3 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 4 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 5 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 6 for all SCSI target IDs, all LUNs

sg7 changed: device 6 0 0 1 ...
from:Direct-Access : 01
to: Direct-Access Model: Compellent Vol Rev: 0602
Type: Direct-Access ANSI SCSI revision: 05

sg10 changed: device 6 0 1 1 ...
from:Direct-Access : 01
to: Direct-Access Model: Compellent Vol Rev: 0602
Type: Direct-Access ANSI SCSI revision: 05

Scanning host 7 for all SCSI target IDs, all LUNs
sg8 changed: device 7 0 1 1 ...
from:Direct-Access : 01
to: Direct-Access Model: Compellent Vol Rev: 0602
Type: Direct-Access ANSI SCSI revision: 05

sg9 changed: device 7 0 2 1 ...
from:Direct-Access : 01
to: Direct-Access Model: Compellent Vol Rev: 0602
Type: Direct-Access ANSI SCSI revision: 05

Scanning host 8 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
sg2 changed: device 8 0 10 0 ...
from:Enclosure : 00
to: Enclosure Model: SAS 6G Expander Rev: 0b05
Type: Enclosure ANSI SCSI revision: 05

sg3 changed: device 8 0 11 0 ...
from:Enclosure : 00
to: Enclosure Model: SAS 6G Expander Rev: 0b05
Type: Enclosure ANSI SCSI revision: 05

sg0 changed: device 8 0 8 0 ...
from:Enclosure : 00
to: Enclosure Model: SAS 6G Expander Rev: 0b05
Type: Enclosure ANSI SCSI revision: 05

sg1 changed: device 8 0 9 0 ...
from:Enclosure : 00
to: Enclosure Model: SAS 6G Expander Rev: 0b05
Type: Enclosure ANSI SCSI revision: 05

sg4 changed: device 8 2 0 0 ...
from:Direct-Access : 00
to: Direct-Access Model: MR9286-8e Rev: 3.29
Type: Direct-Access ANSI SCSI revision: 05

Scanning host 9 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
sg5 changed: device 9 2 0 0 ...
from:Direct-Access : 00
to: Direct-Access Model: MR9261-8i Rev: 2.13
Type: Direct-Access ANSI SCSI revision: 05

sg6 changed: device 9 2 1 0 ...
from:Direct-Access : 00
to: Direct-Access Model: MR9261-8i Rev: 2.13
Type: Direct-Access ANSI SCSI revision: 05

0 new or changed device(s) found.
0 remapped or resized device(s) found.
0 device(s) removed.

Despite some mentions of changed devices, the final summary is of no changes.

There is a pattern to the device numbered 6 however, which is always a Compellent volume.

= Procedure =

The rescan operation is most definitely required. Both Red Hat and Ubuntu guides give the same method of doing this
# echo 1 > /sys/block/device_name/device/rescan

However, under /sys/block/dm-6, there is no device subdirectory, and, needless to say, no rescan file. Mind, this is for SCSI.

Other sites repeat the recipe ad nauseam. There is no mention about whether this can in fact be done live.

== Breakthrough ==

Here is the raw recipe:
# echo 1 > /sys/block/<device_name>/device/rescan
# multipathd -k'resize map mpatha'

The realisation is that the device name is not '''dm-6''', and there could be more than one device. Namely, the sd{d,e,f,g} mentioned in the "multipath -l" command are the referred-to ''devices''. This becomes clear when we see that each have a '''device/rescan''' file which is write-only (we noted above that dm-6 has no such directory nor file). Also, attempts to create such a path/rescan directory and file in '''/sys/block/dm-6''' were fruitless. Therefore '''1''' wa echo'd to each of the four '''rescan''' files in the '''device''' subdirectory of '''/sys/block/sd{d,e,f,g}'''

This was accepted. Then came the resize command in the multipath interpreter. This returned a "fail". This is because '''mpatha''' is a placeholder (recipe was mute about this) and that it should be replaced with the target map in your own configuration. In the interpreter
show maps
will offer the name of the map whihc is marvin's case was '''mapthd'''. So, to resize, the command really is

multipathd -k'resize map mpathd'

This returned a positive message, and finally "multipath -l" gave the new size as 56T, hurrah.

The next thing that has to be done is extend the logical volumes and grow the XFS.

== LVM ==
A quick '''pvscan''' will reveal that the '''/dev/mapper/mpathd''' device has no free space on it. So the extra 6TB has not been recognised? Well, it's because the PV needs to be resized. This can be done with:

pvresize /dev/mapper/mpathd

This will show how one PV was resized and 0 were not changed (unusually phrased message), and a quick pvscan will show that the PV now has 6TB extra space. Next up is the logical volume .. this must be extended into newly resized PV. This is done with the follwong command:

lvextend /dev/STORAGE/storage /dev/mapper/mpathd

== XFS ==

xfs is very easy to resize and can famously be done while filesystem is already mounted.

xfs_growfs /dev/STORAGE/storage

This command returned extremely quickly and showed that the filesystem had been resize from 50T to 56T. Mission accomplished!

= Summary of successful operation =

echo 1 > /sys/block/sdd/device/rescan
echo 1 > /sys/block/sde/device/rescan
echo 1 > /sys/block/sdf/device/rescan
echo 1 > /sys/block/sdg/device/rescan
multipathd -k'resize map mpathd'
pvresize /dev/mapper/mpathd
lvextend /dev/STORAGE/storage /dev/mapper/mpathd
xfs_growfs /dev/STORAGE/storage

= Links=

Both of these are almost the same, but clearly the RedHat was hte preferred reference.

* [https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/DM_Multipath/ Red Hat DM-Multipath guide]
* [https://help.ubuntu.com/lts/serverguide/multipath-admin-and-troubleshooting.html ubuntu troubleshooting guide to DM-Multipath]

This turned out to be not so relevant, included here for interest's sake:

* [http://kb.kristianreese.com/index.php?View=entry&EntryID=127 Kristian Reese's LUN solution]

Expanding Storage - Revision history

Jw297: Created page with "= Introduction = Expanding purchased storage on /storage/ is actually surprisingly easy. This is all taken from Ramon's expansion of 50Tb to 56Tb (page on the wiki). More det..."