RHEL/CentOS Kickstart partition scheme with disk by-path IDs and automation via PXE

So, for several weeks now, I’ve been struggling with disk naming and UDEV as they relate to RHEL or CentOS Kickstart automation via PXE. Basically, what used to be a relatively straight-forward process of setting up partitions based on disk name (e.g. /dev/sda for the primary disk, /dev/sdb for the secondary disk, and so on) has become a little more complicated due to the order in which the kernel may identify disks with UDEV implementations. The originally-used naming conventions of /dev/sdX may no longer be consistent across reboots or across machines. How this problem came to my attention was when I was Kickstarting a bunch of hosts, and realised that several of them were indicating that there wasn’t enough space on the drives that I had referenced. After doing some digging, I realised that some servers were identifying KVM-based media (like via a Cisco UCS CIMC interface), or other USB media as primary disks instead of the actual primary SCSI-based disks (SATA or SAS drives).

Though I agree that identifying disks by their path, their UUID, or an otherwise more permanent name is preferred to the ambiguous /dev/sdX naming scheme, it did cause a problem, and one that didn’t have a readily-identifiable workaround for my situation. My first thought was to use UUID, or gather the disk ID from /dev/disk/by-id/. However, that wouldn’t work because it wouldn’t be the same across all servers, so automatic installations via PXE wouldn’t be feasible. Next, I looked at referencing disks by their PCI location, which can be found within /dev/disk/by-path/. Unfortunately, I found that the PCI location might vary between servers as well. For example, I found:

Server 1:
/dev/disk/by-path/pci-0000:82:00.0-scsi-0:2:0:0 –> primary SCSI disk
/dev/disk/by-path/pci-0000:82:00.0-scsi-0:2:1:0 –> secondary SCSI disk

Server 2:
/dev/disk/by-path/pci-0000:0b:00.0-scsi-0:2:0:0 –> primary SCSI disk
/dev/disk/by-path/pci-0000:0b:00.0-scsi-0:2:1:0 –> secondary SCSI disk

That being said, I did notice one commonality between the servers that I used for testing. The primary and secondary disks (which, by the way, are RAID arrays attached to the same controller) all ended with ‘scsi-0:2:0:0 ‘ for the primary disk, and ‘scsi-0:2:1:0 ‘ for the secondary disk. Thinking that I could possibly just specify using a wildcard, I tried:

part /boot --fstype ext2 --size=100 --ondisk=disk/by-path/*scsi-0:2:0:0

but alas, that caused Anaconda to error out stating that the specified drive did not exist. At this point, I thought that all hope was lost in terms of automation. Then, however, I had a flash of genius (they don’t happen all that often). I could probably gather the full path ID in a pre-installation script, and then get the FULL path ID into the Kickstart configuration file with an include. Here are the code snippets that I wrote:


%pre --interpreter=/bin/bash
## Get the full by-path ID of the primary and secondary disks
osdisk=$(ls -lh /dev/disk/by-path/ | grep 'scsi-0:2:0:0 ' | awk '{print $9}')
datadisk=$(ls -lh /dev/disk/by-path/ | grep 'scsi-0:2:1:0 ' | awk '{print $9}')


## Create a temporary file with the partition scheme to be included in the KS config
echo "# Partitioning scheme" > /tmp/partition_layout
echo "clearpart --all" >> /tmp/partition_layout
echo "zerombr" >> /tmp/partition_layout
echo "part /boot --fstype ext2 --size=100 --ondisk=disk/by-path/$osdisk" >> /tmp/partition_layout
echo "part swap --recommended --ondisk=disk/by-path/$osdisk" >> /tmp/partition_layout
echo "part pv.01 --size=100 --grow --ondisk=disk/by-path/$osdisk" >> /tmp/partition_layout
echo "part pv.02 --size=100 --grow --ondisk=disk/by-path/$datadisk" >> /tmp/partition_layout
echo "volgroup vg_os pv.01" >> /tmp/partition_layout
echo "volgroup vg_data pv.02" >> /tmp/partition_layout
echo "logvol / --vgname=vg_os --size=100 --name=lv_os --fstype=ext4 --grow" >> /tmp/partition_layout
echo "logvol /data --vgname=vg_data --size=100 --name=lv_data --fstype=ext4 --grow" >> /tmp/partition_layout
echo "# Bootloader location" >> /tmp/partition_layout
echo "bootloader --location=mbr --driveorder=/dev/disk/by-path/$osdisk,/dev/disk/by-path/$datadisk --append=\"rhgb\"" >> /tmp/partition_layout
%end

Some notes about the pre-installation script would be: 1) yes, I know that it is a bit clumsy, but it functions despite the lack of elegance; 2) in lines 3 and 4, make sure that the grep statements include a final space. The final space (e.g. grep 'scsi-0:2:0:0 ' instead of just grep 'scsi-0:2:0:0') is important because the machine may have partitions already set up, and in that case, those partitions would be identified as '*scsi-0:2:0:0-part#' where # is the partition number.

So, this pre-installation script generates a file called /tmp/partition_layout that essentially has a normal partition scheme for a Kickstart configuration file, but references the primary and secondary SCSI disks (again, as RAID arrays attached to the same controller) by their full ‘by-path’ IDs. Then, the key is to include that file within the Kickstart configuration via:


# Partitioning scheme information gathered from pre-install script below
%include /tmp/partition_layout

I am happy to report that for the servers that I have tested thus far, this method works. Is it possible that there will be primary and secondary disks that are identified via different path IDs not caught by my pre-installation script? Yes, that is possible. It’s also possible that your situation will require a completely different pre-installation script to gather the unique identifier of your choice. That being said, I hope that this post will help you devise your method of reliably identifying disks so that you can create the partition scheme automatically via your Kickstart configuration file. That way, you can then deploy it to many servers via PXE (or any other implementation that you’re currently utilising for mass deployment).

Cheers,
Zach

15 comments

Skip to comment form

    • Miro on Friday, 10 December 2021 at 06:01
    • Reply

    Hey, not that it would be a big deal but I think you might have a typo in Server 1/2 quote blocks – there is “scsi-0:2:0:0” for both (primary and secondary) disk cases. If I understood it correctly, then for the secondary disk there should be something like “scsi-0:2:1:0” instead. But it was a great article and I am sure it helps/ed a lot of people nevertheless.

      • Zach on Friday, 10 December 2021 at 10:43
        Author
      • Reply

      Hello Miro,

      You’re definitely right about my typo, and thank you for bringing it to my attention! In SCSI notation, the fields are H:B:T:L which stands for HOST:BUS:TARGET:LUN. As such, the third field (TARGET) would be an individual disk attached to the particular SCSI bus. Again, good catch and thank you for making me aware of it. I have updated the post.

      Cheers,
      Zach

    • Gadi on Sunday, 27 January 2019 at 13:56
    • Reply

    Great article, Zach! Very helpful.

    One question though – does this script succeeded to keep the /dev naming scheme ? I’ve tried a few similar scripts in the past, that solved me the issue of the first disk, but once the installation completed, the OS disk was assigned with a different /dev name(not /dev/sda).

    Thanks

      • Zach on Monday, 28 January 2019 at 10:33
        Author
      • Reply

      Hello Gadi,

      I’m glad that you found the article helpful. If you’re calling the disk by path ID, then the name under /dev shouldn’t be referenced. The reason that my script uses path IDs is to avoid name changes. It will, of course, depend on the distribution and disk configuration mechanism, but I think that you’ll likely be safe referencing by path ID.

      Cheers,
      Zach

  1. Zach,

    Really helpful example and forum posts. I found that I could use it for kvm to set the device by UUID in my ks.cfg.


    rrd=$(ls -l /dev/disk/by-uuid/|grep vdb|awk '{print $9}')
    cat <>/etc/fstab
    UUID=${rrd} /var/lib/rrd xfs defaults,noatime,inode64,delaylog 0 0
    EOF

    For reference, CentOS 7 in kvm, the /dev/disk directories looked like this.:


    # ls -l /dev/disk/by-path/
    total 0
    lrwxrwxrwx 1 root root 9 Jan 4 16:24 virtio-pci-0000:00:07.0 -> ../../vda
    lrwxrwxrwx 1 root root 10 Jan 4 16:24 virtio-pci-0000:00:07.0-part1 -> ../../vda1
    lrwxrwxrwx 1 root root 9 Jan 4 16:24 virtio-pci-0000:00:09.0 -> ../../vdb
    # ls -l /dev/disk/by-uuid/
    total 0
    lrwxrwxrwx 1 root root 9 Jan 4 16:24 edffbe64-93f3-444c-be00-8b881b56ff6a -> ../../vdb
    lrwxrwxrwx 1 root root 10 Jan 4 16:24 f4ff231e-9e95-4537-952f-cd14071bff1a -> ../../vda1
    #

      • Zach on Thursday, 4 January 2018 at 21:39
        Author
      • Reply

      Hi Derek,

      I’m glad that you found the article helpful! Thank you for your contributions regarding CentOS 7 as well. I have avoided the RHEL 7.x derivatives due to systemd, but it’s good to see that it’s not completely different. 🙂

      Cheers,
      Zach

    • Steven Langlois on Monday, 18 September 2017 at 15:01
    • Reply

    When I tried this method I got an error about the drive not being partitioned. I have a server with 3 SSD drives and it is UEFI.

    [root@localhost by-path]# ls -l
    total 0
    lrwxrwxrwx 1 root root 9 Sep 18 07:45 pci-0000:00:1f.2-ata-1.0 -> ../../sda
    lrwxrwxrwx 1 root root 9 Sep 18 07:45 pci-0000:00:1f.2-ata-2.0 -> ../../sdb
    lrwxrwxrwx 1 root root 9 Sep 18 07:45 pci-0000:00:1f.2-ata-3.0 -> ../../sdc

    For me, everything is on the osdisk so I modified the pre script to look like:

    %pre –interpreter=/bin/bash
    ## Get the full by-path ID of the primary and secondary disks
    osdisk=$(ls -lh /dev/disk/by-path/ | grep ‘ata-1.0 ‘ | awk ‘{print $9}’)

    ## Create a temporary file with the partition scheme to be included in the KS config
    echo “# Partitioning scheme” > /tmp/partition_layout
    echo “clearpart –all” >> /tmp/partition_layout
    echo “zerombr” >> /tmp/partition_layout
    echo “part /boot/efi –fstype efi –size=200 –ondisk=disk/by-path/$osdisk” >> /tmp/partition_layout
    echo “part /boot –fstype ext2 –size=512 –ondisk=disk/by-path/$osdisk” >> /tmp/partition_layout
    echo “part swap –recommended –ondisk=disk/by-path/$osdisk” >> /tmp/partition_layout
    echo “part pv.01 –size=100 –grow –ondisk=disk/by-path/$osdisk” >> /tmp/partition_layout
    echo “volgroup vg_os pv.01” >> /tmp/partition_layout
    echo “logvol / –vgname=vg_os –size=100 –name=lv_os –fstype=ext4 –grow” >> /tmp/partition_layout
    echo “# Bootloader location” >> /tmp/partition_layout
    echo “bootloader –location=mbr –driveorder=/dev/disk/by-path/$osdisk –append=\”rhgb\”” >> /tmp/partition_layout
    %end

    When I run it, the partition_layout file looks correct when but I get the following error:

    Disk “disk/by-path/pci-0000:00:1f.2-ata-1.0” in part command is not partitioned.

    Any ideas?

    Thanks,

    Steve

      • Zach on Monday, 18 September 2017 at 15:27
        Author
      • Reply

      Hi Steve,

      I’m not sure without seeing more information about what is happening. You may want to set up more verbose logging during the %pre phase:
      https://unix.stackexchange.com/a/127391

      Cheers,
      Zach

    • Odi on Saturday, 6 June 2015 at 04:35
    • Reply

    I found disk labels very useful and simple to boot the same image on physical hw, kvm. Put the label into a kernel cmd line param and use it in your initramfs scripts.

    • Jonathan on Friday, 5 June 2015 at 17:28
    • Reply

    I would have used osdisk=$(find /dev/disk/by-path/ -name '*-scsi-0:2:0:0" -printf "%P") to avoid issues with parsing the output of ls.

      • Zach on Friday, 5 June 2015 at 17:38
        Author
      • Reply

      Very good point, Jonathan! Thanks for the recommendation!

      Cheers,
      Zach

        • Venu on Tuesday, 21 July 2015 at 06:04
        • Reply

        Hi Zach,

        I have following eg.

        part pv.01 –onpart=sda3 –noformat

        In my case partition already exists on and it is pointing to /dev/sda3… Can i use the –ondisk using the way you have identified instead of hard coding sda3.

        [root@end ~]# ls -al /dev/disk/by-path/
        total 0
        drwxr-xr-x 2 root root 120 Jul 21 03:27 .
        drwxr-xr-x 5 root root 100 Jul 21 03:27 ..
        lrwxrwxrwx 1 root root 9 Jul 21 08:27 pci-0000:00:0d.0-scsi-0:0:0:0 -> ../../sda
        lrwxrwxrwx 1 root root 10 Jul 21 08:27 pci-0000:00:0d.0-scsi-0:0:0:0-part1 -> ../../sda1
        lrwxrwxrwx 1 root root 10 Jul 21 08:27 pci-0000:00:0d.0-scsi-0:0:0:0-part2 -> ../../sda2
        lrwxrwxrwx 1 root root 10 Jul 21 08:27 pci-0000:00:0d.0-scsi-0:0:0:0-part3 -> ../../sda3

          • Zach on Tuesday, 21 July 2015 at 11:29
            Author
          • Reply

          Hello Venu,

          If you are wanting to keep that partition, then you should be able to identify the disk itself differently (instead of using /dev/disk/by-path). If you’re wanting to get rid of that partition (and others), then you will want to use clearpart to start fresh. It really depends on what exactly you’re trying to accomplish. If you can give me some more details, I’d be glad to help.

          Cheers,
          Zach

      • Francesco on Thursday, 3 August 2017 at 08:05
      • Reply

      Or also simply $(ls /dev/disk/by-path/ | grep ‘scsi-0:2:0:0$’)

      BTW, thanks Zach. Really good and useful article.

      Cheers,
      Francesco

        • Zach on Thursday, 3 August 2017 at 10:47
          Author
        • Reply

        You’re welcome, Francesco. I’m glad that you found the article helpful.

        Cheers,
        Zach

Leave a Reply

Your email address will not be published.