Storage Node
At ExPECA testbed, we configure Cinder and Swift storage services of Openstack and a seperate storage node is used. We configure Openstack to share the Cinder's block storage (e.g. /dev/sdb
on the storage node) among all worker nodes using iSCSI.
Make sure the following are set in site-config/defaults.yaml
:
enable_cinder: yes
enable_cinder_backend_lvm: yes
enable_swift: yes
Also make sure that a storage node is set in site-config/inventory/hosts
as mentioned in the deployment instructions.
After the deployment, tgtd
service will be running on the storage node. All the volumes created using Cinder will be added as targets in tgt so K8S pods can connect to them to get access to the Cinder's block storage.
Prepare the Storage Node
The storage node on Openstack is a machine with sufficient block storage capacity. Preferably multiple hdds configured in raid format. For instance, the following block storage setup could be used:
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 447.1G 0 disk
|-sda1 8:1 0 1.1G 0 part /boot/efi
|-sda2 8:2 0 512M 0 part
|-sda3 8:3 0 31.3G 0 part [SWAP]
`-sda4 8:4 0 414.3G 0 part /
sdb 8:16 0 2.7T 0 disk
sdc 8:32 0 3.7T 0 disk
The sda
block is the SSD storage that the machine uses for the operating system. Additionally, there is sdb
and sdc
that can be used for Openstack storage node.
ExPECA testbed offers both block storage (Cinder) and object store services (Swift). A simple plan is to use each block for each service. For example sdb
for Cinder and sdc
for Swift.
Set up Linux users and permissions
Make the expeca user password a sudoer. Add the following it to the end of the file:
$ sudo visudo
...
expeca ALL = (ALL) NOPASSWD: ALL
Add controller's public key to ssh authorized keys
Swift preparations
Create labeled partitions
Swift configuration works best if the disk is partitioned. In order to do that according to your needs, you can use online tutorials such as this for the disks smaller than 2TB. However, for disks larger than 2TB, parted
must be used. The following commands will create one primary partition that takes over all the space on sdc
and adds KOLLA_SWIFT_DATA
label to it.
$ sudo parted /dev/sdc -s -- mklabel gpt mkpart KOLLA_SWIFT_DATA 1 -1
Then we format the partition to xfs
file system with label d0
$ sudo mkfs.xfs -f -L d0 /dev/sdc1
meta-data=/dev/sdc1 isize=512 agcount=32, agsize=30515136 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=0
= reflink=1
data = bsize=4096 blocks=976484352, imaxpct=5
= sunit=64 swidth=128 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=476800, version=2
= sectsz=512 sunit=64 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Verify that the partition with file system xfs
and label KOLLA_SWIFT_DATA
$ sudo parted /dev/sdc print
Model: DELL PERC H755 Front (scsi)
Disk /dev/sdc: 4000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 4000GB 4000GB xfs KOLLA_SWIFT_DATA
For evaluation pureposes, loop devices could be created as shown in the upstream references.
Cinder preparations
In this section we describe the commands to prepare a storage device sdb
to serve as an LVM backend for Cinder. Unlike Swift, Cinder needs the disk to be without any partition table. The goal is to create cinder-volumes
LVM volume group.
Make sure there is an unused storage device on the host.
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 447.1G 0 disk
|-sda1 8:1 0 1.1G 0 part /boot/efi
|-sda2 8:2 0 512M 0 part
|-sda3 8:3 0 31.3G 0 part [SWAP]
`-sda4 8:4 0 414.3G 0 part /
sdb 8:16 0 2.7T 0 disk
sdc 8:32 0 3.7T 0 disk
`-sdc1 8:33 0 3.7T 0 part
In this case sda
is the OS device and we consider sdb
for the LVM volume group.
Create the LVM physical volume /dev/vdb
:
$ sudo pvcreate /dev/sdb
WARNING: dos signature detected on /dev/sdb at offset 510. Wipe it? [y/n]: y
Wiping dos signature on /dev/sdb.
Physical volume "/dev/sdb" successfully created.
If you encountered the following error
$ sudo pvcreate /dev/sdb
Device /dev/sdb excluded by a filter.
The reason is that there is partition info on the disk that must be wiped:
$ sudo wipefs -a /dev/sdb
/dev/sdb: 2 bytes were erased at offset 0x000001fe (dos): 55 aa
/dev/sdb: calling ioctl to re-read partition table: Success
$ sudo pvcreate /dev/sdb
Physical volume "/dev/sdb" successfully created.
Create the LVM volume group cinder-volumes:
$ sudo vgcreate cinder-volumes /dev/sdb
Volume group "cinder-volumes" successfully created
Verify the volumes are ready:
$ sudo vgdisplay
--- Volume group ---
VG Name cinder-volumes
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size <2.73 TiB
PE Size 4.00 MiB
Total PE 715007
Alloc PE / Size 0 / 0
Free PE / Size 715007 / <2.73 TiB
VG UUID 7o4Kg1-VxDY-4sBn-FUd4-VzKy-Abzh-Fpu9b0
References
Last updated