> For the complete documentation index, see [llms.txt](https://kth-expeca.gitbook.io/testbedconfig/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://kth-expeca.gitbook.io/testbedconfig/prepare/storage.md).

# Storage Node

At ExPECA testbed, we configure Cinder and Swift storage services of Openstack and a seperate storage node is used. We configure Openstack to share the Cinder's block storage (e.g. `/dev/sdb` on the storage node) among all worker nodes using iSCSI.

Make sure the following are set in `site-config/defaults.yaml`:

```
enable_cinder: yes
enable_cinder_backend_lvm: yes
enable_swift: yes
```

Also make sure that a storage node is set in `site-config/inventory/hosts` as mentioned in the deployment instructions.

After the deployment, `tgtd` service will be running on the storage node. All the volumes created using Cinder will be added as targets in tgt so K8S pods can connect to them to get access to the Cinder's block storage.

## Prepare the Storage Node

The storage node on Openstack is a machine with sufficient block storage capacity. Preferably multiple hdds configured in raid format. For instance, the following block storage setup could be used:

```
$ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 447.1G  0 disk 
|-sda1   8:1    0   1.1G  0 part /boot/efi
|-sda2   8:2    0   512M  0 part 
|-sda3   8:3    0  31.3G  0 part [SWAP]
`-sda4   8:4    0 414.3G  0 part /
sdb      8:16   0   2.7T  0 disk 
sdc      8:32   0   3.7T  0 disk
```

The `sda` block is the SSD storage that the machine uses for the operating system. Additionally, there is `sdb` and `sdc` that can be used for Openstack storage node.

ExPECA testbed offers both block storage (Cinder) and object store services (Swift). A simple plan is to use each block for each service. For example `sdb` for Cinder and `sdc` for Swift.

### Set up Linux users and permissions

Make the expeca user password a sudoer. Add the following it to the end of the file:

```
$ sudo visudo
...
expeca    ALL = (ALL) NOPASSWD: ALL
```

Add controller's public key to ssh authorized keys

### Swift preparations

#### Create labeled partitions

Swift configuration works best if the disk is partitioned. In order to do that according to your needs, you can use online tutorials such as [this](https://www.digitalocean.com/community/tutorials/create-a-partition-in-linux) for the disks smaller than 2TB. However, for disks larger than 2TB, `parted` must be used. The following commands will create one primary partition that takes over all the space on `sdc` and adds `KOLLA_SWIFT_DATA` label to it.

```bash
$ sudo parted /dev/sdc -s -- mklabel gpt mkpart KOLLA_SWIFT_DATA 1 -1
```

Then we format the partition to `xfs` file system with label `d0`

```bash
$ sudo mkfs.xfs -f -L d0 /dev/sdc1
meta-data=/dev/sdc1              isize=512    agcount=32, agsize=30515136 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=1
data     =                       bsize=4096   blocks=976484352, imaxpct=5
         =                       sunit=64     swidth=128 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=476800, version=2
         =                       sectsz=512   sunit=64 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
```

Verify that the partition with file system `xfs` and label `KOLLA_SWIFT_DATA`

```
$ sudo parted /dev/sdc print
Model: DELL PERC H755 Front (scsi)
Disk /dev/sdc: 4000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system  Name              Flags
 1      1049kB  4000GB  4000GB  xfs          KOLLA_SWIFT_DATA
```

For evaluation pureposes, loop devices could be created as shown in the upstream references.

### Cinder preparations

In this section we describe the commands to prepare a storage device `sdb` to serve as an LVM backend for Cinder. Unlike Swift, Cinder needs the disk to be without any partition table. The goal is to create `cinder-volumes` LVM volume group.

Make sure there is an unused storage device on the host.

```
$ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 447.1G  0 disk 
|-sda1   8:1    0   1.1G  0 part /boot/efi
|-sda2   8:2    0   512M  0 part 
|-sda3   8:3    0  31.3G  0 part [SWAP]
`-sda4   8:4    0 414.3G  0 part /
sdb      8:16   0   2.7T  0 disk 
sdc      8:32   0   3.7T  0 disk 
`-sdc1   8:33   0   3.7T  0 part
```

In this case `sda` is the OS device and we consider `sdb` for the LVM volume group.

Create the LVM physical volume `/dev/vdb`:

```
$ sudo pvcreate /dev/sdb
WARNING: dos signature detected on /dev/sdb at offset 510. Wipe it? [y/n]: y
  Wiping dos signature on /dev/sdb.
  Physical volume "/dev/sdb" successfully created.
```

If you encountered the following error

```
$ sudo pvcreate /dev/sdb
  Device /dev/sdb excluded by a filter.
```

The reason is that there is partition info on the disk that must be wiped:

```
$ sudo wipefs -a /dev/sdb
/dev/sdb: 2 bytes were erased at offset 0x000001fe (dos): 55 aa
/dev/sdb: calling ioctl to re-read partition table: Success

$ sudo pvcreate /dev/sdb
  Physical volume "/dev/sdb" successfully created.
```

Create the LVM volume group cinder-volumes:

```
$ sudo vgcreate cinder-volumes /dev/sdb
  Volume group "cinder-volumes" successfully created
```

Verify the volumes are ready:

```
$ sudo vgdisplay
  --- Volume group ---
  VG Name               cinder-volumes
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  1
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               <2.73 TiB
  PE Size               4.00 MiB
  Total PE              715007
  Alloc PE / Size       0 / 0   
  Free  PE / Size       715007 / <2.73 TiB
  VG UUID               7o4Kg1-VxDY-4sBn-FUd4-VzKy-Abzh-Fpu9b0
```

## References

* [docs.openstack.org/cinder/xena/install](https://docs.openstack.org/cinder/xena/install/)
* [docs.openstack.org/cinder/xena/install/cinder-storage-install-obs.html](https://docs.openstack.org/cinder/xena/install/cinder-storage-install-obs.html)
* [docs.openstack.org/kolla-ansible/xena/reference/storage/swift-guide.html](https://docs.openstack.org/kolla-ansible/xena/reference/storage/swift-guide.html)
* [docs.openstack.org/swift/xena/deployment\_guide.html](https://docs.openstack.org/swift/xena/deployment_guide.html)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://kth-expeca.gitbook.io/testbedconfig/prepare/storage.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.