Change hosts in inventory and add storage-01 if you have a separate storage node
# /opt/site-config/inventory/hosts
# These initial groups are the only groups required to be modified. The
# additional groups are for more control of the environment.
[control]
controller-01
[network]
controller-01
[compute]
# No compute node; this is a baremetal-only cluster.
[monitoring]
controller-01
[storage]
storage-01
[deployment]
localhost ansible_connection=local
Change host_vars in inventory for controller
Create a host_vars file for the storage node
In your site config directory, /opt/site-config, you now have a file called defaults.yml You need to modify the networking parameters such as vip addresses and Neutron cidr's according to your public and internal network cidr values.
Add storage-01 host's name and internal ip to /etc/hosts
Bootstrap Servers
Run ./cc-ansible --site /opt/site-config/ bootstrap-servers
This will install docker, configure /etc/hosts, and generally configure the system to run the rest of the installation.
Afterwards, you should see that /etc/hosts contains an entry for your hostname, mapping it to the interface address. Note that this is NOT the haproxy VIP!
Enable ip forwarding by adding the following to /etc/sysctl.conf
Then reload the settings by running sudo sysctl -p.
Create Swift Rings
Before Swift deployment we need to generate rings, which are binary compressed files that at a high level let the various Swift services know where data is in the cluster.
A helper script create_swift_rings.sh is provided at TestbedConfig/tools folder and should be run from the controller node. The commands work with d0 disks with partition table created in preparations section on the storage node. It generates the rings and stores them in /opt/site-config/node_custom_config/swift/.
Modify the script create_swift_rings.sh
with the storage node's internal ip as NODE variable,
and with the storage device name e.g. d0
Create swift folder in node_custom_config and run the script
Verify the generated files
Run Pre-checks
Run cc-ansible --site /opt/site-config/ prechecks
This will warn you about missing configuration, and other common errors.
Downloading Containers
ExPECA uses Dockerhub as the secondry Docker registry. Ensure you have login credentials for it. Insert your Dockerhub username to the site config's defaults.yml.
Add additional passwords
Run ./cc-ansible --site /opt/site-config edit_passwords
Your default editor will open, and you'll see the following. Ensure that docker_registry_password, secondry_docker_registry_password, and tenant_switch_password is configured as follows. The secondry_docker_registry_password and tenant_switch_password lines do not exist, hence must be added. We use passwords to create swift prefix and suffix as well.
The passwords file will be re-encrypted when you exit the editor.
After configuring this password, pull the containers!
Pull container images
Run ./cc-ansible --site /opt/site-config pull
Generate Letsencrypt certificates
Currently, the initial certificate generation is not yet automated. You must perform the following steps:
Copy fullchain.pem to site-config/certificates/haproxy.pem.
Append the private key to ../site-config/certificates/haproxy.pem.
Deploy
You're now ready to run the deploy! This will bring up basic control plane services, listening on the IP addresses you configured above.
NOTE: If you had a break between the pre-check and this deploy step, then first run the pre-check again, followed by making sure that the edits to defaults.yml and config.yml files are accurate.
Run ./cc-ansible --site /opt/site-config deploy
Access your site
After deploy completes, you'll be able to access the horizon webui at http://<kolla_external_vip_address>
The username is admin, and the password can be found by running cc-ansible --site /opt/site-config view_passwords | grep keystone_admin_password
Partial Deploy
If you encounter errors and need to re-run the deploy step, which is expensive, you can skip parts you know have already succeeded. You can watch the Ansible output to see which "role" (service) it is updating. If you know a certain role has completed successfully, you can try skipping it on the next run with the --skip-tags option, e.g. --skip-tags keystone,nova to skip the Keystone and Nova provisioning. You can persist these by uncommenting their lines in kolla-skip-tags
Post-Deploy
Once the deployment is complete, there should be a more or less functional OpenStack deployment that you can log in to. However, much of the bare metal functionality will not work, as there are a few special entities necessary, namely:
A provisioning network on its own VLAN, which Ironic must know about
Access on that VLAN to the Ironic TFTP server
A "baremetal" Nova flavor that is used to allow users to schedule bare metal deployments
Ironic deploy images available in Glance that hold the deployment ramdisk/kernel
A special "freepool" Nova aggregate used by Blazar to manage node reservations
All of these will be provisioned by running the post-deploy script:
Set up admin auth
post-deploy will create a file named admin-openrc.sh in your site-config directory. This file shouldn't be checked into source control, as it contains secrets.
To use it, run source /opt/site-config/admin-openrc.sh and source /opt/chi-in-a-box/venv/bin/activate
This will set a variety of environment variables, all prefixed with OS_
Verify that it works by running openstack token issue
# /opt/site-config/inventory/host_vars/controller-01
---
# Initial assumption is that this is also the deployment node,
# therefore any provisioning can be done locally.
ansible_connection: local
network_interface: brinternal
kolla_external_vip_interface: brpublic
# /opt/site-config/inventory/host_vars/storage-01
---
# Initial assumption is that this is also the deployment node,
# therefore any provisioning can be done locally.
ansible_connection: ssh
network_interface: brinternal
mkdir -p /opt/site-config/node_custom_config/swift
cd /opt/TestbedConfig/tools/
chmod +x create_swift_rings.sh
./create_swift_rings.sh
Device d0r1z1-10.20.111.1:6000R10.20.111.1:6000/d0_"" with 1.0 weight got id 0
Device d0r1z1-10.20.111.1:6001R10.20.111.1:6001/d0_"" with 1.0 weight got id 0
Device d0r1z1-10.20.111.1:6002R10.20.111.1:6002/d0_"" with 1.0 weight got id 0
Reassigned 1024 (100.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
Reassigned 1024 (100.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
Reassigned 1024 (100.00%) partitions. Balance is now 0.00. Dispersion is now 0.00
expeca@controller-01:/opt/site-config/node_custom_config/swift$ ls
account.builder account.ring.gz backups container.builder container.ring.gz object.builder object.ring.gz
# Make a directory for your certs
docker exec -it letsencrypt_certbot mkdir -p /www/data/<kolla_external_fqdn>/
# Request the initial certificate
docker exec -it letsencrypt_certbot certbot certonly --noninteractive --agree-tos
Saving debug log to /var/log/kolla/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for testbed.expeca.proj.kth.se
Waiting for verification...
Cleaning up challenges
cd /opt/chi-in-a-box
mkdir -p ../site-config/certificates/
sudo cp /var/lib/docker/volumes/letsencrypt_certs/_data/live/testbed.expeca.proj.kth.se/fullchain.pem ../site-config/certificates/haproxy.pem
sudo vim /var/lib/docker/volumes/letsencrypt_certs/_data/live/testbed.expeca.proj.kth.se/privkey.pem
vim ../site-config/certificates/haproxy.pem
./cc-ansible --site /opt/site-config post-deploy
# run this in your home directory
cd ~
# create a new virtualenv, and use it
python3 -m venv .venv
source .venv/bin/activate
# update to a recent version of pip
pip install --upgrade pip
# install the openstack client libraries
pip install python-openstackclient python-doniclient python-ironicclient