Worker Nodes
Set up Linux users and permissions
Make the expeca user password a sudoer. Add the following it to the end of the file:
$ sudo visudo
...
expeca ALL = (ALL) NOPASSWD: ALL
Add controller's public key to ssh authorized keys
Install Dependencies
Install Docker (Diginal Ocean Docker installation guide for Ubuntu 20.04)
We install version 20.10.22 of Docker. Use the following commands during the installation instead of
sudo apt install docker-ce
:sudo apt update sudo apt install apt-transport-https ca-certificates curl software-properties-common curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable" sudo apt install docker-ce-cli=5:20.10.22~3-0~ubuntu-focal sudo apt install docker-ce-rootless-extras=5:20.10.22~3-0~ubuntu-focal sudo apt install docker-ce=5:20.10.22~3-0~ubuntu-focal
Check whether docker is installed properly or not
sudo docker version
Install other dependencies:
sudo apt update && sudo apt install -f -y python3 python3-venv python3-pip jq git
Install Docker-py:
First we have to downgrade the
requests
andurllib3
python packages:sudo pip install requests==2.28.1 sudo pip install urllib3==1.26.13
Then install docker-py:
sudo pip install docker
Install GO
wget https://dl.google.com/go/go1.19.2.linux-amd64.tar.gz sudo tar -xvf go1.19.2.linux-amd64.tar.gz sudo mv go /usr/local
if you get “directory not empty”, run the go directory with following commands
cd /usr/local
and `rm -R go/”.Add the following lines to the
.bashrc
file usingvim ~/.bashrc
:GOROOT=/usr/local/go GOPATH=$HOME/Projects/Proj1 PATH=$GOPATH/bin:$GOROOT/bin:$PATH
To apply the new env vars, log out of the server and back in, or type the following:
su - ${USER}
Check whether go is installed properly or not
go version
Configure performance
Enable jumbo frames with mtu set to 9000 on capable interfaces
Disable hyperthreading on BIOS settings or only temporary:
sudo -i echo off > /sys/devices/system/cpu/smt/control exit
Disable power management states of CPU
sudo apt install linux-tools-common linux-tools-5.4.0-155-generic sudo cpupower idle-set -D 2 sudo cpupower idle-set --disable-by-latency 0
Set CPU Governor to
performance
sudo apt install cpufrequtils for cpu in $(seq 0 $(($(nproc) -1))) ; do sudo cpufreq-set -c $cpu -g performance ; done
Check:
sudo cpufreq-info ... analyzing CPU 31: driver: intel_pstate CPUs which run at the same hardware frequency: 31 CPUs which need to have their frequency coordinated by software: 31 maximum transition latency: 4294.55 ms. hardware limits: 800 MHz - 3.40 GHz available cpufreq governors: performance, powersave current policy: frequency should be within 800 MHz and 3.40 GHz. The governor "performance" may decide which speed to use within this range. current CPU frequency is 2.90 GHz.
Stop Ubuntu iscsi services
iscsd process uses configfs which is normally mounted at /sys/kernel/config to store discovered targets information, on centos/rhel type of systems this special file system gets mounted automatically, which is not the case on debian/ubuntu. Since iscsid container runs on every nova compute node, the following steps must be completed on every Ubuntu server targeted for nova compute role.
Add configfs module to
/etc/modules
Rebuild initramfs using:
update-initramfs -u command
Stop
open-iscsi
system service due to its conflicts with iscsid container.$ sudo systemctl stop open-iscsi $ sudo systemctl stop iscsid $ sudo yum remove iscsi-initiator-utils $ sudo systemctl stop iscsid.socket iscsiuio.socket iscsid.service $ sudo systemctl disable iscsid.socket iscsiuio.socket iscsid.service
Make sure configfs gets mounted during a server boot up process. There are multiple ways to accomplish it, one example:
$ mount -t configfs /etc/rc.local /sys/kernel/config
Start PTP Daemons
Linux PTP is an implementation of Precision Time Protocol (PTP). On Ubuntu, LinuxPTP can be downloaded as installed as follows:
wget https://deac-fra.dl.sourceforge.net/project/linuxptp/v3.1/linuxptp-3.1.1.tgz
tar xvf linuxptp-3.1.1.tgz
cd linuxptp-3.1.1/
make
sudo make install
sudo cp ptp4l phc2sys /usr/sbin/
This will install two tools-- ptp4l and phc2sys.
To install ptp4l as a system service, edit '/usr/lib/systemd/system/ptp4l.service' as shown below:
[Unit] Description=Precision Time Protocol (PTP) service Documentation=man:ptp4l [Service] Type=simple EnvironmentFile=-/etc/sysconfig/ptp4l ExecStart=/usr/sbin/ptp4l $OPTIONS [Install] WantedBy=multi-user.target
$OPTIONS is located at the following path at '/etc/sysconfig/ptp4l' with the following:
OPTIONS=-2 -i eth0 -m
The above method can be used to configure linuxptp as PTP slaves with multicast. In order to enable unicast for PTP slaves the following procedure should be followed.
The steps to install ptp4l as a system service are the same as above.
$OPTIONS is located at the following path at '/etc/sysconfig/ptp4l' with the following:
OPTIONS=-f /etc/ptp4l.cfg -m
The content for /etc/ptp4l.cfg are
[global] unicast_req_duration 60 [unicast_master_table] table_id 1 logQueryInterval 2 UDPv4 10.10.80.1 [eno8303] unicast_master_table 1
To start and stop the ptp4l service use:
systemctl start ptp4l systemctl stop ptp4l
To enable pt4l to run after every reboot enable it using systemctl:
systemctl enable ptp4l
Similarly, to install phc2sys as a system service, edit '/usr/lib/systemd/system/phc2sys.service' as shown below:
[Unit] Description=Synchronize system clock or PTP hardware clock (PHC) Documentation=man:phc2sys After=ntpdate.service Requires=ptp4l.service After=ptp4l.service [Service] Type=simple EnvironmentFile=-/etc/sysconfig/phc2sys ExecStart=/usr/sbin/phc2sys $OPTIONS [Install] WantedBy=multi-user.target
$OPTIONS is located at the following path at '/etc/sysconfig/phc2sys' with the following:
OPTIONS=-a -r -m
To start and stop the phc2sys service:
systemctl start phc2sys systemctl stop phc2sys
To enable phc2sys to run after every reboot enable it using systemctl:
systemctl enable phc2sys
Finally, make sure NTP is not active on the machine.
sudo timedatectl set-ntp 0
Last updated