# Worker Nodes

## Set up Linux users and permissions

Make the expeca user password a sudoer. Add the following it to the end of the file:

```
$ sudo visudo
...
expeca    ALL = (ALL) NOPASSWD: ALL
```

Add controller's public key to ssh authorized keys

## Install Dependencies

*   Install Docker ([Diginal Ocean Docker installation guide for Ubuntu 20.04](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04))

    We install version 20.10.22 of Docker. Use the following commands during the installation instead of `sudo apt install docker-ce`:

    ```
    sudo apt update
    sudo apt install apt-transport-https ca-certificates curl software-properties-common
    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
    sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
    sudo apt install docker-ce-cli=5:20.10.22~3-0~ubuntu-focal
    sudo apt install docker-ce-rootless-extras=5:20.10.22~3-0~ubuntu-focal
    sudo apt install docker-ce=5:20.10.22~3-0~ubuntu-focal
    ```
*   Check whether docker is installed properly or not

    ```
    sudo docker version
    ```
*   Install other dependencies:

    ```
    sudo apt update && sudo apt install -f -y python3 python3-venv python3-pip jq git
    ```
*   Install Docker-py:

    First we have to downgrade the `requests` and `urllib3` python packages:

    ```
    sudo pip install requests==2.28.1
    sudo pip install urllib3==1.26.13
    ```

    Then install docker-py:

    ```
    sudo pip install docker
    ```
*   Install GO

    ```
    wget https://dl.google.com/go/go1.19.2.linux-amd64.tar.gz
    sudo tar -xvf go1.19.2.linux-amd64.tar.gz
    sudo mv go /usr/local 
    ```

    if you get “directory not empty”, run the go directory with following commands `cd /usr/local` and \`rm -R go/”.

    Add the following lines to the `.bashrc` file using `vim ~/.bashrc`:

    ```
    GOROOT=/usr/local/go
    GOPATH=$HOME/Projects/Proj1
    PATH=$GOPATH/bin:$GOROOT/bin:$PATH
    ```

    To apply the new env vars, log out of the server and back in, or type the following:

    ```
    su - ${USER}
    ```

    Check whether go is installed properly or not

    ```
    go version
    ```

## Configure performance

* Enable jumbo frames with mtu set to 9000 on capable interfaces
*   Disable hyperthreading on BIOS settings or only temporary:

    ```
    sudo -i
    echo off > /sys/devices/system/cpu/smt/control
    exit
    ```
*   Disable power management states of CPU

    ```
    sudo apt install linux-tools-common linux-tools-5.4.0-155-generic
    sudo cpupower idle-set -D 2
    sudo cpupower idle-set --disable-by-latency 0
    ```
*   Set CPU Governor to `performance`

    ```
    sudo apt install cpufrequtils
    for cpu in $(seq 0 $(($(nproc) -1))) ; do sudo cpufreq-set -c $cpu -g performance ; done
    ```
*   Check:

    ```
    sudo cpufreq-info
    ...
    analyzing CPU 31:
    driver: intel_pstate
    CPUs which run at the same hardware frequency: 31
    CPUs which need to have their frequency coordinated by software: 31
    maximum transition latency: 4294.55 ms.
    hardware limits: 800 MHz - 3.40 GHz
    available cpufreq governors: performance, powersave
    current policy: frequency should be within 800 MHz and 3.40 GHz.
                    The governor "performance" may decide which speed to use
                    within this range.
    current CPU frequency is 2.90 GHz.
    ```

## Stop Ubuntu iscsi services

iscsd process uses configfs which is normally mounted at /sys/kernel/config to store discovered targets information, on centos/rhel type of systems this special file system gets mounted automatically, which is not the case on debian/ubuntu. Since iscsid container runs on every nova compute node, the following steps must be completed on every Ubuntu server targeted for nova compute role.

* Add configfs module to `/etc/modules`
* Rebuild initramfs using: `update-initramfs -u command`
*   Stop `open-iscsi` system service due to its conflicts with iscsid container.

    ```bash
    $ sudo systemctl stop open-iscsi
    $ sudo systemctl stop iscsid
    $ sudo yum remove iscsi-initiator-utils
    $ sudo systemctl stop iscsid.socket iscsiuio.socket iscsid.service 
    $ sudo systemctl disable iscsid.socket iscsiuio.socket iscsid.service  
    ```
*   Make sure configfs gets mounted during a server boot up process. There are multiple ways to accomplish it, one example:

    ```bash
    $ mount -t configfs /etc/rc.local /sys/kernel/config
    ```

## Start PTP Daemons

Linux PTP is an implementation of Precision Time Protocol (PTP). On Ubuntu, LinuxPTP can be downloaded as installed as follows:

```
   wget https://deac-fra.dl.sourceforge.net/project/linuxptp/v3.1/linuxptp-3.1.1.tgz
   tar xvf linuxptp-3.1.1.tgz
   cd linuxptp-3.1.1/
   make
   sudo make install
   sudo cp ptp4l phc2sys /usr/sbin/
 
```

This will install two tools-- ptp4l and phc2sys.

*   To install ptp4l as a system service, edit '/usr/lib/systemd/system/ptp4l.service' as shown below:

    ```
    [Unit]
    Description=Precision Time Protocol (PTP) service
    Documentation=man:ptp4l

    [Service]
    Type=simple
    EnvironmentFile=-/etc/sysconfig/ptp4l
    ExecStart=/usr/sbin/ptp4l $OPTIONS

    [Install]
    WantedBy=multi-user.target
    ```

    $OPTIONS is located at the following path at '/etc/sysconfig/ptp4l' with the following:

    ```
    OPTIONS=-2 -i eth0 -m
    ```

    The above method can be used to configure linuxptp as PTP slaves with multicast. In order to enable unicast for PTP slaves the following procedure should be followed.

    The steps to install ptp4l as a system service are the same as above.

    $OPTIONS is located at the following path at '/etc/sysconfig/ptp4l' with the following:

    ```
    OPTIONS=-f /etc/ptp4l.cfg -m
    ```

    The content for /etc/ptp4l.cfg are

    ```
    [global]

    unicast_req_duration 60

    [unicast_master_table]
    table_id 1
    logQueryInterval 2
    UDPv4 10.10.80.1

    [eno8303]
    unicast_master_table 1
    ```

    To start and stop the ptp4l service use:

    ```
    systemctl start ptp4l 
    systemctl stop ptp4l 
    ```

    To enable pt4l to run after every reboot enable it using systemctl:

    ```
    systemctl enable ptp4l
    ```
*   Similarly, to install phc2sys as a system service, edit '/usr/lib/systemd/system/phc2sys.service' as shown below:

    ```
    [Unit]
    Description=Synchronize system clock or PTP hardware clock (PHC)
    Documentation=man:phc2sys
    After=ntpdate.service
    Requires=ptp4l.service
    After=ptp4l.service

    [Service]
    Type=simple
    EnvironmentFile=-/etc/sysconfig/phc2sys
    ExecStart=/usr/sbin/phc2sys $OPTIONS

    [Install]
    WantedBy=multi-user.target

    ```

    $OPTIONS is located at the following path at '/etc/sysconfig/phc2sys' with the following:

    ```
    OPTIONS=-a -r -m
    ```

    To start and stop the phc2sys service:

    ```
    systemctl start phc2sys 
    systemctl stop phc2sys 
    ```

    To enable phc2sys to run after every reboot enable it using systemctl:

    ```
    systemctl enable phc2sys
    ```

    Finally, make sure NTP is not active on the machine.

    ```
    sudo timedatectl set-ntp 0
    ```
