# During Resource Enrollment

## cc-ansible --playbook ./playbooks/k3s.yml issues

### Unable to read /etc/rancher/k3s/k3s.yaml

```
kubectl get pods -A -o wide
WARN[0000] Unable to read /etc/rancher/k3s/k3s.yaml, please start server with --write-kubeconfig-mode to modify kube config permissions 
error: error loading config file "/etc/rancher/k3s/k3s.yaml": open /etc/rancher/k3s/k3s.yaml: permission denied
```

To overcome this issue, add the following line to `roles/k3s/templates/config.yaml.j2`:

```
write-kubeconfig-mode: "0644"
```

### Wait till the Tigera Operator has fully applied

```
k3s : Wait till the Tigera Operator has fully applied
fatal: [edge -> localhost]: FAILED! => {"changed": false, "msg": "Failed to gather information about TigeraStatus(s) even after waiting for 123 seconds"}
```

By default the timeout for the tigera operator is 120 seconds. If you face this error, increase it 5 mins by adding `wait_timeout: 300` just after `wait: yes` in `roles/k3s/tasks/config-calico.yml` corresponding tasks. Such as below:

```
- name: Wait till the Tigera Operator has fully applied
  delegate_to: "{{ groups['deployment'][0] }}"
  kubernetes.core.k8s_info:
    kind: TigeraStatus
    name: calico
    namespace: default
    wait: yes
    wait_condition:
      type: "Available"
      status: "True"
      reason: "AllObjectsAvailable"
    wait_timeout: 300

- name: Wait till the Calico Installation is created
  delegate_to: "{{ groups['deployment'][0] }}"
  kubernetes.core.k8s_info:
    kind: Installation
    name: default
    namespace: default
    wait: yes
    wait_condition:
      type: "Ready"
      status: "True"
      reason: "AllObjectsAvailable"
    wait_timeout: 300
```

In addition to that, this must be run in a completly clean state.

### Change cluster ip CIDR

In case it is colliding with any of the used subnets, it is better to change it at `roles/k3s/defaults/main.yml`. More info could be find [here](https://docs.k3s.io/reference/server-config#networking).

### Apply Calico network policies

```
TASK [k3s : Apply Calico global network policies] **********************************************************************************************************************************************************
failed: [edge-mv -> localhost] (item=default-deny) => {"ansible_loop_var": "item", "changed": false, "item": "default-deny", "msg": "Failed to find exact match for projectcalico.org/v1.GlobalNetworkPolicy by [kind, name, singularName, shortNames]"}
failed: [edge-mv -> localhost] (item=allow-ping) => {"ansible_loop_var": "item", "changed": false, "item": "allow-ping", "msg": "Failed to find exact match for projectcalico.org/v1.GlobalNetworkPolicy by [kind, name, singularName, shortNames]"}
```

Solution according to [here](https://serverfault.com/questions/1076325/apply-calico-manifest-through-ansible-kubernetes-core-module) is to add `apply: yes` to the `task.kubernetes.core.k8s`.

### Configure Neutron

```
TASK [k3s : Create calico network] *************************************************************************************************************************************************************************
fatal: [edge-mv]: FAILED! => {"action": "os_network", "changed": false, "extra_data": {"data": null, "details": "Running without keystone AuthN requires that tenant_id is specified", "response": "{\"NeutronError\": {\"type\": \"HTTPBadRequest\", \"message\": \"Running without keystone AuthN requires that tenant_id is specified\", \"detail\": \"\"}}"}, "msg": "BadRequestException: 400: Client Error for url: http://10.0.87.254:9696/v2.0/networks, Running without keystone AuthN requires that tenant_id is specified"}
```

Solution: open the webportal, create Calico network, subnet, and router from there. OR, add the following line to 'Create calico network', 'Create calico subnet', and 'Fetch existing NAT router' tasks `<task>.kolla_toolbox.module_args` in `roles/k3s/tasks/config-neutron.yml`:

```
project: "{{ keystone_admin_project }}"
```

### Generate Calico/Neutron connection script

```
TASK [k3s : Generate Calico/Neutron connection script] *****************************************************************************************************************************************************
fatal: [edge-mv]: FAILED! => {"changed": false, "checksum": "0b5e563c74380eba5d02bb1a041e4703aac71e28", "msg": "Destination /etc/rancher/k3s not writable"}
```

The solution is to add `become: yes` to the task `Generate Calico/Neutron connection script` in `roles/k3s/tasks/config-neutron.yml`.

## NOTE: Run these when a new snapshot is loaded:

1. Configs

   ```
   sudo chown root:expeca-deploy /opt
   sudo chmod g+rw /opt
   ```
2. Add `enable_k3s: yes` to

   ```
   sudo vim /opt/site-config/defaults.yml
   ```
3. Add `wait_timeout: 300` to 'Wait till the Tigera Operator has fully applied' and 'Wait till the Calico Installation is created' tasks just after `wait: yes` in `roles/k3s/tasks/config-calico.yml`.
4. Change cluster ip CIDR at `roles/k3s/defaults/main.yml`.
5. Add `apply: yes` to the last task in `roles/k3s/tasks/config-calico.yml`.
6. Add the following line to 'Create calico network', 'Create calico subnet', and 'Fetch existing NAT router' tasks `<task>.kolla_toolbox.module_args` in `roles/k3s/tasks/config-neutron.yml`:

   ```
   project: "{{ keystone_admin_project }}"
   ```
7. Add `become: yes` to the task `Generate Calico/Neutron connection script` in `roles/k3s/tasks/config-neutron.yml`.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://kth-expeca.gitbook.io/testbedconfig/known-issues/k3s-role.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.