SDR Nodes
An SDR node at ExPECA is a USRP E320 which is powered over ethernet.
Make sure the following configurations are done before deplyment. You need to have connect to the SDR with a micro USB cable. Prepare a SD card reader as well.
Download NI Reference Design version 4.3.0.0
NOTE: make sure you disable conda environment conda deactivate
if there is any before the following commands.
In order to change the UHD version on the radio, you must first install the desired UHD version on your host.
sudo apt install -y libboost-all-dev libusb-1.0-0-dev doxygen python3-docutils python3-mako python3-numpy python3-requests python3-ruamel.yaml python3-setuptools cmake build-essential
git clone https://github.com/EttusResearch/uhd.git ~/uhd
cd ~/uhd
git checkout v4.3.0.0
cd host
mkdir build
cd build
cmake ../
make -j $(nproc)
make test # This step is optional
sudo make install
sudo ldconfig
Use uhd_images_downloader
to pull the relevant sd image and fpga bitstream.
expeca@forlong:~$ sudo uhd_images_downloader -t sdimg -t e320
[INFO] Using base URL: https://files.ettus.com/binaries/cache/
[INFO] Images destination: /usr/local/share/uhd/images
The file size for this target (565.1 MiB) exceeds the download limit (100.0 MiB). Continue downloading? [y/N]y
233759 kB / 592569 kB (039%) e3xx_e320_sdimg_default-v4.0.0.0.zip
expeca@forlong:~$ uhd_images_downloader -t e320 -t fpga
[INFO] Using base URL: https://files.ettus.com/binaries/cache/
[INFO] Images destination: /usr/local/share/uhd/images
[INFO] Target e3xx_e320_fpga_default is up to date.
Clean Copy NI Reference Design to SD Card
Insert E320's SD card and unmount all partitions. Running the command lsblk again will show these partitions have been unmounted. If they are not (MOUNTPOINT
is specified) unmount them.
expeca@forlong:~$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:16 1 14.9G 0 disk
├─sda1 8:17 1 16M 0 part
├─sda2 8:18 1 1.9G 0 part
├─sda3 8:19 1 1.9G 0 part
└─sda4 8:20 1 11G 0 part
We write the SD card image using dd to write the disk image, if the sdimg is located on the PC, run
sudo dd if=/usr/local/share/uhd/images/usrp_e320_fs.sdimg | pv | sudo dd of=/dev/sda bs=1M
there is no indication of progress. So be patient.
If the sdimg is located on a USB stick, run
sudo dd if=/media/wlab/cf416480-4366-44bc-8f82-14269c3cde9c/usrp_e320_fs.sdimg | pv | sudo dd of=/dev/sdb bs=10M
To ensure the disk is synchronized, run the sync command:
sync
Copy all the files inside boot
partition to the host. Download micro-controller's new firmware.
sudo cp /media/wlab/boot/* ~/e320_ni_design_4.3.0.0/
curl https://files.ettus.com/binaries/misc/upgrade_mcu_neon_v1.1.7358-a190641-musl-glibc-rev3-7.tar.gz -o ~/e320_ni_design_4.3.0.0/upgrade_mcu_neon_v1.1.7358-a190641-musl-glibc-rev3-7.tar.gz
Re-partition the SD card
In order to make an E320 dual boot, we need to copy Mangocomm's boot files into the boot
partition alongside the reference boot files. However, boot
partition is too small and it needs to be resized. The approach is to append some storage from data
partition which is almost free to boot
partition.
We use Linux's GParted tool. Install it using the package manager and open the SD card using it. NI E320 SD card by default comes with 4 partitions:
expeca@forlong:~$ sudo fdisk -l /dev/sda
Disk /dev/sda: 14,9 GiB, 16022241280 bytes, 31293440 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x2aacd715
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 49152 81919 32768 16M c W95 FAT32 (LBA)
/dev/sda2 81920 4079615 3997696 1,9G 83 Linux
/dev/sda3 4079616 8077311 3997696 1,9G 83 Linux
/dev/sda4 8077312 31293439 23216128 11,1G 83 Linux
expeca@forlong:~$ sudo parted -l
Model: Generic MassStorageClass (scsi)
Disk /dev/sda: 16,0GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 25,2MB 41,9MB 16,8MB primary fat16 boot, lba
2 41,9MB 2089MB 2047MB primary ext4
3 2089MB 4136MB 2047MB primary ext4
4 4136MB 16,0GB 11,9GB primary ext4
Note that partition 1 is named boot
, its type is FAT16
and has boot
and lba
flags.
Make a backup from both boot
and data
partitions that we want to resize. For the boot
partition you can copy the files with cp
command. But for the data
partition use dd
to create an image. We do that to avoid modifing the files permissions on this partition.
expeca@forlong:~$ sudo dd if=/dev/sda4 of=~/Desktop/NI-e320-sdcard/data.img
22323200+0 records in
22323200+0 records out
11429478400 bytes (11 GB, 11 GiB) copied, 132,54 s, 86,2 MB/s
expeca@forlong:~$ mkdir -p ~/Desktop/NI-e320-sdcard/boot
expeca@forlong:~$ cp /media/expeca/boot/* ~/Desktop/NI-e320-sdcard/boot/
Run GParted, unmount all partitions.
Apply the following changes using GParted
Shrink
/dev/sda4
to 10.7G (400MB will be freed)Delete
/dev/sda1
.Create a new FAT16 partition with the label
boot
located after/dev/sda4
using the freed space.
Apply the operations, then don't forget to add the flags: 4. Add flags boot
and lba
to /dev/sda1
The result should look like this
expeca@forlong:~$ sudo fdisk -l /dev/sda
Disk /dev/sda: 14,9 GiB, 16022241280 bytes, 31293440 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x7d6a2a38
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 30400512 31293439 892928 436M e W95 FAT16 (LBA)
/dev/sda2 81920 4079615 3997696 1,9G 83 Linux
/dev/sda3 4079616 8077311 3997696 1,9G 83 Linux
/dev/sda4 8077312 30400511 22323200 10,7G 83 Linux
Partition table entries are not in disk order.
Mount the new BOOT partition
sudo mkdir /media/BOOT
sudo mount /dev/sdb1 /media/BOOT
Copy from the host, boot partition files that you saved before to the new boot
partition. Also create mango_bootbin
and ni_bootbin
folders on this partition. Copy NI's boot.bin
file to its folder.
sudo cp ~/e320_ni_design_4.3.0.0/* /media/BOOT/
sudo mkdir /media/BOOT/mango_bootbin
sudo mkdir /media/BOOT/ni_bootbin
sudo cp ~/e320_ni_design_4.3.0.0/boot.bin /media/BOOT/ni_bootbin/
Create and Copy Mangocomm boot files
Create FPGA image
TODO
Create petalinux image
For each device, a new petalinux image must be built with a new MAC address for the rj45 port. Follow the instructions here to create the petalinux project.
wlab@forlong:~/new_mango_e320$ source ~/petalinux/2019.1/settings.sh
wlab@forlong:~/new_mango_e320$ petalinux-create -t project -s mango_wlan_E320_petalinux_v2.0.2.bsp
INFO: Create project:
INFO: Projects:
INFO: * mango_wlan_E320_petalinux
INFO: has been successfully installed to /home/wlab/new_mango_e320/
INFO: New project successfully created in /home/wlab/new_mango_e320/
Open the project folder
wlab@forlong:~/new_mango_e320$ cd ~/new_mango_e320/mango_wlan_E320_petalinux
Modify
/etc/network/interfaces
file for the static ip ofeth0
vim components/meta-mango-wlan/meta-mango/recipes-core/init-ifupdown/files/interfaces
...
auto eth0
iface eth0 inet static
address 10.30.10.1
netmask 255.255.255.0
Note that auto eth0
is important, otherwise the interface won't come up.
Build
ifupdown
module
petalinux-build -c init-ifupdown
Change ethernet MAC address
Open the petalinux-config
wlab@forlong:~/new_mango_e320/mango_wlan_E320_petalinux$ petalinux-config
Then proceed to "Subsystem AUTO Hardware Settings" -> "Ethernet Settings" for setting the ethernet MAC address and static ip. Set the MAC to randomized address generation and static ip without gateway. Then, save the settings and exit the configuration. Return to the config's "Ethernet Settings" and check the generated MAC address. Write it down.
NOTE: the project creates u-boot with Xilinx's automatic configurations. However, the project does not compile successfully in Ubuntu 18.04. To bypass that, open petalinux-config
and save it again without any change :D. Then build the project again.
Build the project
petalinux-build
Copy Mango boot files
From petalinux project
sudo cp ~/new_mango_e320/mango_wlan_E320_petalinux/images/linux/image.ub /media/BOOT/
From Xilinx SDK
sudo cp ~/new_mango_e320/Mango_802.11_MAC_Software_E320_v2.0.2/bootimg/boot_wlan_lin_dcf.bit /media/BOOT/wlan.bit
sudo cp ~/new_mango_e320/Mango_802.11_MAC_Software_E320_v2.0.2/wlan-sdk/bootimg/BOOT.bin /media/BOOT/mango_bootbin/BOOT.bin
sudo cp ~/new_mango_e320/Mango_802.11_MAC_Software_E320_v2.0.2/wlan-sdk/bootimg/ec-firmware-no-watchdog.bin /media/BOOT/
sudo cp ~/new_mango_e320/Mango_802.11_MAC_Software_E320_v2.0.2/wlan-sdk/bootimg/uEnv.txt /media/BOOT/
Finally boot
should be like:
expeca@forlong:/media/BOOT$ tree
.
├── boot.bin
├── ec-firmware-no-watchdog.bin
├── image.ub
├── mango_bootbin
│ └── BOOT.bin
├── ni_bootbin
│ └── boot.bin
├── u-boot.img
├── uEnv.txt
└── wlan.bit
2 directories, 8 files
Note that we keep boot.bin
files in seperate folders for later use.
Unmount the BOOT partition
sync
sudo umount /media/BOOT
sudo rm -rf /media/BOOT
Bring up and Configure NI design
Now insert back the SD card and turn on the device with micro usb connected to the host. Establish UART connection using the screen
tool.
expeca@forlong:~$ ls /dev/serial/by-id
usb-Digilent_Digilent_USB_Device_251635236A3B-if00-port0 usb-Silicon_Labs_CP2105_Dual_USB_to_UART_Bridge_Controller_011C582C-if00-port0
usb-Digilent_Digilent_USB_Device_251635236A3B-if01-port0 usb-Silicon_Labs_CP2105_Dual_USB_to_UART_Bridge_Controller_011C582C-if01-port0
expeca@forlong:~$ sudo screen /dev/serial/by-id/usb-Silicon_Labs_CP2105_Dual_USB_to_UART_Bridge_Controller_011C1095-if01-port0 115200
ni-e320-3238B97 login: root
root@ni-e320-3238B97:~#
Setup NI design's static ips by editting these files and restarting the networking deamon:
Set the
eth0
andsfp0
ip addresses using these commands:
vim /lib/systemd/network/40-eth0.network
vim /lib/systemd/network/40-sfp0.network
For example eth0
file should be like:
[Match]
Name=eth0
KernelCommandLine=!nfsroot
[Network]
Address=10.30.10.6/24
IPForward=ipv4
sfp0
should be like:
[Match]
Name=sfp0
[Network]
Address=10.30.1.6/24
[Link]
MTUBytes=9000
TxFlowControl=1
Check the eeprom by using
eeprom-dump
command on the USRP.
-- PID/REV: e320 0005
-- MCU_FLAGS[0]: 00000009
-- MCU_FLAGS[1]: 00000000
-- MCU_FLAGS[2]: 00000000
-- MCU_FLAGS[3]: 00000000
-- Serial: 3238B94
-- eth_addr0: 00:80:2f:33:af:97
-- eth_addr1: 00:80:2f:33:af:98
-- eth_addr2: 00:00:00:00:00:00
-- DT-Compat/MCU-Compat: 0000 0002
-- Max-REV: 0004
-- CRC: a3e865db (matches)
Write down eth_addr0
address to set in the next step for eth0
.
Create a
systemd
link file for theeth0
to change the MAC address policy tonone
and set the mac address, speed, and duplex modes.
vim /lib/systemd/network/10-eth0.link
Replace the mac address with the one from eeprom:
[Match]
OriginalName=eth0
[Link]
MACAddressPolicy=none
MACAddress=00:80:2f:33:af:97
AutoNegotiation=yes
BitsPerSecond=1000M
Duplex=full
You can check link file and test it via
udevadm -d test-builtin net_setup_link /sys/class/net/eth0
Remove all tmp backup files created by vim or anythin else in the folder.
rm /lib/systemd/network/.10-eth0.link.un~
rm /lib/systemd/network/.40-eth0.network.un~
rm /lib/systemd/network/.40-sfp0.network.un~
rm /lib/systemd/network/10-eth0.link~
rm /lib/systemd/network/40-eth0.network~
rm /lib/systemd/network/40-sfp0.network~
Restart
systemd-networkd
andreboot
systemctl restart systemd-networkd
Network troubleshoot and checking instructions
Check all interfaces status
root@ni-e320-3238BA5:~# networkctl status
● State: routable
Address: 10.30.10.16 on eth0
10.30.1.16 on sfp0
169.254.0.1 on int0
Check eth0
network config. Verify that it is using 10-eth0.link
and 40-eth0.network
network file.
root@ni-e320-3238B97:~# networkctl status eth0
* 2: eth0
Link File: /lib/systemd/network/99-default.link
Network File: /lib/systemd/network/40-eth0.network
Type: ether
State: carrier (configuring)
Path: platform-e000b000.ethernet
Driver: macb
HW Address: 00:80:2f:33:af:9d (NATIONAL INSTRUMENTS CORP.)
MTU: 1500 (min: 68, max: 1500)
Queue Length (Tx/Rx): 1/1
Auto negotiation: yes
Speed: 1Gbps
Duplex: full
Port: mii
Connected To: poe-switch-01 on port mg2
Check advertised link speeds and connection staatus via:
root@ni-e320-3238BA5:~# ethtool eth0
Settings for eth0:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 100baseT/Half 100baseT/Full
1000baseT/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 1000Mb/s
Duplex: Full
Port: MII
PHYAD: 7
Transceiver: internal
Auto-negotiation: on
Link detected: yes
Update E320 FPGA on NI design
In this section we update the FPGA image to version 4.3.0.0 and the XG version to work with the SFP port and 9000 mtu. Put back the sd card into E320, load NI design if you haven't and make sure the host is connected to it via network. Load fpga image by running the following command on the host
sudo uhd_image_loader --args "type=e3xx,mgmt_addr=192.168.2.4,fpga=XG"
Probe the Device from host if you needed
uhd_usrp_probe --args "mgmt_addr=192.168.10.3,addr=192.168.20.3"
uhd_usrp_probe --args "type=e3xx"
uhd_config_info --print-all
Enable autoboot
This should be done on NI design. Enable autoboot on the SDR as show here: kb.ettus.com/E320_Getting_Started_Guide#Autoboot.
root@ni-e320-3238B97:~# eeprom-set-flags $((0x00000008 | 0x1))
-- PID/REV: e320 0005
-- MCU_FLAGS[0]: 00000008
-- MCU_FLAGS[1]: 00000000
-- MCU_FLAGS[2]: 00000000
-- MCU_FLAGS[3]: 00000000
-- Serial: 3238B97
-- eth_addr0: 00:80:2f:33:af:9d
-- eth_addr1: 00:80:2f:33:af:9e
-- eth_addr2: 00:00:00:00:00:00
-- DT-Compat/MCU-Compat: 0000 0002
-- Max-REV: 0004
-- CRC: 4b477607 (matches)
-- Reading back
-- PID/REV: e320 0005
-- MCU_FLAGS[0]: 00000009
-- MCU_FLAGS[1]: 00000000
-- MCU_FLAGS[2]: 00000000
-- MCU_FLAGS[3]: 00000000
-- Serial: 3238B97
-- eth_addr0: 00:80:2f:33:af:9d
-- eth_addr1: 00:80:2f:33:af:9e
-- eth_addr2: 00:00:00:00:00:00
-- DT-Compat/MCU-Compat: 0000 0002
-- Max-REV: 0004
-- CRC: c41f5914 (matches)
You need to upgrade the embedded controller's firmware so it honors the autoboot setting.
wlab@forlong:~$ curl https://files.ettus.com/binaries/misc/upgrade_mcu_neon_v1.1.7358-a190641-musl-glibc-rev3-7.tar.gz -o upgrade_mcu_neon_v1.1.7358-a190641-musl-glibc-rev3-7.tar.gz
wlab@forlong:~$ scp upgrade_mcu_neon_v1.1.7358-a190641-musl-glibc-rev3-7.tar.gz root@10.30.10.10:/home/root/
root@ni-e320-3238B97:~# tar zxf upgrade_mcu_neon_v1.1.7358-a190641-musl-glibc-rev3-7.tar.gz
root@ni-e320-3238B97:~# cd upgrade_mcu_neon_v1.1.7358-a190641-musl-glibc-rev3-7
root@ni-e320-3238B97:~/upgrade_mcu_neon_v1.1.7358-a190641-musl-glibc-rev3-7# ./flash-firmware.sh
Type y
on both questions. Then reboot shutdown -r now
.
Verify that autoboot is enabled by unplugging and replugging the PoE cable. The SDR must boot up without the need to press the power button.
Switch to Mango Deign and check it
Open UART screen or use SSH to run the following commands.
Switch to Mango WLAN from NI:
root@ni-e320-3238B97:~# cp /uboot/mango_bootbin/BOOT.bin /uboot/
Switch back to NI Reference design from Mango:
root@mango_wlan_linux_E320:~$ cp /media/card/ni_bootbin/boot.bin /media/card/
Then, run a reboot
.
Check the eth static ip:
wlab@forlong:~$ ping 10.30.10.5
PING 10.30.10.5 (10.30.10.5) 56(84) bytes of data.
64 bytes from 10.30.10.5: icmp_seq=1 ttl=64 time=0.515 ms
64 bytes from 10.30.10.5: icmp_seq=2 ttl=64 time=0.275 ms
64 bytes from 10.30.10.5: icmp_seq=3 ttl=64 time=0.272 ms
Run WiFi on Mango Design
Every time mango design boots up, modprobe should be run to bring up the wlan driver, then we change the mac address from default to another one in order to resolve mac address conflicts.
modprobe mango_wlan
iw wlan0 del
iw phy mango-wlan-phy interface add wlan0 type managed addr 40:d8:55:04:20:19
ip link set wlan0 up
./run_ap.sh
Fix UART preventing boot in Mango design
Mango design does not boot without a micro USB cable (UART) connected. In order to resolve that, we use the following hack:
Prepare a microUSB to USB cable, where the +5V wire and GND wire are untouched but data wires are grounded (solder them to the GND wire).
Connect the USB socket to E320's USB port, connect the microUSB socket to E320's microUSB port.
This way the E320's UART transciver which should be powered externally comes up, but since data pins are grounded, it finds nothing and continues.
Read more here: https://support.xilinx.com/s/question/0D52E00006iHma8SAC/zc702-booting-stops-without-uart?language=en_US
Use sdr-tools
package to control the SDR
sdr-tools
package to control the SDRThe sdr-tools
Python package is developed to control the ExPECA SDRs from any device with network connection to the management port. We show how to do that on the controller node. It is containerized:
For example to change the design on SDR 3 to mango, run the following on the controller:
docker run -it --rm -e SERVICE='change_design' -e DESIGN='mango' -e SDR='sdr-03' -e JSON_PATH='sdrs.json' --network host samiemostafavi/sdr-tools
Tenant Network Configuration
When the device is ready to be attached to the network, there will be 2 connections to the tenants switch: one rj45 and one sfp. Write down which ports they are connected to e.g. "te1/0/28" and "te2/0/15". Later, we register these two ports as one network segment in Openstack after deployment. Therefore, you must make sure that on the tenant switch, they are cleared of any default/native access VLANs. This is necessary to avoid access from other open ports on the switch.
Last updated