Pi Racer

Building autonomous car using raspberry pi and a car kit.

Overview

In the small scale AI car projects, AWS DeepRacer Neo is the king with stereo cameras and a LIDAR and overall developer experience, where you would only focus on the Reinforcement Learning function. However this project aims to achieve similar results combining techniques from other similar projects using easily procurable components.

Similar Projects

• WaveShare DonkeyCar

• Pi Car using a different kit but similar configuration

• Pi Pico Car for under $100

• JetRacer using NVIDIA Jetson

• DonkeyCar

• DuckieTown

• FITENTH

Hardware

Total cost : $415

Part	Cost	Rationale
Car base	219	Base includes a servo for steering, motor for throttle, a custom PCB that includes battery case, includes raspberry pi camera and cable with screws and cables to assemble the car. For my order, the car came partially assembled.
Batteries	39	18650 batteries with a flat top.
RPI 4	75	Pi 4 with 8GB RAM
NCS 2	70	Intel VPU - NCS 2 for OpenVINO
microSD Card	10	Recommend at least 32GB card to allow sufficient space for keeping training data.

Software

OS Image

The latest Raspberry Pi OS, BullsEye comes with SSH disabled and installs python 3.9. I had trouble getting and installing Tensorflow and other ML frameworks. So I decided to use the legacy 32-bit OS (Buster). All the instructions below are for a Mac. I have been using Balena Etcher for writing images to microSD card.

# Get the buster image
wget https://downloads.raspberrypi.org/raspios_oldstable_lite_armhf/images/raspios_oldstable_lite_armhf-2021-12-02/2021-12-02-raspios-buster-armhf-lite.zip

WiFi Password and SSH

Once Etcher writes the images to microSD card, it ejects the card, so pull the card out and re-insert to get boot volume mounted in MacOS. If boot volume is not writable, remount the volume for writing.

sudo mount -u -w /Volumes/boot
touch /Volumes/boot/wpa_supplicant.conf
touch /Volumes/boot/ssh

Set ssid and psk appropriate for your setup and add the following contents to /Volumes/boot/wpa_supplicant.conf

country=US
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

network={
    ssid=""
    psk=""
}

Once the above changes are done. Eject the microSD card from PC/Mac and insert that into Raspberry Pi and turn the car board on. Find the RaspberryPi ip address using nmap.

sudo nmap -sP 192.168.68.0/24 | awk '/^Nmap/{ipaddress=$NF}/B8:27:EB/{print ipaddress}'

Once you find the raspberry pi ip address, ssh into it.

Use the command below to enter configuration menu, Enable I2C interface, Enable sshd server, Enable Camera interface, Set the appropriate Timezone.

sudo raspi-config

Install packages

sudo apt-get clean
sudo apt-get autoremove
sudo apt-get update && sudo apt-get upgrade
sudo apt-get install build-essential cmake pkg-config
sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev
sudo apt-get install libfontconfig1-dev libcairo2-dev
sudo apt-get install libgtk2.0-dev libgtk-3-dev
sudo apt-get install libatlas-base-dev gfortran
sudo apt-get install libhdf5-dev libhdf5-serial-dev libhdf5-103
sudo apt-get install libqtgui4 libqtwebkit4 libqt4-test python3-pyqt5
sudo apt-get install python3-dev
sudo apt-get install python-picamera python3-picamera
sudo apt-get install python3-pandas

OpenCV

If you are using Raspberry Pi with < 8GB RAM or in general it is better to increase swap file size to 2GB (2048) by using the following commands:

sudo nano /sbin/dphys-swapfile

sudo nano /etc/dphys-swapfile

sudo reboot

For installing OpenCV on buster, I followed this blog post to get OpenCV successfully installed and working. Direct installation of OpenCV does not work as 32 bit direct installation of OpenCV binaries for arm processor are not available. The shell script below installs pre-requisites and compiles OpenCV from source.

wget https://github.com/Qengineering/Install-OpenCV-Raspberry-Pi-32-bits/raw/main/OpenCV-4-5-5.sh
chmod +x OpenCV-4-5-5.sh
./OpenCV-4-5-5.sh

Install python packages

pip install --upgrade pip
pip install flask
pip install numpy

Install tensorflow

Currently it is difficult to get official Google tensorflow distributions for Raspberry PI. Some developers publish pre-compiled releases for different Raspberry Pi OS distributions as Python wheels. It is easier to use them instead of compiling on our own:

sudo pip install https://github.com/lhelontra/tensorflow-on-arm/releases/download/v1.14.0-buster/tensorflow-1.14.0-cp37-none-linux_armv7l.whl

Install Intel libraries

The Rpi4 due to its power constraints does not include a full blown GPU, so adding the Intel Neural Compute Stick 2 is a great option to offload some image processing.

wget https://storage.openvinotoolkit.org/repositories/openvino/packages/2021.4.2/l_openvino_toolkit_runtime_raspbian_p_2021.4.752.tgz
sudo mkdir -p /opt/intel/openvino_2021
sudo tar -xf  l_openvino_toolkit_runtime_raspbian_p_2021.4.752.tgz --strip 1 -C /opt/intel/openvino_2021
source /opt/intel/openvino_2021/bin/setupvars.sh
echo "source /opt/intel/openvino_2021/bin/setupvars.sh" >> ~/.bashrc

After the above, exit the shell window to pi and do a ssh login again so that .bashrc takes effect. Run the script to update dependencies.

sudo usermod -a -G users "$(whoami)"
sh /opt/intel/openvino_2021/install_dependencies/install_NCS_udev_rules.sh

Run a demo app to make sure Intel NCS2 device and libraries are installed correctly. You need a picture with a human face that when run through the demo, outputs a bmp file which contains a bounding box on the face input image.

mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-march=armv7-a" /opt/intel/openvino_2021/deployment_tools/inference_engine/samples/cpp
make -j2 object_detection_sample_ssd
git clone --depth 1 https://github.com/openvinotoolkit/open_model_zoo
cd open_model_zoo/tools/model_tools
python3 -m pip install -r requirements.in
python3 downloader.py --name face-detection-adas-0001
./armv7l/Release/object_detection_sample_ssd -m ./open_model_zoo/models/intel/face-detection-adas-0001.xml -d MYRIAD -i /home/pi/input.jpeg

Install pi-display

This particular model of the car by WaveShare includes a OLED display on the board. This requires I2C bus to be enabled on Raspberry Pi. By default, the display server shows, the ip-address of the pi, battery level including voltage, amperage, and percent of battery available. CPU, memory and temperature. The code includes an install.sh script which installs dependencies using sudo. This would cause problems with running the included systemd service. To keep the code intact without changes, I had to run the packaged install.sh and change ownership of python libraries to the user pi and modify the systemd service to use User: pi

git clone https://github.com/waveshare/pi-display
cd pi-display/
sudo ./install.sh

pi@raspberrypi:~ $ cat /etc/systemd/system/picard_display.service

[Unit]
Description=JetCard display service

[Service]
Type=simple
# Change to pi instead of the default root
User=pi
ExecStart=/bin/sh -c "python3 -m pidisplay.display_server"
WorkingDirectory=/home/pi
Restart=always

[Install]
WantedBy=multi-user.target

Install dokeycar

git clone https://github.com/autorope/donkeycar
cd donkeycar
git checkout master
pip install -e .[pi]

Pi Racer Test Drive

DonkeyCar lets you create a logical car for customization. Using donkey createcar command you can create as many logical car configurations and customizations for the same physical car.

Create a car

Using donkeycar you create a software car that you use to configure, calibrate, drive, collect data and train models etc.

donkey createcar --path ~/pi-racer

Once you create the car with the above command. Use the following configure servo (steering) and motor (throttle).

Calibration

There is an incorrect command for steering calibration as detailed here: https://www.waveshare.com/wiki/DonkeyCar_for_Pi-Calibrate_DonkeyCar but as per https://github.com/autorope/donkeycar/blob/a26bb7300e0c6ee348876a08491ee172c169203e/donkeycar/templates/cfg_basic.py

The steering channel is 1 and the throttle channel is 0.

Throttle Calibration

donkey calibrate --channel 0 --bus=1

Steering Calibration

donkey calibrate --channel 1 --bus=1

Gamepad Operation

For moving the car using the GamePad (supplied with car kit), use the car software repo you have created earlier. The --js flag enables the gamepad operation, without it, you can operate the car using http://pi-ip-address:8887 webpage.

cd pi-racer/
python3 manage.py drive --js