Eye of the Pi

IP cameras using the Raspberry Pi have been a “thing” since the first Model B released back in 2012. It’s almost an initiation ceremony now to make one!

NoIR cameras are neat. Since they lack an IR filter, they’re able to capture pretty decent images in dark environments (provided that there’s a source of IR light hitting the scene!).

They get a little strange in daytime though. Because of the lacking filter, images tend to have a red hue over everything, which isn’t particularly great!

Alright. So I figured, can the strengths of both “standard” and NoIR cameras be combined into a one “super-camera” that handles most lighting conditions? It turns out that yes, yes they can! And, that camera can be easily streamed over your local network for other applications to use as input.

If you just want to see how it all fits together, check out this video:

Hardware

For this project, you’ll need the following:

• Raspberry Pi Zero W (I used the WH variant to avoid having to solder on the GPIO header)
• Arducam Doubleplexer Stereo Module — make sure this is marked as version UC-444 on the underside
• Raspberry Pi Camera Module
• NoIR Raspberry Pi Camera Module — you DON’T want the one with integrated IR LEDs, since it doesn’t work with the Doubleplexer

Total cost should be around £80.

Alternatively to using the Stereo Module plus the two cameras, you could also use a single IR-CUT camera. You will need to modify the code used later on to toggle it between the day and night modes. (Unfortunately, I only found this exists AFTER finishing this project!).

You’ll also need a way of illuminating the area you want to see at night, with IR light. I did this by cobbling together a USB IR LED, by hacking together an LED from this camera and a spare USB cable. Other options include USB-powered IR LEDs such as this: https://www.amazon.co.uk/Benkeg-Rechargeable-Illuminator-Photography-Accessory-Black/dp/B08869HTH7

Assembly

Putting together the parts is fairly straightforwards:

1. Slot the Stereo Module onto the respective GPIO pins
2. Install the connecting ribbon cable onto the module, and the CSI port of the Raspberry Pi
3. Slot the colour camera’s ribbon cable into the slot marked “Camera A” on the Stereo Module
4. Repeat the above for the NoIR camera into the slot marked “Camera B”

Assembly process

If you want a decent case for this project, you can download 3D-printable files for it here: https://www.thingiverse.com/thing:4866110

Flashing (not the clothes kind)

We’ll be using the latest release of Raspbian as the base OS for this. Download the “Lite” version of it, since we won’t need a desktop running to set everything up.

Have a search for an SD card in the bottom of your drawers. You’ll want one that can store at least 4GB. Using balenaEtcher, flash the downloaded Raspbian image to the card:

Once that finishes, there’s a couple more steps left to do on your computer: enable SSH on the Pi, and configure WiFi.

You want to do the following:

• Navigate to the boot drive that should now show up on your computer. On macOS, this will be found under /Volumes/boot.
• Create an empty file (with no file extension!) named ssh.

Next is WiFi configuration. This is done by creating a file named wpa_supplicant.conf also on this boot drive. It should have the following content:

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=<your country code eg US for United States>

network={
     ssid="Your network name/SSID"
     psk="Your WPA/WPA2 security key"
     key_mgmt=WPA-PSK
} 

Bear in mind that the Raspberry Pi Zero can only connect to 2.4GHz networks with its built-in wireless hardware.

Additionally, if you want to connect to a hidden network, add the following line below key_mgmt:

scan_ssid=1

For more information, see this article.

Now that these two files are created, eject the SD card from your computer. You can now insert it into the Pi, and boot it up!

First things first

Now that the Pi is up and running, its time to connect! This is done using SSH. You can use PuTTY on Windows for this, or on macOS the built-in Terminal app. I’m using macOS for this guide.

You’ll need to find the IP address your wireless router has given your Pi. In my case, it was assigned 192.168.1.5. You should be be able to login to your router to find this out! Otherwise, trial and error is an option you can try.

Using the IP you found for the Pi, its time to login:

ssh pi@<ip address>

You’ll be prompted for a password; this is raspberry by default.

Cool, we’re in. 😎

Photo by Markus Spiske on Unsplash

First thing to do is make sure everything is up to date. Type the following to do an update of all your installed packages:

sudo apt update && sudo apt upgrade -y

Next, we want to enable the camera connector and I2C, adjust the split of RAM assigned to the CPU vs GPU, and also expand the filesystem to match the size of the SD card.

To do this, open the Raspberry Pi configuration utility:

sudo raspi-config

You’ll be greeted by this:

Navigation is done via the keyboard arrow buttons, the TAB key to choose the bottom options, the ENTER key to confirm, and ESC to go back.

First, go into Interface Options > Camera. Answer Yes to the prompt that then shows up:

After confirming, you’ll be taken back to the start.

Go back into Interface Options , and this time select I2C. Again, hit Yes to enable it.

Next, the RAM split needs adjusting. Go to Performance Options > GPU Memory. You should set this to be at least 168, which is needed to handle the overhead of the camera inputs.

My Terminal colour scheme doesn’t like the input field here 😔

Hit TAB after you’re done typing, and then Enter to confirm.

Finally, the filesystem should be expanded to the size of your SD card. From the start, go to Advanced Options > Expand Filesystem. This will immediately start the resize process, and you’ll see something like this:

To prevent some weird and unexpected stability problems, you also should ensure WiFi power management is disabled. This can be done via:

sudo nano /etc/rc.local

And adding iwconfig wlan0 power off above exit 0:

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

# Print the IP address
_IP=$(hostname -I) || true
if [ "$_IP" ]; then
  printf "My IP address is %s\n" "$_IP"
fi
# Disable WiFi power management
iwconfig wlan0 power off

exit 0

Now, reboot, and we’re ready to start installing stuff!

sudo reboot

It’s a good idea at this point to configure a static IP for your Raspberry Pi. This is so you don’t have to re-lookup its IP if it changes on your local network. You can probably do this via your router’s settings, where many let you always assign the same IP to a given MAC address.

The fun part

We’ve gotten the Pi ready for installing things on!

To stream video from our cameras, we need two things: a way of getting the current camera feed, and a way to make it available over the local network. To do this, we’ll be using two utilities: gstreamer and rtsp-simple-server.

In a nutshell, gstreamer will be setup to pipe video into rtsp-simple-server, which then advertises an RTSP stream that other applications (like VLC) can connect to.

If you haven’t already, connect back to the Pi via SSH, then run the following to install all necessary dependencies:

sudo apt install -y gstreamer1.0-tools gstreamer1.0-rtsp gstreamer1.0-plugins-bad git autoconf automake libtool pkg-config libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libraspberrypi-dev

This will install gstreamer, along with build tools we need. However, there is a missing plugin we’ll need to install from source code: https://github.com/thaytan/gst-rpicamsrc

To do this, type the following:

cd ~
git clone https://github.com/thaytan/gst-rpicamsrc.git
cd gst-rpicamsrc
./autogen.sh --prefix=/usr --libdir=/usr/lib/arm-linux-gnueabihf/
make
sudo make install

The autogen and make steps will take some time to run.

Next, we need to download and install rtsp-simple-server. These steps should do the magic for you:

cd ~
wget https://github.com/aler9/rtsp-simple-server/releases/download/v0.15.3/rtsp-simple-server_v0.15.3_linux_arm6.tar.gz
tar -xvf rtsp-simple-server_v0.15.3_linux_arm6.tar.gz

Now, its just a case of editing its respective configuration file:

mv rtsp-simple-server_v0.15.3_linux_arm6 rtsp
cd rtsp
nano rtsp-simple-server.yml

You want to modify this file towards the bottom, in the paths section:

paths:
  stream:
    runOnInit: gst-launch-1.0 rpicamsrc preview=false bitrate=2000000 keyframe-interval=50 ! video/x-h264,width=1280,height=720,framerate=25/1 ! h264parse ! rtspclientsink location=rtsp://localhost:$RTSP_PORT/$RTSP_PATH
    runOnInitRestart: yes

This configures an RTSP stream using gstreamer as the source, which in turn reads from the CSI port for video data. You can modify the video size here, as well as the framerate, if you encounter problems with the stream.

Finally, start the RTSP server by running this command in the same directory:

./rtsp-simple-server

Using an application like VLC, you can now view this stream by connecting to:

rtsp://<ip of pi>:8554/stream

Handling sunrise and sunset

A pre-requisite is node.js , which is installed differently if you’re using the Raspberry Pi Zero. The following instructions assume this; if using any other model you should be able to install node.js from your package manager.

As of v12 onwards, armv6 devices are no longer supported by the node.js team. However, we can still download and install unofficial builds. For this project, we’ll use the last official version, v11.15.0 :

cd ~
mkdir node && cd node
wget https://unofficial-builds.nodejs.org/download/release/v11.15.0/node-v11.15.0-linux-armv6l.tar.xz
tar xvfJ *.xz
sudo cp -R node-v11.15.0-linux-armv6l/* /usr/local

You should now be able to run node --version:

$ node --version
v11.15.0

Download the dependencies needed for the camera toggler:

sudo apt install pigpio git

And now, we can also download the utility itself. We’re using git to clone the relevant repository from GitHub:

cd ~
git clone https://github.com/Matchstic-Project/time-aware-pi-camera.git
cd time-aware-pi-camera
npm install
npm run build
# Test the service starts up
sudo node dist

You should see something like this when trying to run it:

The utility looks for a configuration file where you can specify your own latitude and longitude. This is then used to correctly calculate sunrise and sunset times relative to you; otherwise, it defaults to the times seen by London.

To configure this, create a new file:

sudo nano /boot/camera.json

Paste this into the new file, and modify the latitude and longitude fields to match your location:

{
    "latitude": 51.5007, 
    "longitude": 0.1246 
}

Stayin’ alive

Everything should now be all operational, so up next is making sure it all works after a reboot!

RTSP server

Create a new file with:

sudo nano /lib/systemd/system/rtsp.service

And then paste this:

[Unit]
Description=RTSP server
After=multi-user.target
[Service]
ExecStartPre=/bin/sleep 10
Type=simple
WorkingDirectory=/home/pi/rtsp
ExecStart=/home/pi/rtsp/rtsp-simple-server
Restart=always
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=rtsp-server
[Install]
WantedBy=multi-user.target

Don’t forget to save!

Next, we need to tell systemd to enable this new service, which is done by simply doing the following:

sudo systemctl enable rtsp

It’ll take about 10 seconds to run — this is due to the ExecStartPre line, which waits 10 seconds before starting the server. This is a workaround to ensure the camera hardware is initialised before trying to read video data; without this, the stream usually fails to start on reboot.

Camera toggler

To make this utility persistent, its pretty much the same procedure:

sudo nano /lib/systemd/system/camera-toggle.service

Paste:

[Unit]
Description=Camera toggle utility
After=multi-user.target
[Service]
Type=simple
WorkingDirectory=/home/pi/time-aware-pi-camera
ExecStart=/usr/local/bin/node /home/pi/time-aware-pi-camera/dist
Restart=always
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=camera-toggle-utility
Environment=NODE_ENV=production
User=root
Group=root
[Install]
WantedBy=multi-user.target

And finally, enable it:

sudo systemctl enable camera-toggle

[optional] Motion detection

Want to save video for later viewing when motion is detected? This is how!

Bear in mind though you will need a secondary (Linux) device to run this; the Raspberry Pi Zero is not powerful enough to handle both streaming video, and processing it. I used a Raspberry Pi Model 4B for this.

After setting up your secondary device with its OS image and applying any pending updates via apt, you can proceed to install motion. This is an open-source video monitoring system, handling various different camera sources. In our case, it supports RTSP which means… the camera you just created will work!

The configuration is pretty simple:

sudo apt install motion

Then, edit its configuration file:

sudo nano /etc/motion/motion.conf

Then, adjust the following values — they’re not all in the same place, so don’t just copy and paste!

width 1280
height 720
framerate 5
netcam_url rtsp://<IP of camera>:8554/stream
netcam_keepalive on
target_dir /home/pi/recordings
stream_port 0
webcontrol_port 0 
ffmpeg_output_movies on
ffmpeg_variable_bitrate 75
movie_filename %Y-%m-%d-%H%M%S
start_motion_daemon yes

Note the target_dir field — this specifies where recordings are stored. You can go one step beyond, and use this to auto-upload files to services like Google Drive and DropBox. The general idea is configure such a service to be mounted to a certain folder (e.g., /home/pi/dropbox) and set target_dir to that folder. Tada! 🎉

To finish up the install, just restart the motion daemon:

sudo service motion restart

[optional] HomeKit integration

If you’re on an Apple device, chances are you have access to HomeKit. This means you can do some cool home automation stuff on your local network, including viewing camera streams.

One problem though; your shiny new camera is not HomeKit certified, and cannot be added directly in the Home app. To solve this, we’ll be using HomeBridge and a custom plugin that exposes your camera. This all needs to run on a secondary device for the same reason that motion needs to.

I’m going to skip over the HomeBridge setup process here. So, instead, here’s a link to a useful guide for that: https://github.com/homebridge/homebridge/wiki/Install-Homebridge-on-Raspbian

Alright, so you have a working HomeBridge setup now right?

Cool.

We’ll be using the homebridge-camera-ffmpeg plugin to stream your camera into HomeKit. This is a fairly straightforwards installation:

sudo npm install -g homebridge-camera-ffmpeg --unsafe-perm

Next, you’ll want to update the HomeBridge config file. You can do that via the web interface, or by editing the config file directly ( /var/lib/homebridge/config.json).

The main part you want to add from below is inside the platforms section — I’ve include the full config file so you can see how it should look once done.

{
    "bridge": {
        "name": "Homebridge",
        "username": "00:00:00:00:00:00",
        "port": 51767,
        "pin": "XXX-XX-XXX"
    },
    "platforms": [
        {
          "platform": "Camera-ffmpeg",
          "cameras": [
            {
                "name": "<name of camera shown in Home app>",
                "videoConfig": {
                    "source": "-i rtsp://<camera ip>:8554/stream",
                    "maxStreams": 1,
                    "maxWidth": 1280,
                    "maxHeight": 720,
                    "maxFPS": 15,
                    "maxBitrate": 299,
                    "maxPacketsize": 752,
                    "audio": false,
                    "additionalCommandline": "-fflags +genpts+discardcorrupt"
                },
                "serialNumber": "00-00-00",
                "model": "Raspberry Pi",
                "manufacturer": "Matchstic Project",
                "firmwareRevision": "1.0"
            }
          ]
        }
    ]
}

Make sure to update the source field to specify the IP address of your Raspberry Pi Zero.

Restart HomeBridge, or just restart your secondary device:

sudo reboot

Once HomeBridge is back online, head over to any of your Apple devices, and open the Home app. Navigate to the Default Room page, and you should now see your camera listed as Pi Camera! 🎥