Install dlib on the Raspberry Pi

Source by Adrian Rosebrock 

A few weeks ago I did a blog post on how to install the dlib library on Ubuntu and macOS.

Since Raspbian, the operating system that (most) Raspberry Pi users run is Debian-based (as is Ubuntu), the same install instructions can be used for Raspbian as Ubuntu…

…however, there’s a catch.

The Raspberry Pi 3 ships with only 1GB of RAM. This 1GB of RAM is responsible for all system operations, displaying the GUI/desktop, and handling our compile.

In fact, if you tried to compile dlib on the Raspberry Pi, your compile likely bombed out with an error.

In order to get dlib with Python bindings to successfully install on your Raspberry Pi, you need to update your system to reclaim as much memory as possible as well as update your swap file size.

Luckily, this isn’t as challenging as it sounds, and if you follow the steps detailed in this guide, you’ll have dlib installed on your Raspbian system in the next 45 minutes.

To learn how to install dlib on your Raspberry Pi, just keep reading.

Install dlib on the Raspberry Pi

The first part of this tutorial discusses the common memory error you’ll encounter when trying to compile dlib with Python bindings on your Raspberry Pi.

We then move into my six step guide to installing dlib on the Raspberry Pi.

Finally, we’ll wrap up with an example demonstrating how to detect facial landmarks using dlib, Python, and the Raspberry Pi.

Running into memory errors? System crashing?

If you used the Ubuntu instructions from my previous dlib install tutorial to install dlib on your Raspberry Pi, your compile likely bombed out with an error message similar to this one:

Figure 1: The dlib compile failing on the Raspberry Pi due to the C++ compiler being killed.

The reason for this error can be found in the highlighted box:

c++: internal compiler error: Killed (program cc1plus)

This error message boils down to the dlib compile program using too much memory and the Raspbian operating system killing the process, preventing the Raspberry Pi from seizing.

However, the preventative measures weren’t enough — 3 out of the 5 times I tried to compile dlib on my Raspberry Pi without updating my swap file size, boot options, and memory split (detailed below), my Pi simply locked up and I had to hard reset it.

In order to get dlib with Python bindings installed on Raspbian, we need to reclaim as much available memory as possible. We’ll dive into how to do this in the next section.

Step #1: Update swap file size, boot options, and memory split

Preparing your Raspberry Pi for the dlib compile is divided into three steps, each of them with the same goal — to increase the amount of memory available for the compile:

1. Increase your swap file size.
2. Switch your boot options.
3. Update your GPU/RAM split.

Note: For nearly all screenshots in this tutorial, I was SSH’d into my Raspberry Pi from my host macOS system (that is why the screenshots are from the macOS terminal rather than the Raspbian terminal).

Increase swap file size

Swap files, or swap space, is space on a hard disk/memory card used as virtual memory as an extension to a system’s real memory (RAM).

Enabling swap files allows your operating system to “pretend” like it has more on-board memory than it actually does. Using the swap file architecture, pages in RAM are “swapped out” to disk until they are needed again in which case they are “swapped in”.

Nearly all major operating systems support swapping to some degree.

In Raspbian, the dphys-swapfile solution is used with a default of 100MB dedicated to swap on our card.

In order to update the size of our swap, we need to edit the dphys-swapfile configuration file, which is located in /etc/dphys-swapfile . You can edit the configuration using your favorite text editor:

Install dlib on the Raspberry Pi

Shell

1	$ sudo nano /etc/dphys-swapfile

Scroll down to the configuration which reads:

CONF_SWAPSIZE=100

And then update it to use 1024MB rather than 100MB:

Figure 2: Updating the Raspbian swap size to be 1024MB rather than 100MB.

Notice here how CONF_SWAPSIZE=1024  — this will be more than enough swap for us to compile dlib on our Raspberry Pi.

Note: Increasing swap size is a great way to burn out your Raspberry Pi card. Flash-based storage have limited number of writes you can perform until the card is essentially unable to hold the 1’s and 0’s anymore. We’ll only be enabling large swap for a short period of time, so it’s not a big deal. Regardless, be sure to backup your .img  file after installing dlib just in case your card dies unexpectedly early. You can read more about large swap sizes corrupting memory cards on this page.

After you have updated the /etc/dphys-swapfile  file, run the following two commands to restart the swap service:

Install dlib on the Raspberry Pi

Shell

1 2	$ sudo /etc/init.d/dphys-swapfile stop $ sudo /etc/init.d/dphys-swapfile start

The start  command will likely take a few minutes to finish executing since the swap size is being increased and re-allocated.

You can then run:

Install dlib on the Raspberry Pi

Shell

1

$ free -m

To confirm that your swap size has been increased.

Here you can see that my swap is 1024MB, up from the previous 100MB:

Figure 3: Running free -m confirms that the new swap size of 1024MB is being used.

Change your boot options

By default, your Raspberry Pi will boot into the the PIXEL desktop (assuming you’re using the latest version of Raspbian). This is fine; except that the PIXEL desktop requires memory — RAM that we desperately need for the dlib compile.

Instead of booting into the PIXEL GUI, we should instead boot directly to the terminal. This will save us a substantial amount of memory.

To accomplish this, execute:

Install dlib on the Raspberry Pi

Shell

1	$ sudo raspi-config

And then select Boot Options => Desktop / CLI => Console Autologin :

Figure 4: How to set your Raspberry Pi to boot to a terminal rather than a desktop.

This will ensure that your Raspberry Pi boots directly to a terminal, skipping the PIXEL desktop.

However, before you exit raspi-config , be sure to update your memory split, as detailed below.

Update your memory split

By default, the Raspberry Pi allocates 64MB of memory to the onboard GPU. We should reduce this number to 16MB to reclaim memory for the compile.

This is simple enough using raspi-config . Go back to the main screen and select AdvancedOptions => Memory Split , where you’ll see the 64MB prompt:

Figure 5: Update your GPU/RAM split to be 16MB rather than 64MB.

Update this value to be 16MB and then exit.

Restart your Raspberry Pi

Upon exiting,  raspi-config  will ask if you would like to reboot your system.

Go ahead and reboot, then we can move on with the rest of the install tutorial.

Step #2: Install dlib prerequisites

The dlib library requires four prerequisites:

1. Boost
2. Boost.Python
3. CMake
4. X11

These can all be installed via the following commands:

Install dlib on the Raspberry Pi

Shell

1 2 3 4

$ sudo apt-get update $ sudo apt-get install build-essential cmake $ sudo apt-get install libgtk-3-dev $ sudo apt-get install libboost-all-dev

The pip  command should already be installed on your Raspberry Pi (you can verify this by executing the pip  command and ensuring that it does indeed exist) — otherwise, you can install pip  via:

Install dlib on the Raspberry Pi

Shell

1 2	$ wget https://bootstrap.pypa.io/get-pip.py $ sudo python get-pip.py

Once you’ve verified that pip  is installed, we can move on.

Step #3: Access your Python virtual environment (if you are using them)

All of my OpenCV install tutorials on the PyImageSearch blog make use of Python virtual environments.

Using Python’s virtualenv and virtualenvwrapper libraries, we can create separate Python environments for each project we are working on — this is considered a best practice when developing software in the Python programming language.

I’ve discussed Python virtual environments many times before on the PyImageSearch blog, so I’ll spare any discussion of them here. If you would like to read more about Python virtual environments please refer to any of my installing OpenCV tutorials along with this excellent Python virtual environment primer.

If you would like to install dlib into a pre-existing Python, virtual environment, use the workon  command:

Install dlib on the Raspberry Pi

Shell

1	$ workon <your virtualenv name>

For example, most tutorials here on PyImageSearch create a virtual environment namedcv . We can access the cv  virtual environment via:

Install dlib on the Raspberry Pi

Shell

1	$ workon cv

Otherwise, I suggesting creating an entirely separate virtual environment using themkvirtualenv  command.

The command below will create a Python virtual environment named py2_dlib  with the Python 2.7 interpreter:

Install dlib on the Raspberry Pi

Shell

1	$ mkvirtualenv py2_dlib

While this command will create a Python 3 virtual environment named py3_dlib :

Install dlib on the Raspberry Pi

Shell

1	$ mkvirtualenv py3_dlib -p python3

Please keep in mind that this step is optional, but highly recommended.

For readers who have followed any of my previous OpenCV install tutorials on PyImageSearch, please make sure you access your Python virtual environment before proceeding to Step #4.

Step #4: Use pip to install dlib with Python bindings

We’ll start with the basic NumPy + SciPy stack, followed by scikit-image, a library commonly used in conjunction with dlib:

Install dlib on the Raspberry Pi

Shell

1 2 3

$ pip install numpy $ pip install scipy $ pip install scikit-image

We can then install dlib via pip  as well:

Install dlib on the Raspberry Pi

Shell

1	$ pip install dlib

On my Raspberry Pi 3, this compile took approximately 40 minutes, but as you can see, the library was compiled successfully with no errors:

Figure 6: Successfully compiling dlib with Python bindings on your Raspberry Pi.

Step #5: Test out your dlib install

To test out your dlib install, open up a Python shell (making sure to access your Python virtual environment if you used one), and then try to import dlib :

Install dlib on the Raspberry Pi

Shell

1 2 3 4 5 6

$ python Python 3.4.2 (default, Oct 19 2014, 13:31:11) [GCC 4.9.1] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import dlib >>>

If you would like to access your OpenCV bindings along with your dlib bindings from the same virtual environment, make sure your cv2.so  bindings are properly sym-linked into the site-packages  directory of your Python virtual environment.

Step #6: Reset your swap file size, boot options, and memory split

Important — before you walk away from your machine, be sure to reset your swap file size to 100MB (using the process detailed in the “Step #1: Increase swap file size” section above).

You can then reset your GPU/RAM split to 64MB as well as update the boot options to boot into the desktop interface versus the command line.

After making these changes, reboot your Raspberry Pi to ensure they take affect.

BONUS: Facial landmark detection with dlib and the Raspberry Pi

As a final example of using dlib on the Raspberry Pi, here is a short example I coded up where we are going to detect facial landmarks in an input image:

Install dlib on the Raspberry Pi

Python

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

# import the necessary packages from imutils import face_utils import dlib import cv2 # initialize dlib's face detector (HOG-based) and then create # the facial landmark predictor p = "shape_predictor_68_face_landmarks.dat" detector = dlib.get_frontal_face_detector() predictor = dlib.shape_predictor(p) # load the input image and convert it to grayscale image = cv2.imread("example.jpg") gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) # detect faces in the grayscale image rects = detector(gray, 0) # loop over the face detections for (i, rect) in enumerate(rects): # determine the facial landmarks for the face region, then # convert the facial landmark (x, y)-coordinates to a NumPy # array shape = predictor(gray, rect) shape = face_utils.shape_to_np(shape) # loop over the (x, y)-coordinates for the facial landmarks # and draw them on the image for (x, y) in shape: cv2.circle(image, (x, y), 2, (0, 255, 0), -1) # show the output image with the face detections + facial landmarks cv2.imshow("Output", image) cv2.waitKey(0)

For a detailed review of the code above used for facial landmark prediction, please refer to my previous post on the basics of facial landmarks.

To execute the script:

• Make sure you’ve installed/upgraded the imutils library via pip install --upgradeimutils .
• Use the “Downloads” section below to download the code + example image + pre-trained dlib facial landmark predictor).

From there, you can issue the following command:

Install dlib on the Raspberry Pi

Shell

1	$ python pi_facial_landmarks.py

You should then see the following output with the facial landmarks correctly displayed on the input image:

Figure 7: Detecting facial landmarks on an image using dlib on the Raspberry Pi.

Fun Fact: The picture above is of me during my undergraduate college days — I actually had hair back then!

Unfortunately, from my tests, the Raspberry Pi 3 is not fast enough to perform facial landmark detection in real-time. In my experiments there was a significant lag between polling the next frame from the video stream, followed by applying facial landmark prediction, even when using a threaded video stream for optimal performance. The reason for this is because the face detection process is extremely slow.

Keep in mind that the Raspberry Pi 3 processor runs at only 1.2Ghz while the processor on my Mac (and likely your machine) runs at 3.1GHz (or similar). This is a huge increase in speed.

Methods to speeding up dlib’s facial landmark predictor can be found here — I will be reviewing them in a future blog post with accompanying Python implementations to help us obtain real-time facial landmark prediction performance on the Raspberry Pi.

Summary

In today’s blog post we learned how to compile and install dlib with Python bindings on the Raspberry Pi.

In order to install dlib without error on the Raspberry Pi, we had to take a number of precautions related to memory usage, including:

1. Increasing the amount of swap available to our system.
2. Booting our Raspberry Pi to the terminal rather than the desktop.
3. Reducing memory allocated to the GPU and giving it to main memory instead.

By performing these three steps we were able to reclaim enough memory to install dlib on our Raspberry Pi. However, now we’ve run into the issue that the Pi isn’t quite fast enough for real-time facial landmark prediction. We’ll address how to speedup facial landmark prediction in a future tutorial.

Finally, I wanted to mention that next week’s tutorial is on drowsiness detection while driving a motor vehicle, a heavily requested blog post here on PyImageSearch:

Figure 8: A preview of next week’s tutorial on drowsiness