A Gentle Introduction to Computer Vision, Part 1: Compiling OpenCV on the Jetson Xavier NX

OpenCV Logo

Introduction to OpenCV

OpenCV is for computer vision. There are a number of programming language interfaces: C++, Python, Java and Matlab. OpenCV also runs on various operating systems, such as Windows, Linux, Android and Mac OS, as well as a number of different architectures. The library has a long history of industry adoption and help from Google, Yahoo, Microsoft, Intel, IBM, Sony, Honda, and Toyota, as well as algorithms being implemented from CVPR papers and other academic publications. With more than 2,500 optimized algorithms, it’s got to be one of the most fun libraries to toy with in your spare time or for your work project.

Compiling OpenCV

Compiling OpenCV is a long process, so let’s get started. First, we must install the package dependencies. The goal of the build is to make as complete of a build as possible, leveraging as many features as possible. I’ll also be installing libjpeg-turbo8 instead of the typical libjpeg.

$ sudo apt-get install build-essential unzip pkg-config libavcodec-dev libavformat-dev libswscale-dev libv4l-dev libxvidcore-dev libx264-dev libgtk-3-dev libatlas-base-dev gfortran python3-dev python3-venv libglew-dev libtiff5-dev zlib1g-dev libpng-dev libavcodec-dev libavformat-dev libavutil-dev libpostproc-dev libswscale-dev libeigen3-dev libtbb-dev libgtk2.0-dev qtbase5-dev libavresample-dev libdc1394-22-dev libopenblas-dev liblapack-dev libgflags-dev libgoogle-glog-dev libbenchmark-dev libsuitesparse-dev libmetis-dev ccache libhdf5-dev libhdf5-mpi-dev libhdf5-openmpi-dev libtesseract-dev libprotobuf-dev apt-utils libjpeg-turbo8 libjpeg-turbo8-dev# Upgrade CMake for VTK
$ sudo apt remove --purge cmake
$ hash -r
$ sudo snap install cmake --classic
$ git clone https://github.com/opencv/opencv.git
$ git clone https://github.com/opencv/opencv_contrib.git
$ git clone https://github.com/opencv/opencv_extra.git
$ git clone https://github.com/ceres-solver/ceres-solver.git
$ git clone https://github.com/Kitware/VTK.git
$ sudo /usr/sbin/nvpmodel -m 2  # 15W, 6 core, 1.4GHz each
$ sudo /usr/sbin/nvpmodel -m 0 # 15W, 2 core, 1.9GHz each
$ mkdir VTK-bin && cd VTK-bin
$ cmake ../VTK
$ make -j$((`nproc`-1)) && sudo make install
$ cd .. && mkdir ceres-solver-bin && cd ceres-solver-bin
$ cmake -D BUILD_EXAMPLES=OFF -D BUILD_TESTING=OFF ../ceres-solver
$ make -$((`nproc`-1)) && sudo make install
$ cd .. && mkdir opencv-bin && cd opencv-bin
$ cmake \
-D ENABLE_PRECOMPILED_HEADERS=OFF \
-D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D OPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules \
-D ENABLE_NEON=ON \
-D ENABLE_VFPV3=OFF \
-D CPU_BASELINE_REQUIRE=NEON \
-D BUILD_TESTS=OFF \
-D INSTALL_PYTHON_EXAMPLES=OFF \
-D OPENCV_ENABLE_NONFREE=ON \
-D CMAKE_SHARED_LINKER_FLAGS=-latomic \
-D BUILD_EXAMPLES=OFF \
-D WITH_QT=ON \
-D WITH_CUDA=ON \
-D EIGEN_INCLUDE_PATH=/usr/include/eigen3 \
-D CUDA_ARCH_BIN=7.2 \
-D CUDA_ARCH_PTX="" \
-D WITH_CUDNN=ON \
-D WITH_CUBLAS=ON \
-D ENABLE_FAST_MATH=ON \
-D CUDA_FAST_MATH=ON \
-D OPENCV_DNN_CUDA=ON \
-D WITH_OPENMP=ON \
-D WITH_OPENGL=ON \
-D WITH_FFMPEG=ON \
-D WITH_GSTREAMER=ON \
-D WITH_TBB=ON \
-D BUILD_TBB=ON \
-D WITH_EIGEN=ON \
-D WITH_V4L=ON \
-D WITH_LIBV4L=ON \
-D BUILD_NEW_PYTHON_SUPPORT=ON \
-D BUILD_opencv_python3=TRUE \
-D OPENCV_GENERATE_PKGCONFIG=ON \
-D WITH_JPEG=ON \
-D BUILD_JPEG=OFF \
-D JPEG_INCLUDE_DIR=/usr/include \
-D JPEG_LIBRARY=/usr/lib/aarch64-linux-gnu/libjpeg.a \
../opencv
  • ENABLE_NEON=ON, ENABLE_VFPV3=OFF, CPU_BASELINE_REQUIRE=NEON, CMAKE_SHARED_LINKER_FLAGS=-latomic are enabling NEON and disabling VFPV3. NEON is compatible with the Arm Cortex-A processor series and for Cortex-R52 and Cortex-R82 processors. VFPV3 is not compatible with the Jetson Xavier NX. The atomic linker flag is required when enabling NEON.
  • WITH_QT=ON will enable useful status bar items that show RGB values on mouseover, for example.
  • CUDA_ARCH_BIN=7.2 is the GPU architecture number for the Jetson Xavier NX’s Volta GPU.
  • JPEG_INCLUDE_DIR=/usr/include, JPEG_LIBRARY=/usr/lib/aarch64-linux-gnu/libjpeg.a is here because the JPEG library was not being detected, and I also decided to use libjpeg-turbo8 instead of the standard libjpeg. The “turbo” version will make use of NEON SIMD instructions to to achieve 2–6x faster encode/decode of JPEG images, which happens to be the most common file format.
OpenCV’s got some bloat.
$ make -j$((`nproc`-1)) && sudo make install
$ sudo ln -s /usr/local/lib/python3.6/site-packages/cv2/python-3.6/cv2.cpython-36m-aarch64-linux-gnu.so /usr/local/lib/python3.6/site-packages/cv2/python-3.6/cv2.so
$ ls ~/dev/envs/venv/
bin include lib lib64 pyvenv.cfg share
$ ln -s /usr/local/lib/python3.6/site-packages/cv2/python-3.6/cv2.cpython-36m-aarch64-linux-gnu.so ~/dev/envs/venv/lib/python3.6/site-packages/cv2.so
$ source ~/dev/envs/venv/bin/activate
(venv) $ python
Python 3.6.9 (default, Oct 8 2020, 12:12:24)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.__version__
'4.5.1'

Troubleshooting

“Illegal instruction (core dumped)” when importing cv2

When importing cv2, you may encounter an issue where an illegal instruction is called:

$ python
Python 3.9.1 (tags/v3.9.1:1e5d33e9b9, Jan 11 2021, 22:02:52)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
Illegal instruction (core dumped)
$ sudo sysctl -w kernel.core_pattern=core
kernel.core_pattern = core
$ ulimit -c unlimited
$ python # Run 'import cv2' to generate a core dump
$ gdb python core
GNU gdb (Ubuntu 8.1.1-0ubuntu1) 8.1.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python...done.
warning: core file may not match specified executable file.
[New LWP 17811]
[New LWP 17819]
[New LWP 17818]
[New LWP 17821]
[New LWP 17820]
[New LWP 17817]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
Core was generated by `python'.
Program terminated with signal SIGILL, Illegal instruction.
#0 0x0000007f4c3d5f54 in gotoblas_dynamic_init ()
from /home/username/dev/envs/venv/lib/python3.9/site-packages/numpy/core/../../numpy.libs/libopenblasp-r0-32ff4d91.3.13.so
[Current thread is 1 (Thread 0x7f84cd35c0 (LWP 17811))]
  • gcc 4.x or later
  • FORTRAN 77 compiler
  • OpenBLAS (The choice and location of these libraries as well as include paths and other such build options can be specified in a site.cfg file located in the NumPy root repository or a .numpy-site.cfg file in your home directory. See the site.cfg.example example file included in the NumPy repository or sdist for documentation, and below for specifying search priority from environmental variables)
  • Cython
(venv) $ sudo apt install libfftw3-dev libopenblas-dev
(venv) $ python --version # requires >=3.6
Python 3.9.1
(venv) $ pip install cython
(venv) $ git clone https://www.github.com/numpy/numpy.git
(venv) $ cd numpy
(venv) $ git checkout -b v0.19.5
(venv) $ git reset --hard && git clean -fd
(venv) $ cp site.cfg.example ~/.numpy-site.cfg
[DEFAULT] 
library_dirs = /usr/local/lib64:/usr/local/lib:/usr/lib64:/usr/lib
include_dirs = /usr/local/include:/usr/include
[openblas]
libraries = openblas
library_dirs = /usr/lib/aarch64-linux-gnu
include_dirs = /usr/include/aarch64-linux-gnu
runtime_library_dirs = /usr/lib/aarch64-linux-gnu
[fftw]
libraries = fftw3
(venv) $ python setup.py build -j $(nproc) install
(venv) $ python
Python 3.9.1 (tags/v3.9.1:1e5d33e9b9, Jan 11 2021, 22:59:09)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.__version__
'4.5.1'

Summary

  • Installed dependencies and compiled Ceres Solver and VTK from source.
  • Compiled OpenCV from source.
  • Symlinked compiled library to the Python cv2 module in our virtual environment.
  • Resolved issues with the Numpy package installed from pip by building the Numpy wheel.

Further Resources

Imaging Systems Ninjaneer, Computer Vision, Photographer and Videographer, VR Athlete, Pianist

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store