Build OpenCV with CUDA in docker
If you want to use OpenCV with CUDA, you have to build it yourself. I haven't researched why, but it is very likely related to the Nvidia drivers and software not being open source. You have to download a lot of stuff from the Nvidia servers. Anyway, I needed CUDA acceleration, so here we go.
To build and run OpenCV you need to install a lot of libraries. I don't like that as it clutters my development machine and gives me a build/run environment that is non-repeatable. I have no idea what is floating around on my machine, so when I want to move my OpenCV program to another machine, will it work?
PS. If you don't want to bother building, check this repo: https://github.com/cudawarped/opencv-python-cuda-wheels
Virtual build machine
The problem
Therefor I wanted to build OpenCV on another machine. Preferably a virtual machine. This gives me an isolated environment to work in. Luckily you don't need a GPU inside the VM to build with CUDA. Only the libraries need to be installed.
Initially I used Ubuntu 22.04 which comes with Python 3.10. This was the same version I had on my dev-pc. In the end it was successful and I could copy the build binaries and libraries, link them and run my OpenCV Python program.
Then I decided it was about time to upgrade to 24.04. And everything broke. The Python version is now 3.12 and installing dependencies globally with pip
doesn't work anymore.
joep@ubuntu2404-build:~/repositories/opencv/build$ pip install numpy
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.
If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.
If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.
See /usr/share/doc/python3.12/README.venv for more information.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
This was one of the reasons I wanted to use a VM in the first place. So I didn't need a virtual python environment. But yeah...
Working solution part 0: the environment
- Ubuntu 24.04
- Python 3.12
- OpenCV 4.11.0
- CUDA 12.8
- CUDNN 9.8
- Build using
ninja
Working solution part 1: build path flags
After days of (blindly) changing some build parameters, waiting (~20 minutes) for the build to finish using docker (see below) and quickly finding it didn't work, I took a step back and tried a more targeted approach. First question: "What do all these build flags do exactly?".
There are 5 flags that are given a path: CMAKE_INSTALL_PREFIX
, OPENCV_PYTHON3_INSTALL_PATH
, PYTHON3_EXECUTABLE
, PYTHON3_INCLUDE_DIR
and PYTHON3_PACKAGES_PATH
. All of them are configured in different ways in all the examples online. None of them is explained in detail, so I can't either. However, I tried changing them all and figure out what is changed in the build parameters.
For context, I want the build to end up in /home/joep/repositories/.venv/lib/python3.12/site-packages/cv2
. So I ended up with only setting CMAKE_INSTALL_PREFIX=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
.
==============================================================================================================================
-D CMAKE_INSTALL_PREFIX=$(which python3)
-D OPENCV_PYTHON3_INSTALL_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
-D PYTHON3_EXECUTABLE=$(which python3)
-D PYTHON3_INCLUDE_DIR=$(python3 -c "from sysconfig import get_paths; print(get_paths()['include'])")
-D PYTHON3_PACKAGES_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
-- Python 3:
-- Interpreter: /home/joep/repositories/.venv/bin/python3 (ver 3.12.3)
-- Libraries: /usr/lib/x86_64-linux-gnu/libpython3.12.so (ver 3.12.3)
-- Limited API: NO
-- numpy: /home/joep/repositories/.venv/lib/python3.12/site-packages/numpy/_core/include (ver 2.3.1)
-- install path: /home/joep/repositories/.venv/lib/python3.12/site-packages/cv2/python-3.12
--
-- Python (for build): /home/joep/repositories/.venv/bin/python3
-- Install to: /home/joep/repositories/.venv/bin/python3
==============================================================================================================================
-D CMAKE_INSTALL_PREFIX=/usr/local
-D OPENCV_PYTHON3_INSTALL_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
-D PYTHON3_EXECUTABLE=$(which python3)
-D PYTHON3_INCLUDE_DIR=$(python3 -c "from sysconfig import get_paths; print(get_paths()['include'])")
-D PYTHON3_PACKAGES_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
-- Python 3:
-- Interpreter: /home/joep/repositories/.venv/bin/python3 (ver 3.12.3)
-- Libraries: /usr/lib/x86_64-linux-gnu/libpython3.12.so (ver 3.12.3)
-- Limited API: NO
-- numpy: /home/joep/repositories/.venv/lib/python3.12/site-packages/numpy/_core/include (ver 2.3.1)
-- install path: /home/joep/repositories/.venv/lib/python3.12/site-packages/cv2/python-3.12
--
-- Python (for build): /home/joep/repositories/.venv/bin/python3
-- Install to: /usr/local
==============================================================================================================================
-D CMAKE_INSTALL_PREFIX=$(which python3)
-D OPENCV_PYTHON3_INSTALL_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
-D PYTHON3_EXECUTABLE=$(which python3)
-D PYTHON3_INCLUDE_DIR=$(python3 -c "from sysconfig import get_paths; print(get_paths()['include'])")
-- Python 3:
-- Interpreter: /home/joep/repositories/.venv/bin/python3 (ver 3.12.3)
-- Libraries: /usr/lib/x86_64-linux-gnu/libpython3.12.so (ver 3.12.3)
-- Limited API: NO
-- numpy: /home/joep/repositories/.venv/lib/python3.12/site-packages/numpy/_core/include (ver 2.3.1)
-- install path: /home/joep/repositories/.venv/lib/python3.12/site-packages/cv2/python-3.12
--
-- Python (for build): /home/joep/repositories/.venv/bin/python3
-- Install to: /home/joep/repositories/.venv/bin/python3
==============================================================================================================================
-D CMAKE_INSTALL_PREFIX=$(which python3)
-D OPENCV_PYTHON3_INSTALL_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
-D OPENCV_PYTHON3_INSTALL_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
-D PYTHON3_EXECUTABLE=$(which python3)
-- Python 3:
-- Interpreter: /home/joep/repositories/.venv/bin/python3 (ver 3.12.3)
-- Libraries: /usr/lib/x86_64-linux-gnu/libpython3.12.so (ver 3.12.3)
-- Limited API: NO
-- numpy: /home/joep/repositories/.venv/lib/python3.12/site-packages/numpy/_core/include (ver 2.3.1)
-- install path: /home/joep/repositories/.venv/lib/python3.12/site-packages/cv2/python-3.12
--
-- Python (for build): /home/joep/repositories/.venv/bin/python3
-- Install to: /home/joep/repositories/.venv/bin/python3
==============================================================================================================================
-D CMAKE_INSTALL_PREFIX=$(which python3)
-D PYTHON3_EXECUTABLE=$(which python3)
-- Python 3:
-- Interpreter: /home/joep/repositories/.venv/bin/python3 (ver 3.12.3)
-- Libraries: /usr/lib/x86_64-linux-gnu/libpython3.12.so (ver 3.12.3)
-- Limited API: NO
-- numpy: /home/joep/repositories/.venv/lib/python3.12/site-packages/numpy/_core/include (ver 2.3.1)
-- install path: /home/joep/repositories/.venv/lib/python3.12/site-packages/cv2/python-3.12
--
-- Python (for build): /home/joep/repositories/.venv/bin/python3
-- Install to: /home/joep/repositories/.venv/bin/python3
==============================================================================================================================
-D PYTHON3_EXECUTABLE=$(which python3)
-- Python 3:
-- Interpreter: /home/joep/repositories/.venv/bin/python3 (ver 3.12.3)
-- Libraries: /usr/lib/x86_64-linux-gnu/libpython3.12.so (ver 3.12.3)
-- Limited API: NO
-- numpy: /home/joep/repositories/.venv/lib/python3.12/site-packages/numpy/_core/include (ver 2.3.1)
-- install path: /home/joep/repositories/.venv/lib/python3.12/site-packages/cv2/python-3.12
--
-- Python (for build): /home/joep/repositories/.venv/bin/python3
-- Install to: /home/joep/repositories/.venv/bin/python3
==============================================================================================================================
-- Python 3:
-- Interpreter: /home/joep/repositories/.venv/bin/python3 (ver 3.12.3)
-- Libraries: /usr/lib/x86_64-linux-gnu/libpython3.12.so (ver 3.12.3)
-- Limited API: NO
-- numpy: /home/joep/repositories/.venv/lib/python3.12/site-packages/numpy/_core/include (ver 2.3.1)
-- install path: /home/joep/repositories/.venv/lib/python3.12/site-packages/cv2/python-3.12
--
-- Python (for build): /home/joep/repositories/.venv/bin/python3
-- Install to: /home/joep/repositories/.venv/bin/python3
==============================================================================================================================
-D OPENCV_PYTHON3_INSTALL_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
-- Python 3:
-- Interpreter: /home/joep/repositories/.venv/bin/python3 (ver 3.12.3)
-- Libraries: /usr/lib/x86_64-linux-gnu/libpython3.12.so (ver 3.12.3)
-- Limited API: NO
-- numpy: /home/joep/repositories/.venv/lib/python3.12/site-packages/numpy/_core/include (ver 2.3.1)
-- install path: /home/joep/repositories/.venv/lib/python3.12/site-packages/cv2/python-3.12
--
-- Python (for build): /home/joep/repositories/.venv/bin/python3
-- Install to: /usr/local
==============================================================================================================================
-D CMAKE_INSTALL_PREFIX=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
-- Python 3:
-- Interpreter: /home/joep/repositories/.venv/bin/python3 (ver 3.12.3)
-- Libraries: /usr/lib/x86_64-linux-gnu/libpython3.12.so (ver 3.12.3)
-- Limited API: NO
-- numpy: /home/joep/repositories/.venv/lib/python3.12/site-packages/numpy/_core/include (ver 2.3.1)
-- install path: /home/joep/repositories/.venv/lib/python3.12/site-packages/cv2/python-3.12
--
-- Python (for build): /home/joep/repositories/.venv/bin/python3
-- Install to: /home/joep/repositories/.venv/lib/python3.12/site-packages
Working solution part 2: full build commands
- Install CUDA 12.8
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update
sudo apt install cuda-toolkit-12-8
- Install CUDNN 9.8
sudo apt install cudnn9-cuda-12-8=9.8.0.87-1 libcudnn9-cuda-12=9.8.0.87-1 libcudnn9-dev-cuda-12=9.8.0.87-1 libcudnn9-static-cuda-12=9.8.0.87-1
sudo apt-mark hold cudnn9-cuda-12-8 libcudnn9-cuda-12 libcudnn9-dev-cuda-12 libcudnn9-static-cuda-12
- Add CUDA path to bashrc
echo 'export PATH=/usr/local/cuda-12.8/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.8/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc
- Install build dependencies (source)
sudo apt update
sudo apt install build-essential cmake ninja-build pkg-config git
sudo apt install python3 python3-pip python3-venv python3-dev
# Image I/O libraries
sudo apt install libjpeg-dev libpng-dev libtiff-dev
# Install basic codec libraries
sudo apt install libavcodec-dev libavformat-dev libswscale-dev
# Install GStreamer development libraries
sudo apt install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
# Install additional codec and format libraries
sudo apt install libxvidcore-dev libx264-dev libmp3lame-dev libopus-dev
# Install additional audio codec libraries
sudo apt install libmp3lame-dev libvorbis-dev
# Optional: Install VA-API for hardware acceleration
sudo apt install libva-dev
# Install video capture libraries and utilities
sudo apt install libdc1394-25 libdc1394-dev libxine2-dev libv4l-dev v4l-utils
# GTK lib for the graphical user functionalites coming from OpenCV highghui module
sudo apt-get install libgtk-3-dev
# Parallelism library C++ for CPU
sudo apt-get install libtbb-dev
# Optimization libraries for OpenCV
sudo apt-get install libatlas-base-dev gfortran
- Link libraries
sudo mkdir -p /etc/OpenCL/vendors && echo "libnvidia-opencl.so.1" | sudo tee /etc/OpenCL/vendors/nvidia.icd
sudo ln -s /usr/include/libv4l1-videodev.h /usr/include/linux/videodev.h
- Create virtual environment for Python
cd ~/repositories
python3 -m venv .venv
. .venv/bin/activate
pip install --upgrade pip setuptools wheel numpy
- Set application versions (find your CUDA arch)
CUDA_ARCH=6.1
OPENCV_VERSION=4.11.0
PYTHON_VERSION=3.12
- Get OpenCV
git clone --depth 1 --branch ${OPENCV_VERSION} https://github.com/opencv/opencv.git ~/repositories/opencv
git clone --depth 1 --branch ${OPENCV_VERSION} https://github.com/opencv/opencv_contrib.git ~/repositories/opencv_contrib
mkdir -p ~/repositories/opencv/build
cd ~/repositories/opencv/build
- Prepare build
cmake -G Ninja \
-D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
-D OPENCV_VERSION=${OPENCV_VERSION} \
-D OPENCV_GENERATE_PKGCONFIG=ON \
-D OPENCV_PC_FILE_NAME=opencv4.pc \
-D OPENCV_ENABLE_NONFREE=ON \
-D OPENCV_EXTRA_MODULES_PATH=~/repositories/opencv_contrib/modules \
-D WITH_CUBLAS=1 \
-D WITH_CUDA=ON \
-D WITH_CUDNN=ON \
-D OPENCV_DNN_CUDA=ON \
-D BUILD_opencv_cudacodec=OFF \
-D CUDA_ARCH_BIN=$CUDA_ARCH \
-D CUDA_FAST_MATH=1 \
-D ENABLE_FAST_MATH=1 \
-D WITH_TBB=ON \
-D WITH_V4L=ON \
-D WITH_OPENGL=ON \
-D WITH_GSTREAMER=ON \
-D WITH_QT=OFF \
-D INSTALL_PYTHON_EXAMPLES=OFF \
-D INSTALL_C_EXAMPLES=OFF \
-D BUILD_TESTS=OFF \
-D BUILD_EXAMPLES=OFF ..
- Build OpenCV
ninja
- Install OpenCV
sudo ninja install
- Link libraries
sudo ldconfig
- Test
python3
>>> import cv2
>>> print(cv2.getBuildInformation())
quit()
or
python3 -c "import cv2; print(cv2.__version__); print(cv2.cuda.getCudaEnabledDeviceCount())"
- Create "wheel"
cd ../modules/python/package && python3 setup.py bdist_wheel
mkdir ../../../wheels
cp dist/*.whl ../../../wheels/
Working solution part 3: copy to local machine
Now the cv2
directory should be available in /home/joep/repositories/.venv/lib/python3.12/site-packages/cv2
. Then all supporting binaries, libraries and includes should be copied to /usr/local
. I'm going to copy that folder to a SMB share. And from that share to my dev pc.
- Mount the share
sudo apt install cifs-utils
sudo mkdir /mnt/joep
sudo chown joep:joep /mnt/joep
sudo mount -t cifs -o username=joep,uid=$(id -u),gid=$(id -g) //jpl-truenas20.local.lan/joep /mnt/joep/
- Copy the files to the share
sudo apt install rsync
rsync -avh --progress ~/repositories/.venv/lib/python3.12/site-packages/{bin,cv2,include,lib,share} /mnt/joep/opencv
rsync -avh --progress ~/repositories/opencv/wheels/ /mnt/joep/wheels
- Copy the files on my local dev machine
cd ~/repositories/my-repo
rsync -avh --progress /media/joep/opencv/cv2 .venv/lib/python3.12/site-packages/
rsync -avh --progress /media/joep/opencv/{bin,include,lib,share} /usr/local/
mkdir wheels
rsync -avh --progress /media/joep/wheels .
- Install runtime dependencies
sudo apt update
sudo apt install python3 python3-pip python3-venv
# Image I/O libraries
sudo apt install libjpeg-dev libpng-dev libtiff-dev
# Install basic codec libraries
sudo apt install libavcodec-dev libavformat-dev libswscale-dev
# Install GStreamer development libraries
sudo apt install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
# Install additional codec and format libraries
sudo apt install libxvidcore-dev libx264-dev libmp3lame-dev libopus-dev
# Install additional audio codec libraries
sudo apt install libmp3lame-dev libvorbis-dev
# Install FFmpeg (which includes libavresample functionality)
sudo apt install ffmpeg
# Optional: Install VA-API for hardware acceleration
sudo apt install libva-dev
# Install video capture libraries and utilities
sudo apt install libdc1394-25 libdc1394-dev libxine2-dev libv4l-dev v4l-utils
# GTK lib for the graphical user functionalites coming from OpenCV highghui module
sudo apt-get install libgtk-3-dev
# Parallelism library C++ for CPU
sudo apt-get install libtbb-dev
# Optimization libraries for OpenCV
sudo apt-get install libatlas-base-dev gfortran
- Link libraries
sudo mkdir -p /etc/OpenCL/vendors && echo "libnvidia-opencl.so.1" | sudo tee /etc/OpenCL/vendors/nvidia.icd
sudo ln -s /usr/include/libv4l1-videodev.h /usr/include/linux/videodev.h
- Enable virtual env
cd ~/repositories/my-repo/
. .venv/bin/activate
- Install wheel
pip install ~/repositories/my-repo/wheels/*.whl
- Edit install location. I had to change this. Haven't found how to fix this automatically/programmatically
- Go to
~/repositories/my-repo/.venv/lib/python3.12/site-packages/cv2/config.py
and update the build machine path to your dev machine path. - The same for
~/repositories/my-repo/.venv/lib/python3.12/site-packages/cv2/config-3.12.py
- Go to
# from this:
import os
BINARIES_PATHS = [
os.path.join('/home/joep/repositories/.venv/lib/python3.12/site-packages', 'lib')
] + BINARIES_PATHS
# to this:
import os
BINARIES_PATHS = [
os.path.join('/home/joep/repositories/my-repo/.venv/lib/python3.12/site-packages', 'lib')
] + BINARIES_PATHS
- Test
python3
>>> import cv2
>>> print(cv2.getBuildInformation())
quit()
or
python3 -c "import cv2; print(cv2.__version__); print(cv2.cuda.getCudaEnabledDeviceCount())"
Docker image
To make the installation portable and repeatable I wanted to create a Docker image. With this image file I could also build OpenCV. The Dockerfile
would thus also be the build documentation. In the end, after many trials, this is what I ended up with.
Dockerfile
I removed as many *-dev
libraries as possible. My application still runs, but yout mileage may vary. If you want more info on Nexus repositories, I wrote about it here and here.
Directory tree:
.
├── analyzer
│ ├── .venv
│ ├── src
│ ├── main.py
│ ├── ...
│ └── requirements.txt
├── docker
│ ├── scripts
│ │ └── entrypoint.sh
│ ├── ...
│ └── Dockerfile.analyzer
├── .dockerignore
└── ...
ARG CUDA_VERSION="12.8.0"
ARG OS="ubuntu24.04"
ARG DEBIAN_FRONTEND=noninteractive
ARG CUDA_BUILDER_IMAGE="${CUDA_VERSION}-cudnn-devel-${OS}"
ARG CUDA_RUNTIME_IMAGE="${CUDA_VERSION}-cudnn-runtime-${OS}"
### Builder image
# Use the CUDA builder image as the base for building OpenCV
# This image includes the necessary CUDA and cuDNN libraries for building OpenCV with CUDA support
FROM nexus20.joeplaa.com/nvidia/cuda:${CUDA_BUILDER_IMAGE} AS builder
# setting build related env vars
ARG PYTHON_VERSION=3.12
ARG OPENCV_VERSION=4.11.0
ARG CUDA_DOCKER_ARCH=6.1
ENV GGML_CUDA=1
### Install build dependencies
ENV DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC
RUN echo 'deb https://nexus20.joeplaa.com/repository/ubuntu/ noble main restricted universe' > /etc/apt/sources.list && \
echo 'deb https://nexus20.joeplaa.com/repository/ubuntu/ noble-updates main restricted universe' >> /etc/apt/sources.list && \
echo 'deb https://nexus20.joeplaa.com/repository/ubuntu/ noble-security main restricted universe' >> /etc/apt/sources.list
RUN apt update && apt install -y --no-install-recommends \
ca-certificates \
build-essential cmake ninja-build pkg-config git \
python3 python3-pip python3-venv python3-dev \
# Image I/O libraries
libjpeg-dev libpng-dev libtiff-dev \
# Install basic codec libraries
libavcodec-dev libavformat-dev libswscale-dev \
# Install GStreamer development libraries
libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev \
# Install additional codec and format libraries
libxvidcore-dev libx264-dev libmp3lame-dev libopus-dev \
# Install additional audio codec libraries
libmp3lame-dev libvorbis-dev \
# Optional: Install VA-API for hardware acceleration
libva-dev \
# Install video capture libraries and utilities
libdc1394-dev libxine2-dev libv4l-dev \
# GTK lib for the graphical user functionalites coming from OpenCV highghui module
libgtk-3-dev \
# Parallelism library C++ for CPU
libtbb-dev \
# Optimization libraries for OpenCV
libatlas-base-dev gfortran \
&& rm -rf /var/lib/apt/lists/*
RUN mkdir -p /etc/OpenCL/vendors && echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd
### Create a symbolic link for video device header
RUN ln -s /usr/include/libv4l1-videodev.h /usr/include/linux/videodev.h
### Clone OpenCV and OpenCV Contrib
WORKDIR /app
RUN git clone --depth 1 --branch ${OPENCV_VERSION} https://github.com/opencv/opencv.git /app/opencv
RUN git clone --depth 1 --branch ${OPENCV_VERSION} https://github.com/opencv/opencv_contrib.git /app/opencv_contrib
### Create build folder and install Python dependencies
RUN mkdir -p /app/opencv/build
RUN pip config --user set global.index https://nexus20.joeplaa.com/repository/pypi-proxy/pypi && \
pip config --user set global.index-url https://nexus20.joeplaa.com/repository/pypi-proxy/simple && \
pip config --user set global.trusted-host jovibots
RUN python3 -m venv .venv \
&& . .venv/bin/activate \
&& pip install --upgrade pip setuptools wheel numpy \
&& cd /app/opencv/build \
### Prepare OpenCV build
&& cmake -G Ninja \
-D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
# OpenCV
-D OPENCV_VERSION=${OPENCV_VERSION} \
-D OPENCV_GENERATE_PKGCONFIG=ON \
-D OPENCV_PC_FILE_NAME=opencv4.pc \
-D OPENCV_ENABLE_NONFREE=ON \
-D OPENCV_EXTRA_MODULES_PATH=/app/opencv_contrib/modules \
# CUDA
-D WITH_CUBLAS=1 \
-D WITH_CUDA=ON \
-D WITH_CUDNN=ON \
-D OPENCV_DNN_CUDA=ON \
-D BUILD_opencv_cudacodec=OFF \
-D CUDA_ARCH_BIN=$CUDA_DOCKER_ARCH \
-D CUDA_FAST_MATH=1 \
# Other dependencies
-D ENABLE_FAST_MATH=1 \
-D WITH_TBB=ON \
-D WITH_V4L=ON \
-D WITH_OPENGL=ON \
-D WITH_GSTREAMER=ON \
# Disable unnecessary features
-D WITH_QT=OFF \
-D INSTALL_PYTHON_EXAMPLES=OFF \
-D INSTALL_C_EXAMPLES=OFF \
-D BUILD_TESTS=OFF \
-D BUILD_EXAMPLES=OFF .. \
### Build OpenCV
&& ninja \
&& ninja install
### Create wheel package
RUN cd /app/opencv/modules/python/package && python3 setup.py bdist_wheel
### Strip unneeded symbols to reduce image size
RUN find /usr/local/lib -type f -name "libopencv_*.so*" -exec strip --strip-unneeded {} +
## Runtime image
# Use the CUDA runtime image as the base for the final image
FROM nexus20.joeplaa.com/nvidia/cuda:${CUDA_RUNTIME_IMAGE} AS runner
ARG PYTHON_VERSION="3.12"
### Install build dependencies
ENV DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC
RUN echo 'deb https://nexus20.joeplaa.com/repository/ubuntu/ noble main restricted universe' > /etc/apt/sources.list \
&& echo 'deb https://nexus20.joeplaa.com/repository/ubuntu/ noble-updates main restricted universe' >> /etc/apt/sources.list \
&& echo 'deb https://nexus20.joeplaa.com/repository/ubuntu/ noble-security main restricted universe' >> /etc/apt/sources.list
RUN apt update && \
apt install -y --no-install-recommends \
ca-certificates \
python3 python3-pip python3-venv \
libjpeg-dev libpng-dev libtiff-dev \
libavcodec-dev libavformat-dev libswscale-dev \
libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev \
#libxvidcore-dev libx264-dev libmp3lame-dev libopus-dev \
#libmp3lame-dev libvorbis-dev \
#ffmpeg \
#libva-dev \
#libdc1394-25 libdc1394-dev libxine2-dev libv4l-dev v4l-utils \
libgtk-3-dev \
libtbb-dev \
#libatlas-base-dev \
&& rm -rf /var/lib/apt/lists/*
### Create a symbolic link for video device header
# RUN ln -s /usr/include/libv4l1-videodev.h /usr/include/linux/videodev.h
WORKDIR /app
### Copy nvidia links
COPY --from=builder /etc/OpenCL /etc/OpenCL
### Copy OpenCV system libs
COPY --from=builder /usr/local/bin /usr/local/bin
COPY --from=builder /usr/local/include/opencv4 /usr/local/include/opencv4
COPY --from=builder /usr/local/lib /usr/local/lib
COPY --from=builder /usr/local/share/opencv4 /usr/local/share/opencv4
COPY --from=builder /usr/local/share/licenses/opencv4 /usr/local/share/licenses/opencv4
### Copy OpenCV Python wheel
COPY --from=builder /app/opencv/modules/python/package/dist/*.whl /usr/share/python-wheels/
### Copy analyzer scripts and requirements
COPY analyzer/. .
COPY docker/scripts /scripts
### Create virtual environment and install Python dependencies
RUN pip config --user set global.index https://nexus20.joeplaa.com/repository/pypi-proxy/pypi && \
pip config --user set global.index-url https://nexus20.joeplaa.com/repository/pypi-proxy/simple && \
pip config --user set global.trusted-host jovibots && \
python3 -m venv .venv && \
. .venv/bin/activate && \
pip install --upgrade pip setuptools wheel && \
pip install -r requirements.txt
### Link OpenCV Python bindings
RUN ln -s /usr/local/lib/python3.12/site-packages/cv2 .venv/lib/python3.12/site-packages/
RUN . .venv/bin/activate && pip install /usr/share/python-wheels/opencv-4.11.0-py3-none-any.whl --break-system-packages
### Run entrypoint script
RUN chmod +x /scripts/entrypoint.sh
CMD ["/scripts/entrypoint.sh"]
entrypoint.sh
set -e
python3 -m venv .venv
. .venv/bin/activate
python3 /app/main.py
Build the image
From the root of the repo run these commands:
VERSION=0.1.0
docker build -f docker/Dockerfile.analyzer -t backend-analyzer .
docker tag backend-analyzer:latest demo-backend-analyzer:${VERSION}
Run the image
Go to the analyzer
folder and run the command below.
docker run --rm \
-v "$(pwd)":/app \
-v analyzer-venv:/app/.venv \
--env-file=.env \
--name analyzer-dev-container \
--runtime=nvidia \
--gpus all \
--device=/dev/video0:/dev/video0 \
backend-analyzer:latest
This passes through the Nvidia GPU and the first webcam.
Bonus: use Nexus to upload and share the image (locally)
docker tag backend-analyzer:latest demo-nexus20.joeplaa.com/jovibots/demo-backend-analyzer:latest
docker tag backend-analyzer:latest demo-nexus20.joeplaa.com/jovibots/demo-backend-analyzer:${VERSION}
docker login nexus20.joeplaa.com
docker push nexus20.joeplaa.com/jovibots/demo-backend-analyzer:${VERSION}
docker push nexus20.joeplaa.com/jovibots/demo-backend-analyzer:latest