Efficient Perspective-n-Point (PnP)

Overview

The perspective-n-point algorithm, or PnP for short (also known as the projective-n-point algorithm), is used frequently in computer vision to efficiently and accurately estimate the pose of a camera in 3D space. It accomplishes this by solving a system of equations, which involve the camera’s intrinsic parametersThe internal properties of the camera, such as focal length, the image center, and skew., a collection of $n$ points in 3D space, and $n$ corresponding points in the 2D space of the camera.

The perspective-n-point algorithm

The purpose of the PnP algorithm is to estimate all six degrees of freedom of a calibrated camera. That is, it estimates the rotation $R$ and translation $T$ of the camera. It requires a set of $n$ points with 3D world coordinates and corresponding 2D camera space coordinates. The PnP algorithm finds the $3 \times 3$ rotation matrix $R$ and $1 \times 3$ translation matrix $T$ that minimize the reprojection error between a collection of world points $p_w^i$ and camera points $p_c^i$ where $0 \le i \lt n$.

In the equation above, $s$ is a scale factor and $K$ is the camera intrinsic matrix. The PnP algorithm makes the following assumptions:

• Camera intrinsics matrix $K$ is known (i.e., the focal length, principal point, etc.)

• All point correspondences are not collinear (i.e., we need 3D points with triangulation)

There are several variations of the PnP algorithm depending on how many points we wish to solve for. We first look at the minimal case, where ...