Vision-Based UAV Repositioning

Approach

Rotation from geometry,
translation from the image.

Two-view pose estimators recover translation only as a unit direction, and that direction becomes ill-conditioned exactly where a repositioning loop operates: near the goal, with a baseline that is tiny compared with scenes 150–250 m away. The system therefore separates the two problems.

Correspondences. SuperPoint + LightGlue match the live frame to the reference, spread over the image, refined to sub-pixel accuracy with forward–backward Lucas–Kanade tracking.
Uncertainty. Each match gets an anisotropic structure-tensor covariance, propagated to bearing space with an unscented transform.
Rotation. The Probabilistic Normal Epipolar Constraint (PNEC) estimates the relative rotation, weighting every match by its uncertainty; it drives the gimbal.
Translation. After removing the estimated rotation, a robust shift-and-expansion fit to the remaining displacement field gives a signed pixel error that is zero at the goal.
Control. Saturating (tanh) laws and a per-axis convergence logic close the loop at about 15 Hz, behind the ENGEL DroneManager / PX4 interface.

Results · Simulation (Unreal Engine 5)

Clear-weather accuracy.

Six reference views of a real rural area near Ingolstadt (Google Photorealistic 3D Tiles), 13 configurations, 18 trials each, starting 1.9–3.4 m and 14–28° from the reference. Every number below is the error at the end of the run, over all trials, against simulator ground truth.

0.15°

final rotation error, clear weather (± 0.09°, n = 18)

≈2 px

effective image misalignment (40 cm lateral at ~190 m scene depth)

31 s

mean run duration, rotation below 1° after ~12 s

2.4×

lower rotation error with PNEC uncertainty weighting than with the unweighted NEC

Rotation and translation error over time for 18 clear-weather trials — Simulation Ground-truth rotation and translation error of all 18 clear-weather trials. The rotation decays exponentially (time constant 3.9 s); the translation levels off at 0.5–1 m, explained below.

Reference, start and end frames with error curves of two trials — Simulation Two clear-weather trials: reference, start and end frame, checkerboard of end frame and reference (continuous edges = aligned), and error over time.

Key finding

Why the metric error is not zero.

Rotational image motion does not depend on depth; translational image motion shrinks with it. A residual rotation α is therefore, in the image, almost indistinguishable from a sideways offset of α·Z: at 200 m, 0.1° moves the image by 1.7 px, exactly like 35 cm of translation. An image-only loop aligns the view and comes to rest on this ambiguity.

With rendered ground-truth depth, the final translation error splits into a part that compensates the residual rotation (invisible in the image) and an effective, visible error. In clear weather, the 74 cm metric error contains 39 cm of compensation; the remaining 40 cm ≈ 2 px. Across 215 trials the actual and compensating translations correlate with r = 0.92. A Fisher-information analysis shows that the images would determine the pose to a few centimetres — the residual is set by the stopping rules and the rotation bias, not by missing information.

Actual vs compensating translation, and image misalignment per condition — Simulation (a) Final lateral translation error vs the translation that compensates the residual rotation. (b) Image misalignment caused by the residual rotation alone, by the actual final pose, and with the optimal compensation: in fog, 9.6 px → 4.1 px.

Weather experiments · Simulation

Rain, snow and fog.

Weather is rendered with Ultra Dynamic Sky / Weather; in every run the sky and clouds also change between reference and live view. Bad weather removes matches and degrades the rotation estimate, yet the view is still recovered: the effective lateral error stays at 28–32 cm in every condition, and every trial ends below 1° of rotation error.

The six visibility conditions at the same reference view — Simulation End frames at the Farmland reference view in the six conditions, with the mean number of initial matches.

Condition	Matches	Rotation error	Worst trial	Effective lateral error	Trials < 0.2°
clear	114	0.15 ± 0.09°	0.41°	40 cm	78 %
light rain	112	0.21 ± 0.13°	0.50°	30 cm	61 %
heavy rain	111	0.17 ± 0.11°	0.38°	29 cm	67 %
light snow	100	0.25 ± 0.14°	0.52°	32 cm	44 %
heavy snow	73	0.32 ± 0.21°	0.78°	30 cm	39 %
fog	67	0.51 ± 0.23°	0.93°	28 cm	6 %

Median error over time and final errors per weather condition — Simulation Median errors over time, initial matches and final errors per condition.

Final rotation error per reference view and condition — Simulation Repeatability: final rotation error of every trial per reference view and condition (3 repetitions each).

Ablations · Simulation

What each component contributes.

Rotation estimator: PNEC's uncertainty weighting reduces the final rotation error 2.4× relative to the unweighted NEC; the essential matrix becomes ill-conditioned as the baseline vanishes and ends 7× worse (1.05° vs 0.15°). Translation signal: the signed displacement fit halves the forward-error scatter compared with a parallax magnitude. Control: the saturating law leaves a 30 % smaller rotation error than linear or PD control. Stopping threshold: 0.5 px instead of 1.5 px cuts the effective error by 35 % at the cost of longer runs.

Median error trajectories for rotation estimators and control laws — Simulation Median rotation and translation error over time (shaded: interquartile range) per rotation estimator and control law.

First hardware test · not a full-system validation

Gimbal loop on the real drone.

On the ENGEL quadrotor, with the vehicle stationary and only the gimbal commanded, the rotation loop ran on real camera images in three tests (one outdoors, two in the laboratory). It reduced the camera yaw error from 10–21° to 1–2° within 8–49 s. A residual error of 1–2° remains, larger than in simulation. Translation was not controlled in these tests.

Real-drone gimbal tests: images and camera yaw and pitch error — Real drone · rotation only Reference, start and end frames and camera yaw/pitch error for the three tests (a face in one frame is blurred).

Limitations

What is not shown yet.

The full loop (rotation and translation) has been evaluated in simulation only; on hardware, only the gimbal loop has been tested, with a stationary vehicle.
Metric position accuracy is bounded by scene depth: at 140–230 m, 1 px corresponds to 15–24 cm, so the view can match while the camera is 0.5–1.5 m from the reference position. The forward direction is only weakly observable with one camera.
The loop stops on estimated rotation; its bias near convergence (~0.18°) sets the residual rotation.
All starts overlapped the reference view by 49–71 %; the limit of the convergence basin was not determined. In snow, the strict stopping threshold often lets runs continue to the 180 s limit.

← Back to all work