Target Tracking in the Presence of Occlusions

This target tracking project aimed to avoid losing a target in presence of occlusions chiefly using conformal prediction (CP). A good overview of CP can be found in (Angelopoulos & Bates, 2021), but the technique in a nutshell is that, given a history of predictions and actual outcomes of a model—say, the predicted state of a target at the next time step and its actual state at that time step—you can use a frequentist approach to determine probabilistic bounds to a desired percentile that the target on such a prediction will remain within those bounds. That is, you look at how poorly the model performed in the past and use that to inform how poorly it will perform in the future).

Of course, this technique does require having a decent model of the target’s motion, which I accomplished by employing a Knowledge-based Ordinary Differential Equation/KNODE (Jiahao et al., 2022), which is a neural network approach that attempts to learn the first derivative of a system’s behavior instead of the behavior itself.

Tracking it in the circle works, even with noise causing the perception (and thus the prediction) to jump around severely (I could have used a low-pass filter, but I deliberately wanted error here to show the technique working), but that alone isn’t too difficult:

If something were to block the drone from seeing the target, then the drone might follow the prediction. Unfortuantely, it’s possible that the prediction might have some error, causing the drone to track the prediction and, in the process, lose the target. To this end, I used nonexchangeable conformal prediction (Barber et al., 2023) (arguably ODEs themselves are exchangeable, but that’s a conversation for another time). Essentially, this is how it fits in:

Finally, we can simply use these bounds and ensure the drone can capture the whole area within its field of view. The drone does this by increasing its altitude, as seen in this video, where it loses the target but is able to expand its view such that it doesn’t lose the target under the canopy of grey blocks: