Abstract
3D localisation in GPS-denied environments is challenging. The use of on-board sensors for robot localisation suffers from drift and are limited by the payload capacity of each robotic platform. Off-board localisation solutions that use multiple sensors such as Wi-Fi, multi-camera systems, or UWB often have on-site requirements such as the installation and calibration of sensors which makes them less easy to deploy. On the other hand, off-board monocular methods only use a single camera and have substantially lower on-site requirements while being less susceptible to drift effects but are reliant on their target detection capabilities.
Deep learning-based approaches have also been widely used in monocular tasks due to their robustness to unstructured environments as compared to traditional non-learning-based approaches. However, deep learning-based approaches often require obtaining real-world images for training neural networks to generalise well in real unseen environments and acquiring such images is often expensive and impractical due to resource constraints or limited prior access to deployment sites.
In this thesis, we discuss the use of deep learning and monocular vision without relying on real-world image data prior to deployment. We focus on leveraging low-fidelity synthetic data for fine-tuning neural networks or pretrained models to tackle challenges in both static and dynamic environments, while exploring methods to enhance their robustness and accuracy for localisation tasks.
For static environments, we first explore the use of domain randomisation for generating synthetic data to train a network for camera pose estimation in an unstructured environment with limited access to real data and improve its accuracy by leveraging known geometric information of the target. We then tackle blurring and partial occlusions due to lighting effects by introducing an algorithm to augment synthetic binary training images and leveraging uncertainty propagation when training the network.
For dynamic environments, we propose a marker-based localisation system that uses a CNN-based ellipse detector to detect a circular LED marker against a dynamic background due to tracking using an active PTZ camera. We also demonstrate localisation of a UAV without the requirement of obtaining any image data of the target for fine-tuning a network by proposing the use of a pretrained template-based tracker and a cylindrical target model to improve tracking and localisation.
We evaluate our approaches through experiments in real-world environments. We conclude by summarising our main contributions and suggesting potential future directions for research in monocular 3D localisation.