Thesis: GenAI-Enhanced Multimodal Sensor Fusion for Robust Object Detection in Autonomous Driving

What you will do

At Volvo Autonomous Solutions, we are at the forefront of innovation, designing the autonomous transport solutions of tomorrow. As part of our global and diverse team, you will have the opportunity to work on an industrial research challenge that contributes to the future of safe and reliable autonomous transport.

We are looking for a master’s thesis student to investigate how generative AI can improve multimodal sensor fusion and make 3D object detection more robust under adverse conditions and sensor failures.

The thesis project

Autonomous vehicles rely on multiple sensors, such as cameras, LiDAR and RADAR, to detect and understand their surroundings. Conventional sensor-fusion methods may not fully capture uncertainty or compensate effectively when sensor information is degraded or unavailable.

The goal of this thesis is to investigate whether a generative AI-based fusion module can improve the robustness of multimodal 3D object detection. The project will focus on developing a generative sensor-completion mechanism, for example by reconstructing degraded RADAR or LiDAR bird’s-eye-view features using camera information and other available modalities.

You will evaluate how the proposed approach affects 3D object-detection performance under adverse conditions. You will also investigate whether the system can degrade gracefully when one of the sensors becomes unavailable.

The work will include:

• Implementing a generative sensor-completion mechanism for degraded or missing sensor information.
• Integrating the completion mechanism with a pretrained bird’s-eye-view object detector.
• Training and evaluating the system using synchronized LiDAR, camera and RADAR data.
• Testing the approach under modality-dropout and simulated adverse-weather conditions.
• Measuring detection robustness and graceful degradation when sensor information is unavailable.
• Analysing the results and documenting your findings in a scientific report.

Resources and research environment

You will have access to a pretrained bird’s-eye-view 3D object detector, a generative completion model and synchronized LiDAR, camera and RADAR data. GPU compute resources, modality-dropout tools and adverse-weather simulation capabilities will also be available for developing and evaluating your approach.

You will work closely with experienced engineers and researchers in a collaborative AI and autonomy team, gaining insight into generative AI, multimodal perception and the development of robust autonomous-driving systems.

Who are you?

You are enrolled in a master’s programme in computer science, data science, machine learning, electrical engineering, robotics or a related field. You have solid Python programming skills and hands-on experience with a deep-learning framework such as PyTorch or TensorFlow.

We are looking for someone who is interested in autonomous systems, machine perception, generative AI and working with real-world sensor data.

You should also be:

• Independent, structured and curious.
• Analytical and motivated to solve challenging technical problems.
• Comfortable working with deep-learning models, multimodal data and experimental evaluation.
• Interested in sensor fusion and robust perception under challenging conditions.
• Proficient in written and spoken English.

What’s in it for you?

This thesis gives you the opportunity to work on a real industrial research problem with access to unique multimodal vehicle data and the tools needed to conduct advanced experiments. You will receive close technical supervision from experts in AI and autonomous systems, as well as access to GPU resources and relevant development tooling.

You will also become part of a dynamic and inclusive work environment where new ideas are encouraged and celebrated.

How to apply

To apply, send your CV, grade transcript and a short motivation explaining your interest in the project and relevant experience. Start and scope can be adjusted together with the supervisor.