Cooperative Sensor Fusion ft. Jan Aelterman of Ghent University & imec

[00:00:00] My name is Carl Anthony and I work in the automotive industry in Detroit. Sometimes that work encompasses future vehicle technology, and that's what we talk about here, for the most part anyway. This is AutoVision News Radio.

[00:00:16] It's where the sensors also exchange information on things that they deemed invaluable by themselves but that when combined actually result in a very confident observation. During AutoSense Brussels in September 2023, we conducted our AutoVision News live at AutoSense Interview Series from the exhibition floor.

[00:00:40] As part of the interview series we had the opportunity to talk with some of the world's leading minds in ADAS and autonomous vehicle technology development from inside the warm and nostalgic embrace of the Auto World Museum.

[00:00:54] One of our guests during the event was Jan Aelterman, assistant professor at Ghent University and IMEC. Since earning his PhD in engineering from Ghent University, Jan has built a lengthy academic record in different application fields like MRI, CT, Microscopy, Consumer Video, and Automotive Perception.

[00:01:15] Jan is the co-author of the 2022 AutoSense Academy Module on Sensor Fusion, which we discuss the relevance of when it comes to corner cases. I'm pleased to be able to replay that interview for you now. Jan Aelterman, assistant professor at Ghent University and IMEC. Our guest during AutoSense Brussels 2023.

[00:01:35] From ADAS to electrification this is all division news radio with Karl Anthony in Detroit, Michigan. So I'm an assistant professor at Ghent University as you've explained where I teach and do research on computer vision.

[00:01:49] Initially there was just computer vision, just how do you deal with the camera and the output from that camera? Yes. Before we started to realize that a lot of interesting research questions come from collaboration with industry and research questions that arise from the applications. Sure.

[00:02:07] And in solving those research questions and working with industry, we realized that the answer is not always in just computer vision, not always in just a single camera sensor. Sometimes the answer isn't bringing in other sensors into the solution, which is how you end up with sensor fusion.

[00:02:25] I sort of made a thing of it. And along the way I've also become part of IMEC, the Inter-University Micro-Electronic Center. It's actually an acronym and they're of course world renowned for their work on microelectronics. And they're also now looking for opportunities to combine microelectronics with sensor

[00:02:44] design and sensor processing and sensor fusion to come up with ends to end solutions. Yes. How I end up there as well. Yes. So as we saw in your bio, your research interest is in application-specific challenges for computer vision and sensor fusion.

[00:03:03] Jan, why have you chosen this area of research and what do you find most interesting about it? I like to solve real world problems. I've worked in many different application fields. And what I noticed is that a lot of the time some applications have

[00:03:19] problems that also show up maybe in a different form in other applications. To give you an example, the talks to today just at AutoSense just this morning, they were about image quality. About image quality for automotive.

[00:03:33] Now image quality as you might imagine is also a big thing in consumer video. And so if you work in that field, you will find that solutions in that field about denoising specifically for that application carry over into autonomous driving as well.

[00:03:47] At this point in the interview, Jan and I discussed sensor fusion. In particular, the advantages of sensor fusion when it comes to complex and challenging corner cases. Jan provided an interesting and insightful analogy. So that's actually where I think my research comes in.

[00:04:06] You have to imagine that you're driving the car or a car yourself and you're the only driver. That means that the only sensors in the car are your eyes. And you either see an object or you don't. How do you improve the perception in your car?

[00:04:20] Maybe you have a co-driver at one point. That's another set of eyes. That's another set of eyes seeing things. And then in the most naive sense, sensor fusion would be that you call out the things

[00:04:31] you see and that your co-driver calls out the things that he or she sees. But I would assert that still is not a perfect system because there are things that you might have considered relevant like a silhouette somewhere in the fog.

[00:04:47] And your co-driver had also seen that but also not considered it relevant. Well, though that person was looking from a different angle. And then none of you would have called it out. But still you would have actually seen something.

[00:05:00] And what we try to do in our research is cooperative sensor fusion. It's where the sensors also exchange information on things that they deemed invaluable by themselves, but that when combined actually result in a very confident observation. And we think that's especially useful in corner cases like fog,

[00:05:20] like nighttime, like when there's a lack of light. We had a lot of different sensors, collaborates to see something that they would not have seen by themselves to make a system that is more than some of its parts. Yeah, that's an amazing example. I really like that analogy.

[00:05:36] That's fascinating. Yeah, the driver, the co-driver, but then combining the information, having them communicate with each other. We're speaking with Jan Alterman, assistant professor at Gens University and IMEK. What trends do you see on the horizon, Jan, in the ADAS and autonomous driving space? I think so.

[00:05:57] We are moving towards higher levels of autonomy for self-driving. And just yesterday was a very fascinating talk on functional safety where I think a number was floated that you cannot tolerate autonomous vehicles causing more casualties than one over four billion miles.

[00:06:16] That was 20 times lower, so 20 times higher bar compared to humans drier. And if you want to achieve that level, well then you should prove that your system functions correctly, not in normal circumstances like you would see when you're driving now

[00:06:31] outside, but especially when it's raining and it's night. And the headlights of the cars have just broken down and it's a very exceptional case. Sure. You will not have a lot of data for that, so that's an AI challenge as well.

[00:06:46] But you also need your system to function correctly under those very rare circumstances to avoid hitting somebody under those very specific circumstances. And that's where I think a lot of the research in the next few years

[00:06:59] will be centered around the ones coming up with better solutions for those corner cases. Fascinating. So well said, Jan. What is your favorite car? What is your favorite car and why? That's a tough question. I'm actually more of an airplane's man.

[00:07:13] But when I think about my favorite car, somehow I always think back about my childhood and about cars from fiction. About how you know the cars in James Bond's they always have the right solution for the right situation.

[00:07:28] It's almost as if they knew what was going to happen better than the driver actually knew it. It's not even an autonomous car, but it knows the roads better, the environment better than even the driver.

[00:07:40] I think that's where we want to head towards and we will make strides towards that in that direction in the next few years. And that's what we need a car that knows safety at least better than the human drive.

[00:07:53] Well said on and certainly for any of our James Bond fans, James Bond fans who may be maybe watching 007. Let's talk about the airplane. So you said you have an affinity for airplanes. What do you like about airplanes? What's your favorite one?

[00:08:08] Maybe a bit biased because I'm European. Sure, I really like the Concorde because it's, you know, the supersonic jetliner that used to travel between Europe and the US. Because it was just such a fascinating combination of cutting edge technology. First, first computer controlled inlet ramps.

[00:08:30] The at the time the best performing thermodynamic engine in general, not just airplane engine, but in general, right? A supersonic glimmer, the as well as the technology and it's beautiful streamlines. Yeah, yeah. To learn more about Jan Alterman and his publications, see the links in the show notes.

[00:08:52] AutoVision News Radio is available on Spotify, Apple Podcast, Podbean and more. In Brussels, Belgium at the Auto World Museum alongside Jan Alterman. I'm Carl Anthony, AutoVision News Radio.