High-Performance Image Signal Processing in FPGAs Explained

High-Performance Image Signal Processing in FPGAs Explained: Audio automatically transcribed by Sonix

High-Performance Image Signal Processing in FPGAs Explained: this mp4 audio file was automatically transcribed by Sonix with the best speech-to-text algorithms. This transcript may contain errors.

Ray Hoare:
Quick take - Ray Hoare from Concurrent EDA. I want to talk to you about high performance image signal processing compute in cameras and what does it mean and why do you care. So when you get an image from a sensor and it shows up on your computer or from your iPhone, it actually goes through a whole bunch of processing. And so what I want to show you here is some of the things that you can do for image signal processing that we've done. Put it into an FPGA and is useful for a variety of applications. So in this example there's something called black level subtraction. And I'll go into that. We have flat field correction. And I'll describe that bad pixel correction. Maybe one of the pixels is bad. You need to replace it. The mosaic when we take um a Bayer pattern and turn it into RGB so we can see it, uh, there's white balancing, there's color correction and then we're sharpening. And then we have um, tone curves, gamma and other operations. But let me cover the first few. Black opal subtraction. What is that? So in an image what we really have is, is we put we put a cover over the image so that we get a black image. And now all the sensors on the image may not always be the same level of black. And so we may say, oh okay. Some are when we when they're black, they actually have more signal than they should. And so what we can do is we can subtract things off. And so in this example we have an image.

Ray Hoare:
But if I've already calibrated for a black level and this upper right pixel, maybe it's a little bright when it has the shut the cover on. Well we want to correct for that. And that way we when we see the image we can actually get a good image. So that's fairly simple. Most cameras do this. So it describes how we can do it. You cover it up, you capture it and you can average it. And then you can do regions or you can do per pixel. Now flat field is a whole nother thing. And it's not typically done in cameras. And this is where it's very actually very interesting. So if you have a um, darkening around the edges of of an image of vignetting. We or we have sensitivity variation. So each pixel has a different sensitivity than each other. We try to make them in manufacturing the same, but doesn't always happen. Or if we have dark shadows or dust particles that we see on the if that's in the camera and we can't get rid of it, well, we can get rid of those things with flat field correction. So for example, if we look at this original image and here's some fancy math that describes it, and I did a great job of it. So we're going to give him credit for that on that. So there's the link to to learn some more. But here is an example of a bunch of flags with a light. And you can see okay, here is um, it's not quite uniform and it's kind of dark around the edges if we take the flat field, if we've already done that, you can then see the corrected image.

Ray Hoare:
It's not bright in the center and it's not as dark on the edges. Now the lower right hand side. Well that's kind of occluded from the image. So that's actually part of the image. But you can see I have a much better um image here. So first we're going to get a uniform light. We're going to match real image settings. So we're going to get the right settings in there. We're going to capture the image. And that gives us our flat field. Then we put it into our fancy map. So in this example we kind of took this key and um made the image pretty bad right. So we kind of exaggerated a bit. But before we applied Flat Field we saw this and it really didn't look good after we applied flat field. You can see there's lots of detail on the key. And this actually applies for I if I want to do I, I don't want I looking at this this image and going oh well it has to have a shadow to detect it as a key. Well that's not really part of the real image. So flat field helps us out with that. So flat field in general you have a light that's coming down. It's not dispersing perfectly evenly. We apply flat field correction and then voila we get a beautiful image back out. Thank you very much. Uh visit our website and current NBA.com by.

Sonix is the world’s most advanced automated transcription, translation, and subtitling platform. Fast, accurate, and affordable.

Automatically convert your mp4 files to text (txt file), Microsoft Word (docx file), and SubRip Subtitle (srt file) in minutes.

Sonix has many features that you'd love including secure transcription and file storage, generate automated summaries powered by AI, collaboration tools, share transcripts, and easily transcribe your Zoom meetings. Try Sonix for free today.