When Your CPU Can't Keep Up: FPGA and GPU Acceleration Explained

FPGA-GPU Acceleration Capabilities - July 2026 PG.mp4: Audio automatically transcribed by Sonix

FPGA-GPU Acceleration Capabilities - July 2026 PG.mp4: this mp4 audio file was automatically transcribed by Sonix with the best speech-to-text algorithms. This transcript may contain errors.

Ray Hoare:
Concurrent EDA. What do we do? We produce firmware and electronics that are measurably faster. So you take your algorithms and you put them into electronics, and they run faster than you can on a piece of software. We've been doing it for 20 years. We love high-complexity algorithms, like vision algorithms, tracking algorithms, something that if you put it on your CPU would not meet your real-time requirements. A lot of times that means there's a lot of data or it's a lot of data plus a lot of compute. And so we're big nerds, so that's gigabits per second and gigaops, or operations, per second. And you mix the two of those together and you've moved away from CPUs. GPUs are certainly an option, as well as FPGAs, and we do both. And we've done this for a variety of customers in defense, industrial, commercial, and sports. A lot of big names that we've worked with over the past 20 years and a lot of repeat customers, where we take their algorithms and we move them into either parallel FPGA implementations or GPU implementations. So for 20 years, what do we do? What is really our go-to? Vision and image processing is a great application where there are a lot of data. You're typically doing the same processing on it over and over and over again, whether it's 2,000 frames per second or 80,000 frames per second in a camera, like our GigaSens camera, or you want to do a slower frame rate but maybe you want to do more compute and something like the Alecs camera out of Allied Vision. Networking is also another application where you're doing compute, but it's coming in as packets or you're sending it out as packets. And so a lot of that is high data rate. You need to be doing switching, time-sensitive networks, synchronization, redundancy protocols, a lot of that. We have partners that we work with, IP cores that we integrate specifically on the FPGA space. And then if you look over here on the right, there's all sorts of FPGAs that we can utilize, whether it be the latest Versal FPGAs or SoCs as they call them, system on a chip, to RFSoCs to small, inexpensive Spartan FPGAs. So all of those things together. Interfacing, PCIe, DRAM, high-bandwidth memories, HDMI, DisplayPort, your MGTs, your LVDSes and USB and I-squared-C and SPI, all these weird interfaces that need to talk to other things, whether they be sensors or actuators or lights or streams of data coming in or out or memory going back and forth. And so we utilize those, specifically on the FPGAs for the most part, although we can do that with GPUs to some level. And this Alecs camera can interface directly to Smart Vision Lights and integrate those together, so that one's actually a pretty cool product, and it's almost all done, except for the algorithms inside. And then AI and large language models. What do we do with them? AI is a big topic. How do we deal with it? Well, AI is both a method for processing data and a utility that we utilize to do programming, whether that's FPGAs or GPUs or just algorithm exploration and prototyping. We are heavily into cloud code and generating hundreds of thousands of lines of code a week, which is great. But you got to train it. You got to verify it. There's a whole discipline behind it. So it's a great tool if you can deal with some of its quirks, shall we say. So on the vision side, putting algorithms inside of cameras, putting algorithms inside of FPGA frame grabbers, or putting them into embedded systems. We've done a whole bunch of algorithms, from making sure we can see into shadows, if you will, compression, image classification, laser tracking, 3D metrology, and a host of other things. So if you can prototype it in OpenCV or in Python cv2, we can get it down to the chip. So that's kind of the fun part. On the networking side, we've done some really big beefy switches for some very specialized customers that really need to move a whole lot of data at 10 and 25 gigabits per port, aggregate of 150 gigabits. And then how do you test it? How do you verify it? So we became really good at this. So that involves a net ripper that we created to send traffic in and out of the network, designing the network, increasing its performance. Network interface design, control software and a GUI, and then a digital twin to make sure you can model the whole thing. So this was a big project that we did, and we're really up to speed now on pretty much anything networking. All right, now let's talk about custom interfacing or PCIe cards if you will. On the cost optimize, you have this Artix UltraScale. This is a new chip out that does interfacing to the PCIe. It's a gen 3 or gen 4x8. I have to check that. It's an inexpensive part, but you can get to the processor bus and you can do some custom I/O with it, which is great. Mid-range, which is the Kintex UltraScale. And then you get to all these accelerator cards, which are just beefy and great. Along with that, how do you send the data? Here's an interesting IP core that we've created.

Ray Hoare:
The blue line is sort of your peak bandwidth of your data rate, not the packet data rate, but your payload data rate. How fast can you send data. If I've got a little packet, you've got a lot of overhead. But if you've got a big packet, your overhead is not so much. All the header stuff is not so much. But if you look at what Linux does, and Linux is great, there's a latency between packets, and that latency really bites you. So if you're sending 64 byte packets here, you're not even getting a gigabit on a 10 gig line. And so even if you go to max-size packets, you're only at 3.7 gigabits per second because you have an inter-frame gap that's software, not just a packet. And you just can't keep up. So you need to put something in hardware. So we've done that. And so ours is the green line. And we're 2.10 to 2.5 times, somewhere in that range, depending on the packet size, faster than if you could do it in software. That's a 10 gig. When you go to 25 gig, we're up to almost 20 times faster on the smaller packets and four to five on the larger packets. And then you get to jumbo packets — are still a whole lot better. So being able to move things efficiently across the network from inside of an FPGA to a Python program or a socket or an application layer, that's what this socket stream IP does. So we can incorporate that into your design. The Alecs camera is great for AI acceleration at the edge, where you have a single camera picture you're trying to take. And you prototyped your algorithm in AI or wherever you've prototyped it. And you really just need to get it out there on the edge. You're taking pictures, you're doing some analysis, you're saving those pictures. And the alternative here is, well, you can have a camera and a frame grabber and a PC or you can put it all together in one, where you have a GPU which has ARM cores in it that can do some of that processing. And then on the right-hand side, it can integrate with Smart Vision Lights directly, so there's no additional controller needed. You just wire it in. And so that gets you your lighting, IP67 rating. Put it on the factory floor. Fantastic product. So if you have AI algorithms that you've created or vision algorithms that you've created or analysis algorithms or measurement or inspection and you want to grab pictures, save them, maybe put them into the cloud, maybe do some real-time compute, we've done a bunch of stuff in real-time compute in here. Barcode detection, for example. Twelve megapixel image. Where is the barcode? What is the barcode? Where is the OCR? What's the measurement? We've done some of those algorithms.

Ray Hoare:
If we go a little bit more exotic, the new product out of AMD is this Embedded+ that actually pairs a Ryzen CPU with an FPGA on a single board. And this is great for dedicated applications where you want to do some high-speed I/O, plus you want your x86 processing and you want your Linux running without modification. And this is fantastic. So if you're using AI and you're like, "Hey, I want to run my Python." Great — run your Python on the Ryzen. Works great. Do your I/O, your real time, your dedicated processing, your medical device interface, your sensor interface on that board. And that board is actually in production, and so we're helping customers do that. And to the right is an I/O board. And this is a camera interface from GMSL. Now if you want to go even smaller, same concept but on a really, really small board, this board right here is 6 by 7 centimeters, really small, like a deck of cards. But it has eight ARM cores that are general-purpose Linux or whatever, 10 ARM cores that are specific to a task, plus FPGA logic, plus AI tiles, plus on the bottom here you've got all the I/O that you can imagine. So if you want to put this in your device, all the complexity of creating that system on a module with all the support circuitry and the supply chain misery that goes along with that and the high-speed design that goes along with that — you don't have to do that. You just have to put your I/O board down. And then you get all this horsepower. So that's a great product as well. And there's all sorts of variations of that, from high high end to much lower end and more cost-effective. GigaPitch: This is a multi-camera recording system that we have created for Major League Baseball and Minor League Baseball, where we've taken six cameras from throughout the stadium. They're sending real-time images to a single server, where we're doing the recording. Everything is done through an easy-to-use web interface to that server. We provide real-time support into the server so if people have a problem. And this is an integration example where we can take multiple cameras into a single compute environment. This may not be your environment. Maybe you're not a Major League Baseball team. Maybe you do something on the factory floor or you want to do something in a lab. Same thing applies. We don't have to be all the way above center field and behind home plate and first base and third base. You don't have to be that distributed for us to help you, but we can put together tight systems or highly distributed systems. And these are within one frame of each other. So they're tightly synchronized, and they're doing real-time compression from different angles. So if you're in a factory floor or you're trying to capture an experiment and you want every angle and they've got to be tightly synchronized, this is a great example. If you want to put some smarts in the camera, you want to do some measurement, this is a 3D metrology example. You can see the laser line is pointing straight down, but the camera is at an angle. And so what that does is it shows you —what's above here is that laser line looks like it's higher into the camera, and that height is your Z offset, your third dimension. And you can see it's kind of a sloppy line. So you have to do even more calculation over here of this 3D value, where we're looking at the peak of the laser and we're trying to find the peak of that laser to find the center point of that laser line. It's just so we can get really, really, really sub-pixel accuracy. And so there's a bunch of algorithms that we do in real time. And there's a fun little demo of it just scanning back and forth. So as you can imagine, something coming through is painting the picture.

Ray Hoare:
So we're capturing the line as it comes through. And that's good for an assembly line, something moving, and you want to scan it, like a package or an object. Another cool example of our high-speed processing capabilities is inside of the camera, where if you look over here on the right, there's a little laser dot, down here on the right, and we have a little dancing laser. It's kind of a cute thing, but the point is we can track a laser in real time. And so what you're seeing on the screen is that laser dot. We did this all in the FPGA. So where we're tracking the centroid of the laser. And to illustrate that we've tracked it, we put a bounding box on it. So the computer is not doing anything. It's just saying, "Oh, what's the image?" So the camera is doing all the displaying. So that's kind of fun if you're doing tracking, you're doing additive manufacturing and you want to look at your melt pool monitor. Or if you're doing cell cytometry and you're trying to look at the cells and you're trying to say, "Okay, let me measure that really quickly." These are all the same sorts of things that you can do at 2,000 frames a second, 80,000 frames a second. How fast do you want to go? So we can do that at peak sensor rate.

Ray Hoare:
And that's kind of a fun application we did there. And then a few AI demos that we did — probably really bad video — but here is an example where we're scanning through a bunch of images and it's identifying all those images. So using the AI tiles of the Versal Edge AI. So that's what this is doing here. In the upper right here, this an example of the Alecs camera looking down. We're putting a barcode in there, and you can see it's detecting it. And so we just put a barcode in and it pops up there on the right-hand side. So we can change what kind of barcode we're looking at. Looking for, we can actually do it dynamically. We actually have this working on a 12 megapixel image. So that's kind of fun. And then another fun application for detecting people is this pose detection. You can see that Joseph's there holding his cell phone, moving his arm, and you can see the body is in yellow. And then the left and right foot are red and green, and the arms are detected. So if you want to detect people, you can do that. And that's in a little FPGA. So that's pretty much what we do. If you need something in the FPGA world or GPU world, give us a call. I'd love to talk to you.

About this transcript

This transcript comes from sonix.ai, using English speech-to-text and mp4-to-text conversion to turn recordings into text you can edit, search, and share. See automatic captions for details.

Before you pick a platform, do the homework: AI transcription software compared, VEED review, and this side-by-side comparison of transcription services cover pricing, accuracy, and workflows across the market.

Create a free Sonix account to transcribe your own recordings in minutes — and check our pricing before you commit.