Real-Time Processing With FPGAs: A Strategic Advantage in Defense
When it comes to imaging and perception applications in defense, the cost of latency can mean a strategic advantage or mission failure. Consider defense imaging applications such as situational awareness, missile guidance, and object detection, where a raw image is transformed into knowledge through image/AI processing. Processing latency is a critical component of the system’s reaction time and determines success or failure.
A typical imaging system integrates multiple components, including image sensors (photons to pixels), image transport (pixels to packets), frame grabbers (packets to frames), and software running on CPUs or GPUs (frames to knowledge). In traditional machine vision systems, these components are made with FPGAs and are integrated into servers. This offers flexibility but sacrifices latency. Many defense systems have similar components with comparable latencies — tens to hundreds of milliseconds is common.
Traditional FPGAs only have simple logic, but today’s System-on-a-Chip FPGAs contain logic, DSP processing, AI processing, and multicore CPUs in a single device. This is a latency game changer that provides strategic reaction times and control loops. Latency under 1 ms is common.
Adding SWaP-C constraints? The size, weight, and power of a discrete camera, frame grabber, and CPU can be reduced to a single FPGA System-on-a-Chip with faster processing. Want the FPGA on a 4 x 5 cm printed circuit board that is rugged and leverages the cost advantage of the commercial marketplace? Let’s discuss your 200+ options for System-on-a-Module boards.
Shorter Processing Pipelines = Faster Feedback
While CPUs and GPUs offer a wide range of capabilities and benefits, FPGAs have distinct size, weight, power, and cost (SWaP-C) advantages, as well as performance advantages. Inefficient processing results in increased power consumption, which requires additional cooling, which results in heavy heat sinks, which can be cumbersome in certain applications. With FPGAs, this issue doesn’t exist.
FPGAs can execute custom image and signal processing tasks using tens of thousands of logic cells that are hard-coded to perform computations. Think of an FPGA as a custom-designed factory where raw data enters, powerful processing takes place in an assembly line, and processed knowledge exits.
FPGAs offer the low latency and parallel processing capabilities needed for defense applications. Processing in a typical machine vision application can take 1–10 ms with the type of system shown below (top row). With an FPGA inside a smart camera, on the other hand (bottom row), the processing time is significantly faster (4–20 µs), with 50–250x lower latency.
With several applications in the defense industry, sensors require real-time feedback to be effective. These applications include:
- ISR (intelligence, surveillance, and reconnaissance) systems
- Airborne and ground radar
- Image processing for sensor-enabled weapons systems
- Radar systems
- Sensor fusion
- Missile guidance
- Warning/situational awareness systems (like DARPA’s Luke Binoculars)
- Electronic warfare signal classification and countermeasures
If there is any delay (latency) in processing, failure can be catastrophic. FPGAs offer the real-time edge processing, ultra-low latency, and deterministic, real-time compute needed for mission-critical applications. Concurrent EDA can provide these solutions.
Solving Data Processing Challenges in Defense
An AMD Elite–certified partner, Concurrent EDA specializes in FPGA design services, camera distribution and customization (high speed, high data rate, SWIR, etc.), FPGA module distribution and customization, and more. For 20 years, we’ve been transforming custom algorithms into FPGAs and systems on chips (SoCs) and helping companies solve data processing challenges in mission-critical applications. Customers include the U.S. military (including the U.S. Navy, the U.S. Army, and related companies such as DARPA), major defense contractors, and a variety of prime contractors.
SWaP-C benefits of FPGA-based solutions:
- Size
- As small as 10 x 10 mm for a small FPGA
- FPGA system on a module (SOM) from 4 x 5 cm
- Open FPGA cameras
- Direct RF processing with AMD RFSoCs
- Weight
- Typically 50 g for an SOM
- Efficient processing → less heat → lighter heat sinks
- Power
- Hands-down winner over CPUs and GPUs
- Cost
- Reduced system integration and maintenance costs
- Built-in security means no external security devices
Application examples
- Real-time RF processing in an RFSoC
- Real-time vision processing
- Powerful edge AI capabilities
Image processing is a compute-intensive process, even for simple applications. Pairing a machine vision camera with an FPGA allows more computational tasks to occur in the hardware. This delivers the lower-latency processing required for quicker decisions, even at ultra-high speeds. For example, with the GigaSens HS 2-2247 camera — a 2.1 MP CoaXPress camera capable of frame rates above 2,200 fps at full frame — a 10-core i9 CPU will struggle. But FPGAs can enable real-time processing, even at 2,000-plus fps. So FPGAs that offer ultra-low latency offer a huge benefit for defense applications that need real-time feedback.
Why Latency Cannot Be Tolerated
In the video demonstration below, a high-speed camera tracks a moving laser . In a traditional computer vision setup, the image sensor converts photons into analog signals that are digitized into images. A camera-side FPGA preprocesses the data and sends it over the camera interface to a frame-grabber PCIe card, which creates frames and transfers them into system memory (DRAM). A CPU or GPU then performs processing to determine the control output signal that must travel back across the PCIe bus to an I/O card — often traversing the bus twice.
Because the entire frame must be buffered, moved, and processed before action can occur, end-to-end latency typically reaches tens to hundreds of milliseconds (0.001s). This delay limits how quickly the system can react to and control real-world processes from photon capture to I/O response. An architecture in which processing is moved into the camera itself, with the image sensor paired with an onboard FPGA, can decrease the processing time to as little as a few microseconds (0.000001s), a speed improvement of one-thousand times!
As a pixel stream enters the FPGA, processing takes place in real time, before a full frame is even formed, allowing the system to extract the required information directly from the pixel stream. In the example in the video, the FPGA processes the incoming image data to track a laser beam in real time, significantly reducing latency compared to PC-based processing.
Vision Design Services
Concurrent EDA can help companies using applications that absolutely must have real-time feedback, whether they need camera customization or complete machine vision design services. You can rely on Concurrent for:
- Processing Within the Camera at 40 Gb/s: Our engineers can embed your algorithms into a GigaSens camera, creating a custom OEM solution tailored to your application. Additionally, we can design custom cameras using any commercial image sensor, many of which are already available on a board.
- Processing in the Frame Grabber at 40 Gb/s: We can help you achieve processing speeds of 40 Gb/s using custom logic implemented in the frame grabber. This turns any camera into a customized device using one-, two-, or four-lane CoaXPress or Camera Link.
- Custom Design Using FPGA + Camera Modules: Our team also provides algorithm porting services for your FPGA platform, a service regularly used by OEMs and defense organizations.
FPGA Design Services
Need help turning your algorithms into OEM electronics at speeds ranging from 1 to 200 Gb/s? We utilize the latest FPGA modules, Jetson GPUs, and multicore CPUs. Our AMD Elite–certified engineers can determine which system is best for your application, budget, and deadline while designing and customizing FPGAs tailored to your needs. You can turn to us for:
- Processing in a System on a Chip FPGA: Modern FPGAs incorporate multiple ARM CPUs capable of running Linux, effectively replacing a PC in a compact form factor. Need a customized OEM FPGA module? We sell more than500 FPGA modules, including cutting-edge Versal adaptive compute devices that combine CPU, FPGA, and AI capabilities.
- Processing at the Edge: As an AMD Elite–certified partner, we can specify or customize a solution from AMD’s newEmbedded+ platform, which combines a Ryzen quad-core CPU running Linux with a Versal FPGA featuring FPGA logic and AI-specific tiles for edge computing. This 7 x 7 x 3 inch system consumes under 50 W and functions as a mini-PC with a customizable I/O card. One variant supports up to eight GMSL cameras.
Common Image Signal Processing Tasks in Defense
Systems with FPGAs embedded in cameras can handle a wide range of high-performance image signal processing (ISP), delivering real-time processing across the varied lighting and sensor conditions likely to occur with defense applications. In defense applications that involve tracking, targeting, and control, why are ISP tasks important?
ISP tasks help ensure that incoming pixels are consistent and stable over time, which is crucial for real-time decision-making, tracking accuracy, and closed-loop control. Without high-performance ISP, systems might be noisy, biased, or unstable, which could lead to catastrophic consequences in the event of failure. Below is a list of common ISP tasks and why they are relevant in defense:
- Flat field correction: Corrects pixel-to-pixel sensitivity in images, ensuring a uniform response across the field of view. This removes irregularities or artifacts that could be mistaken for targets.
- Radiometric normalization: Reduces the differences between images taken at different times or by different sensors to ensure repeatable, consistent images. This maintains detection performance across different environments and enables reliable sensor fusion capabilities.
- Temporal noise reduction: Reduces temporal and spatial noise by correlating data across image frames, helping improve detection in low-light or night operations and stabilizing the tracking of dim or distance targets while reducing false positives in EO/IR systems.
- Lens distortion correction: Lens distortion can lead to poor images, which will negatively impact geolocation, sensor fusion, and targeting, for example. ISP corrects lens distortion for a range of use cases, including surveillance, weapons guidance, and missile seeking.
- Image enhancement: Techniques such as sharpening, histogram equalization, and gamma correction help imaging systems obtain optimal images, maintain consistent interpretation across different operators, and avoid hallucinations.
Other ISP techniques relevant in defense applications include feature extraction, black-level subtraction, bad pixel correction, white balancing, color correction, sharpening, motion compensation/image stabilization, laser line/spot isolation, ROI extraction, automatic gain control, and tone mapping. If your business needs high-performance ISP in the FPGA, Concurrent EDA can help.
Faster Response Times in a Low-Power, Compact Form Factor
Imaging systems in the defense space operate under conditions that are drastically different from those on the factory floor. In these applications, imaging and vision systems directly drive action, so latency and jitter can compromise mission success and safety. Putting the processing as close to the camera as possible minimizes the distance data must travel and eliminates unnecessary buffering. FPGA-based solutions deliver the in-camera processing and ultra-fast response times that CPUs and GPUs alone cannot — in a more compact and lower-power solution.
FPGA development requires specialized tools for translating algorithms into difficult Verilog/VHDL programming. Fortunately, at Concurrent EDA, we specialize in this programming. We can integrate your high-speed imaging, FPGAs, and high-performance ISP tasks to ensure you are getting accurate, consistent, and stable data in microseconds — not milliseconds. If designing your system for real-time feedback in the defense industry is a requirement, Concurrent EDA is uniquely positioned to help you succeed.
Contact Us
Email