What the ISP in AMD Versal Gen 2 Actually Does (and How It’s Different From Gen 1)

If you’ve worked with Versal Gen 1, you already know the deal: there was no hardened ISP on the chip. If you wanted an imaging pipeline, you either built it in the programmable logic, bought an external ISP, or stitched something together with PL + AI Engines. It worked, but it wasn’t simple, and it definitely wasn’t light on resources.

Versal Gen 2 changes that. AMD added a real, hardened ISP block inside what they call the MMD tile. It’s built into the silicon, so you don’t burn LUTs, BRAM, or DSPs just to get clean camera data. And because it’s fixed hardware, the timing is predictable and the power draw is lower.

Let’s walk through what it actually does, and clear up a few details that often get mixed up.

What’s inside the Gen 2 ISP

The ISP handles the usual front‑end camera work you’d expect, but with a few specifics worth calling out:

Demosaicing
Turns Bayer into RGB or YUV.
2D Noise Reduction
This is important: it’s spatial only.
There’s no temporal noise reduction block in hardware.
If you need TNR, you’ll be doing it in PL, AIE‑ML, or software.
Lens shading correction
Fixes vignetting and sensor falloff.
Color correction + white balance
Includes a CCM and a dedicated AWB gain block.
Gamma correction
Edge enhancement
Tone mapping and WDR support
Statistics
The ISP sends out a chunk of stats each frame (roughly 16 KB).
These are meant for AE and AWB.
There’s no dedicated AF stats block.
AF usually comes from the sensor, PL logic, or post‑ISP processing.
Flexible streaming modes
MIMO, LILO, LIMO, handy for multi‑camera setups.

Bit depth and formats

Inputs: 8/10/12/14/16‑bit linear, plus compressed formats up to 24‑bit.
Outputs: YUV 4:2:2, YUV 4:2:0, Y‑only, or RGB at 8 or 10 bits.

So yes, it handles the wide range of sensors people actually use in automotive, robotics, and industrial vision.

Throughput

Each ISP instance tops out around 600 megapixels per second.
That covers things like:

3840×2160 @ 60 fps
4096×3072 @ ~40 fps

Some devices have multiple ISP tiles, so the whole chip can push well over 3 gigapixels per second. That’s where the “4K60 and beyond” idea comes from; it’s true at the device level, not per ISP block.

How the software side works

The ISP is tied into a split control model:

On the APU (Arm Cortex‑A78)

Runs Linux
Uses V4L2
Loads JSON configs
Handles buffers and streaming
Talks to the RPU through IPC

On the RPU (Arm Cortex‑R52)

Programs the ISP hardware
Runs AE and AWB
Handles I²C control for the sensor
Can run in lock‑step for safety
Can run multiple ISP tiles in AMP setups

RPU = Realtime Processing Unit

This setup is built for real‑time work, especially in automotive and robotics, where timing and safety matter.

How Gen 2 compares to Gen 1

Here’s the short version.

Gen 1

No hardened ISP
Everything imaging‑related lived in PL or off‑chip
Higher power
More design work
More timing headaches
Throughput depended on how much PL you were willing to burn

Gen 2

Hardened ISP in the MMD tile
Zero PL usage for the core pipeline
Predictable timing
Lower power
Cleaner hand‑off to AIE‑ML
Better CPU setup (A78 + R52)
Faster NoC (NoC2 with DDR5/LPDDR5X)

Quick comparison table

Feature	Versal Gen 1	Versal Gen 2
Hardened ISP	No	Yes
Noise reduction	Custom/PL	2DNR (spatial only)
AF stats	External/PL	Not in ISP (AE/AWB only)
Input bit depth	PL‑dependent	8–16 bit + compressed
Output formats	PL‑dependent	YUV/RGB (8/10 bit)
Per‑ISP throughput	Varies	~600 MP/s
Device throughput	Varies	>3 Gpix/s (multi‑tile)
CPU	A72 + R5F	A78 + R52
NoC	First‑gen	NoC2

Bottom line

Versal Gen 2 finally gives you a real, hardened ISP on the chip. It handles the front‑end camera work cleanly, feeds the AIE‑ML engines with the right formats, and saves you a ton of PL resources. It’s not trying to be a full “everything under the sun” ISP, there’s no temporal NR and no AF block, but it covers the core imaging steps that almost every system needs.

Understanding the ISP in AMD Versal Gen 2