A YouTube creator running the channel "traffic in the brain" has published a detailed walkthrough of building a small-scale AI accelerator from scratch on a budget FPGA board, recreating the core architectural ideas behind Google's Tensor Processing Unit (TPU).
The project, released September 17, 2026, implements a two-layer attention model, the same fundamental building block used in large language models, to recognize hand-drawn shapes on a 16x6 LED grid in real time. Rather than running the neural network entirely on a PC, the creator offloads the attention computation to an FPGA programmed to behave like a miniature TPU.
The core of the design is a systolic array, an architecture where data flows through a grid of multiply-accumulate units in rhythmic pulses rather than each unit independently fetching values from memory. This is the same technique Google uses in its production TPUs, where a much larger 256x256 systolic array forms the chip's matrix-multiply unit. On the FPGA, an 8x8 systolic array handles matrix multiplication using the chip's 64 available DSP slices, with double buffering used to hide memory-load latency behind computation.
To avoid the cost of computing exponentials and division for the model's softmax step, the design substitutes lookup tables, introducing roughly 0.9 percent average error while preserving correct classification about 96 percent of the time. Numeric values throughout the pipeline are quantized to an 8.8 fixed-point format to fit the board's limited on-chip memory, which also caps the model at roughly 69,000 parameters.
