Edge AI Computers Explained: 6 TOPS NPUs, Vision Inference and How to Specify One

AI Edge Computing Box / Multi-channel Video Gateway

"Edge AI computer" is one of the fastest-rising phrases in industrial procurement, and it is also one of the most loosely used. This guide defines what the term means in practice, explains what a 6 TOPS NPU can run, and gives a specification checklist for buying a machine-vision or inspection computer that will actually work in production.

What an edge AI computer is

An edge AI computer runs inference locally, on the device, instead of sending data to a cloud server. The motivation is usually latency, privacy, bandwidth or offline operation: a camera that detects defects on a line cannot wait for a round trip to a data centre, and factory video often cannot leave the premises. The hardware is typically an ARM single-board computer or a fanless box with a dedicated neural processing unit.

NPU TOPS explained, without the marketing

TOPS means tera-operations per second. The number is meaningful only with a data type: 6 TOPS at INT8 is the common rating for the RK3588 NPU. INT8 uses 8-bit integers, which is sufficient for many vision models after quantisation; FP16 offers more precision at lower throughput. When comparing boards, ask whether the quoted figure is on-chip or includes an external accelerator, and which precision it refers to. Vendors quote RK3588 configurations as 6 TOPS on-chip, or higher totals when an M.2 accelerator is added.

What 6 TOPS can realistically run

  • Real-time object detection with YOLO-class models on several camera streams.
  • Face detection and recognition across multiple channels.
  • Speech recognition and keyword spotting.
  • Small quantized language models (typically 1–3B parameters), with larger models needing an accelerator.

The practical ceiling is set as much by memory bandwidth and the software toolchain as by TOPS. A well-supported NPU with RKNN-Toolkit2 and RKLLM will outperform a nominally faster chip with poor tooling.

A vision inference pipeline in practice

A machine-vision computer takes in video from a MIPI-CSI or USB camera, preprocesses frames, runs inference on the NPU, and delivers results over Ethernet, serial or a display interface. Each stage has a specification to check: camera interface and sensor support, memory for frame buffers, NPU throughput, and output interfaces. A bottleneck in any one stage limits the whole system.

Specification checklist

  1. NPU and toolchain. Confirm INT8 TOPS and that a maintained conversion tool exists.
  2. Memory and storage. 8 GB is a practical minimum for vision; 16–32 GB for local LLMs or multi-stream work.
  3. Networking. Dual Gigabit or Gigabit Ethernet if the box is a gateway; 4G/5G options for remote sites.
  4. Industrial I/O. RS-485 and CAN for equipment; note whether ports are isolated.
  5. Thermal and enclosure. Fanless operation and a wide temperature range (−40 to 85 °C) for cabinets and outdoor use.
  6. Long-term support. A documented BSP update and security-patch policy.

TOPEET edge AI products

The AI Edge Computing Box / Multi-channel Video Gateway is built for video analytics at the edge. The Lingqian 4G/5G AI Edge Computing Box adds cellular connectivity for remote deployments. For designers who want to build their own enclosure and I/O, the iTOP-RK3588 SBC provides the 6 TOPS NPU on a carrier-plus-module format, with an optional RK1828 / RK1820 M.2 accelerator for heavier inference.

Frequently asked questions

Do edge AI computers need the cloud? No. Inference runs locally; the cloud is optional for fleet management or model updates.

What is the difference between an edge AI box and an SBC? A box is an integrated, often fanless system ready to deploy; an SBC is a board for building a custom product around your own enclosure and I/O.

How much power does a 6 TOPS board use? Typically in the 5–25 W range depending on load and configuration, far below a GPU workstation.

Related reading: RK3588 compared for edge AI buyers and how to choose an industrial SBC or SoM.

0 comments

Leave a comment

Please note, comments need to be approved before they are published.