The Cerebras CS-1 Chip Is 10,000 Times Faster Than a GPU

Cerebras Systems and its partner, the Department of Energy’s National Energy Technology Laboratory (NETL), revealed that its CS-1 system, featuring a single massive chip that features an innovative design, is 10,000+ times faster than a graphics processing unit (GPU). The CS-1, built around Cerebas’ Wafer-Scale Engine (WSE) and its 400,000 AI cores, was first announced in November 2019. The partnership between the Energy Department and Cerebras includes deployments with the Argonne National Laboratory and Lawrence Livermore National Laboratory.

Los Altos-based Cerebras was founded by SeaMicro’s Andrew Feldman. VentureBeat reports that, rather than dividing the wafer into many chips, Cerebras “makes a single, massive chip out” out of the WSE wherein “each piece of the chip, dubbed a core, is interconnected in a sophisticated way to other cores,” which keeps the cores “functioning at high speeds so the transistors can work together as one.”

The WSE chip has “1.2 trillion transistors, the basic on-off electronic switches that are the building blocks of silicon chips,” compared to Nvidia’s A100 80GB with 54 billion transistors. Because the processor and memory are “closer to each other and have lots of bandwidth to connect them … the CS-1 can deliver performance that is unattainable with any number of central processing units (CPUs) and GPUs, which are both commonly used in supercomputers.”

Feldman reported that, “the CS-1 was also 200 times faster than the Joule Supercomputer, which is No. 82 on a list of the top 500 supercomputers in the world.” The Joule Supercomputer features 84,000 CPU cores, consumes 450 kilowatts of power and cost “tens of millions of dollars to build.”

Cerebras’ CS-1, meanwhile, cost “several million dollars and uses 20 kilowatts of power.” “For these workloads, the wafer-scale CS-1 is the fastest machine ever built,” said Feldman. A single CS-1 is “26 inches tall, fits in one-third of a rack, and is powered by the industry’s only wafer-scale processing engine, Cerebras’ WSE.”

Feldman noted that, “that the CS-1 can finish calculations faster than real time, meaning it can start the simulation of a power plant’s reaction core when the reaction starts and finish the simulation before the reaction ends.” NETL machine learning and data science engineer Dirk Van Essendelft and Cerebras co-founder and chief architecture of advanced technologies Michael James led the research.

Japan Named HPC Leader as World Races to Exascale, SearchDataCenter, 11/16/20
Nvidia Debuts New Version of A100 GPU for Supercomputers, ETCentric, 11/18/20
Global Competition Ramps Up in the Semiconductor Industry, ETCentric, 11/18/20