Deep Learning Programming at Scale
Deep learning has become one of the most important computational workloads of our generation, advancing applications across industries from healthcare to autonomous driving. But it is also profoundly computationally intensive.
June 29, 2021
Limits to Scale-Out for Training Language Models
Natural language processing has revolutionized how data is consumed, meaning that computational demand has skyrocketed. Companies in every industry are using GPU clusters to keep up. But is this really the best solution?
June 24, 2021
Argonne National Laboratory
At Argonne National Laboratory, researchers work to gain a deeper understanding of our planet, our climate, and the cosmos. However, they were running into major challenges associated with scaling large AI models across a cluster of GPUs. Download the case study to find out how the research center overcame these challenges with Cerebras Systems.
June 8, 2021
Memory Efficient 3D U-Net with Reversible Mobile Inverted Bottlenecks for Brain Tumor Segmentation
We propose combining memory saving techniques with traditional U-Net architectures to increase the complexity of the models on the Brain Tumor Segmentation (BraTS) challenge. The BraTS challenge consists of a 3D segmentation of a 240 240 155 4 input image into a set of tumor classes.
March 5, 2021
Mihir Pendse, Vithursan Thangarasa, Vitaliy Chiley, Ryan Holmdahl, Joel Hestness, Dennis DeCoste
Pipelined Backpropagation at Scale: Training Large Models without Batches
New hardware can substantially increase the speed and efficiency of deep neural network training. To guide the development of future hardware architectures, it is pertinent to explore the hardware and machine learning properties of alternative training algorithms.
March 1, 2021
Atli Kosson, Vitaliy Chiley, Abhinav Venigalla, Joel Hestness, Urs Koster
Fast Stencil-Code Computation on a Wafer-Scale Processor
The performance of CPU-based and GPU-based systems is often low for PDE codes, where large, sparse, and often structured systems of linear equations must be solved. Iterative solvers are limited by data movement, both between caches and memory and between nodes.
October 7, 2020
Kamil Rocki, Dirk Van Essendelft, Ilya Sharapov, Robert Schreiber, Michael Morrison, Vladimir Kibardin, Andrey Portnoy, Jean Francois Dietiker, Madhava Syamlal, and Michael James