Song Han, pHd

Song Han is an assistant professor in MIT’s Department of Electrical Engineering and Computer Science. His research focuses on efficient deep learning computing. He proposed “deep compression” technique that can reduce neural network size by an order of magnitude without losing accuracy, and the hardware implementation “efficient inference engine” that first exploited model compression and weight sparsity in deep learning accelerators. Recently he is interested in AutoML and NAS methods for efficient TinyML models. He is a recipient of NSF CAREER Award, MIT Technology Review Innovators Under 35, best paper award at the ICLR’16 and FPGA’17, Facebook Faculty Award, SONY Faculty Award, AWS Machine Learning Award. Many of the pruning, compression, and acceleration techniques have been integrated into commercial AI chips. He was the co-founder and chief scientist of DeePhi Tech (acquired by Xilinx). He earned a PhD in electrical engineering from Stanford University.

Abstract

Efficient Deep Learning with Once-for-All Network

Last June, researchers released a startling report estimating that the amount of power required for training and searching a certain neural network architecture involves the emissions of roughly 626,000 pounds of carbon dioxide. That’s equivalent to nearly five times the lifetime emissions of the average U.S. car, including its manufacturing. This issue gets even more severe in the model deployment phase, where deep neural networks need to be deployed on diverse hardware platforms, each with different properties and computational resources. I will present a new automated AI system for training and running neural networks efficiently, the once-for-all network. Results indicate that, by improving the computational efficiency of the system with weight sharing and progressive shrinking, the system can cut down the pounds of carbon emissions involved in neural architecture search by thousands of times. The produced model consistently outperforms state-of-the-art NAS methods including MobileNet-v3 and EfficientNet, receiving the first place in the 3rd and 4th Low Power Computer Vision Challenge (LPCVC).