PhD student Sean Smithson will present his research on system-level design automation for machine learning this week:
Neural networks designing neural networks: Multi-objective hyper-parameter optimization
The objective of this work is to address an enormous challenge in the design of artificial neural networks: network design. Typically, domain experts perform manual design and optimization of network hyper-parameters (how many layers, how many neurons per layer, etc). The design for one application provides little insight into the design for another, resulting in significant designer effort. Furthermore, most work to date has focused on optimizing accuracy alone, while emerging applications of machine learning target cost-constrained embedded systems. Our work uses a neural network to model a response surface relating hyper-parameters to accuracy in order to quickly identify promising network configurations, resulting in a wider variety of lower cost solutions than could be expected from accuracy optimization alone. This work will be presented at the 2016 International Conference On Computer Aided Design (ICCAD), and will later also appear at Workshop on Hardware and Algorithms for Learning On-a-chip (HALO).