Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Search
Author(s)
Heuillet, Alexandre
Nasser, Ahmad
Arioui, Hichem
Tabia, Hedi
Date Issued
June 28, 2024
Journal
ACM Computing Surveys
Publisher
ACM
Citation
Heuillet, Alexandre, Nasser, Ahmad, Arioui, Hichem and Tabia, Hedi. 2024. "Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Search." ACM Computing Surveys, 56 (11).
Version
Final published version
Abstract
In the past few years, Differentiable Neural Architecture Search (DNAS) rapidly imposed itself as the trending approach to automate the discovery of deep neural network architectures. This rise is mainly due to the popularity of DARTS (Differentiable ARchitecTure Search), one of the first major DNAS methods. In contrast with previous works based on Reinforcement Learning or Evolutionary Algorithms, DNAS is faster by several orders of magnitude and uses fewer computational resources. In this comprehensive survey, we focused specifically on DNAS and reviewed recent approaches in this field. Furthermore, we proposed a novel challenge-based taxonomy to classify DNAS methods. We also discussed the contributions brought to DNAS in the past few years and its impact on the global NAS field. Finally, we concluded by giving some insights into future research directions for the DNAS field.
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3665138
