Deep learning is currently being hyped as an almost magical tool for solving all kinds of difficult problems that computers have not been able to solve in the past. Particularly in the fields of computer vision and natural language processing, spectacular results have been achieved. The hype has now infiltrated several scientific communities. In (nano-)photonics, researchers are trying to apply deep learning to all kinds of forward and inverse problems. A particularly challenging problem is for instance the rational design of nanophotonic materials and devices. In this opinion article, I will first discuss the public expectations of deep learning and give an overview of the quite different scales at which actors from industry and research are operating their deep learning models. I then examine the weaknesses and dangers associated with deep learning. Finally, I'll discuss the key strengths that make this new set of statistical methods so attractive, and review a personal selection of opportunities that shouldn't be missed in the current developments.