PhD Course - Deep Learning - A Computational Efficiency Perspective
Organizer(s):
- Pınar Tözün (Associate Prof., main organizer)
- Ties Robroek (PhD student, helper)
- Ehsan Yousefzadeh-Asl-Miandoab (PhD student, helper)
Course advertisement:
- We have a link at Computer Science Wiki: Link
Lecturer(s):
Date(s) of the course:
- In total 9 lectures occurring every Monday from January 31, 2022 to April 4, 2022. There will be a break week because of instructor attending a Dagstuhl seminar on March 14.
Time:
Room:
- The course will be given in hybrid format with in-person presence at 4A05 and online presence on ITU’s Teams.
Course description:
Data scientists achieve more accurate predictions than ever, even in real-time, for the benefit of our society. Their productivity is fuelled by the exponential evolution of hardware and the surge of deep learning tools, which hide the complexities of hardware. However, there is a widening performance gap between the software tools and modern hardware. Modern tools are not able to utilize modern hardware fully. There is a pressing need for more hardware-conscious deep learning. In this course, our goal is to learn about methods to perform deep learning efficiently on modern hardware.
Intended learning outcomes are the following:
(1) Reflect on the evolution of modern hardware, especially in the context of deep learning applications
(2) Analyze performance metrics relevant for deep learning applications
(3) Reflect on the methods for computationally-efficient deep learning across different hardware platforms
Reading list:
Mandatory reading:
We will be reading one or more chapters from the book below each week. We may complement some of the weeks with a research paper if desired by the participants.
- Efficient Processing of Deep Neural Networks
Optional reading for preparation for the class:
The book below is a short and higher-level introduction to some of the topics we will dig deeper into in this class. It is written to help computer architects to get familiar with deep learning topics. If the participants wish to get a high-level familiarity with some of the terminology earlier on, we recommend them to read this book before the class starts. However, this isn’t mandatory reading for the lectures themselves.
- Deep Learning for Computer Architects
Programme:
- The program can be found here
Prerequisites:
- Some familiarity with machine learning and computer architecture would be useful. However, we want to leave the course open to anyone who are interested in computational efficiency of machine learning workloads. If we realize that a big majority of the people in the course need background material on a certain topic, then we will re-arrange some reading material to include background on that topic.
Exam:
- Oral Exam with Pass/Fail result
Credits:
Amount of hours the student is expected to use on the course:
- Participation: 2 hours – Weekly lectures will be 2 hours of presentation & discussion weekly on that week’s topic
- Preparation: Approx. 4 hours