Deep Learning: Fundamentals and Applications
Prerequisites (knowledge of topic)
- Basic programming skills, Python recommended
- Undergraduate-level linear algebra, analysis and statistics
Hardware
- Personal laptop with Mac OSX or Linux, Windows
- Tablets (iOS, Windows) will not be working with this lecture
Software
- Webbrowser (Chrome, Safari, Firefox)
- Text editor
- Jupyer notebook
- Local Python installation including Numpy, Scipy, scikit-learn, PyTorch
(there will be an installation session on the first day for participants)
Course content
- Machine Learning Refresh
o Supervised Learning vs. Unsupervised Learning
o Traditional Machine Learning vs. End-to-End Learning
- Fundamentals of Neuronal Networks:
o Rosenblatt Perceptron and Neurons
o Network Structure (feed-forward, recurrent), matrix notation, forward evaluation
- Training as optimization
o Loss and Error functions
o Backpropagation
o SGD and other optimizer
- Activation functions and topologies
o Convolutional neural networks
o Generative Adversarial Networks
o Long short-term memory networks
o Special layer types (inception, resnet)
o Embeddings
o Attention Mechanis & Transformer
- Applications to real-world problems:
o Acoustic keyword recognition (audio/speech processing)
o Sentiment analysis (text processing)
o Digit recognition (image processing)
o Tiny Image Recognition (image processing)
o Face Detection and Tracking (image/video processing)
o Stock market prediction (time series prediction)
- Training on large data sets (Hardware, GPU)
- Trustworthy AI
Structure
The course is a theoretical content in the morning and practical exercises in the afternoon in form of lab Jupyter notebook programming.
Literature
Goodfellow I, Benjo Y., Courville A., Courville A, Deep Learning, MIT Press, 2016
Supplementary
• http://jupyter.org/
• http://www.numpy.org/
• https://www.scipy.org/
• https://www.tensorflow.org
• https://pytorch.org
Examination part
- Completed Jupyter notebooks labs 1-8 (40% closed book), in class
- Complete Jupyter Notebook Assignment 1-8 (60%, open-book), at home