Deep Learning: Fundamentals and Applications

Prerequisites (knowledge of topic)
-    Basic programming skills, Python recommended
-    Undergraduate-level linear algebra, analysis and statistics

Hardware
-    Personal laptop with Mac OSX or Linux, Windows
-    Tablets (iOS, Windows) will not be working with this lecture

Software
-    Webbrowser (Chrome, Safari, Firefox)
-    Text editor
-    Jupyer notebook
-    Local Python installation including Numpy, Scipy, scikit-learn, PyTorch
(there will be an installation session on the first day for participants)

Course content
-    Machine Learning Refresh
o    Supervised Learning vs. Unsupervised Learning
o    Traditional Machine Learning vs. End-to-End Learning
-    Fundamentals of Neuronal Networks:
o    Rosenblatt Perceptron and Neurons
o    Network Structure (feed-forward, recurrent), matrix notation, forward evaluation
-    Training as optimization
o    Loss and Error functions
o    Backpropagation
o    SGD and other optimizer
-    Activation functions and topologies
o    Convolutional neural networks
o    Generative Adversarial Networks
o    Long short-term memory networks
o    Special layer types (inception, resnet)
o    Embeddings
o    Attention Mechanis & Transformer
-    Applications to real-world problems:
o    Acoustic keyword recognition (audio/speech processing)
o    Sentiment analysis (text processing)
o    Digit recognition (image processing)
o    Tiny Image Recognition (image processing)
o    Face Detection and Tracking (image/video processing)
o    Stock market prediction (time series prediction)
-    Training on large data sets (Hardware, GPU)
-    Trustworthy AI

Structure
The course is a theoretical content in the morning and practical exercises in the afternoon in form of lab Jupyter notebook programming.

Literature
Goodfellow I, Benjo Y., Courville A., Courville A, Deep Learning, MIT Press, 2016

Supplementary
•    http://jupyter.org/
•    http://www.numpy.org/
•    https://www.scipy.org/
•    https://www.tensorflow.org
•    https://pytorch.org

Examination part
-    Completed Jupyter notebooks labs 1-8 (40% closed book), in class
-    Complete Jupyter Notebook Assignment 1-8 (60%, open-book),  at home