Deep Learning is regarded as a panacea in the era of big data. In this course, you will learn the foundations of deep learning, understand how to interprete others' neural networks, and learn how to build your own neural networks. You will also learn about popular neural network structures including Convolutional neural networks, RNNs, LSTM, and transformers.
- This course meets for in-class lecture Fri 9:10AM - 12:00AM (Seminar room No.2 at KISTI KIUM).
- For all inquiries related to this course, please contact bart7449 AT gmail DOT com
- Kyong-Ha Lee
- Fri. 9:10 AM ~ 12:00PM
- Book: Neural networks and deep learning: a textbook by Charu C. Aggarwal(but not mandatory)
- All slides for this class will be available here.
- Some papers in a reading list will be reviewed and presented by students
- All course announcements take place though this page. Please check this page frequently.
- This course has the following components:
- In-class lecture (1.5~2H) (attendance 10%)
- Paper/code reivew(0~1.5H)(40%)
- A final exam(50%)
| Event | Date | In-class lecture | Materials and Assignments |
|---|---|---|---|
| Lecture 1 | 03/05 | Course Introduction |
|
| Lecture 2 | 03/12 | Topics:
|
|
| Lecture 3 | 03/19 | Topics: Machine Learning with Shallow Neural Networks I(slides) | Paper review:
|
| Lecture 4 | 03/26 | Topics: Machine Learning with Shallow Neural Networks II(slides) | Paper review:
|
| Lecture 5 | 04/02 | Topics: Training Deep Neural Networks I(slides) | Paper review:
|
| Lecture 6 | 04/09 | Topics: Training Deep Neural Networks II(slides) | Paper Review:
|
| Lecture 7 | 04/16 | Topics: Training Deep Neural Networks III(slides) | Paper Review: |
| Lecture 8 | 04/23 | Topics: Teaching Deep Learners to Generalize (slides) | No Assignments |
| Lecture 9 | 04/30 | Topics: Convolutional Neural Networks I(slides) | Paper Reviews:
|
| Lecture 10 | 05/07 | Topics: Convolutional Neural Networks II (slides) | Paper Reviews:
|
| Lecture 11 | 05/14 | Topics: Backpropagations in CNN (slides) | Paper review:
|
| Lecture 12 | 05/21 | Topics: Recurrent Neural Networks (slides) | Paper Reviews:
|
| Lecture 13 | 05/28 | Topics: Attention and Language Model(slides) | Paper Reviews:
|
| Lecture 14 | 06/04 | Topis: Lightweight Model
|
Paper reviews:
|
| Lecture 15 | 06/11 | Topics: Lightweight Model
|
Paper reviews: |
| Lecture 16 | 06/18 | Final Presentations | No Assignments |
- [Ioffe15a] Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, In Proceedings of ICLR 2015.
- (optional)[Ba2016a] Layer Normalization, arXiv:1607.06450v1, 2016.
- (optional)[Glorot10a]Understanding the difficulty of training deep feedforward neural networks, In proceedings of AISTATS 2010.
- (optional)[He2015a] Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification, In Proceedings of ICCV 2015.
- [Hinton15a] Distilling the Knowledge in a Neural Network, arXiv:1503.02531v1, 2015.
- [Alex12a] ImageNet Classification with Deep Convolutional Neural Networks , Advances in Neural Informaiton Processing Systems 25:1098-1105, 2012
- [He16a] Deep Residual Learning for Image Recognition, In Proceedings of CVPR 2016.
- [He16b] Identity mapping in deep residual networks, In Proceedings of ECCV 2016.
- [Huang17a] Densely connected convolutional networks, In Proceedings of CVPR 2017.
- [Hu2018]Squeeze-and-excitation networks, In Proceedings of CVPR 2018.
- [Kipf2017]Semi-supervised classification with graph convolutional networks, In Proceedings of ICLR 2017
- [Mikolov13a] Distributed Representations of Words and Phrases and their Compositionality, Advances in Neural Information Processing Systems 26 (2013): 3111-3119.
- [Rong2016a] word2vec Parameter Learning Explained, arXiv:1411.2738v4
- [Le14a]Distributed Representations of Sentences and Documents, In Proceedings of the ICML 2014.
- [Bojanowski17a] Enriching word vectors with subword information, Transactions of the Association for Computational Linguistics 5 (2017): 135-146.
- [Grover16a] node2vec: Scalable Feature Learning for Networks, In Proceedings of KDD 2016.
- [Badahnau16a] NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE, In Proceedings of ICLR 2015
- [Vaswani17a] Attention is All You Need, In Proceedings of NeurIPS 2017
- [Devlin19a] BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, In Proceedings of ACL 2019
- [Rastegary16a] XNOR-Net: ImageNet classification using binary convolutional Neural Networks, In Proceedings of ECCV 2016
- [Sandler18a] Mobilenetv2: Inverted residuals and linear bottlenecks, In Proceedings of CVPR 2018
- [Howard17a] MobileNets: Efficient convolutional Neural Networks for Mobile Vision Applications, arXiv preprint arXiv:1704.04861, 2017
- [Stock20a] And the bit goes down: revisiting the quantization of neural networks, In Proceedings of ICLR 2020