Deep Learning Terms

Please choose from the following terms as your topic for presentation.

Course Homepage:
http://cs.haifa.ac.il/hagit/courses/seminars/DeepLearningTerms/DeepLearningTerms.html

* Diffusion Networks Omer (16/11/22)
Blog: https://www.assemblyai.com/blog/diffusion-models-for-machine-learning-introduction/
Excellent video: https://www.youtube.com/watch?v=HoKDTa5jHvg
DDPM Paper: https://arxiv.org/abs/2006.11239

* Transformers Chen (23/11/22)
Vision Transformers (VIT)
(Bert)
Blog: https://neptune.ai/blog/comprehensive-guide-to-transformers
https://neptune.ai/blog/bert-and-the-transformer-architecture
VIT (link to paper within) https://towardsdatascience.com/are-transformers-better-than-cnns-at-image-recognition-ced60ccc7c8

* Self-supervised learning (SSL) Gil (30/11/22)
Blog : https://neptune.ai/blog/self-supervised-learning
(also see refs within at the end)
Paper connecting to human brain: https://arxiv.org/abs/2210.09224
Tutorial in ECCV2022 https://gidariss.github.io/ssl-on-wheels-eccv2022/

* Spiking neural nets (snn)
Blog: https://cnvrg.io/spiking-neural-networks/

* Lottery ticket hypothesis - Daniel (14/12/22)
relates to Sparse networks
Blog: https://towardsdatascience.com/demystifying-the-lottery-ticket-hypothesis-in-deep-learning-158570b62674
Blog (paper within) : https://medium.com/dair-ai/the-lottery-ticket-hypothesis-7cd4eae3faaa
Paper: https://arxiv.org/abs/1803.03635

* Meta learning with language models
Deep-meta learning

* Masked Auto-Encoders
Blog: https://www.youtube.com/watch?v=iww7u0uZGH0
ECCV2022 paper: https://arxiv.org/abs/2205.09113

* Neural Radiance Fields (NERF) Azmi (21/12/22)
NeRF https://www.matthewtancik.com/nerf
GQN https://www.deepmind.com/blog/neural-scene-representation-and-rendering
Functa https://arxiv.org/abs/2201.12204