View on GitHub

wiki

A Walk with Deep Learning

Contents

  1. Introduction
  2. Deep Learning Fundamentals
    1. Fundamental Concepts
    2. Architectures
    3. The Deep Learning Revolution
  3. Getting Started
    1. Getting a GPU Deep Learning Server
    2. Deep Learning Libraries
    3. Development Tips
    4. Free Resources
  4. References

Introduction

What is Machine Learning? According to Arthur Samuel (1949): “Suppose we arrange for some automatic means of testing the effectiveness of any current weight assignment in terms of actual performance and provide a mechanism for altering the weight assignment so as to maximize the performance. We need not go into the details of such a procedure to see that it could be made entirely automatic and to see that a machine so programmed would “learn” from its experience”.

There are several types of machine learning designed to address different problems:

In the space domain, these can be used for various tasks, including:

Examples of applications of these different techniques to some of these problems are given in research output.

Deep Learning Fundamentals

The term Deep Learning in general refers to Machine Learning using neural networks.

Fundamental Concepts

Neural networks consist of:

Architectures

Depending on the problem, different deep learning architectures are appropriate. The most typical and popular architectures are:

See: an overview of neural networks architectures for a more extensive list.

The Deep Learning Revolution

Why is DL booming?

DL milestones?

Getting Started

Getting a GPU Deep Learning Server

Free online platforms with GPUs:

Deep Learning Libraries

Deep learning 101 with Pytorch and fastai tutorials.

Deep learning frameworks:

Development Tips

Experiment Tracking

Why should you track your experiments? Any experiment that isn’t tracked is doomed to be repeated…

Single scientists:

Teams:

Tools for ML experiment tracking (See a comparative in: https://neptune.ai/blog/best-ml-experiment-tracking-tools)

  1. Neptune
  2. Weights & Biases
  3. Comet
  4. Sacred
  5. MLFlow
  6. TensorBoard

Weights & Biases (wandb):

See wandb project for this talk: https://wandb.ai/vrodriguezf/TS-III

Software development in Jupyter Notebooks

See an example of a nbdev project in: https://github.com/stardust-r/walk-with-deep-learning

Free Resources

Free online courses:

Free books:

References

[1]: Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … & Polosukhin, I. (2017). Attention is all you need.