NoShade.Vision

Summarized Videos

Jeremy-Howard

Jeremy Howard demo for Mojo launch

Jeremy Howard demonstrated the capabilities of the new language Mojo, which is a super set of Python, showing how it solves the issue of writing concise, flexible and efficient code. He started with matrix multiplication, demonstrating how with a struct, feature vectorization, parallelization, and tiling it was possible to achieve a speed-up of 2200 times over Python. This is built into Mojo to enable users without programming or hardware expertise to optimize and accelerate computations. With Mojo, it's easy to create an auto-tuned version of a program, and compute the Mandelbrot set 35,000 times faster than Python, making it an ideal platform for writing neural networks.

Lesson 1 Deep Learning for Coders (2020)

This video introduces the Deep Learning for Coders course, which is designed to teach deep learning in a top-down approach, using the fastai library and PyTorch. The instructor addresses concerns about learning deep learning without a math background or expensive resources, emphasizing that deep learning is often better than human performance and accessible with Python libraries and GPUs. The course focuses on making the game worth playing, working on the hard parts, and understanding difficult concepts before diving into model building. The instructor explains how to set up a server with Colab and Jupyter Notebook, and introduces the environment that students will use to experiment with deep learning. Additionally, the video explains how machine learning works and how to train deep neural networks by adjusting the weights to solve problems and updating the loss back to the parameters.

Lesson 9: Deep Learning Foundations to Stable Diffusion, 2022

In "Lesson 9: Deep Learning Foundations to Stable Diffusion, 2022", the speaker introduces Part 2 of the "Practical Deep Learning for Coders" series, which focuses on stable diffusion and generative modeling. The course teaches foundational knowledge to keep up with the rapid developments in the field of deep learning. The speaker recommends exploring the "diffusion-nbs" repository and using the Diffusers library from Hugging Face for stable diffusion. Users can adjust the specificity of the output image using the guidance scale and negative prompts, and can fine-tune images using the image to image pipeline. The video also explains a new way of thinking about stable diffusion and how to calculate the gradient of the probability that an image is a handwritten digit with respect to its pixels.

Jeremy Howard and Joshua Browder discuss AI & Jobs with Piers Morgan

During a discussion with Piers Morgan, Joshua Browder expresses his belief that AI will replace many jobs, such as lawyers. However, he also argues that new jobs, like prompt engineers, will be created. Jeremy Howard disagrees, saying that AI and robots are replacing human brains as well, and even if new jobs arise, they might not be enough for everyone, and human roles may change entirely. They also discuss AI's potential benefits, with Joshua seeing it as a tool to empower ordinary people and Jeremy seeing opportunities for democratizing education, such as offering personalized tutoring through robots.

Lesson 11 2022: Deep Learning Foundations to Stable Diffusion

The video covers various topics related to deep learning, including new techniques and tools for reading research papers, an overview of denoising diffusion probabilistic models, and a discussion on the DiffEdit algorithm. The instructor emphasizes the importance of understanding mathematical symbols and recommends resources for conversion and identification. Additionally, the DiffEdit algorithm is explained in detail, providing insight into an algorithm designed to add noise to images and derive a mask based on the denoising results. Overall, the video offers a comprehensive overview of relevant topics and new developments in the field of deep learning.

Lesson 15: Deep Learning Foundations to Stable Diffusion

The video discusses various topics related to convolutional neural networks (CNNs). The instructor explains what convolutions are and how they can be used to convey structural information to the model. They discuss the process of performing convolutions using sliding windows and the "im2col" trick, which allows convolution to be performed using matrix multiplication for faster computation on GPUs. The video also covers stride and padding, as well as reducing dimensionality using max-pooling or stride-2 convolution. The importance of receptive fields is also discussed, and the instructor demonstrates how to calculate the number of parameters in a model using PyTorch and even shows how to perform CNN operations using Microsoft Excel. Finally, the instructor uses the fashion MNIST dataset to illustrate how to create a classifier using a sequential model, along with creating a collate function and data loader function.

Lesson 16: Deep Learning Foundations to Stable Diffusion

In this lesson on deep learning, the instructor demonstrates how to create and use callbacks in fastai to tweak and monitor the training process. The callback class has several methods to handle exceptions, each of which can cancel an epoch, batch, or the whole fitting process. The instructor creates several callbacks, including ones to calculate metrics, track progress, and modify the learning rate schedule. Additionally, the instructor shows how to subclass the Learner class to create a more flexible version with custom functions and parameters. The video also touches on the concepts of momentum and learning rate schedulers and how to diagnose problems with model training.

Lesson 17: Deep Learning Foundations to Stable Diffusion

In this YouTube video, the instructor discusses the minor changes made to the miniai library, including the addition of a TrainLearner and HooksCallback, which simplifies hooks into a callback, resulting in code that is more flexible and simpler. The video also covers the importance of creating visualizations of what is happening inside a deep learning model and the difficulties of determining what is going on with the learning rate finder. Additionally, statistical concepts, including variance, covariance, Pearson correlation coefficient, and Xavier initialization, are discussed, as well as the problem of disappearing gradients when using the ReLU activation function and how Kaiming initialization or He initialization resolves this issue. The importance of initializing neural networks correctly is also covered, which can be achieved using the LSUV method, which initializes any neural network correctly regardless of activation function.

Lesson 18: Deep Learning Foundations to Stable Diffusion

This video provides a detailed explanation of stochastic gradient descent (SGD) and linear regression, demonstrating how calculated and analytical methods can be used to adjust the estimated derivatives for the intercept and slope to improve prediction accuracy. The video also covers the concept of momentum and compares optimization algorithms like momentum, RMSProp, and Adam, as well as how to use PyTorch optimizers and schedulers to adjust learning rates and explore APIs using a REPL environment. Additionally, the video explains the importance of initializing models correctly and explores warmup techniques like T-Fixup. The final section covers updates to the Learner function and the addition of more parameters, as well as ResNets, deeper and wider models that can improve neural nets' ability to learn by providing more opportunities to do so, and the OneCycle learning rate with the scheduler and callbacks they use.

Lesson 19: Deep Learning Foundations to Stable Diffusion

In this video, Jeremy discusses what Dropout is and shows how to implement it in PyTorch. He also introduces a callback that puts the model in training mode all the time, even at evaluation, if the model is a Dropout. The presenters then move on to exploring diffusion models and the backward and forward processes involved in generative modeling, with a focus on the Stable Diffusion model. The simplified training objective needed to train the diffusion model is also explained, including the epsilon-theta function used to predict the noise added to images. Lastly, they discuss the use of Greek letters in code, before moving on to discussing how to predict the amount of noise in the diffusion model.