Sign up for our newsletter and get the latest big data news and analysis.

Backprop Bonanza

Many new data scientists have voiced what they feel is the lack of a satisfying way to learn the concepts of back propagation/gradient computation in neural networks when taking undergrad level ML classes. So I thought I’d put together a number of useful learning resources to jump-start an understanding for this important process. The following list, curated from an informal Twitter poll, appears in no particular order. After consuming one or more of these resources, I’m confident you’ll feel empowered in your ability to take the next step in your development. Many research papers and texts will make a lot more sense. Enjoy!

Video: What is backpropagation really doing?

Lecture Slides: Deep Learning – Spring 2021, Lecture 5 – Intro to Optimization

Lecture Slides: Deep Learning Systems: Algorithms and Implementation, Fall 2021, Carnegie Mellon UniversityAutomatic Differentiation

Blog Post: Yet another backpropagation tutorial

Video: How Backpropagation Works

eBook Chapter: How the backpropagation algorithm works

Blog Post: Backprop and systolic arrays

Lecture slides with Python code: Backprop

Video: Backprop Bootcamp: Introduction to Backpropagation

Blog Post: Calculus on Computational Graphs: Backpropagation

Blog Post: A Visual Explanation of Gradient Descent Methods

Video: Neural Nets via MIT Open Courseware

Blog Post: One LEGO at a Time: Explaining the Math of how Neural Networks Learn with Implementation from Scratch

Blog Post: Reverse-mode automatic differentiation from scratch, in Python

Lecture Video: Backprop by Geoff Hinton

Video: Stanford Winter Quarter 2016 class: CS231n: Convolutional Neural Networks for Visual Recognition. Lecture 4 on Backprop

Blog Post: Hacker’s guide to Neural Networks with Javascript code by Andreg Karpathy

Blog Post: Back-propagation, an introduction

Lecture Slides: Introduction to Artificial Intelligence, CS188 UC Berkeley

Video: Backpropagation Intuition by Andrew Ng

Blog Post: Backpropagation 101

Blog Post: A Step by Step Backpropagation Example

Contributed by Daniel D. Gutierrez, Editor-in-Chief and Resident Data Scientist for insideBIGDATA. In addition to being a tech journalist, Daniel also is a consultant in data scientist, author, educator and sits on a number of advisory boards for various start-up companies. 

Sign up for the free insideBIGDATA newsletter.

Join us on Twitter: @InsideBigData1 –

Leave a Comment


Resource Links: