Course Description
Welcome to Cutting-Edge AI!This is technically Deep Learning in Python part 11 of my deep learning series, and my 3rd reinforcement learning course.
Deep Reinforcement Learning is actually the combination of 2 topics: Reinforcement Learning and Deep Learning (Neural Networks).
While both of these have been around for quite some time, it’s only been recently that Deep Learning has really taken off, and along with it, Reinforcement Learning.
The maturation of deep learning has propelled advances in reinforcement learning, which has been around since the 1980s, although some aspects of it, such as the Bellman equation, have been for much longer.
Recently, these advances have allowed us to showcase just how powerful reinforcement learning can be.
We’ve seen how AlphaZero can master the game of Go using only self-play.
This is just a few years after the original AlphaGo already beat a world champion in Go.
We’ve seen real-world robots learn how to walk, and even recover after being kicked over, despite only being trained using simulation.
Simulation is nice because it doesn’t require actual hardware, which is expensive. If your agent falls down, no real damage is done.
We’ve seen real-world robots learn hand dexterity, which is no small feat.
Walking is one thing, but that involves coarse movements. Hand dexterity is complex - you have many degrees of freedom and many of the forces involved are extremely subtle.
Imagine using your foot to do something you usually do with your hand, and you immediately understand why this would be difficult.
Last but not least - video games.
Even just considering the past few months, we’ve seen some amazing developments. AIs are now beating professional players in CS:GO and Dota 2.
So what makes this course different from the first two?
Now that we know deep learning works with reinforcement learning, the question becomes: how do we improve these algorithms?
This course is going to show you a few different ways: including the powerful A2C (Advantage Actor-Critic) algorithm, the DDPG (Deep Deterministic Policy Gradient) algorithm, and evolution strategies.
Evolution strategies is a new and fresh take on reinforcement learning, that kind of throws away all the old theory in favor of a more "black box" approach, inspired by biological evolution.
What’s also great about this new course is the variety of environments we get to look at.
First, we’re going to look at the classic Atari environments. These are important because they show that reinforcement learning agents can learn based on images alone.
Second, we’re going to look at MuJoCo, which is a physics simulator. This is the first step to building a robot that can navigate the real-world and understand physics - we first have to show it can work with simulated physics.
Finally, we’re going to look at Flappy Bird, everyone’s favorite mobile game just a few years ago.
What do you get if you sign up for the VIP version of this course? A brand new exclusive section covering an entirely new algorithm: TD3! As usual, both theory and code for this powerful state-of-the-art algorithm are provided.
Thanks for reading, and I’ll see you in class!
Suggested Prerequisites:
- calculus
- matrix arithmetic (adding, multiplying)
- probability
- Object-oriented programming
- Python coding: if/else, loops, lists, dicts, sets
- Numpy coding: matrix and vector operations, loading a CSV file
- linear regression, logistic regression
- neural networks and backpropagation
- Know how to build a convolutional neural network (CNN) in TensorFlow
- Markov Decision Proccesses (MDPs), Q-Learning
Lectures
- 10 sections
- 51 lectures
- 8h 39m total length
Reviews
38 reviews for this course
Testimonials and Success Stories
H. Z.
“I am one of your students. Yesterday, I presented my paper at ICCV 2019. You have a significant part in this, so I want to sincerely thank you for your in-depth guidance to the puzzle of deep learning. Please keep making awesome courses that teach us!”
Wade J.
“I just watched your short video on “Predicting Stock Prices with LSTMs: One Mistake Everyone Makes.” Giggled with delight.
You probably already know this, but some of us really and truly appreciate you. BTW, I spent a reasonable amount of time making a learning roadmap based on your courses and have started the journey.
Looking forward to your new stuff.”
Kris M.
“Thank you for doing this! I wish everyone who call’s themselves a Data Scientist would take the time to do this either as a refresher or learn the material. I have had to work with so many people in prior roles that wanted to jump right into machine learning on my teams and didn’t even understand the first thing about the basics you have in here!!
I am signing up so that I have the easy refresh when needed and the see what you consider important, as well as to support your great work, thank you.”
Steve M.
“I have been intending to send you an email expressing my gratitude for the work that you have done to create all of these data science courses in Machine Learning and Artificial Intelligence. I have been looking long and hard for courses that have mathematical rigor relative to the application of the ML & AI algorithms as opposed to just exhibit some 'canned routine' and then viola here is your neural network or logistical regression.
Your courses are just what I have been seeking. I am a retired mathematician, statistician and Supply Chain executive from a large Fortune 500 company in Ohio. I also taught mathematics, statistics and operations research courses at a couple of universities in Northern Ohio.
I have taken many courses and have enjoyed the journey, I am not going to be critical of any of the organizations from whom I have taken courses. However, when I read a review about one of your courses in which the student was complaining that one would need a PhD in Mathematics to understand it, I knew this was the course (or series of courses) that I wanted. (Having advanced degrees in mathematics, I knew that it was highly unlikely that a PhD would actually be required.)”
Saurabh W.
“Hi Sir I am a student from India. I've been wanting to write a note to thank you for the courses that you've made because they have changed my career. I wanted to work in the field of data science but I was not having proper guidance but then I stumbled upon your "Logistic Regression" course in March and since then, there's been no looking back. I learned ANNs, CNNs, RNNs, Tensorflow, NLP and whatnot by going through your lectures. The knowledge that I gained enabled me to get a job as a Business Technology Analyst at one of my dream firms even in the midst of this pandemic. For that, I shall always be grateful to you. Please keep making more courses with the level of detail that you do in low-level libraries like Theano.”
David P.
“I just wanted to reach out and thank you for your most excellent course that I am nearing finishing.
And, I couldn't agree more with some of your "rants", and found myself nodding vigorously!
You are an excellent teacher, and a rare breed.
And, your courses are frankly, more digestible and teach a student far more than some of the top-tier courses from ivy leagues I have taken in the past.
(I plan to go through many more courses, one by one!)
I know you must be deluged with complaints in spite of the best content around That's just human nature.
Also, satisfied people rarely take the time to write, so I thought I will write in for a change. :)”
P. C.
“Hello, Lazy Programmer!
In the process of completing my Master’s at Hunan University, China, I am writing this feedback to you in order to express my deep gratitude for all the knowledge and skills I have obtained studying your courses and following your recommendations.
The first course of yours I took was on Convolutional Neural Networks (“Deep Learning p.5”, as far as I remember). Answering one of my questions on the Q&A board, you suggested I should start from the beginning – the Linear and Logistic Regression courses. Despite that I assumed I had already known many basic things at that time, I overcame my “pride” and decided to start my journey in Deep Learning from scratch.
Course by course, I was renewing the basics and the prerequisites. Thus, in several months, after every day studying under your guidance, I was able to gain enough intuitions and practical skills in order to begin progressing in my research. Having a solid background, it was just a pleasure to read all the relevant papers in the field as well as to make all the experiments needed for achieving my goal – creating a high-performance CNN for offline HCCR.
I believe, the professionalism of any teacher can be estimated by the feedback received from their students, and it’s of the utmost importance for me to thank you, Lazy Programmer!
I want you to know, in spite, that we have never actually met and you haven’t taught me privately, I consider you one of my greatest Teachers.
The most important things I have learned from you (some in the hard way, though) beside many exciting modern Deep Learning/AI techniques and algorithms are:
1) If one doesn’t know how to program something, one doesn’t understand it completely.
2) If one is not honest with oneself about one’s prior knowledge, one will never succeed in studying more advanced things.
3) Developing skills in BOTH Math and Programming is what makes one a good student of this major.
I am still studying your courses, and am certain I will ask you more than just a few technical questions regarding their content, but I already would like to say, that I will remember your contribution to my adventure in the Deep Learning field, and consider it as big as one of such great scientists’ as Andrew Ng, Geoffrey Hinton, and my supervisor.
Thank you, Lazy Programmer! 非常感谢您，Lazy 老师！
If you are interested, you can find my first paper’s preprint here:
https://arxiv.org/abs/xxx”
Dima K.
“By the way, if you are interested to hear. I used the HMM classification, as it was in your course (95% of the script, I had little adjustments there), for the Customer-Care department in a big known fintech company. to predict who will call them, so they can call him before the rush hours, and improve the service. Instead of a poem, I Had a sequence of the last 24 hours' events that the customer had, like: "Loaded money", "Usage in the food service", "Entering the app", "Trying to change the password", etc... the label was called or didn't call. The outcome was great. They use it for their VIP customers. Our data science department and I got a lot of praise.”
Andres Lopez C.
“This course is exactly what I was looking for. The instructor does an impressive job making students understand they need to work hard in order to learned. The examples are clear, and the explanations of the theory is very interesting.”
Mohammed K.
“Thank you, I think you have opened my eyes. I was using API to implement Deep learning algorithms and each time I felt I was messing out on some things. So thank you very much.”
Tom P.
“I have now taken a few classes from some well-known AI profs at Stanford (Andrew Ng, Christopher Manning, …) with an overall average mark in the mid-90s. Just so you know, you are as good as any of them. But I hope that you already know that.
I wish you a happy and safe holiday season. I am glad you chose to share your knowledge with the rest of us.”
Join 30 day bootcamp for free