HomeCoursesOptimization for Machine Learning with NumPy and SciPy

Intermediate

Updated 5 months ago

Optimization for Machine Learning with NumPy and SciPy

Learn optimization for machine learning, including gradients, convex optimization, and gradient descent. Explore constrained optimization and advanced methods using NumPy and SciPy.

Join 2.7 million developers at

Overview

Content

Reviews

In this course, you will learn about optimization, one of the fundamental pillars of mathematics and machine learning. Machine learning depends heavily on optimization because it allows the model to learn from data and generate precise predictions. You will begin by introducing optimization. Then, you will learn about optimization basics, including gradients and integrals. Next, you will cover convex optimization. You will then learn how to compute gradient descent for non-convex optimization. Next, you’ll learn how to perform constrained optimization. You will finish the course by studying the miscellaneous methods of optimization, like Newton’s methods, quasi-Newton methods, and conjugate gradient descent. After completing this course, you’ll have the practical skills to formulate, analyze, and implement optimization algorithms for machine learning using the NumPy and SciPy libraries. This will help you become a highly proficient data scientist or machine learning engineer.

In this course, you will learn about optimization, one of the fundamental pillars of mathematics and machine learning. Machine l...Show More

WHAT YOU'LL LEARN

An understanding of how real-world problems can be framed as optimization problems

Working knowledge about the taxonomy of optimization problems and techniques

Familiarity with the fundamental concepts of optimization

Hands-on experience implementing popular optimization algorithms

The ability to solve popular machine learning problems using optimization

An understanding of how real-world problems can be framed as optimization problems

TAKEAWAY SKILLS

Optimization

Machine Learning

Plotting

Content

62 Lessons1 Project6 Quizzes1 Assessment12 Challenges

Introduction to Optimization

8 Lessons

Get familiar with optimization techniques, algorithms, and their applications in machine learning.

What Is This Course About?

The Motivation for Optimization

Random Search Optimization

Grid Search Optimization

The Nelder-Mead Algorithm

Quiz Yourself on Optimization

Challenge: Grid Search

Solution: Grid Search

Vector Calculus

10 Lessons

Discover the logic behind differentiation, gradients, Hessian, Taylor series, and integrals in vector calculus.

Differentiation of the Univariate Functions

Partial Derivatives and Gradients

Gradients of the Vector-Valued Functions

Gradients of Matrices

Higher-Order Gradients: Hessian

Linearization and the Taylor Series

Integrals and the Trapezoidal Rule

Quiz Yourself on Vector Calculus

Challenge: The Taylor Series

Solution: The Taylor Series

Convex Optimization

14 Lessons

Examine convex optimization techniques, including gradient methods, ridge regression, and Fisher analysis.

Convex Sets and Functions

Gradient-Solving Approach

Solving Ridge Regression

Fisher Discriminant Analysis

Maximum Likelihood Estimation (MLE)

Maximum A Posteriori (MAP)

Alternating Least Squares Regression

Gradient Descent: Logistic Regression

Subgradient Descent—Lasso Regression

Steepest Descent

Coordinate Descent

Quiz Yourself on Convex Optimization

Challenge: Convex Optimization

Solution: Convex Optimization

Gradient Descent for Non-Convex Optimization

10 Lessons

Grasp the fundamentals of advanced gradient descent techniques for optimizing non-convex problems.

Limitations of Gradient Descent

Minibatch Gradient Descent

The Nesterov Momentum

Nesterov Accelerated Gradient (NAG)

Adaptive Gradient Descent

Root Mean Square Propagation (RMSProp)

Adaptive Moment Estimation (Adam)

Quiz Yourself on Non-Convex Optimization

Challenge: Non-Convex Optimization

Solution: Non-Convex Optimization

Use Particle Swarm Optimizer to Optimize a Non-convex Function

Project

Premium

Constrained Optimization

10 Lessons

Explore constrained optimization using linear programming, simplex method, and Lagrange duality.

Linear Programming (LP): Formulation

The Simplex Tableau Method

Quadratic Programming (QP)

Lagrange Duality

The Dual of Linear Programming (LP)

The Dual of Quadratic Programing (QP)

Integer Linear Programming (ILP)

Quiz Yourself on Constrained Optimization

Challenge: Constrained Optimization

Solution: Constrained Optimization

Miscellaneous Methods

9 Lessons

Tackle diverse optimization techniques, including Newton's, Quasi-Newton, barrier methods, and SciPy utilities.

Newton’s Method

Quasi-Newton Methods

Conjugate Gradient Descent

The Log Barrier Method

SciPy Utilities

Interpolation and Curve Fitting

Quiz Yourself on Miscellaneous Methods

Challenge: The Log Barrier Method

Solution: The Log Barrier Method

Course Conclusion

1 Lessons

Build on essential optimization techniques for machine learning, enhancing models with advanced algorithms.

Conclusion

Test Your Concepts of Optimization

Assessment

Training Support Vector Machines (SVMs)

Project

Certificate of Completion

Showcase your accomplishment by sharing your certificate of completion.

Course Author:

Puneet Mangla

Developed by MAANG Engineers

Every Educative lesson is designed by a team of ex-MAANG software engineers and PhD computer science educators, and developed in consultation with developers and data scientists working at Meta, Google, and more. Our mission is to get you hands-on with the necessary skills to stay ahead in a constantly changing industry. No video, no fluff. Just interactive, project-based learning with personalized feedback that adapts to your goals and experience.

Trusted by 2.7 million developers working at companies

"These are high-quality courses. Trust me. I own around 10 and the price is worth it for the content quality. EducativeInc came at the right time in my career. I'm understanding topics better than with any book or online video tutorial I've done. Truly made for developers. Thanks"

Anthony Walker

@_webarchitect_

"Just finished my first full #ML course: Machine learning for Software Engineers from Educative, Inc. ... Highly recommend!"

Evan Dunbar

ML Engineer

"You guys are the gold standard of crash-courses... Narrow enough that it doesn't need years of study or a full blown book to get the gist, but broad enough that an afternoon of Googling doesn't cut it."

Software Developer

Carlos Matias La Borde

"I spend my days and nights on Educative. It is indispensable. It is such a unique and reader-friendly site"

Souvik Kundu

Front-end Developer

"Your courses are simply awesome, the depth they go into and the breadth of coverage is so good that I don't have to refer to 10 different websites looking for interview topics and content."

Vinay Krishnaiah

Software Developer

Hands-on Learning Powered by AI

See how Educative uses AI to make your learning more immersive than ever before.

Personalized Interview Prep

Skip the LeetCode grind with a custom roadmap that adapts to your goals. Hands-on practice for Coding Interviews, System Design, and more.

Mock Interviews

Test your skills in a simulated interview setting. Receive personalized feedback based on your performance. Available for Coding Interviews, System Design, and more.

AI Prompt

Build prompt engineering skills. Practice implementing AI-informed solutions.

Code Feedback

Evaluate and debug your code with the click of a button. Get real-time feedback on test cases, including time and space complexity of your solutions.

Explain with AI

Select any text within any Educative course, and get an instant explanation — without ever leaving your browser.

AI Code Mentor

AI Code Mentor helps you quickly identify errors in your code, learn from your mistakes, and nudge you in the right direction — just like a 1:1 tutor!

Free Resources

FOR TEAMS

Interested in this course for your business or team?

Unlock this course (and 1,000+ more) for your entire org with DevPath