A collection of various gradient descent algorithms implemented in Python from scratch

[ICML 2021] The official PyTorch Implementations of Positive-Negative Momentum Optimizers.

NAG-GS: Nesterov Accelerated Gradients with Gauss-Siedel splitting

Intelligent Detection for RIS-Assisted MIMO Systems: A First-and-Second Momentum Approach

Using Matrix Factorization/Probabilistic Matrix Factorization to solve Recommendation。矩阵分解进行推荐系统算法。

In this project it is used a Machine Learning model based on a method called Extreme Learning, with the employment of L2-regularization. In particular, a comparison was carried out between: (A1) which is a variant of incremental extreme learning machine that is QRIELM and (A2) which is a standard momentum descent approach, applied to the ELM.

Python code for Gradient Descent, Momentum, and Adam optimization methods. Train neural networks efficiently.

Overshoot: Taking advantage of future gradients in momentum-based stochastic optimization

Lightweight neural network library written in ANSI-C supporting prediction and backpropagation for Convolutional- and Fully Connected neural networks

Simple Document Classification using Multi Class Logistic Regression & SVM Soft Margin from scratch

This is an implementation of different optimization algorithms such as: - Gradient Descent (stochastic - mini-batch - batch) - Momentum - NAG - Adagrad - RMS-prop - BFGS - Adam Also, most of them are implemented in vectorized form for multi-variate problems

Generic L-layer 'straight in Python' fully connected Neural Network implementation using numpy.

Numerical Optimization for Machine Learning & Data Science

EE456 2022 mini project implementation of two-moons problem using multi-layer-perceptron with back-propagation with analyzing performance of initializing methods and momentum rule

Machine Learning, Deep Learning Implementations

This repository contains a python implementation of Feed Forward Neural Network with Backpropagation, along with the example scripts for training the network to classify images from mnist and fashion_mnist datasets from keras.

Comparsion of Machine Learning optimization algorithms with MNIST dataset