Keras/TF implementation of AdamW, SGDW, NadamW, Warm Restarts, and Learning Rate multipliers
-
Updated
Jan 6, 2022 - Python
#
adamw
Here are 16 public repositories matching this topic...
Implements https://arxiv.org/abs/1711.05101 AdamW optimizer, cosine learning rate scheduler and "Cyclical Learning Rates for Training Neural Networks" https://arxiv.org/abs/1506.01186 for PyTorch framework
scheduler clr optimizer pytorch restarts triangular adamw cosine-annealing cyclical-learning-rate adamw-optimizer
-
Updated
Jul 14, 2019 - Python
Quasi Hyperbolic Rectified DEMON Adam/Amsgrad with AdaMod, Gradient Centralization, Lookahead, iterative averaging and decorrelated Weight Decay
optimizer ranger adam demon lookahead amsgrad adamw radam adamod gradient-centralization qhadam decay-momentum iterate-averaging qhranger adaptive-optimizer hypergradient-descent hd-adam hd-sgd nosadam nostalgic-adam
-
Updated
Sep 23, 2020 - Python
Pytorch implementation of lookahead optimizer(https://arxiv.org/pdf/1907.08610.pdf) and RAdam(https://arxiv.org/pdf/1908.03265.pdf)
python deep-learning neural-network optimizer pytorch lookahead adamw radam lookahead-optim lookahead-pytorch
-
Updated
Jun 25, 2022 - Jupyter Notebook
Nadir: Cutting-edge PyTorch optimizers for simplicity & composability! 🔥🚀💻
machine-learning optimization pytorch lion adam-optimizer adamax sgd-optimizer amsgrad adamw radam adamp adabelief
-
Updated
Jun 15, 2024 - Python
Literature survey of convex optimizers and optimisation methods for deep-learning; made especially for optimisation researchers with ❤️
-
Updated
Mar 9, 2023
A simple, from-scratch C++ Feedforward Neural Network with zero dependencies, built to demystify the fundamentals of deep learning.
learning machine-learning deep-learning cplusplus neural-network cpp artificial-intelligence sgd feedforward-neural-network no-dependencies backpropagation from-scratch adam feedforward educational-project adamw
-
Updated
Dec 22, 2025 - C++
Kaggle's plant disease image classification competition. Finetuning pre-trained CNN models, loss functions, and optimizers in order to achieve better results.
machine-learning computer-vision deep-learning scheduler cnn plant-pathology resnext adamw efficientnet asymmetric-loss
-
Updated
Aug 4, 2021 - HTML
reproduce Adam, AdamW, Adafactor optimizors with pytorch, and introduce popular optimizers in the training of the LLMs.
-
Updated
Mar 24, 2025 - Python
Clean-room GPT-2/GPT-3 implementation: tokenizers, architecture blocks, training loop with AdamW + cosine decay, CLI scripts, inference tools, and pytest suite. Covers OpenWebText-10k & WikiText-103 workflows. Designed as an academic reference for understanding and scaling decoder-only transformers
nlp transformers pytorch gpu-acceleration language-model adamw gpt2 gpt3 cosine-decay decoder-only educational-implementation
-
Updated
Dec 7, 2025 - Python
-
Updated
Sep 11, 2018 - Python
A repo that contains source code for my blog "Deep Learning Optimizers: A Comprehensive Guide for Beginners (2024)"
-
Updated
Jul 26, 2024 - Python
Super-Convergence on CIFAR10
sophia cifar10 lion second-order-optimization adamw super-convergence weight-decay sharpness-aware-minimization madgrad large-batch-optimization lion-optimizer
-
Updated
Jun 17, 2024 - Python
Improve this page
Add a description, image, and links to the adamw topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the adamw topic, visit your repo's landing page and select "manage topics."