Matlab library for gradient descent algorithms: Version 1.0.1
Alternatives To Gdlibrary
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
4 years ago7mitPython
🎯 A comprehensive gradient-free optimization framework written in Python
2 months ago17February 16, 20223mitPython
A Lightweight Decision Tree Framework supporting regular algorithms: ID3, C4,5, CART, CHAID and Regression Trees; some advanced techniques: Gradient Boosting, Random Forest and Adaboost w/categorical features support for Python
6 years ago1Jupyter Notebook
The materials for the course MTH 594 Advanced data mining: theory and applications (Dmitry Efimov, American University of Sharjah)
Terrain Topology Algorithms298
2 years ago1mitC#
Terrain topology algorithms in Unity
Fmin295366135 years ago2November 25, 20165bsd-3-clauseJavaScript
Unconstrained function minimization in Javascript
Stein Variational Gradient Descent261
4 years ago1mitPython
code for the paper "Stein Variational Gradient Descent (SVGD): A General Purpose Bayesian Inference Algorithm"
Jstarcraft Ai201
2 months ago4apache-2.0Java
目标是提供一个完整的Java机器学习(Machine Learning/ML)框架,作为人工智能在学术界与工业界的桥梁. 让相关领域的研发人员能够在各种软硬件环境/数据结构/算法/模型之间无缝切换. 涵盖了从数据处理到模型的训练与评估各个环节,支持硬件加速和并行计算,是最快最全的Java机器学习库.
Spinning Up Basic164
2 years agomitPython
Basic versions of agents from Spinning Up in Deep RL written in PyTorch
2 years ago8September 19, 20201mitPython
Bilevel Optimization Library in Python for Multi-Task and Meta Learning
Clustering4ever10932 years ago18May 03, 2020apache-2.0Scala
C4E, a JVM friendly library written in Scala for both local and distributed (Spark) Clustering.
Alternatives To Gdlibrary
Select To Compare

Alternative Project Comparisons

GDLibrary : Gradient Descent Library in MATLAB

Authors: Hiroyuki Kasai

Last page update: April 19, 2017

Latest library version: 1.0.1 (see Release notes for more info)


The GDLibrary is a pure-Matlab library of a collection of unconstrained optimization algorithms. This solves an unconstrained minimization problem of the form, min f(x).

Note that the SGDLibrary internally contains this GDLibrary.

List of gradient algorithms available in GDLibrary

  • GD (gradient descent)
    • Standard GD
    • Scaled GD
  • CG (linear conjugate gradient)
    • Standard GD
    • Preconditioned CG
  • NCG (non-linear conjugate gradient)
    • Fletcher-Reeves (FR), Polak-Ribiere (PR)
  • Newton (Netwon's algorithm)
    • Standard Netwon's algorithm
    • Damped Newton's algorithm
    • Cholesky factorizaion based Newton's algorithm
  • BFGS (Broyden-Fletcher-Goldfarb-Shanno algorithm)
    • Standard BFGS
    • Damped BFGS
  • LBFGS (limited-memory BFGS)
    • Standard LBFGS
  • AGD (Accelerated gradient descent, i.e., Nesterov AGD)
    • Standard AGD

List of line-search algorithms available in GDLibrary

Supported problems

Folders and files

./                      - Top directory.
./             - This readme file.
./run_me_first.m        - The scipt that you need to run first.
./demo.m                - Demonstration script to check and understand this package easily. 
|plotter/               - Contains plotting tools to show convergence results and various plots.
|tool/                  - Some auxiliary tools for this project.
|problem/               - Problem definition files to be solved.
|gd_solver/             - Contains various gradient descent optimization algorithms.
|gd_test/               - Some helpful test scripts to use this package.

First to do

Run run_me_first for path configurations.

%% First run the setup script

Usage example 1 (Rosenbrock problem)

Now, just execute demo for demonstration of this package.

%% Execute the demonstration script

The "demo.m" file contains below.

%% define problem definitions
% set number of dimensions
d = 2;    
problem = rosenbrock(d);

%% calculate solution 
w_opt = problem.calc_solution(); 

%% general options for optimization algorithms   
options.w_init = zeros(d,1);
% set verbose mode        
options.verbose = true;  
% set optimal solution    
options.f_opt = problem.cost(w_opt);  
% set store history of solutions
options.store_w = true;

%% perform GD with backtracking line search 
options.step_alg = 'backtracking';
[w_gd, info_list_gd] = gd(problem, options); 

%% perform NCG with backtracking line search 
options.step_alg = 'backtracking';
[w_ncg, info_list_ncd] = ncg(problem, options);     

%% perform L-BFGS with strong wolfe line search
options.step_alg = 'strong_wolfe';                  
[w_lbfgs, info_list_lbfgs] = lbfgs(problem, options);                  

%% plot all
close all;

% display epoch vs cost/gnorm
display_graph('iter','cost', {'GD-BKT', 'NCG-BKT', 'LBFGS-WOLFE'}, {w_gd, w_ncg, w_lbfgs}, {info_list_gd, info_list_ncd, info_list_lbfgs});
% display optimality gap vs grads
display_graph('iter','gnorm', {'GD-BKT', 'NCG-BKT', 'LBFGS-WOLFE'}, {w_gd, w_ncg, w_lbfgs}, {info_list_gd, info_list_ncd, info_list_lbfgs});

% draw convergence sequence
w_history = cell(1);
cost_history = cell(1);
w_history{1} = info_list_gd.w;
w_history{2} = info_list_ncd.w;  
w_history{3} = info_list_lbfgs.w;      
cost_history{1} = info_list_gd.cost;
cost_history{2} = info_list_ncd.cost;  
cost_history{3} = info_list_lbfgs.cost;      
draw_convergence_sequence(problem, w_opt, {'GD-BKT', 'NCG-BKT', 'LBFGS-WOLFE'}, w_history, cost_history);          
  • Output results


The GDLibrary is free and open source for academic/research purposes (non-commercial).

Problems or questions

If you have any problems or questions, please contact the author: Hiroyuki Kasai (email: kasai at is dot uec dot ac dot jp)

Release Notes

  • Version 1.0.1 (Apr. 19, 2017)
    • New solvers and problems are added.
  • Version 1.0.0 (Nov. 04, 2016)
    • Initial version.
Popular Algorithms Projects
Popular Gradient Projects
Popular Computer Science Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Machine Learning
Big Data
Machine Learning Algorithms
Linear Regression
Logistic Regression
Optimization Algorithms
Gradient Descent