|Project Name||Stars||Downloads||Repos Using This||Packages Using This||Most Recent Commit||Total Releases||Latest Release||Open Issues||License||Language|
|Learn2learn||2,283||1||3 months ago||19||February 10, 2022||13||mit||Python|
|A PyTorch Library for Meta-learning Research|
|Metaoptnet||480||9 months ago||11||apache-2.0||Python|
|Meta-Learning with Differentiable Convex Optimization (CVPR 2019 Oral)|
|Torchopt||420||a day ago||10||July 26, 2022||6||apache-2.0||Python|
|TorchOpt is an efficient library for differentiable optimization built upon PyTorch.|
|Automl_survey||173||4 years ago|
|Far Ho||133||4 years ago||2||mit||Jupyter Notebook|
|Gradient based hyperparameter optimization & meta-learning package for TensorFlow|
|Boml||124||2 years ago||8||September 19, 2020||1||mit||Python|
|Bilevel Optimization Library in Python for Multi-Task and Meta Learning|
|Meta Learning Lstm Pytorch||92||5 years ago||2||Python|
|pytorch implementation of Optimization as a Model for Few-shot Learning|
|Optim4rl||17||2 months ago||apache-2.0||Python|
|Optim4RL is a framework of learning to optimize for reinforcement learning.|
|Hyper Representation||16||5 years ago||Python|
|This is the official repo for the experiments in the paper "Bilevel Programming for Hyperparameter Optimization and Meta-Learning"|
|Lois||8||2 years ago||C++|
|[NeurIPS 2019] LOIS: Learning to Optimize In Swarms, guided by posterior estimation|
BOML is a modularized optimization library that unifies several ML algorithms into a common bilevel optimization framework. It provides interfaces to implement popular bilevel optimization algorithms, so that you could quickly build your own meta learning neural network and test its performance.
ReadMe.md contains brief introduction to implement meta-initialization-based and meta-feature-based methods in few-shot classification field. Except for algorithms which have been proposed, various combinations of lower level and upper level strategies are available.
Meta learning works fairly well when facing incoming new tasks by learning an initialization with favorable generalization capability. And it also has good performance even provided with a small amount of training data available, which gives birth to various solutions for different application such as few-shot learning problem.
We present a general bilevel optimization paradigm to unify different types of meta learning approaches, and the mathematical form could be summarized as below:
Here we illustrate the generic optimization process and hierarchically built strategies in the figure, which could be quikcly implemented in the following example.
For more detailed information of basic function and construction process, please refer to our Documentation orProject Page. Scripts in the directory named test_script are useful for constructing general training process.
Here we give recommended settings for specific hyper paremeters to quickly test performance of popular algorithms.
import boml from boml import utils from test_script.script_helper import * dataset = boml.load_data.meta_omniglot( std_num_classes=args.classes, examples_train=args.examples_train, examples_test=args.examples_test, ) # create instance of BOMLExperiment for ong single task ex = boml.BOMLExperiment(dataset)
boml_ho = boml.BOMLOptimizer( method="MetaInit", inner_method="Simple", outer_method="Simple" ) meta_learner = boml_ho.meta_learner(_input=ex.x, dataset=dataset, meta_model="V1") ex.model = boml_ho.base_learner(_input=ex.x, meta_learner=meta_learner)
loss_inner = utils.cross_entropy(pred=ex.model.out, label=ex.y) accuracy = utils.classification_acc(pred=ex.model.out, label=ex.y) inner_grad = boml_ho.ll_problem( inner_objective=loss_inner, learning_rate=args.lr, T=args.T, experiment=ex, var_list=ex.model.var_list, )
loss_outer = utils.cross_entropy(pred=ex.model.re_forward(ex.x_).out, label=ex.y_) # loss function boml_ho.ul_problem( outer_objective=loss_outer, meta_learning_rate=args.meta_lr, inner_grad=inner_grad, meta_param=tf.get_collection(boml.extension.GraphKeys.METAPARAMETERS), )
# Only need to be called once after all the tasks are ready boml_ho.aggregate_all()
with tf.Session() as sess: tf.global_variables_initializer().run(session=sess) for itr in range(args.meta_train_iterations): # Generate the feed_dict for calling run() everytime train_batch = BatchQueueMock( dataset.train, 1, args.meta_batch_size, utils.get_rand_state(1) ) tr_fd, v_fd = utils.feed_dict(train_batch.get_single_batch(), ex) # Meta training step boml_ho.run(tr_fd, v_fd) if itr % 100 == 0: print(sess.run(loss_inner, utils.merge_dicts(tr_fd, v_fd)))
Copyright (c) 2020 Yaohua Liu
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.