Awesome Open Source
Awesome Open Source
https://github.com/unifyai/unifyai.github.io/blob/master/img/externally_linked/logo.png?raw=true

**Were on a mission to unify all ML frameworks + automate code conversions . pip install ivy-core , join our growing community , and lets-unify.ai! **

Contents

Overview

Ivy is an ML framework that currently supports JAX, TensorFlow, PyTorch, and Numpy. Were very excited for you to try it out!

Next on our roadmap is to support automatic code conversions between all frameworks , and add instant multi-framework support for all open-source libraries with only a few lines of code changed! Read on to learn more

The docs are split into a number of sub-pages explaining different aspects of why we created Ivy, how to use it, what weve got planned on our roadmap, and how to contribute! Click on the sub-headings below to check out these pages!

We use to indicate that the feature being discussed is in development. We use to indicate that it is already implemented!

Check out the docs for more info, and check out our Google Colabs for some interactive demos!

Ivy is still at a relatively early stage of development.

Expect breaking changes and sharp edges until we release version 1.2.0 in the next few weeks!

If you would like to contribute, please check out our contributor guide, and take a look at the open tasks if you'd like to dive straight in!

Quick Start

Ivy can be installed like so: pip install ivy-core You can immediately use Ivy to train a neural network, using your favorite framework in the background, like so:

import ivy

class MyModel(ivy.Module):
    def __init__(self):
        self.linear0 = ivy.Linear(3, 64)
        self.linear1 = ivy.Linear(64, 1)
        ivy.Module.__init__(self)

    def _forward(self, x):
        x = ivy.relu(self.linear0(x))
        return ivy.sigmoid(self.linear1(x))

ivy.set_backend('torch')  # change to any backend!
model = MyModel()
optimizer = ivy.Adam(1e-4)
x_in = ivy.array([1., 2., 3.])
target = ivy.array([0.])

def loss_fn(v):
    out = model(x_in, v=v)
    return ivy.mean((out - target)**2)

for step in range(100):
    loss, grads = ivy.execute_with_gradients(loss_fn, model.v)
    model.v = optimizer.step(model.v, grads)
    print('step {} loss {}'.format(step, ivy.to_numpy(loss).item()))

print('Finished training!')

This example uses PyTorch as a backend framework, but the backend can easily be changed to your favorite frameworks, such as TensorFlow, or JAX.

Framework Agnostic Functions

In the example below we show how Ivy's concatenation function is compatible with tensors from different frameworks. This is the same for ALL Ivy functions. They can accept tensors from any framework and return the correct result.

import jax.numpy as jnp
import tensorflow as tf
import numpy as np
import torch

import ivy

jax_concatted   = ivy.concat((jnp.ones((1,)), jnp.ones((1,))), -1)
tf_concatted    = ivy.concat((tf.ones((1,)), tf.ones((1,))), -1)
np_concatted    = ivy.concat((np.ones((1,)), np.ones((1,))), -1)
torch_concatted = ivy.concat((torch.ones((1,)), torch.ones((1,))), -1)

To see a list of all Ivy methods, type ivy. into a python command prompt and press tab. You should then see output like the following:

ivy.Container(                         ivy.general                               ivy.reduce_min(
ivy.abs(                               ivy.get_device(                           ivy.reduce_prod(
ivy.acos(                              ivy.get_num_dims(                         ivy.reduce_sum(
ivy.acosh(                             ivy.gradient_descent_update(              ivy.reductions
ivy.activations                        ivy.gradient_image(                       ivy.relu(
ivy.arange(                            ivy.gradients                             ivy.reshape(
ivy.argmax(                            ivy.identity(                             ivy.round(
ivy.argmin(                            ivy.image                                 ivy.scatter_nd(
ivy.array(                             ivy.indices_where(                        ivy.seed(
ivy.asin(                              ivy.inv(                                  ivy.shape(
ivy.asinh(                             ivy.layers                                ivy.shuffle(
ivy.atan(                              ivy.leaky_relu(                           ivy.sigmoid(
ivy.atan2(                             ivy.linalg                                ivy.sin(
ivy.atanh(                             ivy.linear(                               ivy.sinh(
ivy.bilinear_resample(                 ivy.linspace(                             ivy.softmax(
ivy.cast(                              ivy.log(                                  ivy.softplus(
ivy.ceil(                              ivy.logic                                 ivy.split(
ivy.clip(                              ivy.logical_and(                          ivy.squeeze(
ivy.concatenate(                       ivy.logical_not(                          ivy.stack(
ivy.container                          ivy.logical_or(                           ivy.stack_images(
ivy.conv2d(                            ivy.math                                  ivy.stop_gradient(
ivy.core                               ivy.matmul(                               ivy.svd(
ivy.cos(                               ivy.maximum(                              ivy.tan(
ivy.cosh(                              ivy.minimum(                              ivy.tanh(
ivy.cross(                             ivy.neural_net                            ivy.tile(
ivy.cumsum(                            ivy.nn                                    ivy.to_list(
ivy.depthwise_conv2d(                  ivy.norm(                                 ivy.to_numpy(
ivy.dtype(                             ivy.one_hot(                              ivy.transpose(
ivy.execute_with_gradients(            ivy.ones(                                 ivy.unstack(
ivy.exp(                               ivy.ones_like(                            ivy.variable(
ivy.expand_dims(                       ivy.pinv(                                 ivy.vector_to_skew_symmetric_matrix(
ivy.flip(                              ivy.randint(                              ivy.verbosity
ivy.floor(                             ivy.random                                ivy.where(
ivy.floormod(                          ivy.random_uniform(                       ivy.zero_pad(
ivy.backend_handler                    ivy.reduce_max(                           ivy.zeros(
ivy.gather_nd(                         ivy.reduce_mean(                          ivy.zeros_like(

Background

A huge number of ML tools have exploded onto the scene!

Why should we try to unify them?

Were collaborating with The Consortium for Python Data API Standards

Design

Ivy can fulfill two distinct purposes:

1. Serve as a transpiler between frameworks
2. Serve as a new ML framework with multi-framework support

The Ivy codebase can then be split into three categories, and can be further split into 8 distinct submodules, each of which falls into one of these three categories as follows:
https://github.com/unifyai/unifyai.github.io/blob/master/img/externally_linked/design/submodule_dependency_graph.png?raw=true
Back-end functional APIs
Ivy functional API
Framework Handler
Ivy Compiler

Front-end functional APIs

Ivy stateful API
Ivy Container
Ivy Array

Extensions

Ivy libraries in mechanics, vision, robotics, memory, and other areas

(b) Builder [page coming soon!]
ivy.Trainer, ivy.Dataset, ivy.Dataloader and other helpful classes and functions for creating training workflows in only a few lines of code

Roadmap

We strongly welcome and encourage contributions from the community as we take on this important journey towards ML framework unification. These posts will explain exactly how you can get involved

(a) Standardize [page coming soon!]

(b) Front-Ends [page coming soon!]
Create backend-specific front-ends for each supported ML framework

(c) Transpiler [page coming soon!]
Verify code conversions work for each back-end and front-end combo

(d) Ecosystem [page coming soon!]
Add multi-framework support to popular repos with a few lines changed

Contributing

Join our community as a code contributor, and help accelerate our journey to unify all ML frameworks! Find out more in our Contributing guide!

Citation

@article{lenton2021ivy,
  title={Ivy: Templated deep learning for inter-framework portability},
  author={Lenton, Daniel and Pardo, Fabio and Falck, Fabian and James, Stephen and Clark, Ronald},
  journal={arXiv preprint arXiv:2102.02886},
  year={2021}
}
Related Awesome Lists
Top Programming Languages
Top Projects

Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
Python (837,392
Machine Learning (38,590
Deep Learning (37,652
Tensorflow (22,598
Pytorch (21,659
Neural Network (15,784
Gpu (9,428
Numpy (6,042
Mxnet (1,238
Autograd (282
Jax (279
Ivy (89