Alluxio, data orchestration for analytics and machine learning in the cloud
Alternatives To Alluxio
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Best Of Ml Python13,688
6 days ago19cc-by-sa-4.0
🏆 A ranked list of awesome machine learning Python libraries. Updated weekly.
Ai Learn6,327
a year ago19
人工智能学习路线图,整理近200个实战案例与项目,免费提供配套教材,零基础入门,就业实战!包括:Python,数学,机器学习,数据分析,深度学习,计算机视觉,自然语言处理,PyTorch tensorflow machine-learning,deep-learning data-analysis data-mining mathematics data-science artificial-intelligence python tensorflow tensorflow2 caffe keras pytorch algorithm numpy pandas matplotlib seaborn nlp cv等热门领域
Alluxio6,2533145a day ago54August 05, 2022860apache-2.0Java
Alluxio, data orchestration for analytics and machine learning in the cloud
Danfojs4,199316 days ago36April 20, 202268mitTypeScript
Danfo.js is an open source, JavaScript library providing high performance, intuitive, and easy to use data structures for manipulating and processing structured data.
22 days agomitJupyter Notebook
A comprehensive machine learning repository containing 30+ notebooks on different concepts, algorithms and techniques.
Ml Workspace2,816
6 months ago5apache-2.0Jupyter Notebook
🛠 All-in-one web-based IDE specialized for machine learning and data science.
The Elements Of Statistical Learning Python Notebooks441
2 years ago3Jupyter Notebook
A series of Python Jupyter notebooks that help you better understand "The Elements of Statistical Learning" book
Deeplearning Notes194
10 months agomit
Notes for Deep Learning Specialization Courses led by Andrew Ng.
8 months ago45apache-2.0JavaScript
3 years ago1mitJupyter Notebook
Workshop on basic machine learning, computational modeling, psychophysics, basic data analysis and experiment design
Alternatives To Alluxio
Select To Compare

Alternative Project Comparisons


Slack Release Docker Pulls Documentation OpenSSF Scorecard Twitter Follow License

What is Alluxio

Alluxio (formerly known as Tachyon) is a virtual distributed storage system. It bridges the gap between computation frameworks and storage systems, enabling computation applications to connect to numerous storage systems through a common interface. Read more about Alluxio Overview.

The Alluxio project originated from a research project called Tachyon at AMPLab, UC Berkeley, which was the data layer of the Berkeley Data Analytics Stack (BDAS). For more details, please refer to Haoyuan Li's PhD dissertation Alluxio: A Virtual Distributed File System.

Who Uses Alluxio

Alluxio is used in production to manage Petabytes of data in many leading companies, with the largest deployment exceeding 3,000 nodes. You can find more use cases at Powered by Alluxio or visit our first community conference (Data Orchestration Summit) to learn from other community members!

Who Owns and Manages Alluxio Project

Alluxio Open Source Foundation is the owner of Alluxio project. Project operation is done by Alluxio Project Management Committee (PMC). You can checkout more details in its structure and how to join Alluxio PMC here.

Community and Events

Please use the following to reach members of the community:

Download Alluxio

Binary download

Prebuilt binaries are available to download at .


Download and start an Alluxio master and a worker. More details can be found in documentation.

# Create a network for connecting Alluxio containers
$ docker network create alluxio_nw
# Create a volume for storing ufs data
$ docker volume create ufs
# Launch the Alluxio master
$ docker run -d --net=alluxio_nw \
    -p 19999:19999 \
    --name=alluxio-master \
    -v ufs:/opt/alluxio/underFSStorage \
    alluxio/alluxio master
# Launch the Alluxio worker
$ docker run -d --net=alluxio_nw \
    --name=alluxio-worker \
    -v ufs:/opt/alluxio/underFSStorage \
    -e ALLUXIO_JAVA_OPTS="-Dalluxio.worker.ramdisk.size=${ALLUXIO_WORKER_RAMDISK_SIZE} -Dalluxio.master.hostname=alluxio-master" \
    alluxio/alluxio worker

MacOS Homebrew

$ brew install alluxio

Quick Start

Please follow the Guide to Get Started to run a simple example with Alluxio.

Report a Bug

To report bugs, suggest improvements, or create new feature requests, please open a Github Issue. If you are not sure whether you run into bugs or simply have general questions with respect to Alluxio, post your questions on Alluxio Slack channel.

Depend on Alluxio

Alluxio project provides several different client artifacts for external projects to depend on Alluxio client:

  • Artifact alluxio-shaded-client is recommended generally for a project to use Alluxio client. The jar of this artifact is self-contained (including all dependencies in a shaded form to prevent dependency conflicts), and thus larger than the following two artifacts.
  • Artifact alluxio-core-client-fs provides Alluxio Java file system API) to access all Alluxio-specific functionalities. This artifact is included in alluxio-shaded-client.
  • Artifact alluxio-core-client-hdfs provides HDFS-Compatible file system API. This artifact is included in alluxio-shaded-client.

Here are examples to declare the dependecies on alluxio-shaded-client using Maven:



Contributions via GitHub pull requests are gladly accepted from their original author. Along with any pull requests, please state that the contribution is your original work and that you license the work to the project under the project's open source license. Whether or not you state this explicitly, by submitting any copyrighted material via pull request, email, or other means you agree to license the material under the project's open source license and warrant that you have the legal authority to do so. For a more detailed step-by-step guide, please read how to contribute to Alluxio. For new contributor, please take two new contributor tasks.

For advanced feature requests and contributions, Alluxio core team is hosting regular online meetings with community users and developers to iterate the project in two special interest groups:

  • Alluxio and AI workloads: e.g., running Tensorflow, Pytorch on Alluxio through the POSIX API. Checkout the meeting notes
  • Alluxio and Presto workloads: e.g., running Presto on Alluxio. Checkout the meeting notes

Subscribe our public calendar to join us.

Useful Links

Popular Data Analysis Projects
Popular Tensorflow Projects
Popular Data Processing Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Data Analysis