Awesome Open Source

Programming Languages

Search results for slurm

397 search results found

Ml Engineering ⭐ 6,720

Machine Learning Engineering Open Book

Hollywood ⭐ 2,523

Nextflow ⭐ 2,447

A DSL for data-driven computational pipelines

Slurm ⭐ 2,179

Slurm: A Highly Scalable Workload Manager

Deepops ⭐ 1,074

Tools for building GPU clusters

Submitit ⭐ 980

Python 3.8+ toolbox for submitting jobs to Slurm

A scalable, efficient, cross-platform (Linux/macOS) and easy-to-use workflow engine in pure Python.

Pytorch Distributed ⭐ 859

A quickstart and benchmark for pytorch distributed training.

OpenHPC Integration, Packaging, and Test Repo

Luaunit ⭐ 536

LuaUnit is a popular unit-testing framework for Lua, with an interface typical of xUnit libraries (Python unittest, Junit, NUnit, ...). It supports several output formats (Text, TAP, JUnit, ...) to be used directly or work with Continuous Integration platforms (Jenkins, Maven, ...).

Pyslurm ⭐ 449

Python Interface to Slurm

BERT for Coreference Resolution

Slurm_tools ⭐ 350

My tools for the Slurm HPC workload manager

yet another network load monitor

Elasticluster ⭐ 334

Create clusters of VMs on the cloud and configure them with Ansible.

A One-Click System for Analyzing Loop-Resolution Hi-C Experiments

Carbontracker ⭐ 305

Track and predict the energy consumption and carbon footprint of training deep learning models.

The RPKI-to-Router server used at Cloudflare

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

Batch Shipyard ⭐ 270

Simplify HPC and Batch workloads on Azure

Slurm Docker Cluster ⭐ 241

A Slurm cluster using docker-compose

Dask Jobqueue ⭐ 229

Deploy Dask on job schedulers like PBS, SLURM, and SGE

Scripts and links to recreate the ELI5 dataset.

Clustermanagers.jl ⭐ 210

Container plugin for Slurm Workload Manager

Task Spooler ⭐ 194

A scheduler for GPU/CPU tasks

Magpie contains a number of scripts for running Big Data software in HPC environments, including Hadoop and Spark. There is support for Lustre, Slurm, Moab, Torque. LSF, Flux, and more.

An open-source toolkit for deploying and managing high performance clusters for HPC, AI, and data analytics workloads.

Prometheus Slurm Exporter ⭐ 172

Prometheus exporter for performance metrics from Slurm.

Slurm Gcp ⭐ 172

Slurm on Google Cloud Platform

Hpc Now ⭐ 168

A Cross-Platform, Multi-Cloud High-Performance Computing Platform

Batchtools ⭐ 158

Tools for computation on batch systems

SEML: Slurm Experiment Management Library

Ubuntu Slurm ⭐ 151

Steps to create a small slurm cluster with GPU enabled nodes

Ipython Cluster Helper ⭐ 145

Tool to easily start up an IPython cluster on different schedulers.

Clustermq ⭐ 142

R package to send function calls as jobs on LSF, SGE, Slurm, PBS/Torque, or each via SSH

Grappa: scaling irregular applications on commodity clusters

Milano is a tool for automating hyper-parameters search for your models on a backend of your choice.

Funnel is a toolkit for distributed task execution via a simple, standard API.

Hypertunity ⭐ 114

A toolset for black-box hyperparameter optimisation.

Ansible Playbooks ⭐ 102

ansible playbooks for GC3

A collection of various resources, examples, and executables for the general NREL HPC user community's benefit. Use the following website for accessing documentation.

Cluster Scripts ⭐ 86

A collection of useful scripts, templates, and examples for clusters using SLURM https://slurm.schedmd.com/

Future.batchtools ⭐ 83

🚀 R package future.batchtools: A Future API for Parallel and Distributed Processing using batchtools

Slurm_gpustat ⭐ 82

A simple command line tool to show GPU usage on a SLURM cluster

Docker Centos7 Slurm ⭐ 81

Slurm Docker Container on CentOS 7

Smk Simple Slurm ⭐ 80

A simple Snakemake profile for Slurm without --cluster-config

Coldfront ⭐ 79

HPC Resource Allocation System

Slurm In Docker ⭐ 78

Slurm in Docker - Exploring Slurm using CentOS 7 based Docker images

Slurm Mail ⭐ 76

Slurm-Mail is a drop in replacement for Slurm's e-mails to give users much more information about their jobs compared to the standard Slurm e-mails.

Start_follow_read ⭐ 76

Wlm Operator ⭐ 75

Singularity implementation of k8s operator for interacting with SLURM.

submit jobs to slurm with quick-and-dirty python

Parallel_programming_course ⭐ 67

Parallel-Programming

Functional highly reproducible bioinformatics pipelines

Slurm For Ml ⭐ 65

A Machine Learning workflow for Slurm.

Efficiently generate and analyse high dimensional data.

SLURM: A Highly Scalable Resource Manager

Quickstart ⭐ 61

https://tacc.ust.hk

Drake Examples ⭐ 58

Example workflows for the drake R package

Ansible Slurm ⭐ 57

Ansible role for installing and managing the Slurm Workload Manager

Kernel Gof ⭐ 56

NeurIPS 2017 best paper. An interpretable linear-time kernel goodness-of-fit test.

Fgci Ansible ⭐ 54

🔬 Collection of the Finnish Grid and Cloud Infrastructure Ansible playbooks

Jupyterlab Slurm ⭐ 54

Slurm Gpu ⭐ 54

Scheduling GPU cluster workloads with Slurm

Slurm Examples ⭐ 53

A nextflow pipeline to perform state-of-the-art genome-wide association studies.

Aws Plugin For Slurm ⭐ 51

A sample integration of AWS services with SLURM

SLURM Tools and UBiLities

Distributed Ba3c ⭐ 51

Ensembl Hive ⭐ 50

EnsEMBL Hive - a system for creating and running pipelines on a distributed compute resource

slurmR: A Lightweight Wrapper for Slurm

Submit sbatch jobs without the need to create a script

Slurm Pipeline ⭐ 47

A Python class for scheduling SLURM jobs

A Simple Way of Creating Job Workflows in Go running in Processes, Containers, Tasks, Pods, or Jobs

Easyquake ⭐ 45

Simplified machine-learning driven earthquake detection, location, and analysis.

Cyclecloud Slurm ⭐ 45

Azure CycleCloud project to enable users to create, configure, and use Slurm HPC clusters.

Automated machine learning as an AI-HPC benchmark

Interpretable Test ⭐ 44

NeurIPS 2016. Linear-time interpretable nonparametric two-sample test.

Slurmmon ⭐ 44

gather and plot data about Slurm scheduling and job statistics

Wandb_on_slurm ⭐ 43

Example of how to use Weights & Biases on Slurm

Soopervisor ⭐ 42

☁️ Export Ploomber pipelines to Kubernetes (Argo), Airflow, AWS Batch, SLURM, and Kubeflow.

Slurm Magic ⭐ 41

IPython magic for SLURM.

Documentation of the Snakemake-Profiles project.

Pipelines Nextflow ⭐ 41

A set of workflows written in Nextflow for Genome Annotation.

Cromwell/WDL wrapper for Python

Submit R code to a Slurm cluster

Slurm Drmaa ⭐ 40

DRMAA for Slurm: Implementation of the DRMAA C bindings for Slurm

Tensorflow_on_slurm ⭐ 40

COMP Superscalar (COMPSs) is a framework which aims to ease the development and execution of applications for distributed infrastructures, such as Clusters, Grids and Clouds.

A Minimalist Guide ⭐ 39

Walkthroughs for DSL, AirSim, the Vector Institute, and more

Ansible Role Openhpc ⭐ 38

Ansible role for OpenHPC

Jobschedulers.jl ⭐ 37

A Julia-based job scheduler and workload manager inspired by Slurm, PBS and Crontab.

Compute cluster (HPC) job submission library for Go (#golang) based on the open DRMAA standard.

A toolkit featured artificial intelligence × ab initio for computational chemistry research.

SLURM Example Scripts

Dpdispatcher ⭐ 36

generate HPC scheduler systems jobs input scripts and submit these scripts to HPC systems and poke until they finish

Slurm_showq ⭐ 36

A modification to the slurm_showq code written by TACC.

Slurm.el ⭐ 35

Emacs extension to interact with the SLURM jobs scheduler

Cluster Workers ⭐ 35

a Python client/master/worker system for distributing jobs in a cluster

1-100 of 397 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.