Awesome Open Source

Programming Languages

Search results for database pipeline

38 search results found

Airbyte ⭐ 12,918

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

Roadtools ⭐ 1,540

A collection of Azure AD tools for offensive and defensive security purposes

Datakit ⭐ 1,044

Connect processes into powerful data pipelines with a simple git-like filesystem interface

Neum AI is a best-in-class framework to manage the creation and synchronization of vector embeddings at large scale.

Paddleocr2pytorch ⭐ 553

PaddleOCR inference in PyTorch. Converted from [PaddleOCR](https://github.com/PaddlePaddle/Paddle

Versatile Data Kit ⭐ 389

One framework to develop, deploy and operate data workflows with Python and SQL.

Data Engineering With Python ⭐ 302

Data Engineering with Python, published by Packt

Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!

Logrange ⭐ 192

High performance data aggregating storage

Datajoint Python ⭐ 158

Relational data pipelines for the science lab

VFX & Animation Pipeline Library

Suite2p ⭐ 116

Tools for processing 2P recordings

Superglue ⭐ 102

Superglue is a lineage-tracking tool built to help visualize the propagation of data through complex pipelines composed of tables, jobs and reports.

Mongo Go Pagination ⭐ 102

Golang Mongodb Pagination for official mongodb/mongo-go-driver package which supports both normal queries and Aggregation pipelines with all information like Total records, Page, Per Page, Previous, Next, Total Page and query results.

a Medical Genetics Sequence Analysis Pipeline

Vectordb ⭐ 58

VectorDB is a free analytics DBMS for IoT & Big Data, compatible with ClickHouse.

Kraken2_classification ⭐ 54

Snakemake workflow for metagenomic classification with Kraken2

Airflow Tutorial ⭐ 51

🐍💨 Airflow tutorial for PyCon 2019

A demonstration of a web application with tests

SHallow shOtGUN profiler

Microbiomebestpracticereview ⭐ 45

Current Challenges and Best Practice Protocols for Microbiome Analysis using Amplicon and Metagenomic Sequencing

Scalable creating/updating of metagenome profiling databases

Ley Lab MetaGenome Profiler DataBase generator

Hes_pipeline ⭐ 33

R pipeline to clean and process Hospital Episode Statistics (HES) data

Docker Image with python 3.11 and pyodbc with drivers for MySQL, PostgreSQL and MSSQL

Datayoga ⭐ 27

streaming data pipeline platform

Megasd Db Generator ⭐ 26

MegaSD DB Scripts

Beautiful Algorithms to Normalize Zillions of Astronomical Images

Duplex Seq Pipeline ⭐ 21

A standalone end-to-end data analysis pipeline for Duplex Sequencing

Pipeline V5 ⭐ 19

This repository contains all CWL descriptions of the MGnify pipeline version 5.0.

Azuredevopsextensionsforsqlserver ⭐ 19

Microsoft SQL Server deployment extensions for Azure DevOps Pipelines

Metameta ⭐ 18

A Shotgun Metagenome Annotation Pipeline

Airflowetl ⭐ 16

Blog post on ETL pipelines with Airflow

Annotatevariants ⭐ 14

This is a pipeline for variant annotation in the diagnosis of rare genetic disorders. It relies on open source data and has instructions for software installs.

Nemesyst ⭐ 14

Generalised and highly customisable, hybrid-parallelism, database based, deep learning framework.

Ace Demo Pipeline ⭐ 14

Demo pipeline for App Connect Enterprise

Scrapy Apperyio ⭐ 12

Scrapy pipeline which allows you to store scrapy items in appery.io database.

Personal Identification Pipeline ⭐ 12

Metapathways ⭐ 10

A modular pipeline for constructing Pathway/Genome Databases from environmental sequence information

Uniclust Pipeline ⭐ 10

Snowflake Json Datapipeline ⭐ 9

Building Json data pipeline within Snowflake using Streams and Tasks

Greenplum Database Release ⭐ 9

A repository for code related to creating packages of Greenplum Database

Letterboxd_utility_tools ⭐ 9

Scripts to work with Letterboxd RSS feed and CSV's. From a pipeline to a DB to generating and sorting lists.

Gentb Site ⭐ 8

The genTB project, the Django site, variant calling and prediciton pipeline, and mapping pipeline with hooks to two ravens

Multiphate2 ⭐ 8

multiPhATE with comparative genomics

This is an ETL application on AWS with general open sales and customer data that you can find here: https://github.com/camposvinicius/data/blob/main/A it's a zipped file with some .csvs inside that we will apply transformations.

A Webhook Manager

Datapipelines Python ⭐ 7

Virusscan ⭐ 7

VirusScan Pipeline

Ncbi Eutils ⭐ 7

NCBI E-utilities client for node and the browser

Cincinnati ⭐ 7

DSaPP project with the City of Cincinnati. Building upon the DSSG15 project

Scrapy Pipeline Mongodb ⭐ 7

A pipeline of saving item into MongoDB

Crispr Exposed ⭐ 6

A database of CRISPR information

Asset pipeline like management for database code (triggers, views, functions, etc.)

Dockerized Continuous Delivery Tool

Faas Flow Minio Datastore ⭐ 6

faasflow DataStore implementation that uses either minio DB or amazon s3

Bcdatabaser ⭐ 6

A pipeline to create reference databases for arbitrary markers and taxonomic groups from NCBI data

A blast-based protein homology pipeline

Babykraken ⭐ 5

👶🦑 Very small Kraken2 database for bundling with pipelines

Ensembl Database Loader ⭐ 5

A eHive Pipeline for loading Ensembl and Ensembl Genomes databases to a MySQL compatible server

Eva Pipeline ⭐ 5

Genomic variation pipeline for the European Variation Archive, implemented using Spring Batch

Devopstic ⭐ 5

Devopstic is a service that let you track Devops metrics based on events from your CI/CD pipelines.

Data Factory Copy Blob To Sql ⭐ 5

This sample shows how to copy data from an Azure Blob Storage to an Azure SQL Database

Rase Pipeline ⭐ 5

RASE pipeline for inferring antibiotic resistance and susceptibility using genomic neighbor typing.

Publicly accessible REST API for ReCodEx system

Eagle Api ⭐ 5

EPIC revision 2019

Related Searches

Command Line Database (33,932)

Python Database (10,521)

Javascript Database (9,210)

Database Mysql (6,245)

Php Database (5,990)

Java Database (5,934)

Database Sql (5,557)

Database Postgresql (5,359)

Python Pipeline (4,255)

Docker Database (3,147)

1-38 of 38 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.