Awesome Open Source

Programming Languages

Search results for golang etl

39 search results found

Tidb ⭐ 35,604

TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://tidbcloud.com/free-trial

Benthos ⭐ 7,564

Fancy stream processing made operationally mundane

Steampipe ⭐ 6,061

Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.

Cloudquery ⭐ 5,380

The open source high performance data integration platform built for developers.

Rudder Server ⭐ 3,841

Privacy and Security focused Segment-alternative, in Golang and React

Incubator Devlake ⭐ 2,322

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.

Go Streams ⭐ 1,656

A lightweight stream processing library for Go

💧 Instill VDP (Versatile Data Pipeline) is an open-source tool to seamlessly integrate AI to process unstructured data in the modern data stack

Transporter ⭐ 1,447

Sync data between persistence engines, like ETL only not stodgy

Peerdb ⭐ 1,315

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

Monstache ⭐ 1,208

a go daemon that syncs MongoDB to Elasticsearch in realtime. you know, for search.

Optimus ⭐ 707

Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

Omniparser ⭐ 597

omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.

Bigslice ⭐ 525

A serverless cluster computing system for the Go programming language

Power of appbase.io via CLI, with nifty imports from your favorite data sources

Neosync ⭐ 413

A developer-first way to create high-fidelity synthetic data or anonymize sensitive data and sync it across all environments for testing, fine-tuning or model training.

Conduit ⭐ 321

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

Storagetapper ⭐ 269

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

Substation ⭐ 242

Substation is a cloud-native, event-driven data pipeline toolkit built for security teams.

A fast to develop, fast to run, Go based toolkit for ETL and feature extraction on Hadoop.

Dataplane ⭐ 171

Dataplane is an Airflow inspired unified data platform with additional data mesh and RPA capability to automate, schedule and design data pipelines and workflows. Dataplane is written in Golang with a React front end.

Steampipe Plugin Aws ⭐ 165

Use SQL to instantly query AWS resources across regions and accounts. Open source CLI. No DB required.

A Go native tabular data extraction package. Currently supports .xls, .xlsx, .csv, .tsv formats.

Peaks Consolidation ⭐ 102

The Peaks Consolidation is equipped with state-of-the-art algorithms and data structures that support high-performance databending exercises. It specializes in management accounting and consolidation, with some special topics in machine learning and bioinformatics.

Service for bulk-loading data to databases with automatic schema management (Redshift, Snowflake, BigQuery, ClickHouse, Postgres, MySQL)

Sling Cli ⭐ 84

Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.

go-etl是一个集数据源抽取，转化，加载的工具集，提供强大的离线数据同步能力。

csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.

Beneath is a serverless real-time data platform ⚡️

Zeus + SciFi = the power of the gods, meets the power of science fiction. Designing wisdom into intelligence, through intelligent design.

Steampipe Plugin Github ⭐ 58

Use SQL to instantly query repositories, users, gists and more from GitHub. Open source CLI. No DB required.

SQLpipe makes it easy to move the result of one query from one database to another.

Etl Engine ⭐ 47

etl engine 轻量级跨平台流批一体ETL引擎数据抽取-转换-装载 ETL engine lightweight cross platform batch flow integration ETL engine data extraction transformation loading

Steampipe Plugin Kubernetes ⭐ 41

Use SQL to instantly query Kubernetes API resources. Open source CLI. No DB required.

Steampipe Sqlite ⭐ 39

Steampipe SQLite is a zero-ETL engine for SQLite. Virtual tables translate queries into live API calls for cloud services and APIs. Hundreds of plugins with thousands of documented examples.

Steampipe Plugin Gcp ⭐ 36

Use SQL to instantly query GCP resources across regions, projects and organizations. Open source CLI. No DB required.

CLI and library for diff, patch, and ETL operations on CUE, JSON, and Yaml

Blast is a data orchestration tool that can run SQL and Python against Google BigQuery and Snowflake. It supports templating with Jinja, data quality tests, query validation, environment management and more.

Steampipe Plugin Azure ⭐ 30

Use SQL to instantly query Azure resources across regions and subscriptions. Open source CLI. No DB required.

Cookiecutter R Project ⭐ 28

Basic cookiecutter template for R projects

Stellar Etl ⭐ 27

Stellar ETL will enable real-time analytics on the Stellar network

Steampipe Plugin Terraform ⭐ 27

Use SQL to instantly query resources, data sources and more from Terraform code. Open source CLI. No DB required.

Apache AVRO for go

Steampipe Plugin Sdk ⭐ 25

Steampipe Plugin SDK is a simple abstraction layer to write a Steampipe plugin. Plugins automatically work across all engine types including the Steampipe CLI, Postgres FDW, SQLite extension and the export CLI.

Steampipe Plugin Googlesheets ⭐ 23

Use SQL to instantly query spreadsheets, sheets, and cell data from Google Sheets. Open source CLI. No DB required.

Command line tool to run batch jobs concurrently with ETL framework on AWS or other cloud computing resources

Steampipe Plugin Virustotal ⭐ 22

Use SQL to instantly query file, domain, URL and IP scanning results from VirusTotal.

Go Bqloader ⭐ 21

bqloader is a simple ETL framework to load data from Cloud Storage into BigQuery.

Steampipe Plugin Net ⭐ 20

Use SQL to instantly query DNS records, certificates and other network information. Open source CLI. No DB required.

M-Lab ingestion pipeline

Steampipe Plugin Jira ⭐ 19

Use SQL to instantly query Jira. Open source CLI. No DB required.

⚗️ Instill Model contains components for AI model orchestration

Steampipe Plugin Oci ⭐ 18

Use SQL to instantly query Oracle Cloud resources across regions and accounts. Open source CLI. No DB required.

Steampipe Plugin Whois ⭐ 18

Use SQL to instantly query WHOIS. Open source CLI. No DB required.

ETL/Pipeline library

Steampipe Plugin Csv ⭐ 18

Use SQL to instantly query data from CSV files. Open source CLI. No DB required.

Cq Source Sharepoint ⭐ 18

🔌 CloudQuery SharePoint Source Plugin

📺 Instill AI's official command line tool

Package flow provides support for basic FBP / pipelines

Steampipe Plugin Code ⭐ 14

Use SQL to instantly query secrets and more from source code. Open source CLI. No DB required.

Steampipe Plugin Reddit ⭐ 14

Use SQL to instantly query Reddit posts, comments & more. Open source CLI. No DB required.

Steampipe Plugin Hackernews ⭐ 14

Use SQL to instantly query stories, users and other items from Hacker News. Open source CLI. No DB required.

Steampipe Plugin Twitter ⭐ 13

Use SQL to instantly query tweets, users and followers from Twitter. Open source CLI. No DB required.

uncompress, parse and load tool for csv, xls, xlsx, text

Steampipe Plugin Googleworkspace ⭐ 13

Use SQL to instantly query calendar events, drive files, gmail messages, and more from Google Workspace. Open source CLI. No DB required.

🐟 A streaming ETL for fish

Steampipe Plugin Scaleway ⭐ 12

Use SQL to instantly query instances, networks, databases, and more from Scaleway. Open source CLI. No DB required.

🔮 Instill Core contains components for supporting Instill VDP and Instill Model

Steampipe Plugin Finance ⭐ 11

Use SQL to instantly query financial data including quotes (equities, cryptocurrency, etc) and US public company information. Open source CLI. No DB required.

Etl Gardener ⭐ 11

Gardener provides services for maintaining and reprocessing mlab data.

cross-chain ETL tool for EVM chaindata

Dataframes for Golang

A declarative, SQL-like DSL for data integration tasks.

GoBatch is a batch processing framework in Go like Spring Batch in Java

Steampipe Plugin Trivy ⭐ 8

Use SQL to instantly query advisories, vulnerabilities, packages, findings and more using Trivy. Open source CLI. No DB required.

Steampipe Plugin Bitbucket ⭐ 7

Use SQL to instantly query Bitbucket. Open source CLI. No DB required.

Kassette Server ⭐ 7

Secured pipelines for your reporting and auditing data

Food Plan Organizer ⭐ 7

Stay healthy by checking nutrition facts & manage your favourite paleo diet recipes.

Steampipe Plugin Okta ⭐ 7

Use SQL to instantly query users, groups, applications and more from Okta. Open source CLI. No DB required.

Ratchet Examples ⭐ 7

A golang ETL to collect, filter and output logs/events.

Blockchain sync for the cult of Cardano

Steampipe Plugin Azuread ⭐ 6

Use SQL to instantly query groups, service principals, users and more from Azure Active Directory. Open source CLI. No DB required.

Dsa Workshop ⭐ 6

An example of using microservice architecture powered by Kubernetes and Azure.

Scale Etl ⭐ 6

Partition, Transform, Load, and Search large CSV files.

Complex data processing flows in Go

The import layer will contain strategies for pulling data from existing systems into Coral.

数据同步工具，mysql数据同步到mariadb columnStore，支持全量同步和增量同步，纯go语言实现

ETL for Snowplow and Adjust trackers data with PostgreSQL storage support

Mongo Transporter ⭐ 5

Sync data between two MongoDB deployments

Crunch Starter ⭐ 5

A boilerplate for kickstarting a production Crunch processor

main repo for the churro file/API processor

A workflow based data pipeline framework for golang.

Simple change data capture tool for mysql and kafka. It taps into mysql binlog events, runs the transformation logic on them and produces kafka events

Block Explorer ⭐ 5

Steampipe Plugin Databricks ⭐ 5

Use SQL to instantly query Databricks resources. Open source CLI. No DB required.

Bac Tract ⭐ 5

Extract data from bacpac files

Steampipe Plugin Jenkins ⭐ 5

Use SQL to instantly query Jenkins resources. Open source CLI. No DB required.

Related Searches

Golang Command Line (8,308)

Golang Docker (7,769)

Golang Http (4,290)

Golang Server (4,285)

Javascript Golang (3,372)

Golang Amazon Web Services (3,137)

Golang Database (2,927)

Golang Testing (2,769)

Golang Json (2,652)

Golang Proxy (2,577)

1-39 of 39 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.