Megasparkdiff

A Spark-based data comparison tool at scale which facilitates software development engineers to compare a plethora of pair combinations of possible data sources. Multiple execution modes in multiple environments enable the user to generate a diff report as a Java/Scala-friendly DataFrame or as a file for future use. Comes with out of the box SparkFactory and SparkCompare tools.

Categories > Control Flow > Dataframe

Suggest Alternative

Stars

License

apache-2.0

Open Issues

Most Recent Commit

5 months ago

Programming Language

Scala

Total Releases

Latest Release

October 06, 2018

Categories

Programming Languages > Scala

Data Processing > Spark

Text Processing > Diff

Control Flow > Dataframe

Site

Repo

Suggest An Alternative To MegaSparkDiff

Popular Dataframe Projects

Polars ⭐ 24,900

Dataframes powered by a multithreaded, vectorized query engine, written in Rust

dependent packages 346total releases 328latest release December 01, 2023most recent commit 2 months ago

Modin ⭐ 9,275

Modin: Scale your Pandas workflows by changing a single line of code

dependent packages 25total releases 89latest release November 17, 2023most recent commit 4 months ago

Pygwalker ⭐ 8,698

PyGWalker: Turn your pandas dataframe into an interactive UI for visual analysis

dependent packages 3total releases 124latest release December 10, 2023most recent commit 4 months ago

Vaex ⭐ 8,161

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀

dependent packages 29total releases 69latest release July 21, 2023most recent commit 2 months ago

Cudf ⭐ 6,936

cuDF - GPU DataFrame Library

dependent packages 3total releases 31latest release October 12, 2023most recent commit 4 months ago

Popular Diff Projects

Delta ⭐ 19,661

A syntax-highlighting pager for git, diff, and grep output

total releases 46latest release June 03, 2023most recent commit 4 months ago

Diff So Fancy ⭐ 16,861

Good-lookin' diffs. Actually… nah… The best-lookin' diffs. :tada:

dependent packages 6total releases 39latest release July 27, 2021most recent commit 5 months ago

Difftastic ⭐ 16,695

a structural diff that understands syntax 🟥🟩

total releases 63latest release November 26, 2023most recent commit 4 months ago

Iglistkit ⭐ 12,754

A data-driven UICollectionView framework for building fast and flexible lists.

total releases 10latest release December 01, 2023most recent commit 4 months ago

Learnvue ⭐ 12,190

:octocat:Vue.js 源码解析

most recent commit a year ago

Popular Control Flow Categories