Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Warc Open Source Projects

Open source projects categorized as Warc
Categories > Warc
Archivebox ⭐ 19,721
🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
dependent packages 1total releases 26most recent commit 9 days agopypi archivebox} Downloads
Heritrix3 ⭐ 2,579
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.
dependent packages 2total releases 9most recent commit 6 months ago
Awesome Web Archiving ⭐ 1,669
An Awesome List for getting started with web archiving
most recent commit 3 months ago
Conifer ⭐ 1,434
Collect and revisit web pages.
most recent commit 6 months ago
Grab Site ⭐ 1,254
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
most recent commit a month ago
Ipwb ⭐ 577
InterPlanetary Wayback: A distributed and persistent archive replay system using IPFS
total releases 241most recent commit 3 months ago
Replayweb.page ⭐ 574
Serverless replay of web archives directly in the browser
dependent packages 2total releases 90most recent commit 3 months agonpm replaywebpage} Downloads
Browsertrix Crawler ⭐ 470
Run a high-fidelity browser-based crawler in a single Docker container
most recent commit 3 months ago
Webrecorder Player ⭐ 422
Webrecorder Player for Desktop (OSX/Windows/Linux). (Built with Electron + Webrecorder)
most recent commit 4 years ago
Warcdb ⭐ 380
WarcDB: Web crawl data as SQLite databases.
total releases 4most recent commit 5 months ago
Warcprox ⭐ 348
WARC writing MITM HTTP/S proxy
total releases 24most recent commit 6 months ago
Wail ⭐ 330
🐋 Web Archiving Integration Layer: One-Click User Instigated Preservation
most recent commit 5 months ago
Archivebot ⭐ 328
ArchiveBot, an IRC bot for archiving websites
most recent commit 5 months ago
Cc Pyspark ⭐ 280
Process Common Crawl data with Python and Spark
most recent commit a year ago
Bitextor ⭐ 260
Bitextor generates translation memories from multilingual websites
most recent commit 7 months ago
News Crawl ⭐ 229
News crawling with StormCrawler - stores content as WARC
most recent commit 4 months ago
Obelisk ⭐ 214
Go package and CLI tool for saving web page as single HTML file
most recent commit 3 months ago
Zimit ⭐ 209
Make a ZIM file from any Web site and surf offline!
most recent commit 3 months ago
Warc ⭐ 196
Python library for reading and writing warc files
dependent packages 1total releases 3most recent commit 6 years agopypi warc} Downloads
Webrecorder Desktop ⭐ 192
Webrecorder Desktop App!
most recent commit 3 years ago
Warcreate ⭐ 187
Chrome extension to "Create WARC files from any webpage"
most recent commit 5 months ago
Warcio ⭐ 173
Streaming WARC/ARC library for fast web archive IO
dependent packages 34total releases 22most recent commit 3 years agopypi warcio} Downloads
Cocrawler ⭐ 159
CoCrawler is a versatile web crawler built using modern tools and concurrency.
total releases 8most recent commit 2 years ago
The 10 Most Depended On Warc Open Source Projects
Warcio ⭐ 173
Streaming WARC/ARC library for fast web archive IO
dependent packages 34total releases 22latest release August 11, 2020most recent commit 3 years agopypi warcio} Downloads
Warcio.js ⭐ 25
JS Streaming WARC IO optimized for Browser and Node
dependent packages 10total releases 34latest release November 01, 2023most recent commit 6 months agonpm warcio} Downloads
Warc ⭐ 17
Read and write WARC files in Go
dependent packages 5total releases 1latest release August 06, 2015most recent commit 6 years ago
Warc ⭐ 36
:gear: A Rust library for reading and writing WARC files
dependent packages 5total releases 9latest release March 25, 2022most recent commit 4 months agocargo warc} Downloads
Webarchive ⭐ 15
golang readers for ARC and WARC webarchive formats
dependent packages 4total releases 5latest release April 03, 2023most recent commit a year ago
Node Warc ⭐ 62
Parse And Create Web ARChive (WARC) files with node.js
dependent packages 3total releases 11latest release February 24, 2019most recent commit 3 years agonpm node-warc} Downloads
Warc ⭐ 19
Parse WARC (Web Archive Files) as a node.js stream
dependent packages 3total releases 2latest release October 20, 2014most recent commit 10 years agonpm warc} Downloads
Chatnoir Resiliparse ⭐ 33
A robust web archive analytics toolkit
dependent packages 3total releases 71latest release August 11, 2023most recent commit 5 months agopypi FastWARC} Downloads
Jwarc ⭐ 42
Java library for reading and writing WARC files with a typed API
dependent packages 2total releases 51latest release September 28, 2023most recent commit 4 months ago
Heritrix3 ⭐ 2,579
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.
dependent packages 2total releases 9latest release July 27, 2022most recent commit 6 months ago
The 10 Latest Releases In Warc Open Source Projects
Warcio.js ⭐ 25
JS Streaming WARC IO optimized for Browser and Node
dependent packages 10total releases 34latest release November 01, 2023most recent commit 6 months agonpm warcio} Downloads
Warcdb ⭐ 380
WarcDB: Web crawl data as SQLite databases.
total releases 4latest release October 22, 2023most recent commit 5 months ago
Replayweb.page ⭐ 574
Serverless replay of web archives directly in the browser
dependent packages 2total releases 90latest release October 07, 2023most recent commit 3 months agonpm replaywebpage} Downloads
Jwarc ⭐ 42
Java library for reading and writing WARC files with a typed API
dependent packages 2total releases 51latest release September 28, 2023most recent commit 4 months ago
Warc2zim ⭐ 30
Command line tool to convert a file in the WARC format to a file in the ZIM format
total releases 21latest release September 18, 2023most recent commit 3 months ago
Warc ⭐ 7
Read and write WARC files in Go
total releases 37latest release September 14, 2023most recent commit 7 months ago
Warc Parquet ⭐ 96
🗄️ A simple CLI for converting WARC to Parquet.
total releases 10latest release September 13, 2023most recent commit 3 months ago
Gowarc ⭐ 5
dependent packages 2total releases 35latest release August 24, 2023most recent commit 3 months ago
Ipwb ⭐ 577
InterPlanetary Wayback: A distributed and persistent archive replay system using IPFS
total releases 241latest release August 16, 2023most recent commit 3 months ago
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.