Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Warc Web Archiving Open Source Projects

Open source projects categorized as Warc Web Archiving
Categories > Warc
Categories > Content Management > Web Archiving
Archivebox ⭐ 19,721
🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
dependent packages 1total releases 26most recent commit 4 days agopypi archivebox} Downloads
Awesome Web Archiving ⭐ 1,669
An Awesome List for getting started with web archiving
most recent commit 3 months ago
Conifer ⭐ 1,434
Collect and revisit web pages.
most recent commit 5 months ago
Ipwb ⭐ 577
InterPlanetary Wayback: A distributed and persistent archive replay system using IPFS
total releases 241most recent commit 3 months ago
Replayweb.page ⭐ 574
Serverless replay of web archives directly in the browser
dependent packages 2total releases 90most recent commit 3 months agonpm replaywebpage} Downloads
Browsertrix Crawler ⭐ 470
Run a high-fidelity browser-based crawler in a single Docker container
most recent commit 3 months ago
Webrecorder Player ⭐ 420
Webrecorder Player for Desktop (OSX/Windows/Linux). (Built with Electron + Webrecorder)
most recent commit 4 years ago
Warcdb ⭐ 380
WarcDB: Web crawl data as SQLite databases.
total releases 4most recent commit 4 months ago
Wail ⭐ 330
🐋 Web Archiving Integration Layer: One-Click User Instigated Preservation
most recent commit 5 months ago
Warcreate ⭐ 187
Chrome extension to "Create WARC files from any webpage"
most recent commit 5 months ago
Warcio ⭐ 173
Streaming WARC/ARC library for fast web archive IO
dependent packages 34total releases 22most recent commit 3 years agopypi warcio} Downloads
Cdx_toolkit ⭐ 121
A toolkit for CDX indices such as Common Crawl and the Internet Archive's Wayback Machine
total releases 29most recent commit a year ago
Archivespark ⭐ 118
An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.
total releases 7most recent commit 3 years ago
Wail ⭐ 116
🐋 One-Click User Instigated Preservation
most recent commit 5 years ago
Browsertrix Cloud ⭐ 113
Browsertrix is the hosted, high-fidelity, browser-based crawling service from Webrecorder designed to make web archiving easier and more accessible for all!
most recent commit 23 days ago
Warc Parquet ⭐ 96
🗄️ A simple CLI for converting WARC to Parquet.
total releases 10most recent commit 3 months ago
Wget Lua ⭐ 72
Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.
most recent commit 4 months ago
Node Warc ⭐ 62
Parse And Create Web ARChive (WARC) files with node.js
dependent packages 3total releases 11most recent commit 3 years agonpm node-warc} Downloads
Munin Indexer ⭐ 21
A social media open post web archiving tool
most recent commit a year ago
Metawarc ⭐ 21
metawarc: a command-line tool for metadata extraction from files from WARC (Web ARChive)
most recent commit 5 months ago
Httrack2warc ⭐ 20
Converts HTTrack crawls to WARC files
most recent commit 2 years ago
Cdxj Indexer ⭐ 17
CDXJ Indexing of WARC/ARCs
dependent packages 2total releases 12most recent commit 2 years agopypi cdxj-indexer} Downloads
Ukwa Manage ⭐ 10
Shepherding our web archives from crawl to access.
most recent commit 6 months ago
The 5 Most Depended On Warc Web Archiving Open Source Projects
Warcio ⭐ 173
Streaming WARC/ARC library for fast web archive IO
dependent packages 34total releases 22latest release August 11, 2020most recent commit 3 years agopypi warcio} Downloads
Node Warc ⭐ 62
Parse And Create Web ARChive (WARC) files with node.js
dependent packages 3total releases 11latest release February 24, 2019most recent commit 3 years agonpm node-warc} Downloads
Replayweb.page ⭐ 574
Serverless replay of web archives directly in the browser
dependent packages 2total releases 90latest release October 07, 2023most recent commit 3 months agonpm replaywebpage} Downloads
Cdxj Indexer ⭐ 17
CDXJ Indexing of WARC/ARCs
dependent packages 2total releases 12latest release June 25, 2022most recent commit 2 years agopypi cdxj-indexer} Downloads
The 10 Latest Releases In Warc Web Archiving Open Source Projects
Warcdb ⭐ 380
WarcDB: Web crawl data as SQLite databases.
total releases 4latest release October 22, 2023most recent commit 4 months ago
Replayweb.page ⭐ 574
Serverless replay of web archives directly in the browser
dependent packages 2total releases 90latest release October 07, 2023most recent commit 3 months agonpm replaywebpage} Downloads
Warc Parquet ⭐ 96
🗄️ A simple CLI for converting WARC to Parquet.
total releases 10latest release September 13, 2023most recent commit 3 months ago
Ipwb ⭐ 577
InterPlanetary Wayback: A distributed and persistent archive replay system using IPFS
total releases 241latest release August 16, 2023most recent commit 3 months ago
Cdxj Indexer ⭐ 17
CDXJ Indexing of WARC/ARCs
dependent packages 2total releases 12latest release June 25, 2022most recent commit 2 years agopypi cdxj-indexer} Downloads
Cdx_toolkit ⭐ 121
A toolkit for CDX indices such as Common Crawl and the Internet Archive's Wayback Machine
total releases 29latest release March 02, 2022most recent commit a year ago
Warcio ⭐ 173
Streaming WARC/ARC library for fast web archive IO
dependent packages 34total releases 22latest release August 11, 2020most recent commit 3 years agopypi warcio} Downloads
Node Warc ⭐ 62
Parse And Create Web ARChive (WARC) files with node.js
dependent packages 3total releases 11latest release February 24, 2019most recent commit 3 years agonpm node-warc} Downloads
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.