Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

Openwebtext

Open clone of OpenAI's unreleased WebText dataset scraper. This version uses pushshift.io files instead of the API for speed.
Stars
463
License
gpl-3.0
Open Issues
15
Most Recent Commit
a year ago
Programming Language
Python
Categories
Repo
Alternatives To Openwebtext
Select To Compare
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96most recent commit 3 months agopypi Scrapy} Downloads
Huginn ⭐ 41,465
Create agents that monitor and act on your behalf. Your agents are standing by!
dependent packages 52total releases 8most recent commit 11 days agorubygems huginn_agent} Downloads
Devdocs ⭐ 33,315
API Documentation Browser
most recent commit 3 months ago
Cheerio ⭐ 27,702
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
dependent packages 20,519total releases 70most recent commit 11 days agonpm cheerio} Downloads
Lux ⭐ 24,752
👾 Fast and simple video download library and CLI tool written in Go
dependent packages 8total releases 40most recent commit 18 days ago
Colly ⭐ 21,902
Elegant Scraper and Crawler Framework for Golang
dependent packages 328total releases 22most recent commit a month ago
Newspaper ⭐ 13,147
News, full-text, and article metadata extraction in Python 3. Advanced docs:
dependent packages 97total releases 18most recent commit 7 months agopypi newspaper3k} Downloads
Requests Html ⭐ 13,100
Pythonic HTML Parsing for Humans™
total releases 6most recent commit a year ago


Suggest An Alternative To openwebtext
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Related Searches
Python Scraper
Python Scrape
Scraper Scrape
Python Openai

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Python
Scraper
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.