Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Article Crawler Open Source Projects

Open source projects categorized as Article Crawler
Newspaper ⭐ 13,147
News, full-text, and article metadata extraction in Python 3. Advanced docs:
dependent packages 97total releases 18most recent commit 7 months agopypi newspaper3k} Downloads
News Please ⭐ 1,821
news-please - an integrated web crawler and information extractor for news that just works
dependent packages 4total releases 121most recent commit 4 months agopypi news-please} Downloads
Article Extractor ⭐ 1,297
To extract main article from given URL with Node.js
dependent packages 10total releases 156most recent commit 3 months agonpm article-parser} Downloads
Hacker News Digest ⭐ 620
📰 Let ChatGPT Summarize Hacker News for You
most recent commit 3 months ago
Awesome Scrapy ⭐ 450
A curated list of awesome packages, articles, and other cool resources from the Scrapy community.
most recent commit a year ago
Html2article ⭐ 425
Html网页正文提取
total releases 5most recent commit 7 years ago
Node Readability ⭐ 302
Scrape/Crawl article from any site automatically. Make any web page readable, no matter Chinese or English.
dependent packages 4total releases 67most recent commit 6 years agonpm read-art} Downloads
Koreanewscrawler ⭐ 182
대량의 뉴스 데이터를 수집하기 위해 만들어진 뉴스 크롤러입니다.
total releases 10most recent commit 2 years ago
Selenium Crawler ⭐ 119
Sometimes sites make crawling hard. Selenium-crawler uses selenium automation to fix that.
most recent commit 11 years ago
Strumentalia Seealsology ⭐ 76
see also section scraping on custom levels of depth
most recent commit 5 months ago
Newspaper4k ⭐ 66
📰 Newspaper4k a fork of the beloved Newspaper3k. Extraction of articles, titles, and metadata from news websites.
most recent commit 3 months ago
Newspaperjs ⭐ 63
News extraction and scraping. Article Parsing
dependent packages 2total releases 7most recent commit a year agonpm newspaperjs} Downloads
Risjbot ⭐ 50
A scrapy project to extract the text and metadata of articles from news websites
most recent commit 3 years ago
Wcep Mds Dataset ⭐ 49
most recent commit a year ago
Websecurityarticles ⭐ 45
爬取及整理Freebuf\安全客\先知\知道创宇等站点的”web安全“类优质文章
most recent commit 3 years ago
Camus ⭐ 44
experimental project for crawling articles from a user's twitter feed and re-arranging them in terms of readability attributes
most recent commit 5 years ago
Spider2 ⭐ 42
A 2nd generation spider to crawl any article site, automatic read title and article.
dependent packages 1total releases 6most recent commit 8 years agonpm spider2} Downloads
Wikireverse ⭐ 39
Hadoop jobs for WikiReverse project. Parses Common Crawl data for links to Wikipedia articles.
most recent commit 6 years ago
Ptt Crawler ⭐ 35
crawl ptt articles from its website
most recent commit 8 years ago
Wikiracer ⭐ 32
Finds the shortest path between two Wikipedia articles, using only Wikipedia links.
most recent commit 6 years ago
Wikipedia Crawler ⭐ 25
Extracts plain-text from Wikipedia articles, ideal to perform linguistic analysis
most recent commit 3 years ago
Media Crawler ⭐ 22
Web scraper for generating a graph of media connections via articles, twitter, reddit, and more
most recent commit 7 years ago
Scrapy German News ⭐ 19
Scrapy project with spiders to extract article content from various german news sites
most recent commit 11 years ago
The 10 Most Depended On Article Crawler Open Source Projects
Newspaper ⭐ 13,147
News, full-text, and article metadata extraction in Python 3. Advanced docs:
dependent packages 97total releases 18latest release September 28, 2018most recent commit 7 months agopypi newspaper3k} Downloads
Article Extractor ⭐ 1,297
To extract main article from given URL with Node.js
dependent packages 10total releases 156latest release December 01, 2022most recent commit 3 months agonpm article-parser} Downloads
News Please ⭐ 1,821
news-please - an integrated web crawler and information extractor for news that just works
dependent packages 4total releases 121latest release August 30, 2023most recent commit 4 months agopypi news-please} Downloads
Node Readability ⭐ 302
Scrape/Crawl article from any site automatically. Make any web page readable, no matter Chinese or English.
dependent packages 4total releases 67latest release August 01, 2018most recent commit 6 years agonpm read-art} Downloads
Congregator Sitescraper ⭐ 9
Website crawler
dependent packages 2total releases 2latest release June 28, 2014most recent commit 9 years agonpm congregator-sitescraper} Downloads
Saffron ⭐ 11
A fairly intuitive & powerful framework that enables you to collect & save articles and news from all over the web.
dependent packages 2total releases 102latest release February 07, 2023most recent commit 4 months agonpm @poiw/saffron} Downloads
Newspaperjs ⭐ 63
News extraction and scraping. Article Parsing
dependent packages 2total releases 7latest release May 16, 2020most recent commit a year agonpm newspaperjs} Downloads
Penjabarberita ⭐ 16
Extract the article list from its raw news HTML
dependent packages 2total releases 7latest release July 15, 2022most recent commit a year agonpm @ariya/penjabarberita} Downloads
Webarticlecurator ⭐ 5
Web Article Curator
dependent packages 1total releases 28latest release February 10, 2022most recent commit 2 years agopypi webarticlecurator} Downloads
Spider2 ⭐ 42
A 2nd generation spider to crawl any article site, automatic read title and article.
dependent packages 1total releases 6latest release December 19, 2015most recent commit 8 years agonpm spider2} Downloads
The 10 Latest Releases In Article Crawler Open Source Projects
News Please ⭐ 1,821
news-please - an integrated web crawler and information extractor for news that just works
dependent packages 4total releases 121latest release August 30, 2023most recent commit 4 months agopypi news-please} Downloads
Saffron ⭐ 11
A fairly intuitive & powerful framework that enables you to collect & save articles and news from all over the web.
dependent packages 2total releases 102latest release February 07, 2023most recent commit 4 months agonpm @poiw/saffron} Downloads
Article Extractor ⭐ 1,297
To extract main article from given URL with Node.js
dependent packages 10total releases 156latest release December 01, 2022most recent commit 3 months agonpm article-parser} Downloads
Penjabarberita ⭐ 16
Extract the article list from its raw news HTML
dependent packages 2total releases 7latest release July 15, 2022most recent commit a year agonpm @ariya/penjabarberita} Downloads
Koreanewscrawler ⭐ 182
대량의 뉴스 데이터를 수집하기 위해 만들어진 뉴스 크롤러입니다.
total releases 10latest release March 27, 2022most recent commit 2 years ago
Webarticlecurator ⭐ 5
Web Article Curator
dependent packages 1total releases 28latest release February 10, 2022most recent commit 2 years agopypi webarticlecurator} Downloads
Iloveptt ⭐ 18
我愛批踢踢 A PTT Crawler and Photo downloader which written in Golang
total releases 13latest release February 09, 2021most recent commit 3 years ago
Wikifeedia ⭐ 6
A feed of the daily top articles on Wikipedia in many languages.
total releases 4latest release June 30, 2020most recent commit 3 years ago
Newspaperjs ⭐ 63
News extraction and scraping. Article Parsing
dependent packages 2total releases 7latest release May 16, 2020most recent commit a year agonpm newspaperjs} Downloads
Newspaper ⭐ 13,147
News, full-text, and article metadata extraction in Python 3. Advanced docs:
dependent packages 97total releases 18latest release September 28, 2018most recent commit 7 months agopypi newspaper3k} Downloads
Popular Learning Resources Categories
Tutorials
Paper
Book
Programming
Course
Article
Education
Hack
Cookbook
University
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.