Kotlin/Java library and cli tool for scraping posts and media from various sources with neither authorization nor full page rendering (Facebook, Instagram, Twitter, Youtube, Tiktok, Telegram, Twitch, Reddit, 9GAG, Pinterest, Flickr, Tumblr, Coub, Vimeo, IFunny, VK, Odnoklassniki, Pikabu)
Alternatives To Skraper
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
2 days ago13August 21, 20234apache-2.0Kotlin
Kotlin/Java library and cli tool for scraping posts and media from various sources with neither authorization nor full page rendering (Facebook, Instagram, Twitter, Youtube, Tiktok, Telegram, Twitch, Reddit, 9GAG, Pinterest, Flickr, Tumblr, Coub, Vimeo, IFunny, VK, Odnoklassniki, Pikabu)
7 years ago7otherPHP
jQuery/PHP - Collection of Social Media APIs that display number of your social media fans. Facebook Likes, Twitter Followers, Instagram Followers, YouTube Subscribers, etc..
Network Avatar Picker87122 years ago25January 04, 20221apache-2.0JavaScript
A npm module that returns user's social network avatar. Supported providers: facebook, instagram, twitter, tumblr, vimeo, github, youtube and gmail
Social Circles66
2 years ago1mitCSS
Well designed social media buttons.
Omniauth Rails App32
4 years ago1Ruby
Example Rails 3.2 + OmniAuth application, connected to Facebook, Twitter, Tumblr, YouTube, Vimeo and more
5 months ago1agpl-3.0JavaScript
DONLOD is a social and media platform downloader that doesn't piss you off.
7 years ago
7 years agomitClojure
Alternatives To Skraper
Select To Compare

Alternative Project Comparisons


Here should be some fancy logo

Awesome Kotlin Badge Apache License 2


Kotlin/Java library and cli tool which allows scraping and downloading posts, attachments, other meta from more than 10 sources without any authorization or full page rendering. Based on jsoup, jackson and kotlin-coroutines.

Repository contains:

Current list of implemented sources:


Unfortunately, each web-site is subject to change without any notice, so the tool may work incorrectly because of that. If that happens, please let me know via an issue.

Cli tool

Cli tool allows to:

  • download media with flag --media-only from almost all presented sources.
  • scrape posts meta information


  • Java: 1.8 +
  • Maven (optional)

Build tool

./mvnw clean package -DskipTests=true 


./skraper --help
usage: [-h] PROVIDER PATH [-n LIMIT] [-t TYPE] [-o OUTPUT] [-m]
       [--parallel-downloads PARALLEL_DOWNLOADS]

optional arguments:
  -h, --help                                show this help message and exit

  -n LIMIT, --limit LIMIT                   posts limit (50 by default)

  -t TYPE, --type TYPE                      output type, options: [log, csv, json, xml, yaml]

  -o OUTPUT, --output OUTPUT                output path

  -m, --media-only                          scrape media only

  --parallel-downloads PARALLEL_DOWNLOADS   amount of parallel downloads for media items if
                                            enabled flag --media-only (4 by default)

positional arguments:
  PROVIDER                                  skraper provider, options: facebook, instagram,
                                            twitter, youtube, tiktok, telegram, twitch, reddit,
                                            9gag, pinterest, flickr, tumblr, ifunny, vk, pikabu,
                                            vimeo, odnoklassniki, coub

  PATH                                      path to user/community/channel/topic/trend


./skraper 9gag /hot 
./skraper reddit /r/memes -n 5 -t csv -o ./reddit/posts
./skraper instagram /explore/tags/memes -t json
./skraper flickr /photos/harrythehawk -t yaml
./skraper pinterest /levato/meme -t xml
./skraper youtube /user/JetBrainsTV/videos --media-only -n 2

Kotlin Library




Gradle kotlin dsl:



Instantiate specific scraper

As mentioned before, the provider implementation list is:

After that usage as simple as is:

val skraper = InstagramSkraper(client = OkHttpSkraperClient())

Important moment: it is highly recommended to not use DefaultBlockingSkraperClient . There are some more efficient, non-blocking and resource-friendly implementations for SkraperClient. To use them you just have to put required dependencies in the classpath.

Current http-client implementation list:

Available methods

Each scraper is a class which implements Skraper interface:

interface Skraper {
    val client: SkraperClient
    fun getPosts(path: String): Flow<Post>
    suspend fun getPageInfo(path: String): PageInfo?
    fun supports(media: Media): Boolean
    suspend fun resolve(media: Media): Media

Also, there are some provider-specific kotlin extensions for implementations. You can find them out at the provider implementation package.

Usage from plain Java

Kotlin coroutines is a CPS implementation (aka callbacks). Here is a quite good java side example of how to call kotlin suspend functions from plain Java.

Scrape user/community/channel/topic/trend posts

To scrape the latest posts for specific user, channel or trend use skraper like that:

suspend fun main() {
    val skraper = FacebookSkraper()
    val posts = skraper.getUserPosts(username = "memes").take(2).toList() // extension for getPosts()
    // or 
    val postsDetected = Skrapers.getPosts(url = "") // aggregating singleton

Received data structure is similar to each other provider's. Output data example:

    "id": "5029851093699104",
    "text": "gotta love em!",
    "publishedAt": 1580744400000,
    "statistics": {
      "likes": 79,
      "comments": 3
    "media": [
        "url": "",
        "aspectRatio": 0.864,
        "duration": 10860.000000000
    "id": "4990218157662398",
    "text": "Interesting",
    "publishedAt": 1580742000000,
    "statistics": {
      "likes": 3092,
      "comments": 514
    "media": [
        "url": "",
        "aspectRatio": 0.8960573476702509

You can see the full model structure for posts and others here

Scrape user/community/channel/topic/trend info

It is possible to scrape user/channel/trend info for some purposes:

suspend fun main() {
    val skraper = TwitterSkraper()
    val pageInfo = skraper.getUserInfo(username = "memes") // extension for `getPageInfo()`
    // or 
    val pageInfoDetected = Skrapers.getPageInfo(url = "") // aggregating singleton


  "nick": "memes",
  "name": "",
  "description": " is your number one website for the funniest content on the web. You will find funny pictures, funny memes and much more.",
  "statistics": {
    "posts": 10848,
    "followers": 154718
  "avatar": {
    "url": ""
  "cover": {
    "url": ""

Resolve provider relative url

Sometimes you need to know direct media link:

suspend fun main() {
    val skraper = InstagramSkraper()
    val info = skraper.resolve(Video(url = ""))
    val serializer = JsonMapper().writerWithDefaultPrettyPrinter()


  "url": "",
  "aspectRatio": 0.8010012515644556,
  "thumbnail": {
    "url": "",
    "aspectRatio": 0.8010012515644556

Download media

There is "static" method which allows to download any media from all known implemented sources:

suspend fun main() {
    val tmpDir = Files.createTempDirectory("skraper").toFile()

    val testVideo =
        media = Video(""),
        destDir = tmpDir,
        filename = "Gandalf"

    val testImage =
        media = Image(""),
        destDir = tmpDir,
        filename = "Do_no_harm"




Telegram bot

To use the bot follow the link.

Popular Vimeo Projects
Popular Tumblr Projects
Popular Companies Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.