Awesome Open Source
Awesome Open Source

Worker

Go Report Status Discord Chat Lab release Apache-2.0 License

Worker is a simple HTTP server that accepts FQL queries, executes them and returns their results. OpenAPI v2 schema can be found here.

Quick start

The Worker is shipped with dedicated Docker image that contains headless Google Chrome, so feel free to run queries using cdp driver:

DockerHub

docker run -d -p 8080:8080 montferret/worker

GitHub

docker run -d -p 8080:8080 ghcr.io/montferret/worker

Alternatively, if you want to use your own version of Chrome, you can run the Worker locally.

By installing the binary:

curl https://raw.githubusercontent.com/MontFerret/worker/master/install.sh | sh
worker

Or by building locally:

make

And then just make a POST request:

worker

System Resource Requirements

  • 2 CPU
  • 2 Gb of RAM

Usage

Endpoints

POST /

Executes a given query. The payload must have the following shape:

Query {
    text: String!
    params: Map<string, any>
}

GET /info

Returns a worker information that contains details about Chrome, Ferret and itself. Has the following shape:

Info {
    ip: String!
    version: Version! {
        worker: String!
        chrome: ChromeVersion! {
            browser: String!
            protocol: String!
            v8: String!
            webkit: String!
        }
        ferret: String!
    }
}

GET /health

Health check endpoint (for Kubernetes, e.g.). Returns empty 200.

Run commands

  -cache-size uint
        amount of cached queries. 0 means no caching (default 100)
  -chrome-ip string
        Google Chrome remote IP address (default "127.0.0.1")
  -chrome-port uint
        Google Chrome remote debugging port (default 9222)
  -log-level string
        log level (default "debug")
  -port uint
        port to listen (default 8080)
  -version
        show version
  -help
        show this list

Related Awesome Lists
Top Programming Languages
Top Projects

Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
Golang (160,486
Docker (97,659
Scraper (11,834
Crawler (10,176
Health (6,919
Dsl (5,471
Hacktoberfest2020 (1,928
Scraping Websites (346
Ferret (8