Create full-fledged APIs for slowly moving datasets without writing a single line of code.
Alternatives To Roapi
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Superset52,23626 hours ago3April 29, 20221,343apache-2.0TypeScript
Apache Superset is a Data Visualization and Data Exploration Platform
Datasette7,83235130a day ago120May 02, 2022512apache-2.0Python
An open source multi-tool for exploring and publishing data
Dataset4,5275145111 days ago55December 16, 202134mitPython
Easy-to-use data handling for SQL data stores with support for implicit table creation, bulk loading, and transactions.
3 days ago17March 20, 202235apache-2.0Rust
Create full-fledged APIs for slowly moving datasets without writing a single line of code.
Sql Translator2,821
2 months agomitTypeScript
SQL Translator is a tool for converting natural language queries into SQL code using artificial intelligence. This project is 100% free and open source.
2 days ago4June 07, 202233apache-2.0Java
esProc SPL is a scripting language for data processing, with well-designed rich library functions and powerful syntax, which can be executed in a Java program through JDBC interface and computing independently.
Goqu1,846463 months ago25October 16, 2021104mitGo
SQL builder and query library for golang
10 days ago27bsd-3-clauseHTML
A large annotated semantic parsing corpus for developing natural language interfaces.
Text2sql Data351
7 months ago2otherPython
A collection of datasets that pair questions with SQL queries.
2 months ago5mitJupyter Notebook
🍻 An open-source dataset of breweries, cideries, brewpubs, and bottleshops.
Alternatives To Roapi
Select To Compare

Alternative Project Comparisons


build Documentation

ROAPI automatically spins up read-only APIs for static datasets without requiring you to write a single line of code. It builds on top of Apache Arrow and Datafusion. The core of its design can be boiled down to the following:

  • Query frontends to translate SQL, GraphQL and REST API queries into Datafusion plans.
  • Datafusion for query plan execution.
  • Data layer to load datasets from a variety of sources and formats with automatic schema inference.
  • Response encoding layer to serialize intermediate Arrow record batch into various formats requested by client.

See below for a high level diagram:



Install pre-built binary

# if you are using homebrew
brew install roapi
# or if you prefer pip
pip install roapi

Check out Github release page for pre-built binaries for each platform. Pre-built docker images are also available at

Install from source

cargo install --locked --git --branch main --bins roapi


Quick start

Spin up APIs for test_data/uk_cities_with_headers.csv and test_data/spacex_launches.json:

roapi \
    --table "uk_cities=test_data/uk_cities_with_headers.csv" \
    --table "test_data/spacex_launches.json"

For windows, full scheme(file:// or filesystem://) must filled, and use double quote(") instead of single quote(') to escape windows cmdline limit:

roapi \
    --table "uk_cities=file://d:/path/to/uk_cities_with_headers.csv" \
    --table "file://d:/path/to/test_data/spacex_launches.json"

Or using docker:

docker run -t --rm -p 8080:8080 --addr-http \
    --table "uk_cities=test_data/uk_cities_with_headers.csv" \
    --table "test_data/spacex_launches.json"

For MySQL and SQLite, use parameters like this.

--table "table_name=mysql://username:[email protected]:3306/database"
--table "table_name=sqlite://path/to/database"

Want dynamic register data? Add parameter -d to command. --table parameter cannot be ignored for now.

roapi \
    --table "uk_cities=test_data/uk_cities_with_headers.csv" \

Then post config to /api/table register data.

curl -X POST \
     -H 'Content-Type: application/json' \
     -d '[
         "tableName": "uk_cities2",
         "uri": "./test_data/uk_cities_with_headers.csv"
         "tableName": "table_name",
         "uri": "sqlite://path/to/database"

Query tables using SQL, GraphQL or REST:

curl -X POST -d "SELECT city, lat, lng FROM uk_cities LIMIT 2" localhost:8080/api/sql
curl -X POST -d "query { uk_cities(limit: 2) {city, lat, lng} }" localhost:8080/api/graphql
curl "localhost:8080/api/tables/uk_cities?columns=city,lat,lng&limit=2"

Get inferred schema for all tables:

curl 'localhost:8080/api/schema'

Config file

You can also configure multiple table sources using YAML or Toml config, which supports more advanced format specific table options:

  - name: "blogs"
    uri: "test_data/blogs.parquet"

  - name: "ubuntu_ami"
    uri: "test_data/ubuntu-ami.json"
      format: "json"
      pointer: "/aaData"
      array_encoded: true
        - name: "zone"
          data_type: "Utf8"
        - name: "name"
          data_type: "Utf8"
        - name: "version"
          data_type: "Utf8"
        - name: "arch"
          data_type: "Utf8"
        - name: "instance_type"
          data_type: "Utf8"
        - name: "release"
          data_type: "Utf8"
        - name: "ami_id"
          data_type: "Utf8"
        - name: "aki_id"
          data_type: "Utf8"

  - name: "spacex_launches"
    uri: ""
      format: "json"

  - name: "github_jobs"
    uri: ""

To run serve tables using config file:

roapi -c ./roapi.yml # or .toml

See config documentation for more options including using Google spreadsheet as a table source.

Response serialization

By default, ROAPI encodes responses in JSON format, but you can request different encodings by specifying the ACCEPT header:

curl -X POST \
    -H 'ACCEPT: application/' \
    -d "SELECT launch_library_id FROM spacex_launches WHERE launch_library_id IS NOT NULL" \

REST API query interface

You can query tables through REST API by sending GET requests to /api/tables/{table_name}. Query operators are specified as query params.

REST query frontend currently supports the following query operators:

  • columns
  • sort
  • limit
  • filter

To sort column col1 in ascending order and col2 in descending order, set query param to: sort=col1,-col2.

To find all rows with col1 equal to string 'foo', set query param to: filter[col1]='foo'. You can also do basic comparisons with filters, for example predicate 0 <= col2 < 5 can be expressed as filter[col2]gte=0&filter[col2]lt=5.

GraphQL query interface

To query tables using GraphQL, send the query through POST request to /api/graphql endpoint.

GraphQL query frontend supports the same set of operators supported by REST query frontend. Here how is you can apply various operators in a query:

    filter: { col1: false, col2: { gteq: 4, lt: 1000 } }
    sort: [{ field: "col2", order: "desc" }, { field: "col3" }]
    limit: 100
  ) {

SQL query interface

To query tables using a subset of standard SQL, send the query through POST request to /api/sql endpoint. This is the only query interface that supports table joins.

Key value lookup

You can pick two columns from a table to use a key and value to create a quick keyvalue store API by adding the following lines to the config:

  - name: "launch_name"
    uri: "test_data/spacex_launches.json"
    key: id
    value: name

Key value lookup can be done through simple HTTP GET requests:

curl -v localhost:8080/api/kv/launch_name/600f9a8d8f798e2a4d5f979e
Starlink-21 (v1.0)%

Query through Postgres wire protocol

ROAPI can present itself as a Postgres server so users can use Postgres clients to issue SQL queries.

$ psql -h
psql (12.10 (Ubuntu 12.10-0ubuntu0.20.04.1), server 13)
WARNING: psql major version 12, server major version 13.
         Some psql features might not work.
Type "help" for help.

houqp=> select count(*) from uk_cities;
(1 row)


Query layer:

  • [x] REST API GET
  • [x] GraphQL
  • [x] SQL
  • [x] join between tables
  • [x] access to array elements by index
  • [x] access to nested struct fields by key
  • [ ] column index
  • protocol
    • [ ] gRPC
    • [x] Postgres
  • [x] Key value lookup

Response serialization:

  • [x] JSON application/json
  • [x] Arrow application/
  • [x] Parquet application/vnd.apache.parquet
  • [ ] msgpack

Data layer:

  • [x] filesystem
  • [x] HTTP/HTTPS
  • [x] S3
  • [x] GCS
  • [x] Azure Storage
  • [x] Google spreadsheet
  • [x] MySQL
  • [x] SQLite
  • [ ] Postgres
  • [ ] Airtable
  • Data format


  • [ ] auto gen OpenAPI doc for rest layer
  • [ ] query input type conversion based on table schema
  • [ ] stream arrow encoding response
  • [ ] authentication layer


The core of ROAPI, including query front-ends and data layer, lives in the self-contained columnq crate. It takes queries and outputs Arrow record batches. Data sources will also be loaded and stored in memory as Arrow record batches.

The roapi crate wraps columnq with a multi-protocol query layer. It serializes Arrow record batches produced by columnq into different formats based on client request.

Building ROAPI with simd optimization requires nightly rust toolchain.

Build Docker image

docker build --rm -t .
Popular Dataset Projects
Popular Sql Projects
Popular Data Processing Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Rest Api
Cloud Native