Csvs To Sqlite

Convert CSV files into a SQLite database
Alternatives To Csvs To Sqlite
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
a day ago1,910apache-2.0Java
Free universal database tool and SQL client
Prisma35,08744216 hours ago4,993September 24, 20223,033apache-2.0TypeScript
Next-generation ORM for Node.js & TypeScript | PostgreSQL, MySQL, MariaDB, SQL Server, SQLite, MongoDB and CockroachDB
Typeorm32,5621,9943,6832 days ago784October 05, 20232,296mitTypeScript
ORM for TypeScript and JavaScript. Supports MySQL, PostgreSQL, MariaDB, SQLite, MS SQL Server, Oracle, SAP Hana, WebSQL databases. Works in NodeJS, Browser, Ionic, Cordova and Electron platforms.
Directus23,93723917 hours ago95November 16, 2023391otherTypeScript
The Modern Data Stack 🐰 — Directus is an instant REST+GraphQL API and intuitive no-code data collaboration app for any SQL database.
3 days agoMay 18, 2021640otherC++
Official home of the DB Browser for SQLite (DB4S) project. Previously known as "SQLite Database Browser" and "Database Browser for SQLite". Website at:
Rqlite14,2673317 hours ago42April 14, 202155mitGo
The lightweight, distributed relational database built on SQLite
Beekeeper Studio14,099
17 hours ago517gpl-3.0Vue
Modern and easy to use SQL client for MySQL, Postgres, SQLite, SQL Server, and more. Linux, MacOS, and Windows.
Migrate12,9301,0272 days ago132June 11, 2023315otherGo
Database migrations. CLI and Golang library.
Greendao12,595781618 days ago11November 12, 2015235Java
greenDAO is a light & fast ORM solution for Android that maps objects to SQLite databases.
Sql.js11,83439945611 days ago36November 28, 2023125otherJavaScript
A javascript library to run SQLite on the web.
Alternatives To Csvs To Sqlite
Select To Compare

Alternative Project Comparisons


PyPI Changelog Travis CI License

Convert CSV files into a SQLite database. Browse and publish that SQLite database with Datasette.

Basic usage:

csvs-to-sqlite myfile.csv mydatabase.db

This will create a new SQLite database called mydatabase.db containing a single table, myfile, containing the CSV content.

You can provide multiple CSV files:

csvs-to-sqlite one.csv two.csv bundle.db

The bundle.db database will contain two tables, one and two.

This means you can use wildcards:

csvs-to-sqlite ~/Downloads/*.csv my-downloads.db

If you pass a path to one or more directories, the script will recursively search those directories for CSV files and create tables for each one.

csvs-to-sqlite ~/path/to/directory all-my-csvs.db

Handling TSV (tab-separated values)

You can use the -s option to specify a different delimiter. If you want to use a tab character you'll need to apply shell escaping like so:

csvs-to-sqlite my-file.tsv my-file.db -s $'\t'

Refactoring columns into separate lookup tables

Let's say you have a CSV file that looks like this:

Clark,1,President,,REP,John R. Kasich,5
Clark,2,President,,REP,John R. Kasich,0
Clark,3,President,,REP,John R. Kasich,7

(Real example taken from the Open Elections project)

You can now convert selected columns into separate lookup tables using the new --extract-column option (shortname: -c) - for example:

csvs-to-sqlite openelections-data-*/*.csv \
    -c county:County:name \
    -c precinct:Precinct:name \
    -c office -c district -c party -c candidate \

The format is as follows:


If you just specify the column name e.g. -c office, the following table will be created:

CREATE TABLE "office" (
    "value" TEXT

If you specify all three options, e.g. -c precinct:Precinct:name the table will look like this:

CREATE TABLE "Precinct" (
    "name" TEXT

The original tables will be created like this:

CREATE TABLE "ca__primary__san_francisco__precinct" (
    "county" INTEGER,
    "precinct" INTEGER,
    "office" INTEGER,
    "district" INTEGER,
    "party" INTEGER,
    "candidate" INTEGER,
    "votes" INTEGER,
    FOREIGN KEY (county) REFERENCES County(id),
    FOREIGN KEY (party) REFERENCES party(id),
    FOREIGN KEY (precinct) REFERENCES Precinct(id),
    FOREIGN KEY (office) REFERENCES office(id),
    FOREIGN KEY (candidate) REFERENCES candidate(id)

They will be populated with IDs that reference the new derived tables.


$ pip install csvs-to-sqlite

csvs-to-sqlite now requires Python 3. If you are running Python 2 you can install the last version to support Python 2:

$ pip install csvs-to-sqlite==0.9.2

csvs-to-sqlite --help

Usage: csvs-to-sqlite [OPTIONS] PATHS... DBNAME

  PATHS: paths to individual .csv files or to directories containing .csvs

  DBNAME: name of the SQLite database file to create

  -s, --separator TEXT         Field separator in input .csv
  -q, --quoting INTEGER        Control field quoting behavior per csv.QUOTE_*
                               constants. Use one of QUOTE_MINIMAL (0),
                               QUOTE_ALL (1), QUOTE_NONNUMERIC (2) or
                               QUOTE_NONE (3).
  --skip-errors                Skip lines with too many fields instead of
                               stopping the import
  --replace-tables             Replace tables if they already exist
  -t, --table TEXT             Table to use (instead of using CSV filename)
  -c, --extract-column TEXT    One or more columns to 'extract' into a
                               separate lookup table. If you pass a simple
                               column name that column will be replaced with
                               integer foreign key references to a new table
                               of that name. You can customize the name of the
                               table like so:
                               This will pull unique values from the 'state'
                               column and use them to populate a new 'States'
                               table, with an id column primary key and a
                               state_name column containing the strings from
                               the original column.
  -d, --date TEXT              One or more columns to parse into ISO formatted
  -dt, --datetime TEXT         One or more columns to parse into ISO formatted
  -df, --datetime-format TEXT  One or more custom date format strings to try
                               when parsing dates/datetimes
  -pk, --primary-key TEXT      One or more columns to use as the primary key
  -f, --fts TEXT               One or more columns to use to populate a full-
                               text index
  -i, --index TEXT             Add index on this column (or a compound index
                               with -i col1,col2)
  --shape TEXT                 Custom shape for the DB table - format is
  --filename-column TEXT       Add a column with this name and populate with
                               CSV file name
  --no-index-fks               Skip adding index to foreign key columns
                               created using --extract-column (default is to
                               add them)
  --no-fulltext-fks            Skip adding full-text index on values extracted
                               using --extract-column (default is to add them)
  --just-strings               Import all columns as text strings by default
                               (and, if specified, still obey --shape,
                               --date/datetime, and --datetime-format)

  --version                    Show the version and exit.
  --help                       Show this message and exit.
Popular Database Projects
Popular Sqlite Projects
Popular Data Storage Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Csv Files
Sqlite Database