Awesome Open Source
Awesome Open Source

bookstore πŸ“š

Documentation Status Build Status CircleCI Codecov

bookstore πŸ“š provides tooling and workflow recommendations for storing πŸ’Ώ, scheduling πŸ“†, and publishing πŸ“– notebooks.

The full documentation is hosted on ReadTheDocs.

How does bookstore work

Automatic Notebook Versioning

Every save of a notebook creates an immutable copy of the notebook on object storage.

To simplify implementation, we currently rely on S3 as the object store, using versioned buckets.

Storage Paths

All notebooks are archived to a single versioned S3 bucket with specific prefixes denoting the lifecycle of the notebook:

  • /workspace - where users edit
  • /published - public notebooks (to an organization)

Each notebook path is a namespace that an external service ties into the schedule. We archive off versions, keeping the path intact (until a user changes them).

Prefix Intent
/workspace/kylek/notebooks/mine.ipynb Notebook in β€œdraft”
/published/kylek/notebooks/mine.ipynb Current published copy

Scheduled notebooks will also be referred to by the notebook key. In addition, we'll need to be able to surface version IDs as well.

Transitioning to this Storage Plan

Since most people are on a regular filesystem, we'll start with writing to the /workspace prefix as Archival Storage (writing on save using a post_save_hook for a Jupyter contents manager).

Publishing

The bookstore publishing endpoint is a serverextension to the classic Jupyter server. This means you will need to explicitly enable the serverextension to use the endpoint.

To do so, run:

jupyter serverextension enable --py bookstore

To enable it only for the current environment, run:

jupyter serverextension enable --py bookstore --sys-prefix

Installation

bookstore requires Python 3.6 or higher.

Note: Supports installation on Jupyter servers running Python 3.6 and above. Your notebooks can still be run in Python 2 or Python 3.

  1. Clone this repo.
  2. At the repo's root, enter in the Terminal: python3 -m pip install . (Tip: don't forget the dot at the end of the command)

Configuration

# jupyter config
# At ~/.jupyter/jupyter_notebook_config.py for user installs on macOS
# See https://jupyter.readthedocs.io/en/latest/projects/jupyter-directories.html for other places to plop this

from bookstore import BookstoreContentsArchiver

c.NotebookApp.contents_manager_class = BookstoreContentsArchiver

# All Bookstore settings are centralized on one config object so you don't have to configure it for each class
c.BookstoreSettings.workspace_prefix = "/workspace/kylek/notebooks"
c.BookstoreSettings.published_prefix = "/published/kylek/notebooks"

c.BookstoreSettings.s3_bucket = "<bucket-name>"

# Note: if bookstore is used from an EC2 instance with the right IAM role, you don't
# have to specify these
c.BookstoreSettings.s3_access_key_id = <AWS Access Key ID / IAM Access Key ID>
c.BookstoreSettings.s3_secret_access_key = <AWS Secret Access Key / IAM Secret Access Key>

Developing

If you are developing on bookstore you will want to run the ci tests locally and to make releases.

Use CONTRIBUTING.md to learn more about contributing. Use running_ci_locally.md to learn more about running ci tests locally. Use running_python_tests.md to learn about running tests locally. Use RELEASING.md to learn more about releasing bookstore.


Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
Python (1,141,503)Β 
Data Science (8,797)Β 
Storage (1,691)Β 
Notebook (1,205)Β 
Scheduling (436)Β 
Related Projects
Advertising πŸ“¦Β 9
All Projects
Application Programming Interfaces πŸ“¦Β 120
Applications πŸ“¦Β 181
Artificial Intelligence πŸ“¦Β 72
Blockchain πŸ“¦Β 70
Build Tools πŸ“¦Β 111
Cloud Computing πŸ“¦Β 79
Code Quality πŸ“¦Β 28
Collaboration πŸ“¦Β 30
Command Line Interface πŸ“¦Β 48
Community πŸ“¦Β 81
Companies πŸ“¦Β 60
Compilers πŸ“¦Β 60
Computer Science πŸ“¦Β 74
Configuration Management πŸ“¦Β 39
Content Management πŸ“¦Β 167
Control Flow πŸ“¦Β 197
Data Formats πŸ“¦Β 77
Data Processing πŸ“¦Β 266
Data Storage πŸ“¦Β 132
Economics πŸ“¦Β 60
Frameworks πŸ“¦Β 198
Games πŸ“¦Β 122
Graphics πŸ“¦Β 103
Hardware πŸ“¦Β 148
Integrated Development Environments πŸ“¦Β 47
Learning Resources πŸ“¦Β 147
Legal πŸ“¦Β 28
Libraries πŸ“¦Β 119
Lists Of Projects πŸ“¦Β 21
Machine Learning πŸ“¦Β 336
Mapping πŸ“¦Β 61
Marketing πŸ“¦Β 15
Mathematics πŸ“¦Β 55
Media πŸ“¦Β 228
Messaging πŸ“¦Β 97
Networking πŸ“¦Β 304
Operating Systems πŸ“¦Β 84
Operations πŸ“¦Β 120
Package Managers πŸ“¦Β 52
Programming Languages πŸ“¦Β 229
Runtime Environments πŸ“¦Β 96
Science πŸ“¦Β 42
Security πŸ“¦Β 375
Social Media πŸ“¦Β 26
Software Architecture πŸ“¦Β 70
Software Development πŸ“¦Β 68
Software Performance πŸ“¦Β 57
Software Quality πŸ“¦Β 127
Text Editors πŸ“¦Β 45
Text Processing πŸ“¦Β 131
User Interface πŸ“¦Β 310
User Interface Components πŸ“¦Β 465
Version Control πŸ“¦Β 29
Virtualization πŸ“¦Β 68
Web Browsers πŸ“¦Β 38
Web Servers πŸ“¦Β 25
Web User Interface πŸ“¦Β 194