Deepbedmap

Going beyond BEDMAP2 using a super resolution deep neural network. Also a convenient flat file data repository for high resolution bed elevation datasets around Antarctica.
Alternatives To Deepbedmap
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Data Science Ipython Notebooks25,668
2 months ago34otherPython
Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
Awesome Pytorch List14,103
6 months ago4
A comprehensive list of pytorch related content on github,such as different models,implementations,helper libraries,tutorials etc.
Stanford Cs 230 Deep Learning4,248
4 years ago5mit
VIP cheatsheets for Stanford's CS 230 Deep Learning
D2l Pytorch3,933
a year ago13apache-2.0Jupyter Notebook
This project reproduces the book Dive Into Deep Learning (https://d2l.ai/), adapting the code from MXNet into PyTorch.
Deep Learning Book2,650
5 years ago1December 24, 20162otherJupyter Notebook
Repository for "Introduction to Artificial Neural Networks and Deep Learning: A Practical Guide with Applications in Python"
Datasciencer1,497
6 years ago3mitR
a curated list of R tutorials for Data Science, NLP and Machine Learning
Andrew Ng Notes1,367
a year ago2Jupyter Notebook
This is Andrew NG Coursera Handwritten Notes.
Cracking The Data Science Interview1,291
2 years ago1Jupyter Notebook
A Collection of Cheatsheets, Books, Questions, and Portfolio For DS/ML Interview Prep
Tutorials847
a year ago3otherJupyter Notebook
AI-related tutorials. Access any of them for free → https://towardsai.net/editorial
Numpycnn531
6 months ago3May 24, 20181Python
Building Convolutional Neural Networks From Scratch using NumPy
Alternatives To Deepbedmap
Select To Compare


Alternative Project Comparisons
Readme

DeepBedMap [paper] [poster] [presentation]

Going beyond BEDMAP2 using a super resolution deep neural network. Also a convenient flat file data repository for high resolution bed elevation datasets around Antarctica.

DOI GitHub top language Code style: black Comet.ML: experiments Github Actions Build Status Dependabot Status

DeepBedMap DEM over entire Antarctic continent, EPSG:3031 projection

DeepBedMap Pipeline

Directory structure
  deepbedmap/
    ├── features/ (files describing the high level behaviour of various features)
    │    ├── *.feature... (easily understandable specifications written using the Given-When-Then gherkin language)
    │    └── README.md (markdown information on the feature files)
    ├── highres/ (contains high resolution localized DEMs)
    │    ├── *.txt/csv/grd/xyz... (input vector file containing the point-based bed elevation data)
    │    ├── *.json (the pipeline file used to process the xyz point data)
    │    ├── *.nc (output raster netcdf files)
    │    └── README.md (markdown information on highres data sources)
    ├── lowres/ (contains low resolution whole-continent DEMs)
    │    ├── bedmap2_bed.tif (the low resolution DEM!)
    │    └── README.md (markdown information on lowres data sources)
    ├── misc/ (miscellaneous raster datasets)
    │    ├── *.tif (Surface DEMs, Ice Flow Velocity, etc. See list in Issue #9)
    │    └── README.md (markdown information on miscellaneous data sources)
    ├── model/ (*hidden in git, neural network model related files)
    │    ├── train/ (a place to store the raster tile bounds and model training data)
    │    └── weights/ (contains the neural network model's architecture and weights)
    ├── .env (environment variable config file used by pipenv)
    ├── .<something>ignore (files ignored by a particular piece of software)
    ├── .<something else> (stuff to make the code in this repo look and run nicely e.g. linters, CI/CD config files, etc)
    ├── Dockerfile (set of commands to fully reproduce the software stack here into a docker image, used by binder)
    ├── LICENSE.md (the license covering this repository)
    ├── Pipfile (what you want, the summary list of core python dependencies)
    ├── Pipfile.lock (what you need, all the pinned python dependencies for full reproducibility)
    ├── README.md (the markdown file you're reading now)
    ├── data_list.yml (human and machine readable list of the datasets and their metadata)
    ├── data_prep.ipynb/py (paired jupyter notebook/python script that prepares the data)
    ├── deepbedmap.ipynb/py (paired jupyter notebook/python script that predicts an Antarctic bed digital elevation model)
    ├── environment.yml (conda binary packages to install)
    ├── paper_figures.ipynb/py (paired jupyter notebook/python script to produce figures for DeepBedMap paper
    ├── srgan_train.ipynb/py (paired jupyter notebook/python script that trains the ESRGAN neural network model)
    └── test_ipynb.ipynb/py (paired jupyter notebook/python script that runs doctests in the other jupyter notebooks!)

Getting started

Quickstart

Launch in Binder (Interactive jupyter notebook/lab environment in the cloud).

Binder Open All Collab

Installation

Installation steps

Start by cloning this repo-url

git clone <repo-url>

Then I recommend using conda to install the non-python binaries (e.g. GMT, CUDA, etc). The conda virtual environment will also be created with Python and pipenv installed.

cd deepbedmap
conda env create -f environment.yml

Activate the conda environment first.

conda activate deepbedmap

Then set some environment variables before using pipenv to install the necessary python libraries, otherwise you may encounter some problems (see Common problems below). You may want to ensure that which pipenv returns something similar to ~/.conda/envs/deepbedmap/bin/pipenv.

export HDF5_DIR=$CONDA_PREFIX/
export LD_LIBRARY_PATH=$CONDA_PREFIX/lib/
pipenv install --python $CONDA_PREFIX/bin/python --dev
#or just
HDF5_DIR=$CONDA_PREFIX/ LD_LIBRARY_PATH=$CONDA_PREFIX/lib/ pipenv install --python $CONDA_PREFIX/bin/python --dev

Finally, double-check that the libraries have been installed.

pipenv graph

Syncing/Updating to new dependencies

conda env update -f environment.yml
pipenv sync --dev

Common problems

Note that the .env file stores some environment variables. So if you run conda activate deepbedmap followed by some other command and get an ...error while loading shared libraries: libpython3.7m.so.1.0..., you may need to run pipenv shell or do pipenv run <cmd> to have those environment variables registered properly. Or just run this first:

export LD_LIBRARY_PATH=$CONDA_PREFIX/lib/

Also, if you get a problem when using pipenv to install netcdf4, make sure you have done:

export HDF5_DIR=$CONDA_PREFIX/

and then you can try using pipenv install or pipenv sync again. See also this issue for more information.

Running jupyter lab

conda activate deepbedmap
pipenv shell

python -m ipykernel install --user --name deepbedmap  #to install conda env properly
jupyter kernelspec list --json                        #see if kernel is installed
jupyter lab &

Citing

The paper is published at The Cryosphere and can be referred to using the following BibTeX code:

@Article{tc-14-3687-2020,
    AUTHOR = {Leong, W. J. and Horgan, H. J.},
    TITLE = {DeepBedMap: a deep neural network for resolving the bed topography of Antarctica},
    JOURNAL = {The Cryosphere},
    VOLUME = {14},
    YEAR = {2020},
    NUMBER = {11},
    PAGES = {3687--3705},
    URL = {https://tc.copernicus.org/articles/14/3687/2020/},
    DOI = {10.5194/tc-14-3687-2020}
}

The DeepBedMap_DEM v1.1.0 dataset is available from Zenodo at https://doi.org/10.5281/zenodo.4054246. Neural network model training experiment runs are also recorded at https://www.comet.ml/weiji14/deepbedmap. Python code for the DeepBedMap model here on Github is also mirrored on Zenodo at https://doi.org/10.5281/zenodo.3752613.

Popular Neural Projects
Popular Data Science Projects
Popular Machine Learning Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Jupyter Notebook
Neural
Data Science
Generative Adversarial Network
Deep Neural Networks
Binder
Super Resolution
Remote Sensing
Chainer