Navigation Menu

Skip to content

SatelliteShorelines/CoastSeg

Repository files navigation

CoastSeg

image
image

Last Commit Maintenance Wiki GitHub Wiki

Python TensorFlow Keras

Table of Contents

Useful Links

What is CoastSeg?

Coastseg stands for Coastal Segmentation, it is an interactive jupyter notebook for downloading satellite imagery with CoastSat and applying Doodleverse/Zoo image segmentation models to satellite imagery. CoastSeg provides an interactive interface for drawing Regions of Interest (ROIs) on a map, downloading satellite imagery, loading geojson files, extracting shorelines from satellite imagery, and much more. In a nutshell, it is ...

  • ... an easy way to download time-series of cloud-masked and post-processed satellite imagery anywhere in the world
  • ... a mapping extension for CoastSat using Segmentation Zoo models.
  • ... an interactive interface to download satellite imagery using CoastSat from Google Earth Engine
  • ... an interactive interface for extracting shorelines from satellite imagery
  • ... an interactive interface to apply segmentation models to satellite imagery

In more detail, CoastSeg represents the following advances in satellite-derived shoreline workflows:

  1. An emulation of the CoastSat methodology for defining a shoreline extraction project, image downloading, shoreline extraction, shoreline filtering, and tide correction, all within a single jupyter notebook that can be accessed online
  2. A pip-installable package for CoastSat workflows
  3. Use of concurrency for image downloads from Google Earth Engine
  4. A faster and more convenient API-based methodology for tidal height estimation, using pyTMD
  5. An initial implementation of an alternative shoreline mapping workflow using Zoo models from the Doodleverse
  6. A conda environment that can work on secure networks
  7. Script-based data wrangling utilities
  8. Hyperparameter experimentation and tracking using an organizational workflow idea called ‘Sessions’
  9. Supporting databases for transects, reference shorelines, beach slopes, and other useful metadata variables
  10. A deep-learning based shoreline extraction workflow (still in development)

Version 1 is now stable, and we have a lot of planned new features for version 2.

coastseg_main_flow_updated

  • Create ROIs(regions of interest) along the coast and automatically load shorelines on the map.
  • Use Google Earth Engine to automatically download satellite imagery for each ROI clicked on the map.
  • Coastseg can automatically extract shorelines from downloaded satellite imagery.

Installation Instructions

We recommend that you use Windows 10, Windows 11, or Ubuntu Linux. Mac users, please see here

In order to use Coastseg you need to install Python packages in an environment. We recommend you use Anaconda or Miniconda to install the python packages in an environment for Coastseg.

After you install miniconda/Anaconda on your PC, open the Anaconda prompt or Terminal in Mac and Linux and use the cd command (change directory) to go the folder where you have downloaded the Coastseg repository.

We highly recommend you install CoastSeg using conda following the instructions in Install from conda-forge.

Install from conda-forge

  1. Create an miniconda/Anaconda environment and Activate it
  • This command creates an anaconda environment named coastseg and installs python 3.10 in it.
    conda create --name coastseg python=3.10 -y
    conda activate coastseg
  1. Install coastseg

    conda install -c conda-forge coastseg
  2. (Optional) Install Optional Dependencies

    • Only install these dependencies if you plan to use CoastSeg's Zoo workflow notebook.
    pip install tensorflow
    pip install transformers

Install from Pypi

  1. Create an miniconda/Anaconda environment
  • This command creates an anaconda environment named coastseg and installs python 3.10 in it.
    conda create --name coastseg python=3.10 -y
  1. Activate your conda environment

    conda activate coastseg
  • If you have successfully activated coastseg you should see that your terminal's command line prompt should now start with (coastseg).

coastseg activated in anaconda prompt

  1. Install Conda Dependencies
    • CoastSeg requires jupyterlab and geopandas to function properly so they will be installed in the coastseg environment.
    • Geopandas has GDAL as a dependency so its best to install it with conda.
    • Make sure to install geopandas from the conda-forge channel to ensure you get the latest version.
    • Make sure to install both jupyterlab and geopandas from the conda forge channel to avoid dependency conflicts
      conda install -c conda-forge geopandas -y
  2. Install the CoastSeg from PyPi
    pip install coastseg
  3. Uninstall the h5py installed by pip and reinstall with conda-forge
    • pip install jsonschema==4.19.0 is a temporary command you have to run until issue stac-utils/pystac#1214 is resolved
    pip install jsonschema==4.19.0 --user  
    pip uninstall h5py -y
    conda install -c conda-forge h5py -y

Having Installation Errors?

Use the command conda clean --all to clean old packages from your anaconda base environment. Ensure you are not in your coastseg environment or any other environment by running conda deactivate, to deactivate any environment you're in before running conda clean --all. It is recommended that you have Anaconda prompt (terminal for Mac and Linux) open as an administrator before you attempt to install coastseg again.

Conda Clean Steps

conda deactivate
conda clean --all

Getting Started

Prerequisites

1.Sign up to use Google Earth Engine Python API

Installation & SetUp

1.Activate the coastseg conda environment

conda activate coastseg
  • If you have successfully activated coastseg you should see that your terminal's command line prompt should now start with (coastseg).

coastseg activated in anaconda prompt

2.Download CoastSeg from GitHub

git clone coastseg --depth 1 https://github.com/Doodleverse/CoastSeg.git

Extract Shorelines

1.Launch Jupyter Lab

  • Run this command in the coastseg directory to launch the notebook SDS_coastsat_classifier
conda activate coastseg
jupyter lab SDS_coastsat_classifier.ipynb

2.Authenticate with Google Earth Engine

  • Run the cell located under 'Authenticate with Google Earth Engine (GEE)'

auth_cell_cropped

3.Draw an Bounding Box

  • Draw a bounding box along the coast in this box is where ROIs will be created

Option 1: Draw a Bounding Box with the box tool

draw_bbox

Option 2: Draw a Bounding Box with the polygon tool

  • This is useful if you have back - barrier shorelines that you don't want to include

draw_bbox_polygon_remove_back_barrier

4.Generate ROI (Region of Interest)

  • ROIs can only be generated along a shoreline

  • If no shoreline is found then an error message will appear telling you no ROIs can be created. If this happens create your own reference shoreline following the guide here How to Create Reference Shoreline

generate_roi

5.Load Transects

  • Make sure there are transects inside the ROI you have selected otherwise you won't be able to extract shorelines

  • If there isn't a reference shoreline or any transects available for your site check out the guide on how to upload your own here

load_rois_then_transects_on_map_demo

6.Modify the Settings

  • Change the satellites to L8 and L9

  • Change the dates to 12/01/2023 - 03/01/2024

  • Change the size of the reference shoreline buffer

  • Click Save Settings

save_settings_getting_started_circle

7.Name the Session

  • Let's call this 'demo_session'

  • This is the name of the folder that will be saved in CoastSeg/sessions

  • It will contain a subdirectory for each ROI that shorelines will be extracted for

save_demo_session

8.Preview the available Imagery

  • Preview the amount of available imagery for the selected ROI between the dates

case study 1 preview imagery

9.Download the ROIs

  • Click the ROIs you want to download on the map ( they will turn blue when selected)

  • If no transects or reference shorelines are available for the region you have uploaded follow the guide here

  • NEVER rename your ROIs this is because CoastSeg keeps track of the filename in the 'config.json' and you won't be able to load it into CoastSeg again if you do. The ROI's ID is the filename so that information will be lost if you rename it.

case_study_1_download_roi

10.Extract Shorelines save_settings_download_extract

  • Extracting shorelines works by finding the land water interface in the image and drawing a line along it

  • A time series of shoreline position along each transect is generated as well

shoreline_transect_intersection

Apply Tidal Correction to Extracted Shorelines (Optional)

1.Download the tide model

2.Load the Session with Extracted Shorelines

  • Re-open the jupyter notebook

  • Under the 'Kernel' menu Click 'restart and clear outputs of all cells'

restart kernel and clear outputs of all cells

  • Click 'Load Session' and load 'sample_session'

select load session and tide correct

3.Click Correct Tides

  • Click the ROI ID from the dropdown

     -- You should see some extracted shorelines on the map if you don't then the ROI ID won't appear in the dropdown
    
  • Enter Beach Slope

  • Enter Beach Elevation relative to Mean Sea Level

select roi id for tide correction

load_session_correct_tides_demo

CoastSeg Update Guide

This guide is designed to help you easily update CoastSeg, whether you're updating to a specific version, the latest version, applying a patch, or updating code and notebooks from GitHub.

Step 1: Install CoastSeg from PyPi

Option 1: Update to the latest version

  1. Install CoastSeg from PyPi
    • To ensure you have the latest features and fixes, use this command:
      pip install coastseg --upgrade
  • Don't worry if you see the warning message below. This is normal
  "ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts."
  1. Install jsonschema
    • This is necessary to run coastseg in a jupyter notebook.
 pip install jsonschema==4.19.0 --user

Option 2: Update to a Specific Version

  1. Install CoastSeg from PyPi
    • If you need a specific version of CoastSeg, use this command:
    • Replace with the desired version number (e.g., 1.1.26).
     pip install coastseg==<version>
  • Don't worry if you see the warning message below. This is normal
  "ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts."
  1. Install jsonschema
    • This is necessary to run coastseg in a jupyter notebook.
 pip install jsonschema==4.19.0 --user

Step 2: Update Code and Notebooks from GitHub

(Optional) Follow these steps if you want the latest notebooks or code updates from the CoastSeg GitHub repository.

Click to expand instructions

Step 1: Open CoastSeg in Anaconda

  1. Open Anaconda Prompt
  2. Activate the coastseg environment
  conda activate coastseg
  1. Go to your coastseg location
cd <coastseg location>

Step 2: Check for a Remote Connection to CoastSeg Repository

-Run the command below. In the output of this command you should see origin https://github.com/Doodleverse/CoastSeg.git (fetch)

git remote -v

git remote output

  • If you don't see this output, then run the following command
     git remote add origin  https://github.com/Doodleverse/CoastSeg.git
     git pull origin main

Step 3: Pull the Latest Changes

  1. Run the command below

     git pull origin main
    
  2. If you recieve an error message like the one shown below then proceed to 3, otherwise go to Go to Step 4: Verify Update Success

        Please commit your changes or stash them before you merge
        Aborting
    
    git_pull_fail
  3. Run the command below:

  • WARNING This will clear out anything you have written to the certifications.json make sure to save that file to a new location then move it back when you're done upgrading
       git fetch origin
       git reset --hard origin/main
       git pull origin main

Step 4: Verify Update Success

git status
  • This command should return the following message
  • On branch main
    Your branch is up to date with 'origin/main'.
    

Authors

Package maintainers:

  • @dbuscombe-usgs Contracted to USGS Pacific Coastal and Marine Science Center.
  • @2320sharon : Lead Software Developer / Contracted to USGS Pacific Coastal and Marine Science Center.

Contributions:

  • @ebgoldstein
  • @venuswku
  • @robbibt
  • @edlazarus
  • Beta testers: Catherine Janda, Ann Gibbs, Jon Warrick, Andrea O’Neill, Kathryn Weber, Julia Heslin (USGS)
  • We would like to express our gratitude to all the contributors who made this release possible. Thank you to everyone who tested the beta versions of coastseg and provided us with the feedback we needed to improve coastseg. Thanks also to the developers and maintainers of pyTMD, DEA-tools, xarray, and GDAL, without which this project would be impossible

Citations

Thank you to all the amazing research who contributed their transects to coastseg.

  1. Hawaii small islands https://pubs.usgs.gov/of/2011/1009/data.html
  2. Barter Island, Alaska https://www.sciencebase.gov/catalog/item/5fa1f10ad34e198cb793cee4
  3. Pacific Northwest, Gulf, and SE USA: Dr Sean Vitousek, USGS-PCMSC, based on DSAS transects
  4. Atlantic barrier islands: https://www.sciencebase.gov/catalog/item/5d5ece47e4b01d82ce961e36
  5. Mexico, New Zealand, Japan, Chile, Peru all from: https://zenodo.org/record/7758183#.ZCXZMcrMJPY
  6. Snyder, A.G., and Gibbs, A.E., 2019, National assessment of shoreline change: A GIS compilation of updated vector shorelines and associated shoreline change data for the north coast of Alaska, Icy Cape to Cape Prince of Wales: U.S. Geological Survey data release, https://doi.org/10.5066/P9H1S1PV.
  7. Gibbs, A.E., Ohman, K.A., Coppersmith, R., and Richmond, B.M., 2017, National Assessment of Shoreline Change: A GIS compilation of updated vector shorelines and associated shoreline change data for the north coast of Alaska, U.S. Canadian border to Icy Cape: U.S. Geological Survey data release, https://doi.org/10.5066/F72Z13N1.
  8. Himmelstoss, E.A., Kratzmann, M., Hapke, C., Thieler, E.R., and List, J., 2010, The National Assessment of Shoreline Change: A GIS Compilation of Vector Shorelines and Associated Shoreline Change Data for the New England and Mid-Atlantic Coasts: U.S. Geological Survey Open-File Report 2010-1119, available at https://pubs.usgs.gov/of/2010/1119/.
  9. Kilian Vos. (2023). Time-series of shoreline change along the Pacific Rim (v1.4) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7758183
  10. Vos, Kilian, Wen, Deng, Harley, Mitchell D., Turner, Ian L., & Splinter, Kristen D. (2022). Beach-face slope dataset for Australia (Version 2) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7272538
  11. Gibbs, A.E., Jones, B.M., and Richmond, B.M., 2020, A GIS compilation of vector shorelines and coastal bluff edge positions, and associated rate-of-change data for Barter Island, Alaska: U.S. Geological Survey data release, https://doi.org/10.5066/P9CRBC5I.

To compile into a pdf use

pandoc paper.md -o paper.pdf --bibliography paper.bib

About

An interactive toolbox for downloading satellite imagery, applying image segmentation models, mapping shoreline positions and more. The mapping extension for CoastSat and Zoo.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published