Awesome Open Source
Awesome Open Source

SANSA-Stack

Build Status License Twitter

This project comprises the whole Semantic Analytics Stack (SANSA). At a glance, it features the following functionality:

  • Ingesting RDF and OWL data in various formats into RDDs
  • Operators for working with RDDs and data frames of RDF data at various levels (triples, bindings, graphs, etc)
  • Transformation of RDDs to data frames and partitioning of RDDs into R2RML-mapped data frames
  • Distributed SPARQL querying over R2RML-mapped data frame partitions using RDB2RDF engines (Sparqlify & Ontop)
  • Enrichment of RDDs with inferences
  • Application of machine learning algorithms

For a detailed description of SANSA, please visit http://sansa-stack.net.

Layers

The SANSA project is structured in the following five layers developed in their respective sub-folders:

Release Cycle

A SANSA stack release is done every six months and consists of the latest stable versions of each layer at this point. This repository is used for organising those joint releases.

Usage

Spark

Requirements

We currently require a Spark 3.x.x with Scala 2.12 setup. A Spark 2.x version can be built from source based on the spark2 branch.

Release Version

Some of our dependencies are not in Maven central (yet), so you need to add following Maven repository to your project POM file repositories section:

<repository>
   <id>maven.aksw.internal</id>
   <name>AKSW Release Repository</name>
   <url>http://maven.aksw.org/archiva/repository/internal</url>
   <releases>
      <enabled>true</enabled>
   </releases>
   <snapshots>
      <enabled>false</enabled>
   </snapshots>
</repository>

If you want to import the full SANSA Stack, please add the following Maven dependency to your project POM file:

<!-- SANSA Stack -->
<dependency>
   <groupId>net.sansa-stack</groupId>
   <artifactId>sansa-stack-spark_2.12</artifactId>
   <version>$LATEST_RELEASE_VERSION$</version>
</dependency>

If you only want to use particular layers, just replace $LAYER_NAME$ with the corresponding name of the layer

<!-- SANSA $LAYER_NAME$ layer -->
<dependency>
   <groupId>net.sansa-stack</groupId>
   <artifactId>sansa-$LAYER_NAME$-spark_2.12</artifactId>
   <version>$LATEST_RELEASE_VERSION$</version>
</dependency>

SNAPSHOT Version

While the release versions are available on Maven Central, latest SNAPSHOT versions have to be installed from source code:

git clone https://github.com/SANSA-Stack/SANSA-Stack.git
cd SANSA-Stack

Then to build and install the full SANSA Spark stack you can do

./dev/mvn_install_stack_spark.sh 

or for a single layer $LAYER_NAME$ you can do

mvn -am -DskipTests -pl :sansa-$LAYER_NAME$-spark_2.12 clean install 

Alternatively, you can use the following Maven repository and add it to your project POM file repositories section:

<repository>
   <id>maven.aksw.snapshots</id>
   <name>AKSW Snapshot Repository</name>
   <url>http://maven.aksw.org/archiva/repository/snapshots</url>
   <releases>
      <enabled>false</enabled>
   </releases>
   <snapshots>
      <enabled>true</enabled>
   </snapshots>
</repository>

Then do the same as for the release version and add the dependency:

<!-- SANSA Stack -->
<dependency>
   <groupId>net.sansa-stack</groupId>
   <artifactId>sansa-stack-spark_2.12</artifactId>
   <version>$LATEST_SNAPSHOT_VERSION$</version>
</dependency>

How to Contribute

We always welcome new contributors to the project! Please see our contribution guide for more details on how to get started contributing to SANSA.



Alternative Project Comparisons
Related Awesome Lists
Top Programming Languages
Top Projects

Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
Dataset (33,099
Scala (28,916
Spark (10,763
Rdf (3,092
Apache Spark (1,208
Flink (1,116
Distributed Computing (649
Semantic Web (626
Apache Jena (15