A suite of data mining, analytics, and visualization solutions to create an awesome dashboard for the Museum Barberini, Potsdam, in order to help them analyze and assess customer, advertising, and social media data!
This solution has been originally developed in part of a Data Analytics project run as a cooperation of the Museum Barberini (MB) and the Hasso Plattner Institute (HPI) in 2019/20 (see Credits below). The project comprises a data mining pipeline that is regularly run on a server and feeds several visualization dashboards that are hosted in a Power BI app. For more information, see also the following resources:
While this solution has been tailored for the individual needs of the MB and the overall project is characterized by the structure of a majestic monolith, we think that it contains some features and components that have great potential for being reused as part of other solutions. In particular, these features include the following highlights:
Visitor Prediction: Machine-Learning (ML) based solution to predict the future number of museum visitors by extrapolating historic visitor data.
Credits go to Georg Tennigkeit (@georg.tennigkeit/@georgt99).
Postal Code Cleansing: Collection of heuristics to correct address information entered by humans with errors.
Credits go to Laura Holz (@laura.holz/@lauraholz).
Power BI Crash Tests: Load & crash tests for Power BI visualization reports. See https://awesomeopensource.com/project/LinqLover/pbi-crash-tests.
Credits go to Christoph Thiede (@christoph.thiede/@LinqLover).
Development is currently being continued in a private GitLab instance but a mirror of the repository is available on GitHub.
If you are interested in reusing any part of our solution and have further questions, ideas, or bug reports, please do not hesitate to contact us!
Please note that these instructions are optimized for Ubuntu/amd64.
If you use a different configuration, you may need to adjust the toolchain installation (see
Clone the repository using git
git clone https://github.com/Museum-Barberini-gGmbH/Barberini-Analytics.git
secrets folders (which is not part of the repository) into
Set up the toolchain.
scripts/setup/install_toolchain.sh how to do this.
If you use ubuntu/amd64, you can run the script directly.
sudo to run the commands!
Set up the docker network and add the current user to the
docker user group.
If you cloned the repository in a different folder than
/root/bp-barberini, you may want to adapt the paths in
If no crontab exists before, create it using
make docker-do do='make luigi-scheduler'
This will also start a webserver on http://localhost:8082 where you can trace all running tasks.
make docker-do do='make luigi'
Have a look at our beautiful
To access the luigi docker, do:
make startup connect
Close the session by executing:
Authors: Laura Holz, Selina Reinhard, Leon Schmidt, Georg Tennigkeit, Christoph Thiede, Tom Wollnik (bachelor project BP-FN1 @ HPI, 2019/20).
Organizations: Hasso Plattner Institute, Potsdam; Museum Barberini; Hasso Plattner Foundation.