Awesome Open Source
Awesome Open Source


Twitter users often associate and socialize with other users based on similar interests. The Tweets of these users can be classified using a trained LDA model to automate the discovery of their similarities.


Python 2.7 is recommended since the pattern library is currently incompatible with most Python 3 versions.

Python 3.6 can be used with the pattern library, though it may need to be built from source since most newer Linux distributions don't come with it pre-installed. The commands to build Python 3.6 from source are provided in the script.




git clone

Run bash script:


Python pip requirements included in these files:

# for Python 2.7
pip install -r requirements_py2.txt

# for Python 3
pip install -r requirements_py3.txt

Link to the simple-wikipedia dump:

Mac osx

The installation is very similar to the linux installation:

extra install instructions in

pip install -r requirements_py3_OSX.txt


  1. Get user and follower ids by location -
  2. Download Tweets for each user -
  3. Create an LDA model from a corpus of documents -
  4. Generate topic probability distributions for Tweet documents -
  5. Calculate distances between Tweet documents and graph them -

Sample Visualizations

Built With

  • Gensim - Package for creating LDA model
  • pyLDAvis - Package for visualizing LDA model
  • Tweepy - Package for interacting with Twitter REST API
  • NLTK - Package for stopword management and tokenization

Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
Python (1,123,905
Twitter (3,854
Nltk (776
Tweets (608
Topic Modeling (488
Tweepy (390
Networkx (273
Gensim (235
Community Detection (153
Related Projects