This is our ARTS test set, an enriched test set to probe Aspect Robustness of ABSA.
Alternatives To Arts_testset
Select To Compare


This is the repository for our 2020 paper "Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis".


We provide a Aspect Robustness Probing test set for SemEval 2014 Aspect-Based Sentiment Analysis (ABSA).

Data Generation Process

We generate our new probing test set by three automatic strategies:

  • RevTgt (sentence with a red background): Reverse the sentiment of the Target aspect.
  • RevNon (sentence with a green background): Reverse the sentiment of the Non-target aspect.
  • AddDiff (sentence with a blue background): Add new aspects with Different sentiment.


Aspect Probing Results

We probed nine ABSA models (as mentioned in our paper).

  • Their outputs on SemEval 2014 are in the output folder.

How to Use Our Code

If you have a new ABSA dataset, you can run our code to generate you own aspect robustness probing test set.

python code/ -dataset_name laptop


More Questions

If you have more questions, please feel free to submit a GitHub issue.

Popular Machine Learning Categories

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.