Wonderflow offers a text mining solution. Authors Anna Feltracco, Elisabetta Pisani, and the rest of our dedicated Linguistic team have shared how they perform Aspect-Based Sentiment Analysis to extract value from consumer feedback.
It’s important to understand ABSA’s key challenges for our employees, customers/users, and those in linguistics, customer experience (CX), marketing, etc. They can gain a clearer understanding of this task and possibly draw inspiration from our examples and solutions.
At Wonderflow, we perform Aspect-Based Sentiment Analysis to extract value from consumer feedback, utilizing NLP techniques.
The analysis poses a number of linguistic challenges which could impact the results of an automatic system built to perform the task.
In this article, we will review some of them, aiming at throwing some light on this topic, which can be of interest especially for users of text analytics tools, like our Wonderboard. In particular, we will focus on those challenges concerning the identification of the correct sentiment in feedback texts.
What is Aspect-Based Sentiment Analysis (ABSA)?
Aspect-Based Sentiment Analysis (ABSA) is a type of text analysis that categorizes opinions by aspect and identifies the sentiment related to each aspect.
By aspects, we consider attributes or components of an entity (a product or a service, in our case).
Let’s use an example like the one below. The goal here for the ABSA system is to identify the two aspects – design and price – with their related sentiment. In other words, design:positive, price:negative.
I was tempted to buy this product as I really like its design, but its price is not very good.
Notice that in the same text, different aspects can have different sentiments. In this sense, the output of ABSA is not meant to be a general indication of the sentiment expressed in the text but aims at providing a more granular and detailed level of information.
The following image depicts an example of a customer describing “bad” food in their feedback.
Source: An example of aspect-based sentiment analysis (ABSA). Sentiment expressions such as “bad” appear near their related aspect expressions “sushi rolls,” respectively.
Four Key Challenges in ABSA
From a linguistic perspective, the task of automatically identifying aspects and related sentiments poses many challenges, because we often find complex linguistic phenomena that are not easy to interpret and understand.
In particular, identifying the correct aspect can be considered the hardest subtask of this analysis, because customers can express their opinion about a high number of aspects (e.g. for a product: its price, quality, performance, design, or its usage).
On the other hand, identifying sentiment in consumer feedback could be considered relatively easier because sentiment is generally classified into only three categories – positive, negative, or neutral.
Nevertheless, retrieving the correct sentiment related to the aspect can also be tricky.
Depending on the situation, while performing aspect-based sentiment analysis, we encounter the following challenges:
1. When sentiment is product dependent
Customers can express their appreciation or dissatisfaction about a product or service in an explicit manner. For example, by writing positive adjectives (e.g. good, nice, fantastic) or negative ones (e.g. bad, awful, horrible).
In other cases, the sentiment can be expressed by simply underlining some characteristics of the product, or through a description of the product functions. Consider the following examples:
- The battery lasts long.
- The oven heats up quickly.
- The epilator pulls the hair.
These examples can be interpreted as ‘positive’ product features (e.g. a battery that does last long is considered of ‘good quality’).
However, if similar expressions are used in a text without clearly mentioning the product, they can be ambiguous. Consider the following examples:
a1. It lasts long.
b1. It heats up quickly.
c1. It pulls the hair.
The above examples do not clearly show what is the exact product or service that is being referred to. What if text a1 referred to a “queue,” b1 to a “mobile phone,” or c1 to a “razor”? The sentiment would then be ‘negative.’
Thus the expression “heats up quickly” can be both positive or negative depending on the product it is referred to (oven or mobile phone). That’s why our knowledge of the product itself is crucial to correctly identify the sentiment.
At Wonderflow, we customize our analysis engine according to the product characteristics, in order to correctly detect the sentiment that was meant to be conveyed.
2. When sentiment is target dependent
While describing a product or a service, the customer can also make reference to other entities. For example:
This product has a good price; the one my brother purchased has a good design.
In the above example, it is not enough for the system to be able to retrieve all the aspects mentioned in the text and their related sentiment (i.e. price:positive, design:positive). It’s important that the system is able to only select the pair price:positive, because it is the one related to the product under examination. The pair design:positive should not be selected since it is referring to a different entity.
From a more general point of view, the machine needs to be able to distinguish the “target” (the main product/service of the current analysis) from other entities.
To cope with this kind of challenge, Wonderflow’s NLP engine is provided with the indication of entities that can be mentioned in the text but that is actually not relevant for the analysis (in the aforementioned example, “the one my brother purchased”).
3. When sentiment is rating dependent
Sometimes customers write feedback without explicitly stating a sentiment in the text.
This occurs, for example, in surveys where companies ask the customer to fill in a questionnaire to rate a product/service and also write a comment.
Consider the example below. The customer here answers by stating the aspect s/he liked/disliked without any textual element that can help identify the sentiment of that string of text:
Question 1: How would you rate our service?
Rating Answer: “10”
Question 2: Motivate your answer.
Textual Answer: “Customer care”
The customer is answering the question by mentioning a relevant characteristic of the service, but without expressing the orientation of the opinion in the main text.
Therefore, considering just the textual answer, it’s not possible to determine how they feel about that aspect.
However, this kind of limitation can be handled by taking into account other information provided in the feedback, which is not necessarily verbal. This is the case of texts paired with a rating.
The rating represents an overall sentiment towards the product/service, a score usually ranging from 1 to 10. 1 is the “most negative” and 10 is the “most positive” sentiment. If the rating is encoded, it is possible to retrieve more complete information. Going back to the example above, the short answer “Customer care” is associated with a rating equal to 10. We can combine the text information and the rating, and deduce that the writer appreciates customer care. If the rating had been “3”, we would have interpreted the feedback as a negative one.
At Wonderflow, we can manage this kind of feedback by linking structured feedback texts without sentiment to the rating provided by the customer. Thus encoding the sentiment rating and providing a complete analysis of the feedback.
4. When sentiment is not as it looks!
So far, we have seen examples of feedback texts in which the sentiment is implicitly conveyed (and retrieved, thanks to our knowledge of the product), explicitly conveyed (but sometimes does not refer to our target), or totally absent (and conveyed thanks to non-textual information).
Let’s now consider another interesting scenario where the sentiment is conveyed explicitly, but should not be taken literally.
We are mainly referring to the use of irony, where the real sentiment is opposite to the one conveyed at the textual level. For example:
This toothbrush broke after 3 days! Really fantastic!
It’s great if you are searching for skin irritation!
Although the expressions “Really fantastic” and “It’s great” are clearly positive per se, the context suggests that the customer wants to convey the opposite sentiment, thus giving negative feedback.
These cases are strongly context dependent, and human interpretation is fundamental.
Although a large-scale solution is not applicable to identify ironic linguistic phenomena, at Wonderflow we put constant effort into checking samples where the rating does not match the identified sentiment at the textual level since there is a high probability of detecting such phenomena.
For instance, we manually check low rating reviews (i.e., reviews with an overall negative sentiment) in which we initially automatically identified a positive sentiment. By doing so, we spot cases of irony and correct them.
This manual fine-tuning step that we perform on samples allows us to retrieve the real sentiment conveyed in the customer feedback.
