What is Sentiment Analysis?

Sentiment analysis, also known as opinion mining or emotion AI, is a field that uses natural language processing (NLP) to detect and interpret people’s attitudes, emotions and stances in text. It's widely applied to various materials like customer reviews, survey responses, social media, and healthcare data for purposes ranging from marketing to customer service and clinical medicine. 

The most common method detects whether a text expresses positive, negative or neutral sentiment (some methods dispense with neutral labelling under the assumption that neutral texts sit close to the binary of either positive or negative). A numerical score (e.g., -10 to +10 or 0 to +4) can then be assigned to words associated with negative, neutral, or positive sentiment, giving a more nuanced understanding of sentiment by adjusting scores based on a variety of contexts.

In practice, standard NLP algorithms treat language as a series of data points, where certain words provoke sentiment scores in isolation of the wider narrative. For example, if the input sentence contains the words “good”, “love” and “excellent” the algorithm will output a “positive” sentiment annotation. Conversely, if the input sentence contains the words “bad”, “love” and “terrible”, the algorithm will generate a “negative” output annotation. This is why most robust systems rely on supervised learning: humans annotate examples to create labelled data; a model is trained on that labelled data; the model then predicts sentiment for new, unlabeled texts based on the similarity of the input text to annotated examples in the training data. 

Key Realities of Sentiment Analysis

The accuracy of a sentiment analysis system is typically measured by how well it agrees with human judgments, but human raters often disagree (around 80% agreement). This means a system achieving 70% accuracy is performing quite well. However, computer systems make different errors to humans, struggling with negations, exaggerations, jokes, or sarcasm. Whilst sentiment analysis systems can roughly match human judgments, each implementation often requires a separate training model for accurate sentiment representation.

• Domain limitation: Sentiment analysis can be highly effective for specific task-based purposes, such as the effect of text on brand reputation. However, most APIs are optimised for product and marketing reviews, making them inappropriate for more context specific scenarios such as jokes and sarcasm, or to the analysis of texts with ideological or ethical contexts. 

• Black-box risk: Annotation schemas, training data, and methodological documentation is often not shared by vendors of sentiment analysis systems and customers have no reliable way of understanding how a sentiment score has been assigned. 

• Word-based NLP limitation: basing a sentiment score on the similarity of particular word use to training data may provide comparable sentiment accuracy results when compared to human raters, but what does it actually explain? Standard NLP algorithms treat words as singular data points, unlike humans who see words as parts of a wider grammatical pattern. As in the example above, an algorithm may interpret a sentence containing the words “bad”, “love” and “terrible” as negative, when a human will immediately understand the positive sentiment of the sentence: “It might have been considered a bad choice, terrible even, to bring a dog into our busy home, but the whole family fell in love with Bruno”. No matter how good the training data, if the algorithm computes sentiment on the basis of words in isolation rather than as part of a grammatical pattern, the outcomes will be questionable.

• No Standard Unit: There is no agreed meaning/scale for numerical scores across vendors of sentiment analysis systems making comparison across systems difficult. If certain sentences are rated positive under one system, but less positive under another, choosing a system is based on best case for purpose rather than on the overall merits of sentiment analysis.

• An absence of Sociological Theory. No widely adopted sociological theory of ‘sentiment’ requires the development of purpose built methodology, documentation and audit for responsible use.

Conclusion

Using sentiment analysis to understand ideologically or ethically sensitive texts gives unreliable results that are hard to compare across systems, or understand within systems given the lack of methodological transparency. Even with responsible, human intervention and transparency in methodology, without an agreed theory of ‘sentiment’, results remain limited.