New AI Model Detects Sarcasm With 86 Percent Accuracy
Researchers in China created an AI language model capable of identifying sarcasm with 86% accuracy.
If “crowdsourcing sarcasm” feels like a Twitter bio, then it’s not hard to understand how crucial sarcasm is to modern language — which is why researchers in China have developed an AI capable of detecting sarcasm, according to a recent paper published on the online journal ACL Web, which is totally fine.
However, to say whether our enthusiasm is sarcastic would be telling.
The researchers in China claim they’ve developed a state-of-the-art sarcasm detection AI — which performed wonderfully on a dataset pulled from Twitter.
Sarcasm detection can help in the analysis of sentiment, and enable scientists to build crowdsourced comprehension of public attitudes on various subjects, the researchers argue. Of course, this isn’t exactly a new idea — earlier this year, Facebook began using multimodal AI to identify memes deemed in violation of the social media company’s terms of service.
The sarcasm-detecting AI hones in on differences between text and imagery, then combines the results to generate predictions. It even compares hashtags to tweet text to help assess the subtextual contrast toward which the user is aiming.
“Particularly, the input tokens will give high attention values to the image regions contradicting them, as incongruity is a key character of sarcasm,” reads the recent paper. “As the incongruity might only appear within the text (e.g., a sarcastic text associated with an unrelated image), it is necessary to consider the intra modality incongruity.”
Using a Twitter-based dataset, the AI model scored 2.74% better on sarcasm detection F1 score, compared to HFM — a different multimodal detection model initially introduced in 2019. The new model from China also reached an 86% accuracy rate, higher than the HFM’s 83%.
The new paper came jointly from the Institute of Information Engineering and the Chinese Academy of Sciences — both based in Beijing, China. The results were also presented this week during the virtual Empirical Methods in Natural Language Processing (EMNLP) conference, Venture Beat reports.
This study’s AI is the latest and one of the most advanced examples of multimodal sarcasm detection to hit the world stage since AI researchers began initial attempts to apply multimodal strategies to content on Tumblr, Instagram, and Twitter in 2016.
Another paper from the University of Singapore and the University of Michigan — titled “Towards Multimodal Sarcasm Detection (An Obviously Perfect Paper)” — used language models combined with computer vision to identify sarcasm in or on television shows. That work was recognized as part of the Association for Computational Linguistics (ACL) in 2019.
As social media companies continue to monitor, moderate, and modify the modern commons we call social media — it may feel like there will soon be no way to express things freely in any public forum as machines (in this case, AI language models) categorize our public speech as okay to say or not, like a digital panopticon. But we’re not there yet, and humans — as the most interesting users of language in the world — will continue to find creative ways of outwitting the algorithms.
Source:https://interestingengineering.com
