Multilingual Structured Sentiment Analysis for Environmental Sustainability

Abstract

To effectively address global environmental challenges, we must have tools that allow us to carefully monitor how citizens, policy makers and other stakeholders debate sustainability. However, there are currently very few NLP resources and tools specialized for this topic. This paper presents EnviS, a multilingual corpus (Italian, English, and Indonesian) for investigating the debate on environmental sustainability in social media using Structured Sentiment Analysis. We introduce a framework for the automatic aggregation of span-level annotations that preserves the annotators’ perspective and avoids manual intervention by safeguarding the quality of the annotations. We performed a series of experiments with four open-source instruction-based Large Language Models in zero-shot and few-shot settings, where we have measures the impact of the order and number of shots. The results further confirm the ineffectiveness of LLMs in extracting fine-grained sentiment information, being outperformed by a supervised state-of-the-art neural method trained on very few data. This questions the suitability of LLMs for rich knowledge/information extraction tasks requiring manipulation of text spans. In particular, our error analysis indicates that LLMs mostly struggle in identifying the sentiment term or its associated polarity, failing to extract full sentiment triples.

Resources

Details