Parameter-Efficient Few-Shot Sentiment Analysis Using LoRA-Enhanced Transformers

Abstrak

Sentiment analysis in low-resource languages is often limited by scarce annotated data and the high computational cost of fine-tuning large language models. This study proposes a parameter-efficient framework that integrates Low-Rank Adaptation (LoRA) with lightweight transformer architectures, including AfriBERTa, DistilBERT, and MiniLMv2, for Hausa sentiment analysis using the NaijaSenti dataset. The framework is designed to address three key challenges: effective few-shot learning, robustness under extreme data scarcity, and mitigation of language-specific linguistic errors. Experimental results demonstrate that AfriBERTa-LoRA achieves 69.0% accuracy, only 4.8 percentage points below a fully fine-tuned XLM-RoBERTa baseline, while utilizing just 1.06% of trainable parameters and reducing GPU memory consumption by approximately 50%. Performance improves consistently with increasing data, indicating strong scalability under few-shot conditions. Linguistic error analysis reveals four dominant Hausa-specific failure modes accounting for 71.5% of misclassifications. Targeted mitigation strategies yield an 8.7 percentage point reduction in error rate (28% relative reduction, p < 0.01), with each individual strategy demonstrating statistical significance. These findings establish LoRA as an effective and efficient paradigm for low-resource natural language processing, providing a scalable and reproducible framework for sentiment analysis in underrepresented African languages.