مجلة آمنير للدراسات والبحوث
Volume 3, Numéro 1, Pages 79-89
2026-07-26

Defining And Measuring Quality In Automatically Generated Language: A Systematic Review



Authors : Tahri Abderrahmen .

Abstract

The rapid proliferation of natural language generation (NLG) systems has intensified the need for robust frameworks to define and assess the quality of their content. This systematic review synthesizes research published between 2018 and 2023 to examine how quality is conceptualized and operationalized across key NLG tasks, including machine translation, summarization, dialogue systems and creative text generation. A total of 87 peer-reviewed studies were analysed, revealing three dominant evaluation paradigms: human-centred approaches (e.g., fluency, adequacy, and coherence), automated metrics (e.g., BLEU and BERTScore), and task-specific composite measures. The findings highlight persistent discrepancies between human judgments and automated scores, an overreliance on surface-level metrics, and a lack of standardized taxonomies that incorporate domain-specific and ethical dimensions, such as factual consistency and bias. In response, this review proposes a multilayered evaluation framework that integrates quantitative metrics with qualitative human assessment. This study provides a foundation for developing more reliable and comprehensive quality assurance practices for applied natural language processing systems.

Keywords

Natural language generation ; Evaluation metrics ; Content quality assessment ; Human–AI collaboration ; Systematic review

Les articles similaires

Ai-generated Animated Visuals And Their Effect On Efl Learning Outcomes: A Systematic Review With Pedagogical Implications For Practice

Djekidel Oumaima Nour Elhouda .  Rogti Maroua . 
pages 257-278.

Early Implementation Of Ai-generated Art In Tesol: An Exploration Of Ai-generated Art For Language Acquisition

Moody Morgan .  Lopez Victor .  Johnson Phillip . 
pages 209-228.

Code-switching In The Middle School Ef Classes: The Case Study Of The First Year Learners Must It Be Perceived As An Asset To Ease Or As A Constraint To Obstruct Foreign Language Learning? (1) 1 - Efl: English As A Foreign Language /ell: English Language Learning/ 1fl1: Foreign Language 1/fl2: Foreign Language 2 / L: Learner/ Ls: Learners/ Mt: Mother Tongue/ T: Teacher/ Tie: Technological Institute Of Education/ Tl: Target Language.

Ammar Benabed .  Victor Segalen . 
pages 13-30.

إثارة القاضي تلقائيا لعدم دستورية القوانين بين الحظر والجواز The Automatically Rising By The Judge Of Unconstitutionality Of The Laws Between Prohibition And Permissibility

مرابط حسان . 
ص 90-109.

Total Quality Management In Higher Education Institutions: A Systematic Literature Review

Dieb Omar .  Boumedien Youcef . 
pages 364-378.

Formats de citation