Large Language Models (llms) In Audiovisual Transcription And Translation: A Case Study Of Whisper
Auteurs : Qian Duoxiu .
Résumé
Following the breakthrough with the transformer architecture in 2017 and the subsequent release of generative models such as GPT-3 and ChatGPT, modern Large Language Models (LLMs) have reshaped almost every field of study. By enabling neural networks trained on massive datasets, LLMs have also informed research and supported professional practice in unprecedented ways. In audiovisual (AV) transcription and translation training and practice, LLMs have been increasingly deployed to provide substantial assistance and streamline the entire workflow in delivering AV products in other languages. However, research addressing the pedagogical implications of these developments remains relatively limited. This paper aims to examine the transformative role of LLMs in the process of AV transcription and translation, using the most recent functions in Subtitle Edit as an example. This versatile tool integrates multiple LLMs to support AV production and translation, including Whisper for speech to text transcription and Cantonese/Chinese-English translation, as well as advanced AI engines that help to generate automatic translation through APIs. The paper mainly compares the performances of two LLMs (one small and one big) in Whisper in transcribing from Cantonese/Chinese to English. It also illustrates how cloud based or locally deployed LLM-supported systems can be employed for translation, either via APIs or through direct system integration. Based on the demonstration, it becomes evident that LLMs have had a profound impact on traditional audiovisual (AV) teaching and professional practice. The author is able to propose a human-in-the-loop pipeline relating to LLM-supported AV transcription and translation and suggests that scholars and educators in this field should recognize the importance of embracing emerging technologies and leveraging freely available applications as they arise so as to address practical challenges more effectively in AVT education.
Mots clés
audiovisual ; LLMs ; transcription ; translation ; Whisper ; Subtitle Edit
