LT-Bridge Winter School 2024

5 March

9:30 – 10:00 Registration
10:00 – 10:30 Welcome
10:30 – 12:00 Overview lecture: Conversational AI (Prof. Günter Neumann, DFKI)
12:00 – 13:00 Lunch break
13:00 – 14:30 Keynote: Prof. Milica Gašić (University Düsseldorf)
Continually learning Conversational AI (see abstract)Large language models have achieved impressive performance across the NLP task spectrum and even appear to have superhuman conversational capabilities. However, they suffer from hallucinations, lack of transparency, and limited ability to improve over time without needing full and rather expensive retraining. Modular dialogue systems, on the other hand, offer an interpretable underlying state and action and a set-up for long-term reward optimisation via reinforcement learning. This talk will explain the steps taken towards building a continually learning task-oriented dialogue system consisting of a dynamic policy model, a data-driven user simulator, and a challenging environment to study the ability of the system to learn in a world that is continuously changing.
14:30 – 15:00 Coffee break
15:00 – 15:30 Lightning presentations
15:30 – 17:00 Keynote: Mikel L. Forcada (University of Alicante)
Large, neural probabilistic language models (see abstract)Many call them “artificial intelligence“ but large language models (LLMs) such as ChatGPT or Google Bard are basically very large probabilistic language models that have been first trained to generate a continuation for a text prompt, and then fine-tuned to behave as conversational models providing acceptable, useful responses. Starting from traditional n-gram models, this talk will move on to neural LLMs. It will give the possibility to discuss their neural architecture, how text is represented in them, and how they are trained and fine-tuned. Finally, it will briefly present existing language models, discuss their availability and usage rights, and some of the ethical and environmental issues associated to training and using them.
17:00 – 19:00 Posters and Reception

6 March

10:00 – 10:30 Takeaways from poster session
10:30 – 12:00 Overview lecture: Low-Resource NLP (Dr. Simon Ostermann, DFKI)
12:00 – 13:00 Lunch break
13:00 – 14:30 Keynote: Prof. Jan Niehues (KIT)
Approaching the Babel Fish? Current research questions in Speech Translation (see abstract)Speech translation holds the transformative potential to realize the age-old aspiration of global, multilingual communication, reminiscent of the iconic Babel fish. Recent advancements in the field, driven by deep learning and self-supervised learning within foundation models, have brought this dream closer to reality, paving the way for an array of applications where seamless cross-language interaction is within reach. However, a variety of open questions persists. This talk offers a comprehensive overview on the current state-of-the-art in speech translation technology, delving into its capabilities and limitations. Emphasizing the quest for low-latency solutions, the challenge of accommodating multilingual diversity, and the challenges of integrating pre-trained models, we navigate through the complexities and opportunities that lie ahead.
14:30 – 15:00 Coffee break
15:00 – 17:30 Hands-on Lab: Low-Resource NLP with Adapters and Prompts (Dr. Simon Ostermann, Tatjana Anikina, DFKI)
18:30 Social Dinner: Restaurant Die Kartoffel, Saarbrücken

7 March

10:00 – 12:00 Overview lecture: Machine Translation (Prof. Josef van Genabith, DFKI)
12:00 – 13:00 Lunch break
13:00 – 14:30 Keynote: Mikel Artetxe (Reka)
Revisiting Cross-Lingual Transfer Learning (see abstract)Given downstream training data in one language (typically English), the goal of cross-lingual transfer learning is to perform the task in another language. Existing approaches have been broadly classified into 3 categories: zero-shot (fine-tune a multilingual language model in English and zero-shot transfer into the target language), translate-train (translate the training data into the target language through machine translation and fine-tune a multilingual language model), and translate-test (translate the evaluation data into English through machine translation and use an English model). In this talk, we will critically revisit some the fundamentals of this problem, with a special focus on the interaction between cross-lingual transfer learning and machine translation.
14:30 – 18:00 Excursion: Tour to the World Heritage Site Völklinger Hütte

8 March

10:00 – 12:00 Hands-on Lab: Machine Translation pt. 1 (Dr. Cristina España i Bonet, DFKI)
12:00 – 13:00 Lunch break
13:00 – 15:00 Hands-on Lab: Machine Translation pt. 2 (Dr. Cristina España i Bonet, DFKI)
15:00 – 15:30 Closing remarks