Introduction to Large Language Models

This comprehensive workshop guides participants through the technical foundations of large language models, beginning with neural networks and advancing to sophisticated concepts like transfer learning and instruction tuning. The curriculum covers essential prompting techniques and strategies, helping participants understand how to effectively communicate with language models. Participants learn about model alignment through RLHF and gain awareness of current limitations while exploring practical tools like retrieval and API integration. The workshop emphasizes hands-on learning, combining theoretical knowledge with practical exercises to ensure participants can apply their understanding in real-world scenarios.

This workshop provides a comprehensive introduction to large language models, starting with the fundamental concept of how neural networks process and generate text. Participants learn about the core principles of language models and how they leverage transfer learning to apply knowledge across different tasks. The workshop delves into practical prompting strategies, exploring various techniques and parameters that can be adjusted to optimize model outputs. Through hands-on exercises, attendees discover different prompting approaches and understand how instruction tuning enables models to better follow user directions. The curriculum covers the crucial role of Reinforcement Learning from Human Feedback (RLHF) in aligning language models with human values and preferences. Participants gain insights into the current limitations of large language models, fostering a realistic understanding of their capabilities and constraints. The workshop then transitions into the practical application of tools, emphasizing how retrieval mechanisms can enhance model responses with accurate citations. Participants learn how language models interact with other programs through APIs, expanding their potential applications. Throughout the workshop, theoretical concepts are reinforced through interactive exercises that give participants direct experience with the techniques they learn. The combination of theoretical background and practical application ensures participants gain both fundamental understanding and hands-on skills.

Joel Niklaus:
 

Joel Niklaus is a Research Scientist at Harvey, where he focuses on developing and evaluating LLM systems in the legal domain. He also serves as a Lecturer at the Bern University of Applied Sciences, teaching continuous education courses on NLP. Prior to his current roles, Joel was an AI Resident at (Google) X, where he trained multi-billion parameter LLMs on hundreds of TPUs, achieving state-of-the-art performance on LegalBench. His experience also includes investigating efficient domain-specific pretraining approaches at Thomson Reuters Labs.

Joel’s academic journey led him to Stanford University, where he conducted research on LLMs in the legal domain under the supervision of Prof. Dan Ho and Prof. Percy Liang. He has served as an advisor to companies specializing in the applications of modern NLP to legal challenges and has led research projects for the Swiss Federal Supreme Court. With extensive experience in pretraining and finetuning LLMs for diverse tasks across various compute environments, his research primarily focuses on dataset curation to train and evaluate language models multilingually for the legal domain. His datasets have laid the groundwork for legal NLP in Switzerland.

Joel’s research has been published at leading Natural Language Processing and Machine Learning conferences. It has been covered by the Swiss National Radio & Television and honored with an Outstanding Paper Award at ACL. He holds a PhD in Natural Language Processing, a Master’s in Data Science, and a Bachelor’s in Computer Science from the University of Bern.

weniger anzeigen