LLM-driven Instruction Following: 

Progresses & Concerns

(Dec. 6, 2023; EMNLP'23 Tutorial)

Contact: wenpeng@psu.edu


The progress of classic NLP is primarily driven by machine learning that optimizes a system on a large-scale set of task-specific labeled examples. This learning paradigm limits the ability of machines to have the same capabilities as humans in handling new tasks since humans can often solve unseen tasks with a couple of examples accompanied by task instructions. In addition, we may not have a chance to prepare task-specific examples of large-volume for new tasks because we cannot foresee what task needs to be addressed next and how complex to annotate for it. Therefore, task instructions act as a novel and promising resource for supervision. This tutorial presents a diverse thread of instruction-driven NLP studies that try to answer the following questions: (i) What is task instruction? (ii) How is the process of creating datasets and evaluating systems conducted? (iii) How to encode task instructions? (iv) When and why do some instructions work better? (v) What concerns remain in LLM-driven instruction following? We will discuss several lines of frontier research that tackle those challenges and will conclude the tutorial by outlining directions for further investigation.

Hinrich Schütze

Introduction (Slides)

Wenpeng Yin

Datasets & Evaluations for Instruction Following (Slides)

Qinyuan Ye

Xiang Ren

Methodology---Prompting (Slides)

Pengfei Liu

Methodology---Alignment in LLMs (Slides)

Wenpeng Yin

Challenges & Issues in Instruction Following (Slides)

Hinrich Schütze

Conclusion & Future Directions (Slides)

               Overall slides             Recording