I am a Junior Research Fellow starting from October 2024 at Trinity College, University of Cambridge. Before that, I worked with Prof Phil Woodland as a research associate at the Machine Intelligence Laboratory, the University of Cambridge. I am also closely collaborating with Prof. Mark Gales at the University of Cambridge and Prof Chao Zhang at Tsinghua University. My research interest is controllable and reliable audio-visual conversational AI with large language models. Specifically, this includes audio-visual contextual knowledge integration and editing, hallucination, and multimodal contextualised AI safety. My research interest and experience also include speaker representation, speech synthesis and source separation.

I completed my Ph.D. in June 2023. My PhD at the University of Cambridge supervised by Prof. Phil Woodland (advisor Prof Mark Gales) on contextual knowledge integration in end-to-end neural-based conversational AI systems. I held a research internship at Google Brain with Dr Yu Zhang in 2019 and ByteDance with Dr Wei Li in 2023. I was also grateful to have collaborated with Poly AI Ltd working with Dr Ivan Vulić and Dr Paweł Budzianowski in 2023. I received by BA and MEng degree in 2019 at Trinity College, University of Cambridge.

My goal is to make multimodal large language models (LLMs) that understand audio and video inputs, such as OpenAI GPT-4o and Google Gemini series, more reliable and controllable. Specifically, my ongoing research investigates the following aspects of multimodal LLMs:

1. Chain-of-thought reasoning of LLMs for general video understanding tasks: To solve a question about a video clip, the model should not only give the answer but also explain how this answer is derived step-by-step. This explanation provides more transparency to the model behaviour and hence enables us to better answer questions like “Why does it do this” or “What goes wrong” about the model,

2. Knowledge integration and removal in multimodal LLMs: In addition to the reasoning process, I also want to control what knowledge the model has by explicitly integrating knowledge into the model, or removing any knowledge for reasons such as privacy concerns or controversial issues.

3. Other safety and reliability issues, including hallucination detection in LLMs with audio and video inputs and contextualised LLM safety evaluation.