Ruotong Liao

About Me

I am a Ph.D. student at LMU Munich supervised by Prof. Volker Tresp. I am an enthusiastic AI researcher in the field of Generative AI, foundational multi-modal models, and large language models, with a special focus on reasoning on temporal data such as videos, dynamic graphs, etc. Prior to my PhD, I finished my Bachelor’s at Beijing Institute of Technology (BIT) and my Master’s at Technical University of Munich (TUM) in Electrical Engineering. After that, I started my PhD program fully funded by Munich Center of Machine Learning (MCML), the German permanent AI center. I am awarded additional personal funding of 130,000 Euro from the German Ministry of Education and Research (BMBF) and German Aerospace Center (DLR), in collaboration with Huawei Munich Research Center aiming for supporting my research and cultivating me as the future tech lead.

Please reach out if you are interested in collaborations, master thesis, and research assistants!

Research Interests

• Generative AI: LLM Agents, Multimodal Reasoning
• Multi-modal Foundation Models: Visual-Language Models (VLMs), Video understanding & Generation
• Structured Graph Data: (Temporal) Knowledge graphs, Dynamic Graphs, Scene graphs

News

• [Dec. 2024] Invited Talk of Reasoning with LLMs with a Temporal View at Technical University of Dresden in Dresden.
• [Nov. 2024] I finished the Volkswagen Diversity & Inclusivity training in Wolfsburg.
• [Nov. 2024] I finished the HUAWEI Forsight-driven Innovation training in Munich.
• [Nov. 2024] Happy to be in Miami and present our work VideoINSTA at EMNLP2024.
• [Oct. 2024] Happy to be in Milan and present our work EchoScene at ECCV2024.
• [Sep. 2024] 🎉One paper is accepted to EMNLP 2024!
• [Jul. 2024] I finished the BOSCH Innovation & Entrepreneurship training in Stuttgart.
• [Jul. 2024] 🎉One paper is accepted to ECCV 2024!
• [Jul. 2024] 🎉Accepted to ELLIS Summer School of Collaborative and Generative AI (CoGenAI) in Helsinki!
• [Jun. 2024] We organized the ELLIS Workshop of Semantic, Symbolic and Interpretable Machine Learning (SSIML@ELLIS) together with ELISE Wrap Up Conference & ELLIS Community Event in Helsinki.
• [Jun. 2024] Happy to be in Mexico and present our work at NAACL2024.
• [Api. 2024] Joint project with Huawei Munich Research Center kicked off.
• [Mar. 2024] 🎉Two papers are accepted to NAACL2024!
• [Feb. 2024] Invited Talk of Foundation Models on Temporal Graphs at Semantic, Symbolic and Interpretable Machine Learning workshop hosted at the Oberwolfach Research Institute for Mathematics (MFO), Germany.
• [Dec. 2023] Invited Spotlight Talk of Generative Forcasting on Temporal Knowlege Graph at TGL@NeurIPS2023. See you in New Orleans!
• [Jul. 2023] 🎉Awarded with personal funding of 130,000 euro by BMBF and DLR to support my research and aim for future tech lead!
• [May. 2023] 🎉One paper is accepted as Finding of ACL 2023!

Selected Awards

• 2023 Funding Award of 130,000 euro by BMBF(German Federal Ministry of Education and Research) and DLR (German Aerospace Center).
• 2019 Full Scholarship for KSOP Master Programm at Karlsruhe Institute of Technology (KIT).
• 2019 Full Scholarship for Master Programm in Optics at Friedrich Schiller University Jena.
• 2018 Outstanding Student Leader as Vice Student President at Beijing Insitiute of Technology.
• 2018 Chinese National third Prize in the National Photo-electric Design Competition.
• 2017 Funding Award for National College Students’ Innovation and Entrepreneurship Programm.
• 2016 Outstanding Student for Undergraduates.
• 2015 Scholarship for First Prize of Excellent Students for Undergraduates.

Publications

Academic Services

Conference Reviewers

• The 2023-2024 Conference on Empirical Methods in Natural Language Processing (EMNLP)
• The 61st Annual Meeting of the Association for Computational Linguistics (ACL)

Journal Reviewers

• IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASL)

Workshop Reviewers

• Workshop on robustness of zero/few-shot learning in foundation models (R0-FoMo)

Workshop Committee

• ELLIS Workshop Semantic, Symbolic and Interpretable Machine Learning (SSIML@ELLIS)

Teaching & Supervision

Teaching

• Foundation Models on AI. Lecturer, Ludwig-Maximilians-University of Munich, WiSe2023
• Machine Learning with Knowledge Graphs. Lecturer, Ludwig-Maximilians-University of Munich, SoSe2023, WiSe2023, SoSe2024

Supervision

• Prompt Tuning for LLM-based Multi-Agent Systems . Master Thesis @ LMU Munich & MSG systems AG, WiSe2024
• Invstigating Temporal Reasoning on Large Language Models. Master Thesis @ Technical University of Munich (TUM), SoSe2023
• Video Understanding with Large Language Models (I). Research Intern @ Universit of Edingburgh, SoSe2023
• Video Understanding with Large Language Models (II). Master Thesis @ LMU Munich, WiSe2023
• Cross-domain and In-domain Generalizibility of Large Language Models on Temporal Data. Research Intern @ Technical University of Munich (TUM), WiSe2023
• Knowledge Graphs for Question Answering with Language Models. Master Thesis @ LMU Munich & Inovex GmbH, SoSe2023
• Knowledge Augmented Generation for Large Language Models. Master Thesis @ TUM, WiSe2022
• Temporal Knowledge Graph for Temporal Question Answering. Master Thesis @ LMU Munich & TUM, WiSe2022

Fun Facts

• I served as a coach for the National Olympiad in Informatics Programming for three years and was the co-founder of a startup for computer science education.
• I once led an organization committee for a concert for more than 3,000 people.
• I won the Fifth Prize in a video-making competition on Bilibili (Chinese YouTube).

Powered by Jekyll and Minimal Light theme.