Zongyu Lin

I am currently a member of Meta TBD lab. I co-led one model series of Muse Spark (🥑), which was landed in Meta AI Voice & Wearables. I was a CS Ph.D at UCLA (on-leave), co-advised by Prof. Yizhou Sun and Prof. Kaiwei Chang. I was the founding member in Kimi/Moonshot.AI, working as a research scientist working on LLM pretrain and long context. I was the major contributor of pre-training large language models with extremely long context, leading to KIMI CHAT, achieving the state-of-the-art performance on many long context tasks compared with GPT4 and Claude2 at that time. I completed my bachelor's degree of Electronic Enginnering at Tsinghua University. I was lucky to work with Zhilin Yang from 2021 to 2023. I contributed to Muse Spark 🥑 (26), Kimi-Linear (25), Kimi-K2 (25), Kimi-Long (23) GROOT-N1 (25) and DreamGen (25), etc.

Email: lzyxx17 [at] gmail.com

Twitter   Google Scholar  

My research interest lies broadly in natural language processing and general machine learning. Most Recently, I am interested in

(1) exploring scalable architectures and recipes for LLMs / VLMs and multi-modal generation;

(2) improving the self-evolution of LLMs / VLMs;

(3) improve better alignment with the physical world for vision generative models and vision language models

Interested in the combination of Inference time scaling + LLM Agent?🤖💭 Announcing QLASS (Q-guided Language Agent Stepwise Search), a framework that supercharges language agents at inference time. ⚡In this work, we build a process reward model to guide open language agents on complex interactive tasks by estimating the Q-value of each step, without any human annotation for process rewards.

Apple's latest recipe and studies for scalable video generation models🔥🔥🔥. In this work, we aim at providing a transparent and detailed recipe 📖 for model architecture, training strategy and data for scalable text-image conditioned video generation. The pretrained T2V and TI2V models outperform SOTA open-sourced, close-sourced models like Gen-3, PIKA, KLING and CogVideoX-5B on VBench.

We present VideoPhy, a benchmark designed to assess whether the generated videos follow physical commonsense for real-world activities (e.g. marbles will roll down when placed on a slanted surface). Specifically, we curate a list of 688 captions that involve interactions between various material types in the physical world (e.g., solid-solid, solid-fluid, fluid-fluid).

This research aims to comprehensively explore building a multimodal foundation model for egocentric video understanding, covering training data construction, model and evaluation benchmark.

We propose a novel sparsity-aware contrastive method to solve contradiction retrieval problem, outperforming traditional bi-encoder and cross encoder baselines, which can benefit fact checking and pretraining data filtering.

We train a generative judge (critique model) to improve visual programs of CodeLLMs.

We are the very pioneering work to study how to train the LLM as a reward model and study its zero-shot generalization on different NLP tasks.

We are the very pioneering work to study the task transfer between different NLP tasks using LLMs. We propose several methods to improve the generalization of LLMs on zero-shot tasks from a data-centric perspective.

We are the early work to do self-training on LLMs, and we propose a novel model-based feature to detect noisy labels in the self-training process.

Comprehensive Outstanding Scholarship(~10/280), Tsinghua University. 2020

Excellent Technology Innovation Scholarship, Tsinghua University. 2020

First Prize in Software Design Contest, Department of Electronic Enginnering, Tsinghua University. 2018

Teaching Assistant

• Natural Language Processing Seminar(Spring 2022). Teaching assistant. Instructor: Prof. Zhilin Yang. Tsinghua University

Reviewer

• 2025: ICML, ICLR, ACL, NeurIPS

Sports! I really enjoy playing ballgames like football and tennis. I am a big fan of Lionel Messi, Rafael Nadal and Stephen Curry! Also, I love running, swimming and hiking.