李东｜西安外国语大学｜学者主页

2026

Universal YOCO for Efficient Depth Scaling

Yutao Sun, Li Dong, Tianzhu Ye, Shaohan Huang, Jianyong Wang, Furu Wei

arXiv preprint

2026

LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks

Tianzhu Ye, Li Dong, Guanheng Chen, He Zhu, Xun Wu, Shaohan Huang, Furu Wei

arXiv preprint

2026

You Only Index Once: Cross-Layer Sparse Attention with Shared Routing

Yutao Sun, Yanqi Zhang, Li Dong, Jianyong Wang, Furu Wei

arXiv preprint

2026

Online Experiential Learning for Language Models

Tianzhu Ye, Li Dong, Qingxiu Dong, Xun Wu, Shaohan Huang, Furu Wei

arXiv preprint

2026

Can Machines Really See Objects in Images? A Study Based on Syntactic Distance and Visual Self-Referential Instances

Xingyu Peng, Junran Wu, Yue Hou, Zhongliang Qiao, Jiaheng Liu, Shangzhe Li, Jichang Zhao, Wenjun Wu, Xianglong Liu, Yongxin Tong, Li Dong, Ke Xu

arXiv preprint

2025

Reward Reasoning Model

Jiaxin Guo, Zewen Chi, Li Dong, Qingxiu Dong, Xun Wu, Shaohan Huang, Furu Wei

arXiv preprint

2025

VibeVoice Technical Report

Zhiliang Peng, Jianwei Yu, Wenhui Wang, Yaoyao Chang, Yutao Sun, Li Dong, Yi Zhu, Weijiang Xu, Hangbo Bao, Zehua Wang, Shaohan Huang, Yan Xia, Furu Wei

arXiv preprint

2025

On-Policy RL with Optimal Reward Baseline

Yaru Hao, Li Dong, Xun Wu, Shaohan Huang, Zewen Chi, Furu Wei

arXiv preprint

2025

Black-Box On-Policy Distillation of Large Language Models

Tianzhu Ye, Li Dong, Zewen Chi, Xun Wu, Shaohan Huang, Furu Wei

arXiv preprint

2025

Reinforcement Pre-Training

Qingxiu Dong, Li Dong, Yao Tang, Tianzhu Ye, Yutao Sun, Zhifang Sui, Furu Wei

arXiv preprint

2024

Towards Optimal Learning of Language Models

Yuxian Gu, Li Dong, Yaru Hao, Qingxiu Dong, Minlie Huang, Furu Wei

arXiv preprint

2024

Multimodal Latent Language Modeling with Next-Token Diffusion

Yutao Sun, Hangbo Bao, Wenhui Wang, Zhiliang Peng, Li Dong, Shaohan Huang, Jianyong Wang, Furu Wei

arXiv preprint

2024

You Only Cache Once: Decoder-Decoder Architectures for Language Models

Yutao Sun, Li Dong, Yi Zhu, Shaohan Huang, Wenhui Wang, Shuming Ma, Quanlu Zhang, Jianyong Wang, Furu Wei

arXiv preprint

2024

Differential Transformer

Tianzhu Ye, Li Dong, Yuqing Xia, Yutao Sun, Yi Zhu, Gao Huang, Furu Wei

arXiv preprint

2023

Large Language Model for Science: A Study on P vs. NP

Qingxiu Dong, Li Dong, Ke Xu, Guangyan Zhou, Yaru Hao, Zhifang Sui, Furu Wei

arXiv preprint

2023

KOSMOS-2.5: A Multimodal Literate Model

Tengchao Lv, Yupan Huang, Jingye Chen, Yuzhong Zhao, Yilin Jia, Lei Cui, Shuming Ma, Yaoyao Chang, Shaohan Huang, Wenhui Wang, Li Dong, Weiyao Luo, Shaoxiang Wu, Guoxin Wang, Cha Zhang, Furu Wei

arXiv preprint

2023

Retentive Network: A Successor to Transformer for Large Language Models

Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, Furu Wei

arXiv preprint

2023

Kosmos-2: Grounding Multimodal Large Language Models to the World

Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Furu Wei

arXiv preprint

2023

Language Is Not All You Need: Aligning Perception with Language Models

Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, Qiang Liu, Kriti Aggarwal, Zewen Chi, Johan Bjorck, Vishrav Chaudhary, Subhojit Som, Xia Song, Furu Wei

arXiv preprint

2023

Augmenting Language Models with Long-Term Memory

Weizhi Wang, Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, Furu Wei

arXiv preprint

2022

Prototypical Calibration for Few-shot Learning of Language Models

Zhixiong Han, Yaru Hao, Li Dong, Yutao Sun, Furu Wei

arXiv preprint

2022

Visually-Augmented Language Modeling

Weizhi Wang, Li Dong, Hao Cheng, Haoyu Song, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, Furu Wei

arXiv preprint

2022

A Length-Extrapolatable Transformer

Yutao Sun, Li Dong, Barun Patra, Shuming Ma, Shaohan Huang, Alon Benhaim, Vishrav Chaudhary, Xia Song, Furu Wei

arXiv preprint

2022

Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks

Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhiliang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, Furu Wei

arXiv preprint

2022

BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers

Zhiliang Peng, Li Dong, Hangbo Bao, Qixiang Ye, Furu Wei

arXiv preprint

2022

A Unified View of Masked Image Modeling

Zhiliang Peng, Li Dong, Hangbo Bao, Qixiang Ye, Furu Wei

arXiv preprint

2022

Language Models as Inductive Reasoners

Zonglin Yang, Li Dong, Xinya Du, Hao Cheng, Erik Cambria, Xiaodong Liu, Jianfeng Gao, Furu Wei

arXiv preprint

2022

On the Representation Collapse of Sparse Mixture of Experts

Zewen Chi, Li Dong, Shaohan Huang, Damai Dai, Shuming Ma, Barun Patra, Saksham Singhal, Payal Bajaj, Xia Song, Xian-Ling Mao, Heyan Huang, Furu Wei

arXiv preprint

2022

VL-BEiT: Generative Vision-Language Pretraining

Hangbo Bao, Wenhui Wang, Li Dong, Furu Wei

arXiv preprint

2022

Language Models are General-Purpose Interfaces

Yaru Hao, Haoyu Song, Li Dong, Shaohan Huang, Zewen Chi, Wenhui Wang, Shuming Ma, Furu Wei

arXiv preprint

2022

StableMoE: Stable Routing Strategy for Mixture of Experts

Damai Dai, Li Dong, Shuming Ma, Bo Zheng, Zhifang Sui, Baobao Chang, Furu Wei

arXiv preprint

2022

Optimizing Prompts for Text-to-Image Generation

Yaru Hao, Zewen Chi, Li Dong, Furu Wei

arXiv preprint

2022

Structured Prompting: Scaling In-Context Learning to 1,000 Examples

Yaru Hao, Yutao Sun, Li Dong, Zhixiong Han, Yuxian Gu, Furu Wei

arXiv preprint

2022

Kformer: Knowledge Injection in Transformer Feed-Forward Layers

Yunzhi Yao, Shaohan Huang, Li Dong, Furu Wei, Huajun Chen, Ningyu Zhang

arXiv preprint

2022

DeepNet: Scaling Transformers to 1,000 Layers

Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Furu Wei

arXiv preprint

2022

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers

Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, Furu Wei

arXiv preprint

2022

Controllable Natural Language Generation with Contrastive Prefixes

Jing Qian, Li Dong, Yelong Shen, Furu Wei, Weizhu Chen

arXiv preprint

2022

Non-Contrastive Learning Meets Language-Image Pre-Training

Jinghao Zhou, Li Dong, Zhe Gan, Lijuan Wang, Furu Wei

arXiv preprint

2022

TorchScale: Transformers at Scale

Shuming Ma, Hongyu Wang, Shaohan Huang, Wenhui Wang, Zewen Chi, Li Dong, Alon Benhaim, Barun Patra, Vishrav Chaudhary, Xia Song, Furu Wei

arXiv preprint

2021

s2s-ft: Fine-Tuning Pretrained Transformer Encoders for Sequence-to-Sequence Learning

Hangbo Bao, Li Dong, Wenhui Wang, Nan Yang, Furu Wei

arXiv preprint

2021

XLM-E: Cross-lingual Language Model Pre-training via ELECTRA

Zewen Chi, Shaohan Huang, Li Dong, Shuming Ma, Bo Zheng, Saksham Singhal, Payal Bajaj, Xia Song, Xian-Ling Mao, Heyan Huang, Furu Wei

arXiv preprint

2021

Consistency Regularization for Cross-Lingual Fine-Tuning

Bo Zheng, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei

arXiv preprint

2021

BEiT: BERT Pre-Training of Image Transformers

Hangbo Bao, Li Dong, Songhao Piao, Furu Wei

arXiv preprint

2021

Knowledge Neurons in Pretrained Transformers

Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, Furu Wei

arXiv preprint

2021

Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-training

Bo Zheng, Li Dong, Shaohan Huang, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei

arXiv preprint

2021

VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts

Hangbo Bao, Wenhui Wang, Li Dong, Qiang Liu, Owais Khan Mohammed, Kriti Aggarwal, Subhojit Som, Furu Wei

arXiv preprint

2021

Learning to Sample Replacements for ELECTRA Pre-Training

Yaru Hao, Li Dong, Hangbo Bao, Ke Xu, Furu Wei

arXiv preprint

2021

Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment

Zewen Chi, Li Dong, Bo Zheng, Shaohan Huang, Xian-Ling Mao, Heyan Huang, Furu Wei

arXiv preprint

2021

MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs

Zewen Chi, Li Dong, Shuming Ma, Shaohan Huang Xian-Ling Mao, Heyan Huang, Furu Wei

arXiv preprint

2021

DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders

Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei

arXiv preprint

2020

Harvesting and Refining Question-Answer Pairs for Unsupervised QA

Zhongli Li, Wenhui Wang, Li Dong, Furu Wei, Ke Xu

arXiv preprint

2020

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training

Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou

arXiv preprint

2020

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers

Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, Ming Zhou

arXiv preprint

2020

Self-Attention Attribution: Interpreting Information Interactions Inside Transformer

Yaru Hao, Li Dong, Furu Wei, Ke Xu

arXiv preprint

2020

Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks

Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, Jianfeng Gao

arXiv preprint

2020

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training

Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

arXiv preprint

2019

Cross-Lingual Natural Language Generation via Pre-Training

Zewen Chi, Li Dong, Furu Wei, Wenhui Wang, Xian-Ling Mao, Heyan Huang

arXiv preprint

2019

Can Monolingual Pretrained Models Help Cross-Lingual Classification?

Zewen Chi, Li Dong, Furu Wei, Xian-Ling Mao, Heyan Huang

arXiv preprint

2019

Visualizing and Understanding the Effectiveness of BERT

Yaru Hao, Li Dong, Furu Wei, Ke Xu

arXiv preprint

2019

Unified Language Model Pre-training for Natural Language Understanding and Generation

Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

arXiv preprint

2019

Learning to Ask Unanswerable Questions for Machine Reading Comprehension

Haichao Zhu, Li Dong, Furu Wei, Wenhui Wang, Bing Qin, Ting Liu

arXiv preprint

2019

Learning A Unified Named Entity Tagger From Multiple Partially Annotated Corpora For Efficient Adaptation

Xiao Huang, Li Dong, Elizabeth Boschee, Nanyun Peng

arXiv preprint

2019

Transforming Wikipedia into Augmented Data for Query-Focused Summarization

Haichao Zhu, Li Dong, Furu Wei, Bing Qin, Ting Liu

arXiv preprint

2019

Data-to-text Generation with Entity Modeling

Ratish Puduppully, Li Dong, Mirella Lapata

arXiv preprint

2018

Coarse-to-Fine Decoding for Neural Semantic Parsing

Li Dong, Mirella Lapata

arXiv preprint

2018

Confidence Modeling for Neural Semantic Parsing

Li Dong, Chris Quirk, Mirella Lapata

arXiv preprint

2017

Learning to Paraphrase for Question Answering

Li Dong, Jonathan Mallinson, Siva Reddy, Mirella Lapata

arXiv preprint

2016

Language to Logical Form with Neural Attention

Li Dong, Mirella Lapata

arXiv preprint

2014

A Statistical Parsing Framework for Sentiment Classification

Li Dong, Furu Wei, Shujie Liu, Ming Zhou, Ke Xu

arXiv preprint

精选作品

期刊论文

Research on the Current Status of Joint Brand Marketing in Tourism Activities: A Factor Detection Analysis Based on 10 Prefecture-Level Cities in Shaanxi Province

旅游促进民族交往交流交融量表研究：基于目的地居民视角

预印本

Universal YOCO for Efficient Depth Scaling

LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks

You Only Index Once: Cross-Layer Sparse Attention with Shared Routing

Online Experiential Learning for Language Models

Can Machines Really See Objects in Images? A Study Based on Syntactic Distance and Visual Self-Referential Instances

Reward Reasoning Model

VibeVoice Technical Report

On-Policy RL with Optimal Reward Baseline

Black-Box On-Policy Distillation of Large Language Models

Reinforcement Pre-Training

Towards Optimal Learning of Language Models

Multimodal Latent Language Modeling with Next-Token Diffusion

You Only Cache Once: Decoder-Decoder Architectures for Language Models

Differential Transformer

Large Language Model for Science: A Study on P vs. NP

KOSMOS-2.5: A Multimodal Literate Model

Retentive Network: A Successor to Transformer for Large Language Models

Kosmos-2: Grounding Multimodal Large Language Models to the World

Language Is Not All You Need: Aligning Perception with Language Models

Augmenting Language Models with Long-Term Memory

Prototypical Calibration for Few-shot Learning of Language Models

Visually-Augmented Language Modeling

A Length-Extrapolatable Transformer

Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks

BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers

A Unified View of Masked Image Modeling

Language Models as Inductive Reasoners

On the Representation Collapse of Sparse Mixture of Experts

VL-BEiT: Generative Vision-Language Pretraining

Language Models are General-Purpose Interfaces

StableMoE: Stable Routing Strategy for Mixture of Experts

Optimizing Prompts for Text-to-Image Generation

Structured Prompting: Scaling In-Context Learning to 1,000 Examples

Kformer: Knowledge Injection in Transformer Feed-Forward Layers

DeepNet: Scaling Transformers to 1,000 Layers

Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers

Controllable Natural Language Generation with Contrastive Prefixes

Non-Contrastive Learning Meets Language-Image Pre-Training

TorchScale: Transformers at Scale

s2s-ft: Fine-Tuning Pretrained Transformer Encoders for Sequence-to-Sequence Learning

XLM-E: Cross-lingual Language Model Pre-training via ELECTRA

Consistency Regularization for Cross-Lingual Fine-Tuning

BEiT: BERT Pre-Training of Image Transformers

Knowledge Neurons in Pretrained Transformers

Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-training

VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts

Learning to Sample Replacements for ELECTRA Pre-Training

Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment

MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs

DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders

Harvesting and Refining Question-Answer Pairs for Unsupervised QA

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers

Self-Attention Attribution: Interpreting Information Interactions Inside Transformer

Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training

Cross-Lingual Natural Language Generation via Pre-Training

Can Monolingual Pretrained Models Help Cross-Lingual Classification?

Visualizing and Understanding the Effectiveness of BERT

Unified Language Model Pre-training for Natural Language Understanding and Generation

Learning to Ask Unanswerable Questions for Machine Reading Comprehension

Learning A Unified Named Entity Tagger From Multiple Partially Annotated Corpora For Efficient Adaptation

Transforming Wikipedia into Augmented Data for Query-Focused Summarization

Data-to-text Generation with Entity Modeling

Coarse-to-Fine Decoding for Neural Semantic Parsing

Confidence Modeling for Neural Semantic Parsing

Learning to Paraphrase for Question Answering

Language to Logical Form with Neural Attention

A Statistical Parsing Framework for Sentiment Classification