Dongyue Li

I am a Ph.D. Student in the Khoury College of Computer Sciences at Northeastern University starting from Fall 2021. I am fortunate to work with Professor Hongyang R. Zhang. I am honored to receive the 2024 JP Morgan Chase PhD Fellowship.

In my research, I have been designing scalable approaches for training large models on multiple data sources, such as the training and fine-tuning of foundation models. I also have been developing robust fine-tuning algorithms to adapt pretrained models to downstream applications, based on the Hessians of large neural networks.

I obtained my bachelor's degree in Computer Science from Shanghai Jiao Tong University and minored in Mathematics and Applied Mathematics.

Email  /  Google Scholar  /  Github

Multitask Learning

• Scalable Multitask Learning Using Gradient-based Estimation of Task Affinity
  Dongyue Li, Aneesh Sharma, Hongyang R. Zhang
  SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2024
  [Code][Video]

• Boosting Multitask Learning on Graphs through Higher-Order Task Affinities
  Dongyue Li, Haotian Ju, Aneesh Sharma, Hongyang R. Zhang
  SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2023
  [Code]

• Identification of Negative Transfers in Multitask Learning Using Surrogate Models
  Dongyue Li, Huy L. Nguyen, Hongyang R. Zhang
  Transactions on Machine Learning Research (TMLR), 2023 (Featured Certification)
  [Code] [Presented in NeurIPS Workshop on Distribution Shifts (DistShift), 2022]

Fine-tuning

• Scalable Fine-tuning From Multiple Data Sources: A First-Order Approximation Approach
  Dongyue Li*, Ziniu Zhang*, Lu Wang, Hongyang R. Zhang
  Empirical Methods in Natural Language Processing (EMNLP Findings), 2024
  [Code]

• Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees
  Haotian Ju*, Dongyue Li*, Hongyang R. Zhang
  International Conference on Machine Learning (ICML), 2022
  [Code] [Presented in ICML Workshop on Updatable Machine Learning (UpML), 2022]

• Improved Regularization and Robustness for Fine-tuning in Neural Networks
  Dongyue Li, Hongyang R. Zhang
  Advances in Neural Information Processing Systems (NeurIPS), 2021
  [Code]

Generalization

• Noise Stability Optimization for Finding Flat Minima: A Hessian-based Regularization Approach
  Hongyang R. Zhang, Dongyue Li, Haotian Ju
  Transactions on Machine Learning Research (TMLR), 2024
  [Code]

• Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusion
  Haotian Ju, Dongyue Li, Aneesh Sharma, Hongyang R. Zhang
  International Conference on Artificial Intelligence and Statistics (AISTATS), 2023
  [Code]

Data Augmentation, Transportation Networks, Mobility Networks

• Learning Tree-Structured Composition of Data Augmentation
  Dongyue Li, Kailai Chen, Predrag Radivojac, Hongyang R. Zhang
  Transactions on Machine Learning Research (TMLR), 2024
  [Code]

• Graph Neural Networks for Road Safety Modeling: Datasets and Evaluations for Accident Analysis
  Abhinav Nippani*, Dongyue Li*, Haotian Ju, Haris N. Koutsopoulos, Hongyang R. Zhang
  Advances in Neural Information Processing Systems (NeurIPS), 2023, Datasets and Benchmarks Track
  [ML4RoadSafety Package]

• Optimal Intervention on Weighted Networks via Edge Centrality
  Dongyue Li, Tina Eliassi-Rad, Hongyang R. Zhang
  SIAM International Conference on Data Mining (SDM), 2023
  [Code] [Presented in KDD Workshop on Epidemiology meets Data Mining and Knowledge Discovery (epiDAMIK), 2022]

Asterisk(*) indicates equal contribution

Applied Research Intern
Capital One, New York, Jun. 2024 to Aug. 2024
Research Topics: Customer behavior modeling, pretraining transformers for tabular time series data, diffusion Models
Mentors: Nam Nguyen, Gang Mei, Sam Sharpe, Senthil Kumar, and Bayan Bruss

Full-time Researcher
Shanghai Qi Zhi Institute, Shanghai, Aug. 2020 to Jun. 2021
Research Topics: Model compression and system co-design
Mentors: Li Jiang

Research Intern
DiDi Chuxing AI Labs, Beijing, Jul. 2019 to Sep. 2019
Research Topics: Spatio-temporal data mining of traffic networks
Mentors: Yan Liu

Credit to this webpage template