Sheng Li

Hi! I am Sheng Li. I am currently a Ph.D. student in Department of Computer Science at the University of Pittsburgh, advised by Dr. Xulong Tang. I received my B.E. degree in Software Engineering from Sichuan University, China, in 2020. My research focuses on designing and optimizing efficient machine learning algorithms and systems, spanning applications in diffusion-based video generation, self-supervised contrastive learning, and supervised model training. I also work on 3D-related algorithms and systems, such as efficient 3D Gaussian Splatting. My research targets diverse platforms, ranging from powerful cloud clusters to resource-constrained edge devices.

News

05/2025	Joining Adobe Research as a Research Scientist/Engineer Intern!
01/2025	One paper is accepted to ICLR 2025. Thanks to all collaborators!
09/2024	One paper is accepted to ASP-DAC 2025. Thanks to all collaborators!
01/2024	One paper is accepted to ICLR 2024. Thanks to all collaborators!
01/2023	One paper is accepted to ICLR 2023 (with Spotlight award). Thanks to all collaborators!

Selected Publications

ICLR

Mutual Effort for Efficiency: A Similarity-based Token Pruning for Vision Transformers in Self-Supervised Learning

Sheng Li^*, Qitao Tan^*, Yue Dai, Zhenglun Kong, Tianyu Wang, Jun Liu, Ao Li, Ninghao Liu, Yufei Ding, Xulong Tang, and Geng Yuan

In The Thirteenth International Conference on Learning Representations, 2025

Bib HTML

@inproceedings{li2025mutual,
  title = {Mutual Effort for Efficiency: A Similarity-based Token Pruning for Vision Transformers in Self-Supervised Learning},
  author = {Li, Sheng and Tan, Qitao and Dai, Yue and Kong, Zhenglun and Wang, Tianyu and Liu, Jun and Li, Ao and Liu, Ninghao and Ding, Yufei and Tang, Xulong and Yuan, Geng},
  booktitle = {The Thirteenth International Conference on Learning Representations},
  year = {2025},
  url = {https://openreview.net/forum?id=GTcEe5fayC}
}

ASP-DAC

A Computation and Energy Efficient Hardware Architecture for SSL Acceleration

Huidong Ji, Sheng Li, Yue Cao, Chen Ding, Jiawei Xu, Qitao Tan, Ao Li, Jun Liu, Xulong Tang, Lirong Zheng, Geng Yuan, and Zhuo Zou

In 30th Asia and South Pacific Design Automation Conference, 2025

Bib HTML

@inproceedings{ji2025acomp,
  title = {A Computation and Energy Efficient Hardware Architecture for SSL Acceleration},
  author = {Ji, Huidong and Li, Sheng and Cao, Yue and Ding, Chen and Xu, Jiawei and Tan, Qitao and Li, Ao and Liu, Jun and Tang, Xulong and Zheng, Lirong and Yuan, Geng and Zou, Zhuo},
  booktitle = {30th Asia and South Pacific Design Automation Conference},
  pages = {23--29},
  year = {2025},
  url = {https://dl.acm.org/doi/10.1145/3658617.3697548}
}

ICLR

Waxing-and-Waning: a Generic Similarity-based Framework for Efficient Self-Supervised Learning

Sheng Li, Chao Wu, Ao Li, Yanzhi Wang, Xulong Tang, and Geng Yuan

In The Twelfth International Conference on Learning Representations, 2024

Bib HTML

@inproceedings{li2024waxingandwaning,
  title = {Waxing-and-Waning: a Generic Similarity-based Framework for Efficient Self-Supervised Learning},
  author = {Li, Sheng and Wu, Chao and Li, Ao and Wang, Yanzhi and Tang, Xulong and Yuan, Geng},
  booktitle = {The Twelfth International Conference on Learning Representations},
  year = {2024},
  url = {https://openreview.net/forum?id=TilcG5C8bN}
}

ICLR

SmartFRZ: An Efficient Training Framework using Attention-Based Layer Freezing

Sheng Li^*, Geng Yuan^*, Yue Dai^*, Youtao Zhang, Yanzhi Wang, and Xulong Tang

In The Eleventh International Conference on Learning Representations , 2023

Spotlight Bib HTML

@inproceedings{li2023smartfrz,
  title = {Smart{FRZ}: An Efficient Training Framework using Attention-Based Layer Freezing},
  author = {Li, Sheng and Yuan, Geng and Dai, Yue and Zhang, Youtao and Wang, Yanzhi and Tang, Xulong},
  booktitle = {The Eleventh International Conference on Learning Representations },
  year = {2023},
  url = {https://openreview.net/forum?id=i9UlAr1T_xl}
}

IEEE Micro

Sustainable AI processing at the edge

Sébastien Ollivier, Sheng Li, Yue Tang, Stephen Cahoon, Ryan Caginalp, Chayanika Chaudhuri, Peipei Zhou, Xulong Tang, Jingtong Hu, and Alex K Jones

IEEE Micro, 2022

Bib HTML

@article{ollivier2022sustainable,
  title = {Sustainable AI processing at the edge},
  author = {Ollivier, S{\'e}bastien and Li, Sheng and Tang, Yue and Cahoon, Stephen and Caginalp, Ryan and Chaudhuri, Chayanika and Zhou, Peipei and Tang, Xulong and Hu, Jingtong and Jones, Alex K},
  journal = {IEEE Micro},
  volume = {43},
  number = {1},
  pages = {19--28},
  year = {2022},
  publisher = {IEEE}
}

NeurIPS

Layer freezing & data sieving: missing pieces of a generic framework for sparse training

Geng Yuan^*, Yanyu Li^*, Sheng Li, Zhenglun Kong, Sergey Tulyakov, Xulong Tang, Yanzhi Wang, and Jian Ren

Advances in Neural Information Processing Systems, 2022

Bib HTML

@article{yuan2022layer,
  title = {Layer freezing \&amp; data sieving: missing pieces of a generic framework for sparse training},
  author = {Yuan, Geng and Li, Yanyu and Li, Sheng and Kong, Zhenglun and Tulyakov, Sergey and Tang, Xulong and Wang, Yanzhi and Ren, Jian},
  journal = {Advances in Neural Information Processing Systems},
  volume = {35},
  pages = {19061--19074},
  year = {2022},
  url = {https://proceedings.neurips.cc/paper_files/paper/2022/file/794a425a2e47e05d29d30f79b79a692d-Paper-Conference.pdf}
}