Publications

QSpec: Speculative Decoding with Complementary Quantization Schemes
Quantization has been substantially adopted to accelerate inference and reduce memory consumption of large language models (LLMs). …
Juntao Zhao, Wenhao Lu, Sheng Wang, Lingpeng Kong, Chuan Wu
PDF 引用
QSpec: Speculative Decoding with Complementary Quantization Schemes
ProReason: Multi-Modal Proactive Reasoning with Decoupled Eyesight and Wisdom
Large vision-language models (LVLMs) have witnessed significant progress on visual understanding tasks. However, they often prioritize …
Jingqi Zhou, Sheng Wang, Jingwei Dong, Lei Li, Jiahui Gao, Lingpeng Kong, Chuan Wu
PDF 引用
ProReason: Multi-Modal Proactive Reasoning with Decoupled Eyesight and Wisdom
Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Exploration
Large language models (LLMs) have significantly benefited from training on diverse, high-quality task-specific data, leading to …
Qintong Li, Jiahui Gao, Sheng Wang, Renjie Pi, Xueliang Zhao, Chuan Wu, Xin Jiang, Zhenguo Li, Lingpeng Kong
PDF 引用
Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Exploration
MoS: Unleashing Parameter Efficiency of Low-Rank Adaptation with Mixture of Shards
The rapid scaling of large language models necessitates more lightweight finetuning methods to reduce the explosive GPU memory overhead …
Sheng Wang, Liheng Chen, Pengan Chen, Jingwei Dong, Boyang Xue, Jiyue Jiang, Lingpeng Kong, Chuan Wu
PDF 引用
MoS: Unleashing Parameter Efficiency of Low-Rank Adaptation with Mixture of Shards
How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models
The rapid evolution of large language models (LLMs) has transformed the competitive landscape in natural language processing (NLP), …
Jiyue Jiang, Pengan Chen, Liheng Chen, Sheng Wang, Qinghang Bao, Lingpeng Kong, Yu Li, Chuan Wu
PDF 引用
How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models
PRoLoRA: Partial Rotation Empowers More Parameter-Efficient LoRA
With the rapid scaling of large language models (LLMs), serving numerous LoRAs concurrently has become increasingly impractical, …
Sheng Wang, Boyang Xue, Jiacheng Ye, Jiyue Jiang, Liheng Chen, Lingpeng Kong, Chuan Wu
PDF 引用
PRoLoRA: Partial Rotation Empowers More Parameter-Efficient LoRA
LoRA Meets Dropout under a Unified Framework
With the remarkable capabilities, large language models (LLMs) have emerged as essential elements in numerous NLP applications, while …
Sheng Wang, Liheng Chen, Jiyue Jiang, Boyang Xue, Lingpeng Kong, Chuan Wu
PDF 引用
LoRA Meets Dropout under a Unified Framework
A Cognitive Stimulation Therapy Dialogue System with Multi-Source Knowledge Fusion for Elders with Cognitive Impairment
When communicating with elders with cognitive impairment, cognitive stimulation (CS) help to maintain the cognitive health of elders. …
Jiyue Jiang, Sheng Wang, Qintong Li, Lingpeng Kong, Chuan Wu
PDF 引用
A Cognitive Stimulation Therapy Dialogue System with Multi-Source Knowledge Fusion for Elders with Cognitive Impairment