MixGradient: A gradient-based re-weighting scheme with mixup for imbalanced data streams

doi:10.1016/j.neunet.2023.02.017

CORC > 自动化研究所 > 中国科学院自动化研究所 > 多模态人工智能系统全国重点实验室

	MixGradient: A gradient-based re-weighting scheme with mixup for imbalanced data streams
	Peng, Xinyu 2; Wang, Fei-Yue 3; Li, Li 1,2
刊名	NEURAL NETWORKS
	2023-04-01
卷号	161 页码:525-534
关键词	Deep learning Imbalanced data streams Sample gradient Typical samples Mixup
ISSN号	0893-6080
DOI	10.1016/j.neunet.2023.02.017
通讯作者	Li, Li(li-li@tsinghua.edu.cn)
英文摘要	A challenge for contemporary deep neural networks in real-world problems is learning from an imbalanced data stream, where data tends to be received chunk by chunk over time, and the prior class distribution is severely imbalanced. Although many sophisticated algorithms have been derived, most of them overlook the importance of gradient information. From this perspective, the difficulty of learning from imbalanced data streams lies in the fact that the gradient estimated on an uneven class distribution is not informative enough to reflect the critical pattern of each class. To this end, we propose to assign higher weights on the training samples whose gradients are close to the gradient of corresponding typical samples, thus highlighting the important samples in minority classes and suppressing the noisy samples in majority classes. Such an idea can be combined with Mixup, which exploits the interpolation information of data to further compensate for the information of sample space that the typical samples do not provide and expand the role of the proposed re -weighting scheme. Experiments on artificially induced long-tailed CIFAR data streams and long-tailed MiniPlaces data stream show that the resulting method, termed MixGradient, boosts the generalization performance of DNNs under different imbalance ratios and achieves up to 10% accuracy improvement.(c) 2023 Elsevier Ltd. All rights reserved.
资助项目	National Key Research and Development Program of China[2020AAA0108104]
WOS关键词	EXTREME LEARNING-MACHINE ; NEURAL-NETWORKS
WOS研究方向	Computer Science ; Neurosciences & Neurology
语种	英语
出版者	PERGAMON-ELSEVIER SCIENCE LTD
WOS记录号	WOS:000990516400001
资助机构	National Key Research and Development Program of China
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/53311]
专题	多模态人工智能系统全国重点实验室
通讯作者	Li, Li
作者单位	1.Tsinghua Univ, Natl Lab Informat Sci & Technol, Beijing 100084, Peoples R China 2.Tsinghua Univ, Dept Automat, Beijing 100084, Peoples R China 3.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100080, Peoples R China
推荐引用方式 GB/T 7714	Peng, Xinyu,Wang, Fei-Yue,Li, Li. MixGradient: A gradient-based re-weighting scheme with mixup for imbalanced data streams[J]. NEURAL NETWORKS,2023,161:525-534.
APA	Peng, Xinyu,Wang, Fei-Yue,&Li, Li.(2023).MixGradient: A gradient-based re-weighting scheme with mixup for imbalanced data streams.NEURAL NETWORKS,161,525-534.
MLA	Peng, Xinyu,et al."MixGradient: A gradient-based re-weighting scheme with mixup for imbalanced data streams".NEURAL NETWORKS 161(2023):525-534.