The scaling limit of high-dimensional online independent component analysis

doi:10.1088/1742-5468/ab39d6

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 模式分析与学习团队

	The scaling limit of high-dimensional online independent component analysis
	Wang, Chuang 1,2; Lu, Yue M.2
刊名	JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT
	2019-12-01
卷号	2019 期号:12 页码:13
关键词	machine learning
ISSN号	1742-5468
DOI	10.1088/1742-5468/ab39d6
通讯作者	Wang, Chuang(wangchuang@ia.ac.cn)
英文摘要	We analyze the dynamics of an online algorithm for independent component analysis in the high-dimensional scaling limit. As the ambient dimension tends to infinity, and with proper time scaling, we show that the time-varying joint empirical measure of the target feature vector and the estimates provided by the algorithm will converge weakly to a deterministic measured-valued process that can be characterized as the unique solution of a nonlinear PDE. Numerical solutions of this PDE, which involves two spatial variables and one time variable, can be efficiently obtained. These solutions provide detailed information about the performance of the ICA algorithm, as many practical performance metrics are functionals of the joint empirical measures. Numerical simulations show that our asymptotic analysis is accurate even for moderate dimensions. In addition to providing a tool for understanding the performance of the algorithm, our PDE analysis also provides useful insight. In particular, in the high-dimensional limit, the original coupled dynamics associated with the algorithm will be asymptotically 'decoupled', with each coordinate independently solving a 1D effiective minimization problem via stochastic gradient descent. Exploiting this insight to design new algorithms for achieving optimal trade-offs between computational and statistical efficiency may prove an interesting line of future research.
资助项目	US Army Research O.ce[W911NF-16-1-0265] ; US National Science Foundation[CCF-1319140] ; US National Science Foundation[CCF-1718698]
WOS关键词	SOLVABLE MODEL ; DYNAMICS
WOS研究方向	Mechanics ; Physics
语种	英语
出版者	IOP PUBLISHING LTD
WOS记录号	WOS:000510503800010
资助机构	US Army Research O.ce ; US National Science Foundation
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/28586]
专题	自动化研究所_模式识别国家重点实验室_模式分析与学习团队
通讯作者	Wang, Chuang
作者单位	1.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing, Peoples R China 2.Harvard Univ, John A Paulson Sch Engn & Appl Sci, 33 Oxford St, Cambridge, MA 02138 USA
推荐引用方式 GB/T 7714	Wang, Chuang,Lu, Yue M.. The scaling limit of high-dimensional online independent component analysis[J]. JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT,2019,2019(12):13.
APA	Wang, Chuang,&Lu, Yue M..(2019).The scaling limit of high-dimensional online independent component analysis.JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT,2019(12),13.
MLA	Wang, Chuang,et al."The scaling limit of high-dimensional online independent component analysis".JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2019.12(2019):13.