Kernel-based online gradient descent using distributed approach

Xiaming Chen

doi:10.3934/mfc.2019001

Article Contents

2019, Volume 2, Issue 1: 1-9. Doi: 10.3934/mfc.2019001

This issue Previous Article Next Article Comparisons of different methods for balanced data classification under the discrete non-local total variational framework

Kernel-based online gradient descent using distributed approach

Xiaming Chen^,

Shantou University, No. 243 Daxue Rd., Shantou, Guangdong, China

^* Corresponding author: Xiaming Chen
^* Corresponding author: Xiaming Chen

Published: March 2019

The first author is supported by STU Scientific Research Foundation for Talents grant (NTF-18022).

Abstract / Introduction Full Text(HTML) Related Papers Cited by

Abstract

In this paper we study the kernel-based online gradient descent with least squares loss without an explicit regularization term. Our approach is novel by controlling the expectation of the K-norm of $ f_t $ using an iterative process. Then we use distributed learning to improve our result.

Keywords:

Mathematics Subject Classification: Primary: 68Q32, 68T05; Secondary: 62J02.

Citation:

Full Text(HTML)

Related Papers

Cited by

References

[1]	X. Chen and Y. Lei, Refined Bounds for online pairwise learning algorithm, Neurocomputing, 275 (2018), 2656-2665. doi: 10.1016/j.neucom.2017.11.049.
[2]	F. Cucker and S. Smale, On the mathematical foundations of learning, Bull. Amer. Math. Soc., 39 (2001), 1-49. doi: 10.1090/S0273-0979-01-00923-5.
[3]	J. Duchi, E. Hazan and Y. Singer, Adaptive subgradient methods for online learning and stochastic optimization, Journal of Machine Learning Research, 12 (2011), 2121-2159.
[4]	T. Hu, Online regression with varying gaussians and non-identical distributions, Analysis and Applications, 9 (2011), 395-408. doi: 10.1142/S0219530511001923.
[5]	J. Kiefer and J. Wolfowitz, Stochastic estimation of the maximum of a regression function, Ann. Math. Statist., 23 (1952), 462-466. doi: 10.1214/aoms/1177729392.
[6]	S. B. Lin, X Guo and D. X. Zhou, Distributed learning with regularized least squares, Journal of Machine Learning Research, 18 (2017), Paper No. 92, 31 pp.
[7]	M. Pontil, Y. Ying and D. X. Zhou, Error analysis for online gradient descent algorithms in reproducing kernel Hilbert space, Technical Report, Department of Computer Science, University College London, (2005).
[8]	H. Robbins and S. Monro, A stochastic approximation method, Ann. Math. Statist., 22 (1951), 400-407. doi: 10.1214/aoms/1177729586.
[9]	S. Smale and Y. Yao, Online learning algorithms, Found. Comput. Math., 6 (2006), 145-170. doi: 10.1007/s10208-004-0160-z.
[10]	S. Smale and D. X. Zhou, Online learning with Markov sampling, Analysis and Applications, 7 (2009), 87-113. doi: 10.1142/S0219530509001293.
[11]	S. Vijayakumar, A. D'Souza and S. Schaal, Incremental online learning in high dimensions, Neural Computation, 17 (2005), 2602-2634. doi: 10.1162/089976605774320557.
[12]	C. Wang and T Hu, Online minimum error entropy algorithm with unbounded sampling, Analysis and Applications, 17 (2019), 293-322. doi: 10.1142/S0219530518500148.
[13]	J. Xu, Z. Zheng, Z. Fan and W. Liu, Online personalized QoS prediction approach for cloud services, 4th International Conference on Cloud Computing and Intelligence Systems, 2016. doi: 10.1109/CCIS.2016.7790220.
[14]	Y. Yao, L. Losasco and A. Caponnetto, Early stopping in gradient descent boosting, Constr. Approx., 26 (2007), 289-315. doi: 10.1007/s00365-006-0663-2.
[15]	Y. Ying and M. Pontil, Online pairwise learning algorithms, Found. Comput. Math., 28 (2016), 743-777. doi: 10.1162/NECO_a_00817.
[16]	Y. Ying and D. X. Zhou, Online regularized classification algorithm, IEEE, trans. Inform. Theory, 52 (2006), 4775-4788. doi: 10.1109/TIT.2006.883632.
[17]	Z. H. Zhou, N. V. Chawla, Y. Jin and G. J. Williams, Big data opportunities and challenges: Discussions from data analytics perspective, IEEE Computational Intelligence Magazine, 9 (2014), 62-74.