Kernel based regression with robust loss function via iteratively reweighted least squares

preprint2020arXivOpen access

Abstract

Least squares kernel based methods have been widely used in regression problems due to the simple implementation and good generalization performance. Among them, least squares support vector regression (LS-SVR) and extreme learning machine (ELM) are popular techniques. However, the noise sensitivity is a major bottleneck. To address this issue, a generalized loss function, called ℓs\ell_s-loss, is proposed in this paper. With the support of novel loss function, two kernel based regressors are constructed by replacing the ℓ2\ell_2-loss in LS-SVR and ELM with the proposed ℓs\ell_s-loss for better noise robustness. Important properties of ℓs\ell_s-loss, including robustness, asymmetry and asymptotic approximation behaviors, are verified theoretically. Moreover, iteratively reweighted least squares (IRLS) is utilized to optimize and interpret the proposed methods from a weighted viewpoint. The convergence of the proposal are proved, and detailed analyses of robustness are given. Experiments on both artificial and benchmark datasets confirm the validity of the proposed methods.

Explore this paper’s authors, topics and related work.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Reviews 0

Write a reviewWrite

No reviews yet.

Discussion 0

Add a commentWrite

No comments yet.