Rescale Hinge Loss Support Vector Data Description
Pub. online: 23 April 2025
Type: Statistical Data Science
Open Access
Received
17 August 2024
17 August 2024
Accepted
7 April 2025
7 April 2025
Published
23 April 2025
23 April 2025
Abstract
Significant attention has been drawn to support vector data description (SVDD) due to its exceptional performance in one-class classification and novelty detection tasks. Nevertheless, all slack variables are assigned the same weight during the modeling process. This can lead to a decline in learning performance if the training data contains erroneous observations or outliers. In this study, an extended SVDD model, Rescale Hinge Loss Support Vector Data Description (RSVDD) is introduced to strengthen the resistance of the SVDD to anomalies. This is achieved by redefining the initial optimization problem of SVDD using a hinge loss function that has been rescaled. As this loss function can increase the significance of samples that are more likely to represent the target class while decreasing the impact of samples that are more likely to represent anomalies, it can be considered one of the variants of weighted SVDD. To efficiently address the optimization challenge associated with the proposed model, the half-quadratic optimization method was utilized to generate a dynamic optimization algorithm. Experimental findings on a synthetic and breast cancer data set are presented to illustrate the new proposed method’s performance superiority over the already existing methods for the settings considered.
Supplementary material
Supplementary MaterialWe have provided all the supplementary materials necessary to successfully reproduce this work, including the simulation data, corresponding code, and illustrative examples.
References
Cha M, Kim JS, Baek JG (2014). Density weighted support vector data description. Expert Systems with Applications, 41(7): 3343–3350. https://doi.org/10.1016/j.eswa.2013.11.025
Donoho DL (1982). Breakdown properties of multivariate location estimators. Technical report, Harvard University, Boston. http://www-stat.stanford.edu/~
Erfani SM, Rajasegarar S, Karunasekera S, Leckie C (2016). High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning. Pattern Recognition, 58: 121–134. https://doi.org/10.1016/j.patcog.2016.03.028
Hu W, Hu T, Wei Y, Lou J, Wang S (2021). Global plus local jointly regularized support vector data description for novelty detection. IEEE Transactions on Neural Networks and Learning Systems, 34(9): 6602–6614. https://doi.org/10.1109/TNNLS.2021.3129321
Khan NM, Ksantini R, Ahmad IS, Guan L (2014). Covariance-guided one-class support vector machine. Pattern Recognition, 47(6): 2165–2177. https://doi.org/10.1016/j.patcog.2014.01.004
Khan SS, Madden MG (2014). One-class classification: Taxonomy of study and review of techniques. Knowledge Engineering Review, 29(3): 345–374. https://doi.org/10.1017/S026988891300043X
Kivinen J, Smola AJ, Williamson RC (2004). Online learning with kernels. IEEE Transactions on Signal Processing, 52(8): 2165–2176. https://doi.org/10.1109/TSP.2004.830991
Liu W, Pokharel PP, Principe JC (2007). Correntropy: Properties and applications in non-Gaussian signal processing. IEEE Transactions on Signal Processing, 55(11): 5286–5298. https://doi.org/10.1109/TSP.2007.896065
Maboudou-Tchao EM (2018). Kernel methods for changes detection in covariance matrices. Communications in Statistics. Simulation and Computation, 47(6): 1704–1721. https://doi.org/10.1080/03610918.2017.1322701
Maboudou-Tchao EM (2020). Change detection using least squares one-class classification control chart. Quality Technology & Quantitative Management, 17(5): 609–626. https://doi.org/10.1080/16843703.2019.1711302
Maboudou-Tchao EM (2021a). High-dimensional data monitoring using support machines. Communications in Statistics. Simulation and Computation, 50(7): 1927–1942. https://doi.org/10.1080/03610918.2019.1588312
Maboudou-Tchao EM (2021b). Monitoring the mean with least-squares support vector data description. Gestão & Produção, 28(3): e019. https://doi.org/10.1590/1806-9649-2021v28e019
Maboudou-Tchao EM (2021c). Support tensor data description. Journal of Quality Technology, 53(2): 109–134. https://doi.org/10.1080/00224065.2019.1642815
Maboudou-Tchao EM (2023). Least-squares support tensor data description. Communications in Statistics. Simulation and Computation, 52(7): 3026–3042. https://doi.org/10.1080/03610918.2021.1926500
Maboudou-Tchao EM, Hampton HD (2025). Deep least squares one-class classification. Journal of Quality Technology, 57(1): 68–92. https://doi.org/10.1080/00224065.2024.2421164
Maboudou-Tchao EM, Silva IR, Diawara N (2018). Monitoring the mean vector with Mahalanobis kernels. Quality Technology & Quantitative Management, 15(4): 459–474. https://doi.org/10.1080/16843703.2016.1226707
Nikolova M, Ng MK (2005). Analysis of half-quadratic minimization methods for signal and image recovery. SIAM Journal on Scientific Computing, 27(3): 937–966. https://doi.org/10.1137/030600862
Schölkopf B, Platt JC, Shawe-Taylor J, Smola AJ, Williamson RC (2001). Estimating the support of a high-dimensional distribution. Neural Computation, 13(7): 1443–1471. https://doi.org/10.1162/089976601750264965
Seliya N, Abdollah Zadeh A, Khoshgoftaar TM (2021). A literature review on one-class classification and its potential applications in big data. Journal of Big Data, 8: 1–31. https://doi.org/10.1186/s40537-020-00387-6
Singh A, Pokharel R, Principe J (2014). The c-loss function for pattern classification. Pattern Recognition, 47(1): 441–453. https://doi.org/10.1016/j.patcog.2013.07.017
Sun R, Tsung F (2003). A kernel-distance-based multivariate control chart using support vector methods. International Journal of Production Research, 41(13): 2975–2989. https://doi.org/10.1080/1352816031000075224
Tax DM, Duin RP (2004). Support vector data description. Machine Learning, 54: 45–66. https://doi.org/10.1023/B:MACH.0000008084.60811.49
Wang CD, Lai J (2013). Position regularized support vector domain description. Pattern Recognition, 46(3): 875–884. https://doi.org/10.1016/j.patcog.2012.09.018
Wang K, Lan H (2020). Robust support vector data description for novelty detection with contaminated data. Engineering Applications of Artificial Intelligence, 91: 103554. https://doi.org/10.1016/j.engappai.2020.103554
Wolberg WH, Mangasarian OL (1990). Multisurface method of pattern separation for medical diagnosis applied to breast cytology. Proceedings of the National Academy of Sciences, 87(23): 9193–9196. https://doi.org/10.1073/pnas.87.23.9193
Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2008). Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(2): 210–227. https://doi.org/10.1109/TPAMI.2008.79
Xu G, Cao Z, Hu BG, Principe JC (2017). Robust support vector machines based on the rescaled hinge loss function. Pattern Recognition, 63: 139–148. https://doi.org/10.1016/j.patcog.2016.09.045