JDS Journal of Data Science 1683-86021680-743X1680-743X School of Statistics, Renmin University of China JDS1162 10.6339/24-JDS1162 Computing in Data Science ReLU-ReHU Representations of Piecewise Linear-Quadratic Losses GaoTingxian1 DaiBenbendai@cuhk.edu.hk1∗ QiuYixuan2 Department of Statistics, The Chinese University of Hong Kong, China School of Statistics and Management, Shanghai University of Finance and Economics, China Corresponding author. Email: bendai@cuhk.edu.hk. 20252012025234648658 Supplementary Material

The data, code, and README are all available at https://github.com/keepwith/PLQComposite.

279202425122024 2025 The Author(s). Published by the School of Statistics and the Center for Applied Statistics, Renmin University of China.2025 Open access article under the CC BY license.

Piecewise linear-quadratic (PLQ) functions are a fundamental function class in convex optimization, especially within the Empirical Risk Minimization (ERM) framework, which employs various PLQ loss functions. This paper provides a workflow for decomposing a general convex PLQ loss into its ReLU-ReHU representation, along with a Python implementation designed to enhance the efficiency of presenting and solving ERM problems, particularly when integrated with ReHLine (a powerful solver for PLQ ERMs). Our proposed package, plqcom, accepts three representations of PLQ functions and offers user-friendly APIs for verifying their convexity and continuity. The Python package is available at https://github.com/keepwith/PLQComposite.

 empirical risk minimization optimization piecewise linear-quadratic function Python package ReHLine References Bandler J, Chen SH, Biernacki R, Gao L, Madsen K, Yu H (1993). Huber optimization of circuits: a robust approach. IEEE Transactions on Microwave Theory and Techniques, 41(12): 2279–2287. https://doi.org/10.1109/22.260718 Benine-Neto A, Scalzi S, Mammar S (2011). Vehicle lane keeping control based on piecewise affine regions. In: International IEEE Conference on Intelligent Transportation Systems (ITSC), 907–912. IEEE. Dai B, Qiu Y (2024). ReHLine: regularized composite ReLU-ReHU loss minimization with linear computation and linear convergence. Advances in Neural Information Processing Systems, 36. Fukushima K (1969). Visual feature extraction by a multilayered network of analog threshold elements. IEEE Transactions on Systems Science and Cybernetics, 5(4): 322–333. https://doi.org/10.1109/TSSC.1969.300225 Garcia-Rubio R, Bayón L, Grau JM (2014). Generalization of the firm’s profit maximization problem: An algorithm for the analytical and nonsmooth solution. Computational Economics, 43(1): 1–14. https://doi.org/10.1007/s10614-013-9378-7 Gardiner B, Lucet Y (2010). Convex hull algorithms for piecewise linear-quadratic functions in computational convex analysis. Set-Valued and Variational Analysis, 18(3–4): 467–482. https://doi.org/10.1007/s11228-010-0157-5 Hein M, Andriushchenko M, Bitterwolf J (2019). Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 41–50. IEEE Computer Society, Los Alamitos, CA, USA. Huber PJ (1964). Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1): 73–101. https://doi.org/10.1214/aoms/1177703732 Jensen DL, King AJ (1992). Frontier: A graphical interface for portfolio optimization in a piecewise linear-quadratic risk framework. IBM Systems Journal, 31(1): 62–70. https://doi.org/10.1147/sj.311.0062 Portnoy S, Koenker R (1997). The Gaussian hare and the Laplacian tortoise: Computability of squared-error versus absolute-error estimators. Statistical Science, 12(4): 279–300. https://doi.org/10.1214/ss/1030037960 Rockafellar RT (1988). First- and second-order epi-differentiability in nonlinear programming. Transactions of the American Mathematical Society, 307(1): 75–108. https://doi.org/10.1090/S0002-9947-1988-0936806-9 Vapnik V (1998). Statistical Learning Theory, volume 2, 831–842. John Wiley & Sons. Vapnik V (2006). Estimation of Dependences Based on Empirical Data. Springer Science & Business Media.