ReLU-ReHU Representations of Piecewise Linear-Quadratic Losses
Pub. online: 20 January 2025
Type: Computing In Data Science
Open Access
Open Access
Received
27 September 2024
27 September 2024
Accepted
25 December 2024
25 December 2024
Published
20 January 2025
20 January 2025
Abstract
Piecewise linear-quadratic (PLQ) functions are a fundamental function class in convex optimization, especially within the Empirical Risk Minimization (ERM) framework, which employs various PLQ loss functions. This paper provides a workflow for decomposing a general convex PLQ loss into its ReLU-ReHU representation, along with a Python implementation designed to enhance the efficiency of presenting and solving ERM problems, particularly when integrated with ReHLine (a powerful solver for PLQ ERMs). Our proposed package, plqcom, accepts three representations of PLQ functions and offers user-friendly APIs for verifying their convexity and continuity. The Python package is available at https://github.com/keepwith/PLQComposite.
Supplementary material
Supplementary MaterialThe data, code, and README are all available at https://github.com/keepwith/PLQComposite.
References
Bandler J, Chen SH, Biernacki R, Gao L, Madsen K, Yu H (1993). Huber optimization of circuits: a robust approach. IEEE Transactions on Microwave Theory and Techniques, 41(12): 2279–2287. https://doi.org/10.1109/22.260718
Fukushima K (1969). Visual feature extraction by a multilayered network of analog threshold elements. IEEE Transactions on Systems Science and Cybernetics, 5(4): 322–333. https://doi.org/10.1109/TSSC.1969.300225
Garcia-Rubio R, Bayón L, Grau JM (2014). Generalization of the firm’s profit maximization problem: An algorithm for the analytical and nonsmooth solution. Computational Economics, 43(1): 1–14. https://doi.org/10.1007/s10614-013-9378-7
Gardiner B, Lucet Y (2010). Convex hull algorithms for piecewise linear-quadratic functions in computational convex analysis. Set-Valued and Variational Analysis, 18(3–4): 467–482. https://doi.org/10.1007/s11228-010-0157-5
Huber PJ (1964). Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1): 73–101. https://doi.org/10.1214/aoms/1177703732
Jensen DL, King AJ (1992). Frontier: A graphical interface for portfolio optimization in a piecewise linear-quadratic risk framework. IBM Systems Journal, 31(1): 62–70. https://doi.org/10.1147/sj.311.0062
Portnoy S, Koenker R (1997). The Gaussian hare and the Laplacian tortoise: Computability of squared-error versus absolute-error estimators. Statistical Science, 12(4): 279–300. https://doi.org/10.1214/ss/1030037960
Rockafellar RT (1988). First- and second-order epi-differentiability in nonlinear programming. Transactions of the American Mathematical Society, 307(1): 75–108. https://doi.org/10.1090/S0002-9947-1988-0936806-9
