• Journal of Electronic Science and Technology
  • Vol. 22, Issue 1, 100246 (2024)
Peng Wang1,2,*, Ji Guo1, and Lin-Feng Li3
Author Affiliations
  • 1School of Finance and Economics, Xizang Minzu University, Xianyang, 712082, China
  • 2Research Center for Quality Development of Xizang Special Industries, Xianyang, 712082, China
  • 3School of Computer Science and Technology, Xinjiang University, Urumqi, 830017, China
  • show less
    DOI: 10.1016/j.jnlest.2024.100246 Cite this Article
    Peng Wang, Ji Guo, Lin-Feng Li. Machine learning model based on non-convex penalized huberized-SVM[J]. Journal of Electronic Science and Technology, 2024, 22(1): 100246 Copy Citation Text show less
    Comparison of the hinge loss with the huberized hinge loss.
    Fig. 1. Comparison of the hinge loss with the huberized hinge loss.
    Flowchart of the algorithm.
    Fig. 2. Flowchart of the algorithm.
    Algorithm
    1. Initialize $ {\tilde \beta _0} $ and $ \tilde {\Cambriabfont\text{β}} $.
    2. Iterate 1) and 2) until convergence:
       1) Cyclic coordinate descent: j = 1, 2, ···, p.
        i) Calculate $ {r_i} = {y_i}({\tilde \beta _0} + {\mathbf{x}}_i^{\text{T}}\tilde {\Cambriabfont\text{β}}) $;
        ii) Calculate $ \hat \beta _j^{{\text{new}}} = \left( {{\delta \mathord{\left/ {\vphantom {\delta 2}} \right. } 2}} \right)S\left( {z{\rm{,}}{\text{ }}{P_{\lambda _1}'}\left( {\left| {{{\tilde \beta }_j}} \right|} \right)} \right) $;
        iii) Let $ {\tilde \beta _j} = \hat \beta _j^{{\text{new}}}{\rm{,}}{\text{ }}j = j + 1 $.
       2) Update the intercept term:
        i) Calculate $ {r_i} = {y_i}\left( {{{\tilde \beta }_0} + {\mathbf{x}}_i^{\text{T}}\tilde {\Cambriabfont\text{β}}} \right) $;
        ii) Calculate $ \hat \beta _0^{{\text{new}}} = {\tilde \beta _0} - \left( {{\delta \mathord{\left/ {\vphantom {\delta {2n}}} \right. } {2n}}} \right)\sum\limits_{i = 1}^n {{l_\delta' }\left( {{r_i}} \right){y_i}} $;
        iii) Let $ {\tilde \beta _j} = \hat \beta _j^{{\text{new}}}{\rm{,}}{\text{ }}j = j + 1 $.
    3. If $\mathop {{\rm{max}} }\limits_{0 \leq j \leq p} \left| {\hat \beta _j^{{\text{new}}} - {{\tilde \beta }_j}} \right| < \varepsilon $, stop the iteration; otherwise, return to step 2.
    Table 1. Desciption of the resulted algorithm for solving the model (1).
    IndicatorsMethodsMin1st QuMedianMean3rd QuMax
    ACCSVM0.70160.74350.80100.80000.85340.8952
    LASSO-SVM0.76120.81140.86800.86530.90250.9589
    MCP-HSVM0.77730.82600.87210.87490.91800.9708
    SCAD-HSVM0.78420.83960.88620.88840.93720.9804
    AUCSVM0.78400.81700.86270.86080.90210.9335
    LASSO-SVM0.84360.89160.93780.92900.96570.9902
    MCP-HSVM0.84890.88010.92390.92390.96250.9969
    SCAD-HSVM0.85020.88670.93280.92560.96260.9974
    TPRSVM0.66910.70750.75460.76020.80550.8645
    LASSO-SVM0.75530.79880.83700.84630.88090.9500
    MCP-HSVM0.76020.80680.84340.85120.89780.9581
    SCAD-HSVM0.75950.79250.85250.85420.91020.9555
    FPRSVM0.21010.23420.25350.25710.28070.3097
    LASSO-SVM0.14700.17100.19740.19600.21950.2456
    MCP-HSVM0.16430.18680.21000.21210.23810.2634
    SCAD-HSVM0.16830.19820.22390.22090.24570.2670
    Table 2. Results of evaluation indicators for the classification prediction of each model.
    IndicatorsMethodsMin1st QuMedianMean3rd QuMax
    NVLASSO-SVM5.000010.000017.000016.640024.250038.0000
    MCP-HSVM6.00009.000015.000019.600025.750047.0000
    SCAD-HSVM8.00009.000010.000010.260011.000013.0000
    FNRLASSO-SVM0.01760.10290.22940.25120.32470.5112
    MCP-HSVM0.07650.12520.20000.21820.30590.4824
    SCAD-HSVM0.10290.22940.25120.28240.34710.5647
    FDRLASSO-SVM0.00000.07640.22480.26790.35640.4578
    MCP-HSVM0.00000.11930.25560.28800.38100.4610
    SCAD-HSVM0.00000.10000.15770.17350.26620.3500
    Table 3. Results of evaluation indicators for variable selection of LASSO-SVM, MCP-HSVM, and SCAD-HSVM.
    ρIndicatorsSVMLASSO-SVMMCP-HSVMSCAD-HSVM
    ρ = 0.3ACC0.77580.83390.84360.8508
    AUC0.83510.88020.90500.8956
    TPR0.72900.79960.83060.8260
    FPR0.28270.21030.22060.2230
    ρ = 0.5ACC0.74960.81880.83480.8310
    AUC0.78280.86500.88610.8853
    TPR0.69390.76470.82450.8130
    FPR0.32380.23570.23940.2345
    ρ = 0.8ACC0.67600.77860.82000.8196
    AUC0.72090.82120.87010.8635
    TPR0.63310.72110.81350.7992
    FPR0.33010.25670.24120.2451
    Table 4. Results of evaluation indicators for the classification prediction of each model under different correlations.
    ρIndicatorsLASSO-SVMMCP-HSVMSCAD-HSVM
    ρ = 0.3NV15.820018.480014.2600
    FNR0.24240.16120.2118
    FDR0.23600.20570.1885
    ρ = 0.5NV12.900016.080013.7600
    FNR0.31170.22940.2941
    FDR0.32980.23250.2916
    ρ = 0.8NV10.240015.760012.9800
    FNR0.37290.28000.3329
    FDR0.16550.23970.3244
    Table 5. Means of evaluation indicators for variable selection of LASSO-SVM, MCP-HSVM, and SCAD-HSVM.
    Level 1 indicatorsLevel 2 indicators
    SolvencyCurrent ratio: X1Quick ratio: X2Equity ratio: X3Total equity/liabilities attributable to shareholders of the parent company: X4Earnings before interest, taxes, depreciation, and amortization/total liabilities: X5Net cash flow from operating activities/total liabilities: X6Interest coverage ratio (earnings before interest and taxes/interest expense): X7
    Indicators per shareEarnings per share (RMB): X8Operating cash flow per share (RMB): X9Net assets per share (net of minority interests) (RMB): X10
    Revenue qualityNet income from operating activities/total profit (%): X11Net gain from changes in value/total profit (%): X12Net non-operating income and expenses/total profit (%): X13Net income after deducting non-recurring gain and losses/net income (%): X14
    ProfitabilityGross profit margin on sales (%): X15Net sales margin (%): X16Net return on assets (%): X17Net margin on total assets (%): X18
    Operating capacityInventory turnover ratio (times): X19Accounts receivable turnover ratio (times): X20Current assets turnover ratio (times): X21Fixed assets turnover ratio (times): X22Total assets turnover ratio (times): X23
    Capital structureAsset-liability ratio (%): X24Equity multiplier: X25Current assets/total assets (%): X26Non-current assets/total assets (%): X27Current liabilities/total liabilities (%): X28Non-current liabilities/total liabilities (%): X29
    Table 6. Financial indicators used in this paper.
    MethodsACCAUCTPRNV
    SVM0.87760.76760.7057/
    Lasso-SVM0.86320.79670.730117.8
    MCP-HSVM0.86520.81870.768223.6
    SCAD-HSVM0.87540.83850.749422.1
    Table 7. Prediction results of each method (average of 100 predictions).
    VariablesLASSO-SVMMCP-HSVMSCAD-HSVMVariablesLASSO-SVMMCP-HSVMSCAD-HSVM
    X101511X16100100100
    X2100100100X17093100
    X3405045X18100100100
    X4080X190055
    X5100100100X20100100100
    X65010058X21100100100
    X7005X22100100100
    X8100100100X239510055
    X9100100100X249195100
    X10100100100X250400
    X119295100X26100100100
    X125095100X27100100100
    X13003X288595100
    X14100100100X2905576
    X150510
    Table 8. Selection frequency of each financial indicator.
    Selection frequencyLevel 2 indicators
    100X2 (quick ratio), X5 (earnings before interest, taxes, depreciation, and amortization/total liabilities), X6 (net cash flow from operating activities/total liabilities), X8 (earnings per share), X9 (operating cash flow per share), X10 (net assets per share), X14 (net income after deducting non-recurring gain and losses/net income), X16 (net sales margin), X18 (net margin on total assets), X20 (accounts receivable turnover ratio), X21 (current assets turnover ratio), X22 (fixed assets turnover ratio), X23 (total assets turnover ratio), X26 (current assets/total assets), and X27 (non-current assets/total assets)
    90−100X11 (net income from operating activities/total profit), X12 (net gain from changes in value/total profit), X17 (return on net assets), X24 (asset-liability ratio), and X28 (current liabilities/total liabilities)
    60−90None
    40−60X3 (equity ratio), X15 (gross profit margin on sales), X25 (equity multiplier), and X29 (non-current liabilities/total liabilities)
    20−40None
    1−20X1 (current ratio) and X4 (total equity/liabilities attributable to shareholders of the parent company)
    0X7 (interest coverage ratio (earnings before interest and taxes/interest expense)), X13 (net non-operating income and expenses/total profit), and X19 (inventory turnover ratio)
    Table 9. Classification of variables by the selection frequency of MCP-HSVM.
    Peng Wang, Ji Guo, Lin-Feng Li. Machine learning model based on non-convex penalized huberized-SVM[J]. Journal of Electronic Science and Technology, 2024, 22(1): 100246
    Download Citation