Fedot

View the Project on GitHub ITMO-NSS-team/FEDOT.Docs

Evaluation on benchmarks

The results of PMLB benchmarks are the follows:

Quality
metric
Pipeline and
framework
Regression data sets        
    1027_ESL 1096_FacultySalaries 227_cpu_small 228_elusage 605_fri_c2_250_25
MAE Static (XGBoost) 0.441±0.010 1.597±0.027 2.006±0.023 10.484±0.037 0.349±0.019
  Composite (FEDOT) 0.430±0.009 1.120±0.012 1.931±0.007 9.451±0.020 0.371±0.033
  Variable (TPOT) 0.432±0.004 1.508±0.023 2.064±0.011 11.051±0.031 0.373±0.014
  Linear (MLBox) 0.434±0.018 3.670±0.048 1.934±0.005 21.075±0.054 0.423±0.076
RMSE Static (XGBoost) 0.588±0.028 2.199±0.037 2.799±0.023 14.053±0.023 0.448±0.025
  Composite (FEDOT) 0.566±0.013 1.403±0.024 2.759±0.007 11.558±0.020 0.466±0.035
  Variable (TPOT) 0.570±0.009 1.789±0.033 2.886±0.023 14.863±0.032 0.465±0.014
  Linear (MLBox) 0.628±0.052 4.894±0.061 2.793±0.023 26.141±2.27 0.505±0.089
Quality
metric
Pipeline and
framework
Classification data sets        
    flare ionosphere labor magic spect
F1 Static (XGBoost) 0.299±0.110 0.939±0.026 0.895±0.023 0.793±0.013 0.837±0.077
  Composite (FEDOT) 0.332±0.057 0.919±0.026 0.931±0.013 0.817±0.004 0.893±0.062
  Variable (TPOT) 0.292±0.034 0.774±0.052 0.840±0.023 0.809±0.004 0.830±0.046
  Linear (MLBox) 0.181±0.097 0.933±0.088 0.840±0.069 0.812±0.017 0.845±0.071
ROC
AUC
Static (XGBoost) 0.693±0.025 0.956±0.007 0.923±0.023 0.915±0.015 0.736±0.075
  Composite (FEDOT) 0.708±0.033 0.951±0.011 0.958±0.019 0.930±0.004 0.779±0.043
  Variable (TPOT) 0.701±0.008 0.954±0.079 0.958±0.023 0.928±0.004 0.657±0.039
  Static (XGBoost) 0.509±0.012 0.907±1.036 0.515±0.058 0.856±0.026 0.628±0.031