Closed
Description
First of all, I am using
python 3.8.5
pycaret 2.2.3
scikit-learn 0.23.2
Win10 Pro
The add_metric() works fine on the built-in dataset as shown in below.
from pycaret.datasets import get_data
from pycaret.classification import *
from sklearn.metrics import average_precision_score
data1 = get_data('juice')
clf = setup(data1, target='Purchase', session_id=123, silent=True, verbose=False)
add_metric('auprc', 'AUPRC', average_precision_score, target = 'pred_proba')
best = compare_models(include=['rf', 'catboost', 'xgboost', 'lightgbm'], fold=3)
However, when I use the following multiclass dataset, it stops working for some reason. I got a blank table instead. Here is my code.
import pandas as pd
import numpy as np
data2 = pd.DataFrame(np.random.randint(0, 2, size=(1000,20)))
data2.columns = [f'f{ea:02d}' for ea in range(data2.shape[1])]
data2['target'] = np.random.choice(['A', 'B', 'C'], data2.shape[0])
data2.head()
... the following code simply return a blank table
clf = setup(data2, target='target', session_id=123, silent=True, verbose=False)
add_metric('auprc', 'AUPRC', average_precision_score, target = 'pred_proba')
best = compare_models(include=['rf', 'catboost', 'xgboost', 'lightgbm'], fold=3)But then I had decided to try another scikit built-in function log_loss on the same dataset, it works fine.
from sklearn.metrics import log_loss
clf = setup(data2, target='target', session_id=123, silent=True, verbose=False)
add_metric('logloss', 'LogLoss', log_loss, target = 'pred_proba')
best = compare_models(include=['rf', 'catboost', 'xgboost', 'lightgbm'], fold=3)
Can someone please shed some light on this issue? Thanks