Stacking Classification

This example uses the ‘iris’ dataset and performs a complex stacking classification. We will use two different classifiers, one applied to petal features and one applied to sepal features. A final logistic regression classifier will be applied on the predictions of the two classifiers.

# Authors: Federico Raimondo <f.raimondo@fz-juelich.de>
#
# License: AGPL
from seaborn import load_dataset
from julearn import run_cross_validation
from julearn.pipeline import PipelineCreator
from julearn.utils import configure_logging

Set the logging level to info to see extra information

configure_logging(level="INFO")

2023-07-19 12:41:59,876 - julearn - INFO - ===== Lib Versions =====
2023-07-19 12:41:59,876 - julearn - INFO - numpy: 1.25.1
2023-07-19 12:41:59,876 - julearn - INFO - scipy: 1.11.1
2023-07-19 12:41:59,876 - julearn - INFO - sklearn: 1.3.0
2023-07-19 12:41:59,876 - julearn - INFO - pandas: 2.0.3
2023-07-19 12:41:59,876 - julearn - INFO - julearn: 0.3.1.dev1
2023-07-19 12:41:59,876 - julearn - INFO - ========================

df_iris = load_dataset("iris")

The dataset has three kind of species. We will keep two to perform a binary classification.

df_iris = df_iris[df_iris["species"].isin(["versicolor", "virginica"])]

As features, we will use the sepal length, width and petal length. We will try to predict the species.

X = ["sepal_length", "sepal_width", "petal_length", "petal_width"]
y = "species"

# Define our feature types
X_types = {
    "sepal": ["sepal_length", "sepal_width"],
    "petal": ["petal_length", "petal_width"],
}


# Create the pipeline for the sepal features, by default will apply to "sepal"
model_sepal = PipelineCreator(problem_type="classification", apply_to="sepal")
model_sepal.add("filter_columns", apply_to="*", keep="sepal")
model_sepal.add("zscore")
model_sepal.add("svm")

# Create the pipeline for the petal features, by default will apply to "petal"
model_petal = PipelineCreator(problem_type="classification", apply_to="petal")
model_petal.add("filter_columns", apply_to="*", keep="petal")
model_petal.add("zscore")
model_petal.add("rf")

# Create the stacking model
model = PipelineCreator(problem_type="classification")
model.add(
    "stacking",
    estimators=[[("model_sepal", model_sepal), ("model_petal", model_petal)]],
    apply_to="*",
)

scores = run_cross_validation(
    X=X, y=y, X_types=X_types, data=df_iris, model=model
)

print(scores["test_score"])

2023-07-19 12:41:59,880 - julearn - INFO - Adding step filter_columns that applies to ColumnTypes<types={'*'}; pattern=.*>
2023-07-19 12:41:59,880 - julearn - INFO - Setting hyperparameter keep = sepal
2023-07-19 12:41:59,880 - julearn - INFO - Step added
2023-07-19 12:41:59,881 - julearn - INFO - Adding step zscore that applies to ColumnTypes<types={'sepal'}; pattern=(?:__:type:__sepal)>
2023-07-19 12:41:59,881 - julearn - INFO - Step added
2023-07-19 12:41:59,881 - julearn - INFO - Adding step svm that applies to ColumnTypes<types={'sepal'}; pattern=(?:__:type:__sepal)>
2023-07-19 12:41:59,881 - julearn - INFO - Step added
2023-07-19 12:41:59,881 - julearn - INFO - Adding step filter_columns that applies to ColumnTypes<types={'*'}; pattern=.*>
2023-07-19 12:41:59,881 - julearn - INFO - Setting hyperparameter keep = petal
2023-07-19 12:41:59,881 - julearn - INFO - Step added
2023-07-19 12:41:59,881 - julearn - INFO - Adding step zscore that applies to ColumnTypes<types={'petal'}; pattern=(?:__:type:__petal)>
2023-07-19 12:41:59,881 - julearn - INFO - Step added
2023-07-19 12:41:59,881 - julearn - INFO - Adding step rf that applies to ColumnTypes<types={'petal'}; pattern=(?:__:type:__petal)>
2023-07-19 12:41:59,882 - julearn - INFO - Step added
2023-07-19 12:41:59,882 - julearn - INFO - Adding step stacking that applies to ColumnTypes<types={'*'}; pattern=.*>
2023-07-19 12:41:59,882 - julearn - INFO - Setting hyperparameter estimators = [('model_sepal', <julearn.pipeline.pipeline_creator.PipelineCreator object at 0x7f7f628d4340>), ('model_petal', <julearn.pipeline.pipeline_creator.PipelineCreator object at 0x7f7f628d5720>)]
2023-07-19 12:41:59,882 - julearn - INFO - Step added
2023-07-19 12:41:59,882 - julearn - INFO - ==== Input Data ====
2023-07-19 12:41:59,882 - julearn - INFO - Using dataframe as input
2023-07-19 12:41:59,882 - julearn - INFO -      Features: ['sepal_length', 'sepal_width', 'petal_length', 'petal_width']
2023-07-19 12:41:59,882 - julearn - INFO -      Target: species
2023-07-19 12:41:59,882 - julearn - INFO -      Expanded features: ['sepal_length', 'sepal_width', 'petal_length', 'petal_width']
2023-07-19 12:41:59,882 - julearn - INFO -      X_types:{'sepal': ['sepal_length', 'sepal_width'], 'petal': ['petal_length', 'petal_width']}
2023-07-19 12:41:59,883 - julearn - INFO - ====================
2023-07-19 12:41:59,883 - julearn - INFO -
2023-07-19 12:41:59,885 - julearn - INFO - = Model Parameters =
2023-07-19 12:41:59,885 - julearn - INFO - ====================
2023-07-19 12:41:59,885 - julearn - INFO -
2023-07-19 12:41:59,886 - julearn - INFO - = Model Parameters =
2023-07-19 12:41:59,886 - julearn - INFO - ====================
2023-07-19 12:41:59,886 - julearn - INFO -
2023-07-19 12:42:00,015 - julearn - INFO - = Model Parameters =
2023-07-19 12:42:00,015 - julearn - INFO - ====================
2023-07-19 12:42:00,015 - julearn - INFO -
2023-07-19 12:42:00,015 - julearn - INFO - = Data Information =
2023-07-19 12:42:00,015 - julearn - INFO -      Problem type: classification
2023-07-19 12:42:00,015 - julearn - INFO -      Number of samples: 100
2023-07-19 12:42:00,015 - julearn - INFO -      Number of features: 4
2023-07-19 12:42:00,015 - julearn - INFO - ====================
2023-07-19 12:42:00,015 - julearn - INFO -
2023-07-19 12:42:00,015 - julearn - INFO -      Number of classes: 2
2023-07-19 12:42:00,015 - julearn - INFO -      Target type: object
2023-07-19 12:42:00,016 - julearn - INFO -      Class distributions: species
versicolor    50
virginica     50
Name: count, dtype: int64
2023-07-19 12:42:00,016 - julearn - INFO - Using outer CV scheme KFold(n_splits=5, random_state=None, shuffle=False)
2023-07-19 12:42:00,016 - julearn - INFO - Binary classification problem detected.
0    1.00
1    0.85
2    0.95
3    0.95
4    0.95
Name: test_score, dtype: float64