Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

How to specify the levels to iterate in a grid search with an ensemble classifier?

I have the following setup but I can’t find a way to pass levels to explore in the Grid search for svm* and mlp*:

steps = [('preprocessing', StandardScaler()),
         ('feature_selection', SelectKBest(mutual_info_classif, k=15)),
         ('clf', VotingClassifier(estimators=[("mlp1", mlp1),
                                              ("mlp2", mlp2),
                                              ("mlp3", mlp3),
                                              ("svm1", svm1),
                                              ("svm2", svm2)
                                             ], voting='soft'))
        ]

model = Pipeline(steps=steps)
params = [{
    'preprocessing': [StandardScaler(), MinMaxScaler(), MaxAbsScaler()],
    'feature_selection__score_func': [f_classif, mutual_info_classif]
}]

grid_search = GridSearchCV(model, params, cv=10, scoring='balanced_accuracy', verbose=1, n_jobs=20, refit=True)

>Solution :

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

Within VotingClassifier, nested parameters can be specified using the following structure:

clf__<estimator_name>__<hyperparameter_name>

clf refers to the VotingClassifier step in the pipeline.
<estimator_name> refers to the name of the estimator in the VotingClassifier (e.g. mlp1, svm1).
<hyperparameter_name> refers to the parameter of the individual estimator (e.g. hidden_layer_sizes for MLP or C for SVM).

Now you can include hyperparameters mlp1, mlp2, mlp3 (for MLP classifiers) and svm1, svm2 (for SVM classifiers) in the params dictionary. e.g.

params = [{
    # Preprocessing variations
    'preprocessing': [StandardScaler(), MinMaxScaler(), MaxAbsScaler()],
    
    # Feature selection variations
    'feature_selection__score_func': [f_classif, mutual_info_classif],
    
    # Hyperparameters for MLP classifiers
    'clf__mlp1__hidden_layer_sizes': [(50,), (100,), (50, 50)],
    'clf__mlp1__activation': ['relu', 'tanh'],
    'clf__mlp2__hidden_layer_sizes': [(50,), (100,)],
    'clf__mlp2__activation': ['relu', 'tanh'],
    'clf__mlp3__hidden_layer_sizes': [(50, 50), (100, 50)],
    'clf__mlp3__activation': ['relu', 'logistic'],
    
    # Hyperparameters for SVM classifiers
    'clf__svm1__C': [0.01, 0.1, 1, 10],
    'clf__svm1__kernel': ['linear', 'rbf'],
    'clf__svm2__C': [0.1, 1, 10],
    'clf__svm2__kernel': ['rbf', 'sigmoid']
}]
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading