A question about MLP-What does this line mean?

July 16, 2022

I am new to NN. I AM TRYING TO TUNE MY MLP REGRESSOR MODEL. I don’t understand what does this line mean "’hidden_layer_sizes’: [(100,), (50,100,), (50,75,100,)]" does this mean that we’re asking the model to check if the model will perform better in case it has only one hidden layer with 100 hidden points, or two hidden layers(with how many neurons?)or three hidden layers?

from sklearn.neural_network import MLPRegressor
from sklearn.model_selection import GridSearchCV
X, y = scaled_df[[ "Part's Z-Height (mm)","Part's Weight (N)","Part's Volume (cm^3)","Part's Surface Area (cm^2)","Part's Orientation (Support's height) (mm)","Part's Orientation (Support's volume) (cm^3)","Layer Height (mm)","Printing/Scanning Speed (mm/s)","Infill Density (%)"]], scaled_df [["Climate change (kg CO2 eq.)","Climate change, incl biogenic carbon (kg CO2 eq.)","Fine Particulate Matter Formation (kg PM2.5 eq.)","Fossil depletion (kg oil eq.)","Freshwater Consumption (m^3)","Freshwater ecotoxicity (kg 1,4-DB eq.)","Freshwater Eutrophication (kg P eq.)","Human toxicity, cancer (kg 1,4-DB eq.)","Human toxicity, non-cancer (kg 1,4-DB eq.)","Ionizing Radiation (Bq. C-60 eq. to air)","Land use (Annual crop eq. yr)","Marine ecotoxicity (kg 1,4-DB eq.)","Marine Eutrophication (kg N eq.)","Metal depletion (kg Cu eq.)","Photochemical Ozone Formation, Ecosystem (kg NOx eq.)","Photochemical Ozone Formation, Human Health (kg NOx eq.)","Stratospheric Ozone Depletion (kg CFC-11 eq.)","Terrestrial Acidification (kg SO2 eq.)","Terrestrial ecotoxicity (kg 1,4-DB eq.)"]]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

mlp = MLPRegressor()
param_grid = {'hidden_layer_sizes': [(100,), (50,100,), (50,75,100,)],
              'activation': ['tanh','relu','lbfgs'],
              'solver': ['sgd', 'adam'],
              'learning_rate': ['constant','adaptive','invscaling'],
              'alpha': [0.0001, 0.05],
              'max_iter': [10000000000],
              'early_stopping': [False],
              'warm_start': [False]}
GS = GridSearchCV(mlp, param_grid=param_grid,n_jobs= -1,cv=5, scoring='r2')
                  
                  
GS.fit(X_train, y_train)

print(GS.best_params_)

>Solution :

Yes, you are performing a Grid Search hyper param optimization.
You are trying with:

Just one hidden layer of 100 units
Two hidden layers of 50, 100
3 hidden layers of 50, 75, 100

Your code will extract all combinations of hyperparameters and tell you which one performed best. If you want to extract only some of these combinations, you could look up RandomSearch.
Otherwise, I suggest the hyperopt library to properly optimize your hyperparams with bayesian methods (e.g. TPE)