I am using Numpy as part of a neural network, and when updating the weights I am struggling to implement a step in a natural way.
The step works for an input rho_deltas (shape: (m,)) and self.node_layers[i-1].val (shape: (n,)) and outputs self.previous_edge_layer[i - 1] (shape: (m,n))
It should be such that self.previous_edge_layer[i - 1][j][k] == rho_deltas[j] * self.node_layers[i - 1].vals[k]
Example working inputs and outputs here.
(I’ll try to update these so it is easier to copy and paste for testing your methods.)
I have managed to get it working well like:
self.previous_edge_layer[i - 1] = np.array([rho_delta * self.node_layers[i - 1].vals for rho_delta in rho_deltas])
However, it feels to me as though there is a Numpy operator/function that should be able to do this without the iteration over the full list. My inclination is matrix multiplication (@) however, I have not been able to get this to work. Or perhaps, dot product (*), however for n != m this fails.
Furthermore, I struggled to come up with a useful name for this question so feel free to rename so something better :).
>Solution :
Matrix multiplication is the right idea: the preliminary is to form matrices from your 1D vectors. We need 2D matrices here, even though one dimension will be of size 1. Something like this:
import numpy as np
rho_deltas = np.array([7.6, 12.3, 11.1]) # example data with m = 3
layer_vals = np.array([1.5, 20.9, -3.5, 7.0]) # example with n = 4
rho_deltas_row_mat = rho_deltas.reshape(-1, 1) # m rows, 1 column
layer_vals_col_mat = layer_vals.reshape(1, -1) # 1 row, n columns
res = rho_deltas_row_mat @ layer_vals_col_mat
print(res.shape)
print(all(res[j][k] == rho_deltas[j] * layer_vals[k] for j in range(rho_deltas.shape[0]) for k in range(layer_vals.shape[0])))
prints:
(3, 4)
True
Alternatively, you could reshape both of them to row matrices and use transposition, something like:
rho_deltas_row_mat = rho_deltas.reshape(-1, 1)
layer_vals_row_mat = layer_vals.reshape(-1, 1)
res = rho_deltas_row_mat @ layer_vals_row_mat.T