I’ve written linear regression from scratch. The fit function calculates partial derivatives at each epoch for slope (m) and bias (b) and updates these variables using their partial derivatives.
My loss decreases, but it gets stuck at a curious place.
The code:
class LinearRegression:
def __init__(self):
self.m = 3
self.b = 2
def fit(self, x, y, epochs, lr):
for epoch in range(epochs):
pred = (self.m * x) + self.b
cur_loss = np.sum((y - pred)**2)
if epoch % 1000 == 0:
print(cur_loss)
pd_wrt_m = (-2/x.size) * np.sum((y - pred) * x)
pd_wrt_b = (-2/x.size) * np.sum(y - pred)
self.m -= (lr * pd_wrt_m)
self.b -= (lr * pd_wrt_b)
def predict(self, x):
return (self.m * x) + self.b
lr_model = LinearRegression()
lr_model.fit(x, y, 10000, 0.01)
Y_pred = lr_model.predict(x)
plt.scatter(x, y)
plt.plot([min(x), max(y)], [min(Y_pred), max(Y_pred)], color='red') # regression line
plt.show()
And this is the output when the above code is executed.
I’ve tried smaller and larger learning rates, smaller and larger epoch counts, but it seems to be the case that wherever the initial m and b values start out, they always stop converging at this particular position.
I’ve reread my code and could not find a mistake. What could I be doing wrong?
>Solution :
You wrote max(y) instead of max(x) lol.
Replace the plt.plot line with:
plt.plot([min(x), max(x)], [min(Y_pred), max(Y_pred)], color='red') # regression line
