Formula for SGD
Why is a munis there? Can I replace the minus with a plus? I never thought about it, but now I’m interested.
>Solution :
The gradient tells us the direction to move to increase the error (gradient ascent / uphill). But, since we want to reduce the error, we add the minus. You can find an in-depth explanation in this medium article.