Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Gradient descent extended function example

Here is my gradient descent code:

import numpy
def gradient_descent(func, grad_func, w_init, n_epochs=100, lr=0.001, verbose=0):
    
    i = 0
    w = w_init
    while i < n_epochs:
        
        delta_w = -lr * grad_func(w)
        w = w + delta_w
                
        if verbose > 0:
            print("f={}; w: {}".format(func(w), w))

        i += 1    
    
    return w

Now the example that was explained to me is with simple function i.e:
f(p,q) = p^2 + q^2.
The partial derivates are in vector: [2p,2q] and that is clear.
The code for that function and its gradient is:

def f(w):
    return numpy.sum(w*w)

def grad(w):
    return 2*w

So the calculation goes:

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

# starting point
w_init = numpy.array([10,10])

# learning rate
lr = 0.1

# apply gradient descent
w_opt = gradient_descent(f, grad, w_init, n_epochs=25, lr=lr, verbose=1)

My problem is calculating it for another function such as: f(p,q) = (p^2 + 2q^3). I know the values for partial derivative, but how to implement those values to this particular code?
How to write new function f and gradient function grad for it and to compute it with main function?

>Solution :

Probably you’re looking for something like this:

def f(w):
    return w[0] ** 2 + 2 * w[1] ** 3

def grad(w):
    return np.array([2 * w[0], 3 * w[1] ** 2])
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading