Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Faster for loop with only if in python

I’m dealing with a big dataset and want to basically this:

test = np.random.rand(int(1e7))-0.5
def test0(test):
    return [0 if c<0 else c for c in test]

which is doing this:

def test1(test):
    for i,dat in enumerate(test):
        if dat<0: 
            test[i] = 0
        else:
            test[i] = dat
    return test

Is there a way to modify test0 to skip the else request so i works like this:

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

def test1(test):
    for i,dat in enumerate(test):
        if dat<0: test[i] = 0
    return test

Thanks in advance!

>Solution :

just do which seems to be fastest option:

test[test < 0] = 0

test:

import numpy as np
import timeit
from copy import copy
from functools import partial


def create_data():
    return np.random.rand(int(1e7))-0.5


def func1(data):
    data[data < 0] = 0


def func2(data):
    np.putmask(data, data < 0, 0)


def func3(data):
    np.maximum(data, 0)


if __name__ == '__main__':
    n_loops = 1000
    test = create_data()

    t1 = timeit.Timer(partial(func1, copy(test)))
    t2 = timeit.Timer(partial(func2, copy(test)))
    t3 = timeit.Timer(partial(func3, copy(test)))

    print(f"func1 timeit {t1.timeit(n_loops)} num test loops {n_loops}")
    print(f"func2 timeit {t2.timeit(n_loops)} num test loops {n_loops}")
    print(f"func3 timeit {t3.timeit(n_loops)} num test loops {n_loops}")

test results:

execution1:

func1 timeit 6.547473503 num test loops 1000
func2 timeit 12.467706045000002 num test loops 1000
func3 timeit 22.388469115 num test loops 1000

execution2:

func1 timeit 6.785610083000002 num test loops 1000
func2 timeit 12.850003099999999 num test loops 1000
func3 timeit 24.912014532 num test loops 1000
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading