Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Why does size of the objects in a list affect the speed to step through the list?

I am learning python after having some java in school back in the day. If there are 2 lists, one with single digit integers and one with objects with a bunch of attributes, they appear to be a different speed to step through them? I was under the understanding that the objects are a memory pointer.

Example:

int_list = [1] * 1000
obj_list = [CustObject(0,1,2,3)] * 1000

total = 0
for i in int_list:
    total += i
print(total)

total = 0
for o in obj_list:
    total += o.int_variable
print(total)

I ran this with some tile objects that have about 20 attributes, one of them being an image. Obviously the tile list creation takes a lot more time, which is expected. But stepping through them is slower as seen from the output:

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

print("starting test")

ints = [[1 for i in range(100)] for j in range(80)]
print("created ints")

tiles = [[Tile(i, j, 30, 1) for i in range(100)] for j in range(80)]
print("created tiles")

# step through the ints
start_time = time.perf_counter()
total = 0
for i in range(0, len(ints)):
    for j in range(0, len(ints[i])):
        total += ints[i][j]
end_time = time.perf_counter()
elapsed_time = end_time - start_time
print("Ints Elapsed time: ", elapsed_time, total)

# step through the Tiles
start_time = time.perf_counter()
total = 0
for i in range(0, len(tiles)):
    for j in range(0, len(tiles[i])):
        total += tiles[i][j].tile_type
end_time = time.perf_counter()
elapsed_time = end_time - start_time
print("Tiles Elapsed time: ", elapsed_time, total)

Output

starting test
created ints
created tiles
Ints Elapsed time:  0.0011542000574991107 8000
Tiles Elapsed time:  0.002249700017273426 8000

>Solution :

Accessing ints[i][j] needs:

  • lookup for ints variable
  • lookup for i & j variables
  • array boundary check

Accessing tiles[i][j].tile_type needs the above plus name lookup for tile_type, so it’s slower.

Anyway, instead of trying to figure out why all this is slower, you can optimize this code a lot by forgetting about indices. This code:

for i in range(0, len(tiles)):
    for j in range(0, len(tiles[i])):
        total += tiles[i][j].tile_type

becomes (after directly iterating on the objects, not the indices)

for ta in tiles:
    for e in ta:
        total += e.tile_type

or even faster probably with sum and double flat comprehension:

total = sum(e.tile_type for for ta in tiles for e in ta)
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading