loop doesn't work for TextIOWrapper for a certain situation

February 3, 2022

The following code gives an empty chunk list when I use the second line ‘text = f.read()’ after opening the file as a TextIOWrapper. However, it runs properly when I omit the ‘text=f.read()’ line. Can anyone please explain why it behaves like this?

with open(r'LDcal20_220127.3dm','r') as f:
    text = f.read()
    
    chunk = [] 
    for line in f:
        if line.startswith('ND'):
            chunk.append(line)

>Solution :

text = f.read() leaves the file pointer at the end of the file. The first time next(f) is called by the for loop, a StopIteration exception is raised because there’s nothing left to read from the file, terminating the loop immediately.

If you want to iterate over the file, you need to either close it and creat a fresh object:

with open(r'LDcal20_220127.3dm','r') as f:
    text = f.read()
    
with open(r'LDcal20_220127.3dm','r') as f:
    chunk = [] 
    for line in f:
        if line.startswith('ND'):
            chunk.append(line)

or reset the pointer with seek:

with open(r'LDcal20_220127.3dm','r') as f:
    text = f.read()
    
    f.seek(0)
    chunk = [] 
    for line in f:
        if line.startswith('ND'):
            chunk.append(line)

I would recommend the first solution, though. Mixing the iterator protocol (which itself calls read on the file object) with direct file reads can produce unpredictable results.