Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Python conditional loop evaluating backwards

So I’ve got this loop I’m working on, and it is supposed to check n number of csv files, look for "TD" in cell A1, and if it doesn’t find it move the file to the badPath. But the it evaluates exactly backwards, meaning that files with "TD" get dumped into the badPath. This is my ‘working’ code. With the shutil.move(os.path.join(path,f),badPath) call outside of the with loop the code executes, only it identifies files with this identifier as not having it.

data = []

for f in TD_files:    
    with open(f, newline='', encoding='utf-8') as g: 
        r = csv.reader((line.replace('\0', '') for line in g))
        data = [line for line in r] 
        if data[0] != 'TD': 
            print('Not A Trend File') 
    shutil.move(os.path.join(path,f),badPath)

Code I have tried which hasn’t worked:

data = []

for f in TD_files:    
    with open(f, newline='', encoding='utf-8') as g: 
        r = csv.reader((line.replace('\0', '') for line in g))
        data = [line for line in r] 
        if data[0] != 'TD': 
            print('Not A Trend File') 
            shutil.move(os.path.join(path,f),badPath)

This produces a PermissionError because the processes are all tied up in one another. I have included an image of the error and called it error1.

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

data = []

for f in TD_files:    
    with open(f, newline='', encoding='utf-8') as g: 
        r = csv.reader((line.replace('\0', '') for line in g))
        data = [line for line in r if data[0] != 'TD']  
        print('Not A Trend File')
        shutil.move(os.path.join(path,f),badPath)

This code was my first inclination and it produces an IndexError: list out of range which leads me to assume that my evaluations are incorrect.

Question

Am I having syntactical errors, logical errors with my understanding of the python language and its general logic and rules, or is there something simple here I’m missing? I am running my tests on a batch on 127 files, each range from 10,000 to 50,000 rows & 26 to 50 columns. But they each have ‘TD’ in cell A1, so this leads me to assume I’m having a logical misunderstanding here, because each and every file is being moved to the badPath.

error1

>Solution :

This produces a PermissionError because the processes are all tied up in one another.

We cannot move the file until it is closed; and it is open for the duration of the with block. However, with does not create a new scope in Python (very few things do), so we can easily compute values and then use them afterwards.

Rather than read the entire file in order to use indexing to get the first line, we can just read the first line out of the csv.reader, check whether the file is "bad", and set a flag to use afterwards:

for f in TD_files:    
    with open(f, newline='', encoding='utf-8') as g: 
        r = csv.reader((line.replace('\0', '') for line in g))
        is_trend = next(r)[0] == 'TD'
    if not is_trend:
        print('Not A Trend File')
        shutil.move(os.path.join(path, f), badPath)

(For that matter, we can probably just check whether the actual text of the file starts with TD, rather than parsing it as a CSV…)

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading