Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Trim leading zero's using python pandas without changing the datatype of any columns

I have a csv file of around 42000 lines and around 80 columns, from which I need to remove leading Zero’s, hence I am using Pandas to_csv and saving it back to text file by which leading Zero’s are removed.

Any column may contain null values in any row, but those columns are getting converted to Float datatype and getting decimal values as output, I want to avoid that scenario.

For example, below is a sample of my original file.

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

0000055|O|Price Rite Marketplace|361|1600 Memorial Dr|Chicopee|MA|010203933|Chicopee|25|013|USA|05|1|H|C|42.2001|-72.5731|A|250138113012012|||10
0000071|O|Big Es Supermarket|189|11 Union St|Easthampton|MA|010271417|Easthampton|25|015|USA|05|5|A|I|42.2697|-72.6717|A|250158224021037||
0000084|O|Big Y Supermarket|14|441 N Main St|East Longmeadow|MA|010281804|East Longmeadow|25|013|USA|05|5|G|C|42.0788|-72.5280|A|250138134012011|||15
0000101|O|Stop & Shop|95|440 Russell St|Hadley|MA|010359566|Hadley|25|015|USA|05|5|K|C|42.3644|-72.5382|A|250158214004004|||14
0000139|O|Key Food Marketplace|2508|13 Cabot St|Holyoke|MA|010406055|Holyoke|25|013|USA|05|5|A|C|42.1980|-72.6042|A|250138115002019|||06
0000149|O|Stop & Shop|9|28 Lincoln St|Holyoke|MA|010403472|Holyoke|25|013|USA|05|5|K|C|42.2150|-72.6172|A|250138118005012|||13

I used the below method to convert to

import pandas as pd
df = pd.read_csv(r"/home/ter/stest/cminxte1.txt", sep="|")
df.to_csv(r"/home/ter/stest/cminxte.txt", sep='|', index=False)

The output file looks like

55|O|Price Rite Marketplace|361|1600 Memorial Dr|Chicopee|MA|10203933|Chicopee|25|13|USA|5|1|H|C|42.2001|-72.5731|A|250138113012012|||10.0
71|O|Big Es Supermarket|189|11 Union St|Easthampton|MA|10271417|Easthampton|25|15|USA|5|5|A|I|42.2697|-72.6717|A|250158224021037||
84|O|Big Y Supermarket|14|441 N Main St|East Longmeadow|MA|10281804|East Longmeadow|25|13|USA|5|5|G|C|42.0788|-72.528|A|250138134012011|||15.0
101|O|Stop & Shop|95|440 Russell St|Hadley|MA|10359566|Hadley|25|15|USA|5|5|K|C|42.3644|-72.5382|A|250158214004004|||14.0
139|O|Key Food Marketplace|2508|13 Cabot St|Holyoke|MA|10406055|Holyoke|25|13|USA|5|5|A|C|42.198|-72.6042|A|250138115002019|||6.0
149|O|Stop & Shop|9|28 Lincoln St|Holyoke|MA|10403472|Holyoke|25|13|USA|5|5|K|C|42.215|-72.6172|A|250138118005012|||13.0

It has removed all the leading Zero’s in all the columns as expected, however, at the last column, it is converting to float with decimal values as that column has got null values. Any idea on how can this be rectified?

>Solution :

First convert all values to strings and in next step remove trailing zeros:

df = pd.read_csv(r"/home/ter/stest/cminxte1.txt", sep="|", dtype=str) 
df = df.apply(lambda x: x.str.lstrip('0'))
df.to_csv(r"/home/ter/stest/cminxte.txt", sep='|', index=False)
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading