Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Updating a pandas dataframe that exports Twitter Data to CSV

I am currently exploring the Twitter and wanted to save my results to a csv file.
I used:

#create dataframe
columns = ['source', 'created_at', 'ID', 'user', 'tweet', 'likes', 'location', 'coordinates']
data = []

for tweet in tweets:
    data.append([tweet.source, tweet.created_at, tweet.id, tweet.user.screen_name, tweet.full_text, tweet.favorite_count, tweet.user.location, tweet.coordinates])

df = pd.DataFrame(data, columns=columns)

df.to_csv('tweets.csv', index=True)

to make some appropriate columns and it works really well! Now I want to keep adding to this csv, but everytime I run the code it remakes the file instead of adding /to/ it. Is there a way I can make it so it would update the file instead of replacing it?

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

>Solution :

you can use the mode argument of the to_csv() method to specify that you want to append data to the file instead of overwriting it. To do this, you can use the mode=’a’ argument when calling the to_csv() method, like this:

df.to_csv('tweets.csv', index=True, mode='a')

This should work:

#create dataframe
columns = ['source', 'created_at', 'ID', 'user', 'tweet', 'likes', 'location', 'coordinates']
data = []

for tweet in tweets:
    data.append([tweet.source, tweet.created_at, tweet.id, tweet.user.screen_name, tweet.full_text, tweet.favorite_count, tweet.user.location, tweet.coordinates])

df = pd.DataFrame(data, columns=columns)

df.to_csv('tweets.csv', index=True, mode='a')
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading