Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

pandas concating two default-indexed dataframes

I have data like (simplified for the sake of example):

list = [
    {
     age: 1234,
     val: 0.5,
     val2: 0.2
    },
    {
     age: 1234,
     val: 0.2,
     val2: 0.8
    },
]

I create pandas dataframe by frame = pandas.DataFrame(list) and it creates unnamed index from 0 to len(list) – 1.
Frame looks like:

     age  val  val2
0    1234 0.5  0.2
1    1234 0.2  0.8

Then I save it to csv by frame.to_csv('file.csv') – it goes ok.
But then, I want to create another frame exactly like this, load previous frame from csv file and add them together, so new data comes after old. I don’t care about the index too, preferably it could go from 0 to new length with added data.
I tried doing it by pd.concat([old_frame, new_frame], ignore_index=True) but final data has now 2 columns with index values like this:

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

           age  val  val2
0     0    1234 0.5  0.2
1     1    1234 0.2  0.8

How to properly concat the frames without creating additional index column each time?

>Solution :

You need to change this part of code frame.to_csv('file.csv', index=False)

This is how it will look

import pandas as pd

my_list = [
    {
     "age": 1234,
     "val": 0.5,
     "val2": 0.2
    },
    {
     "age": 1234,
     "val": 0.2,
     "val2": 0.8
    },
]

frame = pd.DataFrame(list)
frame.to_csv('file.csv', index=False) # <- change here


old_frame = pd.read_csv('file.csv')
new_frame = pd.DataFrame(list)



combined_frame = pd.concat([old_frame, new_frame], ignore_index=True)

The result will be:

age  val  val2
0  1234  0.5   0.2
1  1234  0.2   0.8
2  1234  0.5   0.2
3  1234  0.2   0.8
Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading