I have a sample file like this sample.json { "school":[ { "testId":123, "testName":"test1", "Teachers":[ { "Tid":111, "Tname":"aaa" }, { "Tid":222, "Tname":"bbb" }, { "Tid":333, "Tname":"ccc" }, { "Tid":444, "Tname":"ddd" } ], "location":"India" } ] } i need to normalize the Teachers list from this file, my code: import json import pandas as pd with open(‘sample.json’,… Read More How to normalize a list from a json file using python?

I have a dataframe and want to randomly select 5 rows from it and have them in another dataframe, then remove those rows from my first dataframe. For example in this dataframe: Randomly select these rows: And have first dataframe without them : How Can I do this action? >Solution : You can use sample:… Read More Select and remove rows in pandas dataframe

Trying to reduce the overhead of iterrows by changing it to itertuples. (there are many columns) I’m trying to turn this with iterrows def named_tuple_issue_iterrows(df: pd.DataFrame, column_name: str): for index, series in df.iterrows(): result = series[column_name] # Do something with result Into itertuples. def named_tuple_issue_itertuples(df: pd.DataFrame, column_name: str): for namedtuple in df.itertuples(): result = namedtuple[column_name]… Read More Converting iterrows into itertuples and accessing namedtuples

I have 2 dataframes as such: DF1: Name Age Score A 20 55 B 21 60 C 22 20 D 23 25 E 24 30 The second dataframe will contain similar data but with less info. Crucially, all the names in the the second dataframe are in the first one and I am taking a… Read More Check for matching columns and sum values over

The way I’ve done this is rather cumbersome. How would I be able to adjust the below code for the link to be amended with the country codes in the list, and load these json links, rather than unioning different dataframes. Your help is much appreciated! import urllib.request import json import pandas as pd from… Read More Basic Python Questions

I have two dataframe df1 and df2, df1 & df2 have column "A". I want output df3 has column "A" which has values of df1 not exist on df2. df1 A I-13856942 I-13856914 I-13861633 I-13875002 I-13875673 df2 A I-13856942 I-13856914 I-13861633 output df3 A I-13875002 I-13875673 >Solution : A possible solution: df1.loc[df1.merge(df2, how=’left’, indicator=True)[‘_merge’].eq(‘left_only’),:] Output:… Read More Get column values not exist in another column pandas

Having tried all available solutions that I could find both on StackOverflow and elsewhere, I still find myself stuck with this challenge. I want to group time by the Hour in other to obtain 12 different hours, and I have used several groupby() syntaxes but all keep throwing KeyError: "time" at me. After double-checking the… Read More Impossible to group by the hour – KeyError: "time"

I have two dataframes. The first one has thousands of columns which represent a given city followed by the year, like "London_2001", "London_2002", and some measurements in the rows. The second dataframe has two columns. The first one is a territorial region, and the second one a list of cities. Something like: df1 <- data.frame(London_2021… Read More How to create a list of dataframes based on two other dataframes

I read a pdf file into a DataFrame using tabula and used .concat() to combine it all into one DataFrame by doing the following: import pandas as pd import tabula df = tabula.read_pdf(‘card_details.pdf’, pages=’all’) df = pd.concat(df, ignore_index=True) I want to clean some of this data as a column which contains card numbers also has… Read More Pandas DataFrame: .replace() and .strip() methods returning NaN values

Im very new to pandas I have a CSV that contains 43 states and a count of how many times something has happened in that state. STATE,Count AL,1 AK,4 AZ,7 My CSV does not contain every state, how can I ensure that every state is accounted for? If its not in the original dataframe it… Read More How can I ensure every us state is accounted for in a pandas dataframe?