I have the following data frame: import pandas as pd data = { "id_1": [1, 1, 1, 2, 2, 2], "id_2": [1, 1, 1, 2, 2, 2], "foo": [0.1, 0.1, 0.1, 0.2, 0.2, 0.2], } df = pd.DataFrame(data) df = df.set_index(["id_1", "id_2"]) which looks like this: foo id_1 id_2 1 1 0.1 1 0.1 1… Read More Getting index counter on a data frame

I have a DataFrame with multiple index levels. I define some subset by selecting multiple combinations of all levels but the last. Then I want to slice the original DataFrame with that subset, but I cannot find how. Best is to look at a simple example: In [1]: import pandas as pd In [2]: df… Read More Slice MultiIndex by multiple tuples

I am trying to filter my multi-indexed data df based on the column value employ_start_date and index employ_class . data = pd.DataFrame({‘agcy_nbr’: {0: 166, 1: 435, 2: 129, 3: 129, 4: 129}, ’employ_class’: {0: ‘Corr’, 1: ‘LE’, 2: ‘Corr’, 3: ‘Corr’, 4: ‘Corr’}, ’employ_start_date’: {0: 1999, 1: 2001, 2: 2002, 3: 204, 4: 1995}, ‘person_nbr’:… Read More Boolean filtering of different lengths with Multi Indexes

I have a double Multiindex dataframe as follows. I slice the rows with idx = pd.IndexSlice but I dont know how to do the same with the columns so provided this data: df = pd.DataFrame(data=pd.DataFrame(data=np.random.randint(0, 10, size=(9, 5)))) # rows list1 = [‘2021-01-01′,’2022-02-01′,’2022-03-01’] list2 = [‘PHOTO’, ‘QUE’,’TXR’] combinations = [(x, y) for x in list1… Read More python pandas: using pd.IndexSlice for both rows and columns in a double multiindex dataframe

Suppose that I have a dataframe which can be created using code below df = pd.DataFrame(data = {‘date’:[‘2021-01-01’, ‘2021-01-02’, ‘2021-01-05′,’2021-01-02’, ‘2021-01-03’, ‘2021-01-05’], ‘product’:[‘A’, ‘A’, ‘A’, ‘B’, ‘B’, ‘B’], ‘price’:[10, 20, 30, 40, 50, 60] } ) df[‘date’] = pd.to_datetime(df[‘date’]) I want to create an empty dataframe let’s say main_df which will contain all dates between… Read More Create Multi-Index empty DataFrame to join with main DataFrame

The code below shows what I would like to do. Is it possible to do it without the iteritems iteration? Basically I have a DataFrame df with a MultiIndex, and I have a smaller, boolean DataFrame sdf, without the MultiIndex. I would to use sdf as a preselection/mask to set values in df. mi =… Read More Pandas DataFrame subselection

Am trying to create a multi-column table, meaning a column has three sub-columns. I tried a solution from this: How to Insert a List of Data into Pandas Multi-Index Dataframe But the thing is that it passes a list, each item in the list corresponds to a column; i want to pass a data frame… Read More Pandas, Multi Column Table

I am calculating concentrations from a file that has separate Date and Time columns. I set the date and time columns as indexes as they are not suppose to change. However when I print the new dataframe it only prints the "date" one time like this: 55… Date Time 2020-12-30 10:37:04 0.000000 … 10:37:07 0.000000… Read More Pandas: Printing all the values from an index

I’m trying to add a third column "Productivity" so every role like Admin would have three sub columns produktiv, unproduktiv and Productivity. Productivity would be calculated as follows: Productivity = Produktiv / (Produktiv + Unproduktiv) * 100 (don’t mind the s, I had to anonymize the data) Here is the output of df.columns Any help… Read More How to add another subcolumn to dataframe with multi-index

What I want to do I have two pandas.DataFrame, df1 and df2. Both have the same columns. All indices in df2 are also found in df1, but there are some indices that only df1 has. Rows with an index that is owned by both df1 and df2, use rows of df2. Rows with an index… Read More Replace values in pandas.DataFrame using MultiIndex