I have about 300 csv files that I want to merge into a single one using pandas. All of them have 3 rows for variable names (project in the first one, Device_1 in the second one and variable in the third one). The first column goes something like (‘Asset’,’Element’,’Date’) and continues the timeseries values. Sometimes… Read More Replace MultiIndex names for the nth position in all levels
Let’s say I have a dataframe like this: data = {‘City’: [‘Rochester’, ‘Anaheim’, ‘Toledo’, ‘Rochester’, ‘Anaheim’, ‘Anaheim’, ‘Toledo’, ‘Rochester’, ‘Rochester’, ‘Rochester’, ‘Toledo’, ‘Toledo’, ‘Toledo’, ‘Anaheim’], ‘PersonID’: [4930, 7343, 4368, 6909, 4574, 4086, 5024, 3642, 9997, 4745, 1207, 6081, 7832, 6309], ‘MoneySpent’: [100, 1710, 20, 910, 2040, 1100, 490, 70, 1940, 100, 1240, 80, 1420, 2090],… Read More How to change the grouping of a column multi-index in a pandas pivot table?
I have a json which is in the following format : data = { ‘Low’: { ‘2023.07.01’: {"u": "mo", ‘N’: 1, ‘O’: 2, "PN": 22, "PO": 34}, ‘2023.07.02’: {"u": "no", ‘N’: 1, ‘O’: 2, "PN": 22, "PO": 34} }, ‘Medium’: { ‘2023.07.01’: {"u": "no", ‘N’: 1, ‘O’: 2, "PN": 22, "PO": 34}, ‘2023.07.02’: {"u": "mo",… Read More Pandas Multi-index on 2 Levels
How do I put labels on a DataFrame columns. I have a list with names of the label columns I have to put. This is the data frame And I should make it look like this I tried columns = [ (‘Age’, ‘0-18’),(‘Age’, ’19-25′), (‘Education’, ‘Low’),(‘Education’, ‘Medium’),(‘Education’, ‘High’)] column_names = [‘Age’,’Education’] labels = pd.MultiIndex.from_tuples(df.columns, columns)… Read More Put common label on columns in DataFrame with MultiIndex
I’m trying to learn how to use Pandas crosstab functionality but I can’t find way to access ‘upper level name’ of multi-index dataframe that crosstab produces. Simple example: df_test = pd.DataFrame.from_dict({‘A’: [1, 2, 3], ‘B’: [4, 5, 6]}, orient=’index’) df_test2 = pd.crosstab(df_test[0], df_test[1], margins=True) print(df_test2.index.names) Index.names gives me only ‘0’ but want to get list… Read More Accessing 'upper level name' of pandas multi-index
I have a pandas data frame where every row has a some readings from a sensor at different altitudes. I managed to get these readings in a matrix using the df.pivot function, where the index the identifier is, the columns the different altitude and three values of data at those altitudes. My matrix looks like… Read More Reorganise Columns in MultiIndex Matrix Pandas
I have the following data frame: import pandas as pd data = { "id_1": [1, 1, 1, 2, 2, 2], "id_2": [1, 1, 1, 2, 2, 2], "foo": [0.1, 0.1, 0.1, 0.2, 0.2, 0.2], } df = pd.DataFrame(data) df = df.set_index(["id_1", "id_2"]) which looks like this: foo id_1 id_2 1 1 0.1 1 0.1 1… Read More Getting index counter on a data frame
I have a DataFrame with multiple index levels. I define some subset by selecting multiple combinations of all levels but the last. Then I want to slice the original DataFrame with that subset, but I cannot find how. Best is to look at a simple example: In [1]: import pandas as pd In [2]: df… Read More Slice MultiIndex by multiple tuples
I am trying to filter my multi-indexed data df based on the column value employ_start_date and index employ_class . data = pd.DataFrame({‘agcy_nbr’: {0: 166, 1: 435, 2: 129, 3: 129, 4: 129}, ’employ_class’: {0: ‘Corr’, 1: ‘LE’, 2: ‘Corr’, 3: ‘Corr’, 4: ‘Corr’}, ’employ_start_date’: {0: 1999, 1: 2001, 2: 2002, 3: 204, 4: 1995}, ‘person_nbr’:… Read More Boolean filtering of different lengths with Multi Indexes
I have a double Multiindex dataframe as follows. I slice the rows with idx = pd.IndexSlice but I dont know how to do the same with the columns so provided this data: df = pd.DataFrame(data=pd.DataFrame(data=np.random.randint(0, 10, size=(9, 5)))) # rows list1 = [‘2021-01-01′,’2022-02-01′,’2022-03-01’] list2 = [‘PHOTO’, ‘QUE’,’TXR’] combinations = [(x, y) for x in list1… Read More python pandas: using pd.IndexSlice for both rows and columns in a double multiindex dataframe