I’m working on a project where I have a pyspark dataframe of two columns (word, word count) that are string and bigint respectively. The dataset is dirty such that some words have a non-letter character attached to them (ex. ‘date’, ‘[date’, ‘date]’ and ‘_date’ are all separate items but should be just ‘date’) print(dirty_df.schema) output—>… Read More Pyspark Compare column strings, grouping if alphabetic character sets are same, but avoid similar words?

I have an element with title, type, description and so on. I have a DAO that defines all that and functions already created that extract the type for one element only: it’s getById(). How can I loop through all the elements and create groups of them by the same id? $group = array(); foreach ($announcement… Read More PHP group by similar value called by function

I have the following list. I’m trying to split it into odd & even lists using GroupBy: var x = new int[] { 2, 4, 6, 8, 10 }; var y = x .GroupBy(x => x % 2 == 0) .Select(x => x.ToList()) .ToList(); This outputs a List<List<int>>, the problem is that it doesn’t insert… Read More C# Linq GroupBy Return Default Value Instead Of Null

I am trying to replace a string for specific groups of customer ids/order dates. The example below may better illustrate my question. I have a dataframe: customerid <- c("A1", "A1", "A2", "A2", "A3", "A3", "A3", "A4") orderdate <- c("2018-09-14", "2018-09-14", "2018-09-15", "2018-09-15", "2020-08-21", "2020-08-21","2020-08-21", "2018-08-10") status <- c("review", "review", "review", "negative", "positive", "review", "review", "review")… Read More R – how to replace strings in a column based on 2 or more values

I have a table with the column "X". "X" contains int numbers. I would like to count <0, 0 and >0. Sure I can fire three statements like var p = t.Count(a => a.X < 0); var q = t.Count(b => b.X == 0); var r = t.Count(c => c.X > 0); But I believe… Read More C# Grouping with multiple conditions

I want to apply a conditional rowMeans to each group of IDs using dplyr. The reproducible table follows: dat <- as.Date("2021/08/04") len <- 4 seq(dat, by = "day", length.out = len) input <- data.frame( date = c(seq(dat, by = "day", length.out = len) , seq(dat, by = "day", length.out = len)), user_id = c(rep("aa",4),rep("bb",4)), var1… Read More Conditional rowMeans with dplyr

I have the following pandas dataframe: import pandas as pd df4 = pd.DataFrame({‘timestamp’:[‘2022-10-01 01:00:00’, ‘2022-10-02 01:00:00’, ‘2022-10-03 01:00:00’, ‘2022-10-04 01:00:00’, ‘2022-10-05 01:00:00’, ‘2022-10-01 02:00:00’, ‘2022-10-02 02:00:00’, ‘2022-10-03 02:00:00’, ‘2022-10-04 02:00:00’, ‘2022-10-05 02:00:00’], ‘A’: [1,2,3,4,5,6,7,8,9,10], ‘B’: [10,9,8,7,6,5,4,3,2,1]} ) df4[‘timestamp’] = df4[‘timestamp’].astype(‘datetime64’) df4 that gives the following data frame: | timestamp | A| B | |——————–|–| –|… Read More Pandas data frame with time-series data – grouping without aggregating data

I have a list like this: ls = [0, 1, 2, 4, 6, 7] # it won’t have duplicates and it is sorted Now I want to group this list into bins based on a offset (in this example offset=1) which should return this: [[0, 1, 2], [4], [6, 7]] # Note: # The offset… Read More Group numbers into bins based on offset with Python

I need to filter a data frame with different groups. The data frame looks as follows: df = pd.DataFrame({"group":[1,1,1, 2,2,2,2, 3,3,3, 4,4], "percentage":[70,70,70, 45,80,60,70, 71,85,90, np.nan, np.nan]}) My goal is to return a data frame containing only groups that satisfy one of the two following conditions: All observations of the group have percentage > 70… Read More Filtering DataFrame on groups where all elements of one group fullfills a one of various conditions

For each element in a group determine if it is present in the next group (in the order as these groups appear). For the last group – all False. Example: df = pd.DataFrame({‘group’: [ 0, 1, 1, 0, 2 ], ‘val’: [‘a’, ‘b’, ‘a’, ‘c’, ‘c’]}) grouped = df.groupby(‘group’) print(result) 0 True 1 False 2… Read More Can .apply use information from other groups?