pyspark pivot row without aggrefation

I have Pyspark Dataframe named df as below,

enter image description here

I need to pivot the data based on ProducingMonth and classification column and need to produce the following output

enter image description here

I am using the following pyspark code


while I am displaying the data I am getting error "’GroupedData’ object has no attribute ‘display’"

>Solution :

You need to perform the aggregation after.

from pyspark.sql import functions as F

pivotDF = df.groupBy("WELL_ID","producing_month").pivot("CLASSIFICATION").agg(

Then you can probably use display pivotDF.display()

Leave a Reply