how to use nvl in pyspark for variable?

June 20, 2023

i have a scenario like getting the maximum date from the table in hive and if the table is empty then max(column) will be null/none in that scenario replace that with "2022-01-01"

i am getting the max from

df1 = spark.table()
max_date_to=df1.agg(max(col("date"))).collect()[0].asDict()[‘max(date)’]

but my table is empty so i what to get the value as "2022-01-01"

>Solution :

You could simply use if else in this scenario,

df1 = spark.table()
check_date = df1.agg(max(col("date"))).collect()[0].asDict()['max(date)']
# Check if the check_date variable is not null if yes then use as it is else use the other date.
max_date_to = check_date if check_date else "2022-01-01"