I try to execute this query in pyspark i get all the time error. I have looked everywhere but I don’t know or it doesn’t work if someone can help me. the goal of this request is to update a new column that I will later create called temp_ok :
this my code:

CASE WHEN _temp_ok_calculer='non' AND Operator level 2 ="XXX" OR  Operator level 2= "AAA" AND Auto Cleaning criteria !="YYY" Auto Cleaning criteria <> "AA"  AND Workstation Type = "Chaine A" THEN 'ok' ELSE CASE WHEN _temp_ok_calculer='ok' THEN 'ok' ELSE 'ko' END END

My table contains this columns: _temp_ok_calculer,Operator level 2, Auto Cleaning criteria, Workstation Type

>Solution :

Spark SQL uses back-tick ` for delimiters, and identifiers with spaces in them require using delimiters SQL. So

CASE WHEN _temp_ok_calculer='non' AND `Operator level 2` ="XXX" OR . . .