Apologies this is probably something very obvious. I don’t understand why this isn’t working:
val df = spark.createDataFrame([("A",1),("B",2),("C",3)],["Letter","Number"])
When I run this I get the below error:
error: illegal start of simple expression
Why isn’t this working?
>Solution :
This is Pyspark syntax not Spark Scala. Try this instead:
val df = spark.createDataFrame(Seq(("A",1),("B",2),("C",3))).toDF("Letter", "Number")
Or:
import spark.implicits._
val df = Seq(("A",1),("B",2),("C",3)).toDF("Letter", "Number")