4/8/2023 0 Comments Scala arrayexplode – array column exampleĭf.select($"name",explode($"knownLanguages"))ĭf.select($"name",explode($"properties"))Įxplode_outer – Create rows for each element in an array or map. from the above example, Washington and Jefferson have null or empty values in array and map, hence the following snippet out does not contain these rows. This will ignore elements that have null or empty. When a map is passed, it creates two new columns one for key and one for value and each element in map split into the row. When an array is passed to this function, it creates a new default column “col1” and it contains all array elements. Spark function explode(e: Column) is used to explode or create array or map columns to rows. Val df = spark.createDataFrame((arrayData),arraySchema)Įxplode – spark explode array or map column to rows add("properties", MapType(StringType,StringType)) add("knownLanguages", ArrayType(StringType))
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |