spark.read.option(header,inferschema) .csv example

Helpful Hamster answered on April 25, 2021 Popularity 7/10 Helpfulness 4/10

answer spark.read.option(header,inferschema) .csv example

spark.read.option(header,inferschema) .csv example

Comment

val customSchema = StructType(Array(
    StructField("numicu", StringType, true),
    StructField("fecha_solicitud", TimestampType, true),
    StructField("codtecnica", StringType, true),
    StructField("tecnica", StringType, true),
    StructField("finexploracion", TimestampType, true),
    StructField("ultimavalidacioninforme", TimestampType, true),
    StructField("validador", StringType, true)))

val df_explo = spark.read
        .format("csv")
        .option("header", "true")
        .option("delimiter", "\t")
        .option("timestampFormat", "yyyy/MM/dd HH:mm:ss") 
        .schema(customSchema)
        .load(filename)

Popularity 7/10 Helpfulness 4/10 Language whatever

Source: stackoverflow.com

Tags: whatever

Link to this answer
Share Copy Link