schema for csv file in spark

Shiny Swiftlet answered on March 30, 2021 Popularity 8/10 Helpfulness 4/10

answer schema for csv file in spark

related schema for csv file in spark

schema for csv file in spark

Comment

import org.apache.spark.sql.types._

val customSchema = StructType(Array(
  StructField("project", StringType, true),
  StructField("article", StringType, true),
  StructField("requests", IntegerType, true),
  StructField("bytes_served", DoubleType, true))
)

val pagecount = sqlContext.read.format("csv")
  .option("delimiter"," ").option("quote","")
  .option("header", "true")
  .schema(customSchema)
  .load("dbfs:/databricks-datasets/wikipedia-datasets/data-001/pagecounts/sample/pagecounts-20151124-170000")

Popularity 8/10 Helpfulness 4/10 Language whatever

Source: stackoverflow.com

Tags: csv file schema whatever

Link to this answer
Share Copy Link

Contributed on Mar 30 2021

Shiny Swiftlet

0 Answers Avg Quality 2/10

Closely Related Answers

schema for csv file in spark

Comment

val pagecount = sqlContext.read.format("csv")
  .option("delimiter"," ").option("quote","")
  .option("schema","project: string ,article: string ,requests: integer ,bytes_served: long")
  .load("dbfs:/databricks-datasets/wikipedia-datasets/data-001/pagecounts/sample/pagecounts-20151124-170000")

Popularity 8/10 Helpfulness 3/10 Language whatever

Source: stackoverflow.com

Tags: file whatever schema csv whate

Link to this answer
Share Copy Link

Contributed on Mar 30 2021

Shiny Swiftlet

0 Answers Avg Quality 2/10

schema for csv file in spark

Contents

More Related Answers

schema for csv file in spark

Closely Related Answers

schema for csv file in spark

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.