spark read delta

Cristian Agudelo answered on June 23, 2022 Popularity 7/10 Helpfulness 5/10

answer spark read delta

spark read delta

Comment

# Define the input and output formats and paths and the table name.
read_format = 'delta'
write_format = 'delta'
load_path = '/databricks-datasets/learning-spark-v2/people/people-10m.delta'
partition_by = 'gender'
save_path = '/tmp/delta/people-10m'
table_name = 'default.people10m'

# Load the data from its source.
people = spark \
  .read \
  .format(read_format) \
  .load(load_path)

# Write the data to its target.
people.write \
  .partitionBy(partition_by) \
  .format(write_format) \
  .save(save_path)

# Create the table.
spark.sql("CREATE TABLE " + table_name + " USING DELTA LOCATION '" + save_path + "'")

Popularity 7/10 Helpfulness 5/10 Language whatever

Source: docs.databricks.com

Tags: delta whatever

Link to this answer
Share Copy Link