spark get year from date

Matheus Batista answered on June 29, 2020 Popularity 8/10 Helpfulness 1/10

answer spark get year from date

spark get year from date

Comment

-1

import datetime
from pyspark.sql.functions import year, month, dayofmonth

elevDF = sc.parallelize([
    (datetime.datetime(1984, 1, 1, 0, 0), 1, 638.55),
    (datetime.datetime(1984, 1, 1, 0, 0), 2, 638.55),
    (datetime.datetime(1984, 1, 1, 0, 0), 3, 638.55),
    (datetime.datetime(1984, 1, 1, 0, 0), 4, 638.55),
    (datetime.datetime(1984, 1, 1, 0, 0), 5, 638.55)
]).toDF(["date", "hour", "value"])

elevDF.select(
    year("date").alias('year'), 
    month("date").alias('month'), 
    dayofmonth("date").alias('day')
).show()

Popularity 8/10 Helpfulness 1/10 Language whatever

Source: stackoverflow.com

Tags: date get whatever

Link to this answer
Share Copy Link