foreach batch spark scala

Ogun OZALP answered on May 17, 2022 Popularity 6/10 Helpfulness 3/10

answer foreach batch spark scala

foreach batch spark scala

Comment

streamingDF.writeStream.foreachBatch { (batchDF: DataFrame, batchId: Long) =>
  batchDF.persist()
  batchDF.write.format(...).save(...)  // location 1
  batchDF.write.format(...).save(...)  // location 2
  batchDF.unpersist()
}

Popularity 6/10 Helpfulness 3/10 Language scala

Source: docs.databricks.com

Tags: foreach scala

Link to this answer
Share Copy Link

Contributed on May 17 2022

Ogun OZALP

0 Answers Avg Quality 2/10

Continue with Google

By continuing, I agree that I have read and agree to Greppers's Terms of Service and Privacy Policy.

foreach batch spark scala

Contents

More Related Answers

foreach batch spark scala

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.