8. Persisting data

Finally, you can persist a dataset to storage using any of the “write” functions that Ray Data supports.

Lets write our predictions to a parquet dataset.

local_pred_folder = f"{storage_folder}/mnist_preds" # Change this to your local path if needed
ds_preds.write_parquet(local_pred_folder)

Refer to the Input/Output docs for a comprehensive list of write functions.