Local storage encryption in Spark 3
Spark ensures encryption for RPC Connections and it allows users to encrypt data they are storing on local drive (includes shuffle files, shuffle spills, and data blocks).
Note: Spark does not support the encryption of data generated by applications with APIs, such as “saveAsHadoopFile”, or temporary files created by the user.
The following table summarizes how to enable encryption for data that is written to script:
Free Resources
Copyright ©2024 Educative, Inc. All rights reserved
TRENDING TOPICS
