A readable and writable data store that will persist across tasks
Beam allows you to create persistent volumes that can be used across tasks. You might use persistent volumes for things like storing model artifacts created during training, or caching model weights to avoid having to download them each time your function is invoked.
You can read and write to the persistent volume like an ordinary Python file:
run.py
Copy
Ask AI
def get_model_weights(): with open("./weights/my_file.pkl", "r") as f: f.read() f.close()def update_model_weights(): with open("./weights/my_file.pkl", "w") as f: f.write("update I want to persist") f.close()
A good use case for Persistent Volumes is caching your ML model in order to avoid downloading it each time you make a prediction.If you’re using a Huggingface model, you can set the Huggingface cache location in app.py.