Do you ever start training a machine learning model on the Google Colab and the runtime disconnects before you can save your results? It use to happen to me and I would get frustrated. I would leave my model training to come back a few hours later and find it finished without saving!
Image by author.
So what can you do about this? Use Rclone¹, Google Drive², and Google Colab³ together with the following 3 major steps:
1) Install Rclone.
2) Configure Rclone.
3) Export with Rclone.
Step 1. Install Rclone.
Do this in the very first cell of your Google Colab Jupyter Notebook.
Step 2. Configure Rclone.
Configure the newly installed Rclone.
Step 2.1. Make a new remote
Step 2.2. Name it “remote”
Step 2.3. Enter “15” for the Google Drive
Step 2.4. Press Enter for default on client_id
Step 2.5. Press Enter for default on client_secret
Step 2.6. Enter “1” for Full access all files on your Google Drive.
Step 2.7. Press Enter for default on root_folder_id.
Step 2.8. Press Enter for default on service_account_file.
Step 2.9. Enter “n” for default on advanced config.
Step 2.10. Enter “n” for non-default on Remote config.
Go to the link it generates for you.
Copy the code from your Google Drive Sign in for Rclone access.
Paste it where it tells you to Enter verification code
Step 2.11. Enter “n” for default on Shared Drive.
Step 2.12. Enter “y” for default on remote settings.
Step 2.13. Enter “q” for Quit config.
Step 3. Export with Rclone.
Add an export after your model finishes training.
Copy your current “/content/” directory in the Google Colab to wherever you want on your Google Drive through the Rclone remote path we just established.
Summary
So after following those 3 major steps outlined above, you should be able to back-up whatever models you are training with Google Colab (assuming they finish).
Here is an example Jupyter Notebook that implements theses steps from start to finish.
You can see for that example, my Google Drive stored the trained models as such under content:
Image by author.
You can find more examples in this Github repository I made for various TensorFlow machine learning problem notebooks.
I hope you found this article helpful! Thank you for reading and hit the like if so. Follow me on here for more machine learning based content coming soon!
