Deep learning with PyTorch: step-by-step on CUDO Compute

The simplicity and Pythonic nature of PyTorch make developing simple and complex deep-learning models easy. As discussed previously, PyTorch's use of a dynamic computational graph, which allows a model’s structure to change during runtime, provides flexibility and enables real-time adjustments during training.

This flexibility has led to PyTorch being successfully used to develop a wide range of deep learning models, from simple image classifiers to complex large language models (LLMs) like OpenAI's GPT-2 and GPT-3.

However, training these models often demands significant computational resources that many projects and individual developers find too expensive to acquire and maintain. Utilizing a cloud service like CUDO Compute, which offers scalable GPU resources and a cost-effective pricing model, becomes a compelling solution.

If you're eager to use PyTorch without breaking the bank or struggling with limited compute resources, this guide is for you. Here, we’ll cover how to build and deploy your PyTorch application on CUDO Compute. Before we begin, let’s talk about what you might need to know to follow this guide:

• A computer for accessing CUDO Compute: This can be any computer with an internet connection and a web browser. However, to secure your account credentials, it would be best to use a personal computer.
• A CUDO Compute account and access to a CUDO Compute server: To follow along with this guide, you'll need to create an account on CUDO Compute and deploy a server. After creating an account, make sure you add a billing account. If you do not have an account, click here to sign up.
• Basic knowledge of SSH and terminal commands: We'll be using these to interact with your CUDO Compute server. Don't worry if you're not an expert; we'll explain each command as we go along.

With all of these checked, let’s dive into creating a project.

Step 1: Creating a project

After signing into your new account, you will see this screen.

First, you must set up a billing account to pay for your projects. Once completed, click on ‘Projects’ and then ‘Create a Project’.

This will bring up a sidebar where you need to add a Project ID and select a billing account. Add those, and then create the project.

With these simple steps, you have created a project. Next, let’s discuss creating a virtual machine (VM) and a hard disk.

Step 2: Create a VM.

Before we create the VM, we need to generate an SSH key pair to help you securely access it. We will use simple steps that are easy to follow regardless of your operating system.

2.1 Create SSH key pair

Open your terminal, then type this command and press enter:

ssh-keygen -t ed25519 -C "your\[email protected]"

This command generates a new SSH key pair using the ED25519 algorithm and associates it with your email address for identification.

You will be prompted to enter a file name for the key. This will be the result on Windows:

C:\Users\test-user>ssh-keygen -t ed25519 -C "your\[email protected]"

Generating public/private ed25519 key pair.

Enter file in which to save the key (C:\Users\test-user/.ssh/id\_ed25519):

This will be the result on Linux or Mac:

testuser@users:~$ ssh-keygen -t ed25519 -C "your\[email protected]"

Generating public/private ed25519 key pair.

Enter file in which to save the key (/home/testuser/.ssh/id\_ed25519):

The files will be saved in your .ssh folder. On Windows, your keys will be:

• Your public key will be saved as C:\Users<YourUsername>.ssh\id_ed25519.pub.
• Your private key will be saved as C:\Users<YourUsername>.ssh\id_ed25519.

On Mac and Linux, they will be saved as:

• Your public key will be saved as ~/.ssh/id_ed25519.pub.
• Your private key will be saved as ~/.ssh/id_ed25519.

You can protect your key with a passphrase for added security or press Enter to skip this step.

The next step is to copy the Public Key to the CUDO Compute. On your CUDO Compute dashboard, click on SSH Keys.

Click ‘Add an SSH key’ and paste the public key into the dialog box, as shown in the image below.

Now that you have added your public key, you can create your virtual machine and disk.

2.2 Create a virtual disk and VM

To begin, click on ‘Projects’, then select the project for which you want to create the VM. This project-based separation helps to keep your resources organized.

The first step is to create a disk, which acts as your project's hard drive. It is very important to get the right amount of storage space when you create it. Your models, data, and applications will be saved on the disk, so you want to get enough storage to handle them but not too much so you don’t have too much idle space, raising your cost.

This step is not strictly necessary, but if you want to download a lot of models and data, and still have the option of destroying your VM without losing them, then you need this, which is good for small projects with intermittent VM use. You can also use the disk to make an identical copy of one VM and then replicate it many times.

Click on ‘Disks’, then ‘Create a Disk’. Configure the disk setting to fit your demand. Be sure the data center selected is the exact location of your GPU. The disk holds all the packages and images you need for your project. Make sure you have enough space on your disk.

Create the disk and wait for it to get to a ready state.

Next, you can set up a private network and create security groups. We will skip those steps for this guide, but in a production environment, ensure you set both up.

Now, we will create the VM. Click on ‘Virtual Machines’.

Click on ‘Create a Virtual Machine’ to open the configuration dialog box. Provide the name of the VM, then select a datacenter. You can either select the data center you want or leave it as any data center.

Next, configure the VM specification. Select the GPU you want, the number of vCPUs, and the memory you need for your project.

Next, add the boot disk image you want. For PyTorch projects, the recommended image is the Ubuntu 22.04 + NVIDIA drivers + Docker. You can customize the size of the boot disk depending on the size of your project. We recommend a minimum of 50GB (like we've done in this guide) but, if you want to download pretrained models like LLMs or large datasets, you should get 100GB.

Next, add the storage created earlier as well if you created one. We’re not adding a network and security group as we did not create any, but add the ones you created earlier.

Manage the security settings for the virtual machine by choosing which SSH public keys can access it. Set the root password and add any startup script you need to run as soon as you boot the VM. You can also edit the metadata if necessary.

Finally, deploy your VM and wait for the status to become Active.

Step 3: SSH into the server from your computer

Your VM is now ready, and you are ready to log into it. Click on the ‘copy SSH’ command (circled in blue in the image below), then copy your VM’s IP address.

Open the terminal and type the following on Windows:

ssh -i C:\Users\<YourUsername>\.ssh\id\_ed25519 root@server\_ip\_or\_hostname

On Linux or Mac:

ssh -i ~/.ssh/id\_ed25519 username@server\_ip\_or\_hostname

This command specifies the path to your private key file and initiates a connection to the server.

There are numerous ways log in to your VM, but for ease, we will show you a very easy way. On your computer, open the config file in your .ssh folder. Add the following lines for Windows:

Host myserver
HostName <server\_ip\_address>

User root
IdentityFile "C:\Users\Your-User-Name\.ssh\id\_ed25519"

On Linux or Mac:

Host myserver
HostName <server\_ip\_address>

User root
IdentityFile ~/.ssh/id\_ed25519

NOTE: HostName will be your VM’s IP address, IdentityFile should be the path to your Private Key, and the Host can be any name you can easily remember.

With this, you can simply type:

ssh myserver

This establishes a secure connection to the CUDO Compute server, and you will be logged into your VM automatically.

Step 3: Install and use PyTorch on CUDO Compute

The first step is to update and upgrade your VM, ensuring the server has the latest updates and security patches. Using this command:

sudo apt update && sudo apt upgrade -y

Then install pip, which we will use when installing other Python libraries and frameworks with this command:

apt install python3-pip

Next, we install PyTorch and related packages using the following command:

pip3 install torch torchvision torchaudio3

This will take a moment to run.

When it is done, test if you now have PyTorch running in your VM by running these commands sequentially:

$ python3>>> import torch>>> print(torch.cuda.is_available())

It should look like the image below.

Conclusion

In this guide, we have accomplished several key steps:

• Generated SSH Keys: We created a secure key pair to access your CUDO Compute server without repeatedly entering passwords. This ensures a safe and convenient connection.
• Connected to CUDO Compute Server: We established a secure connection to your virtual machine (VM) on CUDO Compute using the SSH keys. This VM is your dedicated space in the cloud where you'll be running your PyTorch projects.
• Installed PyTorch: We updated and upgraded the VM's software and then installed PyTorch, which will be the backbone of your projects.

The possibilities are endless now that you have PyTorch up and running on your CUDO Compute VM. You can experiment with various deep learning models, train them on large datasets, and use our scalable GPU resources to accelerate your research or development.

Check out our docs and resources, try different architectures, and tackle challenging deep-learning tasks with the right support.