ComfyUI Setup: A GPU Server Guide for Stable Diffusion

Trying image generation models on your own computer is fun; but when you want to do the same job every day, share it with others or run it automatically, a home computer falls short. The graphics card is constantly busy, the computer sleeps or shuts down, and workflows stay tied to one machine. ComfyUI running on a GPU server solves these problems: the server is on around the clock, you reach it from anywhere through the browser, you keep workflows as files and automate them through the API. In this guide we install ComfyUI on a GPU server running Ubuntu 24.04 LTS and turn it into a service, work out the VRAM and disk you need, and show how to reach the interface safely without exposing it to the internet.

Why ComfyUI?

ComfyUI is a workflow interface in which you build image generation out of nodes. Every step, such as loading a model, writing a prompt, sampling, upscaling and saving, is a box on the screen and you connect the boxes to each other. This approach has three concrete benefits for server use. First, a workflow is a file: moving the same flow to another server or sharing it with a teammate is as easy as copy and paste. Second, ComfyUI has an HTTP API; you can run a workflow from a script instead of the interface and queue hundreds of images one after another. Third, because it runs only the steps that are needed, it is usually frugal with memory, which is an advantage on cards with limited VRAM.

VRAM and disk: how much do you need?

In image generation the most important resource is again VRAM. The table below gives a rough idea; the real need changes with resolution, batch size and the extensions you use.

Disk should not be underestimated either. A single SDXL checkpoint takes a few gigabytes; large models such as Flux can reach twenty gigabytes or more. As LoRAs, VAE files, control networks and upscaling models are added, tens of gigabytes pile up, and the images you generate add to that. Gen4/Gen5 M.2 NVMe disks meet both the capacity and the model loading speed. System RAM matters too: large models pass through it while loading, so 32 GB or more is a comfortable start. How the cards behave in image generation is covered in our RTX 5060 Ti guide and RTX 5090 guide.

1. Driver and base packages

First update the server, install the NVIDIA driver and add the required packages. For driver steps in detail, see our Ubuntu 24.04 guide:

sudo apt update && sudo apt -y upgrade
sudo ubuntu-drivers install
sudo apt -y install git python3-venv python3-pip build-essential
sudo reboot

After the reboot, confirm with nvidia-smi that the card is visible. Running ComfyUI under a separate user is good security practice; that way a problem with an extension affects only this user's area, not the whole system:

sudo adduser --disabled-password --gecos "" comfy
sudo -iu comfy

2. Download ComfyUI and set up the virtual environment

Run the following commands as the comfy user. First clone the repository and create an isolated Python environment:

git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip

The next step is installing PyTorch with the CUDA version that matches your graphics card, and this is where mistakes are most common. PyTorch's official install selector gives the right command for your operating system and CUDA version. RTX 50 series cards need a CUDA 12.8 or newer build; the command looks like this, and you should replace the version code with whatever the selector recommends:

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt

On older generation cards such as the Tesla P40, the current PyTorch wheel may not include that architecture. After installation, use the small test below to confirm both that CUDA works and that your card's compute capability is present in the build (for the P40 that value is 6.1):

python -c "import torch; print(torch.__version__, torch.cuda.is_available(), torch.cuda.get_device_name(0)); print(torch.cuda.get_arch_list())"

If you see True and your card's name, the foundation is ready. If it says False, the driver, the CUDA version or the card architecture does not match; review the nvidia-smi output and your PyTorch version.

3. Prepare the first model and the folder structure

ComfyUI looks for model files in subfolders under models: checkpoints for main models, loras for fine-tuning files, vae for decoders, controlnet for control networks and upscale_models for upscaling models. It is enough to put the .safetensors file of the model you downloaded in the right folder. When you download model files from a model repository, first read that model's licence; some models do not allow commercial use or ask you to accept terms before downloading. This article is not legal advice, and the terms of each model can differ.

Downloading the file straight to the server with wget or curl is much faster than downloading to your own computer first and uploading, because the server's gigabit connection is used directly. Do not forget to watch disk usage with df -h.

4. Starting it and reaching it through an SSH tunnel

Start ComfyUI so that it listens only on the local address. This prevents the interface from being exposed to the internet in the first place:

python main.py --listen 127.0.0.1 --port 8188

Open an SSH tunnel from your own computer and use the local address in your browser:

ssh -L 8188:127.0.0.1:8188 user@server-address

When you open http://localhost:8188 in your browser, the interface appears. Load a first workflow, write a prompt and generate. The first generation is slower because the model is loaded into memory; later ones are much faster. During generation, run nvidia-smi in another terminal and watch VRAM use: it shows how well the model fits your card.

5. A systemd service: start by itself

You do not want to step in by hand when the server restarts or ComfyUI crashes. For that, write a systemd service. With your administrator user, create the file /etc/systemd/system/comfyui.service:

[Unit]
Description=ComfyUI
After=network.target

[Service]
User=comfy
WorkingDirectory=/home/comfy/ComfyUI
ExecStart=/home/comfy/ComfyUI/venv/bin/python main.py --listen 127.0.0.1 --port 8188
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now comfyui
sudo systemctl status comfyui --no-pager
journalctl -u comfyui -f

While the service runs, you can always connect to the same address through the SSH tunnel. You can follow the logs with journalctl and start from that output when something goes wrong.

6. Tips for working with low VRAM

If your card's memory barely fits the model, a few methods help. First, lower the resolution and the batch size; memory use climbs fast with these two. Generating at a small resolution and then raising it with an upscaling model needs far less memory than generating directly at a large resolution. Second, start ComfyUI with the --lowvram flag; it works by moving model parts between memory and system RAM, so it slows down but fits. Third, prefer quantized model versions. Fourth, if VAE decoding runs out of memory, use the tiled VAE node. If none of these solves the problem, the real fix is a card with more VRAM.

7. Security: extensions run code

ComfyUI's power comes from community extensions (custom nodes), but every extension runs Python code on the server. Do not install extensions from sources you do not trust, look at what the repository does before installing it, and update regularly. The interface itself does not require authentication, so do not expose it to the internet. If several people will use it, set up a reverse proxy that provides HTTPS and authenticates users. For the server's basic security, apply our SSH and Linux security guide. Generated images fill the disk over time; clean the output folder regularly or point it to a separate disk.

8. Automation through the API

The real power of ComfyUI is that you can run a workflow from a script. If you turn on developer mode in the interface settings, you can save a workflow in "API format". You can trigger generation by sending this JSON to ComfyUI's /prompt address. Jobs such as producing a different prompt for each product in a catalogue or queueing a set of images overnight become automatic this way. If you also run your script on the server, no data leaves for the outside world.

First workflows and the right expectations

When you first open ComfyUI you see an empty canvas, but you do not have to connect nodes from scratch. The interface offers a ready default workflow, and you can load workflow files from community shares by dragging and dropping them into the interface. The best way to start is to run a simple text-to-image flow that matches the model family you use, then change values such as the step count, the sampler and the resolution one by one and observe the result. If a workflow does not run, there is usually a missing model file or a missing extension; ComfyUI tells you in the error message which node needs what.

Set expectations correctly from the start too: the first generation is noticeably slower than the ones after it because the model is loaded into memory. As long as you keep using the same model and settings, later generations speed up. When you switch models, a new load into memory is needed. That is why choosing one model for a job and producing the series with it is much more efficient than changing the model for every image.

Card choice and generation speed

Generation time depends on the card's compute power and its memory. The same SDXL workflow takes noticeably less time on a newer and more powerful card, but the real difference appears when resolution and batch size grow. On an 8 GB card you need --lowvram to reach a large resolution, while on a 24-32 GB card the same job runs directly and fast. If you will do regular, high-volume production, for example hundreds of images a day for a catalogue, a more powerful card saves time. If you will only generate a few images now and then, starting with an entry-level card and growing as the need arises makes sense.

Backup: what should you keep?

Model files can be downloaded again; what is really valuable is your workflows, the LoRA files you trained yourself and the images you produced. Back up workflow JSON files regularly to a repository, for example a Git repository, or to a backup kept off the server. Store the LoRA files you train separately. Do not leave generated images on the server for long; move the images you have delivered or archived off the disk.

9. Common problems

• CUDA out of memory: Lower the resolution, drop the batch size to 1 or start with --lowvram.
• Black or corrupted image: It usually comes from the VAE or the precision setting; try a different VAE or run the VAE at higher precision.
• The interface will not open: Confirm the SSH tunnel is open and that the output of systemctl status comfyui shows it running.
• PyTorch does not see the GPU: Check the value of torch.cuda.is_available() and whether the CUDA build is compatible with your card.
• The disk is full: Clean unused models and old outputs; du -sh models/* output shows which folder has grown.

Related Guides

• GPU server rental packages
• Dedicated servers located in Turkey
• RTX 5090 server guide
• How much VRAM does a local LLM need?
• What is a GPU VDS and what is it for?
• GPU rental guide: GPU server, GPU VDS and cloud compared
• Ollama and Open WebUI setup on an Ubuntu GPU server
• nvidia-smi errors and GPU server troubleshooting
• Blender GPU render server guide
• When do you need a dual GPU server?

Frequently Asked Questions