Before we start
- You need a PC with at least 8GB RAM (16 GB recommended to run 7B models efficiently)
- Nvidia GPU to run models (with official drivers)
- Linux/WSL installation
- Docker installed
Checking Docker installation
“docker run hello-world” is enough to check the docker installation.
You should see a screen like this. (Docker installation tutorial on WSL2 and Linux is coming soon!)
You can skip this step if you will install with "CPU only" option
Checking Nvidia drivers
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
##Test
docker run --gpus all nvidia/cuda:11.5.2-base-ubuntu20.04 nvidia-smi
You should see a confirmation text about nvidia toolkit
Make your choice and proceed with one of the options below
1) GPU installation
wget -0 ollama-docker-compose.yaml https://raw.githubusercontent.com/KaanErgun/scripts/main/docker-compose-ollama-cpu-easy.yaml
2) CPU only installation
wget -0 ollama-docker-compose.yaml https://raw.githubusercontent.com/KaanErgun/scripts/main/docker-compose-ollama-gpu-easy.yaml
3) Ollama without UI installation
!!! Prefer this option if you know what are you doing !!!
wget -0 ollama-docker-compose.yaml https://raw.githubusercontent.com/KaanErgun/scripts/main/docker-compose-ollama-without-UI.yaml
Composing the docker that you chose
docker compose up -d
Final step!
docker compose pull
docker compose up --force-recreate -d
You can use this one to update your local docker as well.
How to access to Web UI
Open your browser and browse to 3000 port of your localhost
Or just click here!