Before we start
- You need a PC with at least 8GB RAM (16 GB recommended to run 7B models efficiently)
- Nvidia GPU to run models (with official drivers)
- Linux/WSL installation
- Docker installed
Checking Docker installation
“docker run hello-world” is enough to check the docker installation.
You should see a screen like this. (Docker installation tutorial on WSL2 and Linux is coming soon!)
You can skip this step if you will install with "CPU only" option
Checking Nvidia drivers
nvidia-container-toolkit.bash
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
nvidia-container-toolkit.bash
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
##Test
docker run --gpus all nvidia/cuda:11.5.2-base-ubuntu20.04 nvidia-smiYou should see a confirmation text about nvidia toolkit
Make your choice and proceed with one of the options below
1) GPU installation
Ollama-GPU.bash
wget -0 ollama-docker-compose.yaml https://raw.githubusercontent.com/KaanErgun/scripts/main/docker-compose-ollama-cpu-easy.yaml2) CPU only installation
Ollama-CPU.bash
wget -0 ollama-docker-compose.yaml https://raw.githubusercontent.com/KaanErgun/scripts/main/docker-compose-ollama-gpu-easy.yaml3) Ollama without UI installation
!!! Prefer this option if you know what are you doing !!!
Ollama-without-UI.bash
wget -0 ollama-docker-compose.yaml https://raw.githubusercontent.com/KaanErgun/scripts/main/docker-compose-ollama-without-UI.yamlComposing the docker that you chose
docker-compose.bash
docker compose up -dFinal step!
Pull-and-run.bash
docker compose pull
docker compose up --force-recreate -dYou can use this one to update your local docker as well.
How to access to Web UI
Open your browser and browse to 3000 port of your localhost
Or just click here!
