Oobabooga Text Generation WebUI Complete Guide

The Power User’s Local LLM Interface

Oobabooga’s Text Generation WebUI (often called text-generation-webui or simply “ooba”) is one of the most feature-rich frontends for running local language models. While tools like LM Studio and GPT4All target mainstream users, Oobabooga is built for power users who want granular control over inference parameters, model loading backends, and extensibility through a robust plugin system.

In 2026, it remains the go-to choice for running large models, experimenting with fine-tunes, building character chatbots, and integrating local LLMs into custom pipelines.

---

Key Features

• Supports GGUF (via llama.cpp), GPTQ, AWQ, EXL2, and full-precision Transformers models
• Multiple UI modes: chat, default (instruct), notebook
• Character/persona system with custom avatars and backgrounds
• Extensions system for adding capabilities (TTS, STT, code execution, image gen)
• OpenAI-compatible API for seamless integration
• Training and fine-tuning LoRA support built in

---

Installation

Prerequisites

• Python 3.11
• Git
• NVIDIA GPU with CUDA (optional but recommended) or CPU-only mode
• 8+ GB RAM minimum, 16 GB recommended

One-Click Installers (Windows/Linux/macOS)

The easiest method — download the appropriate script from the GitHub releases page:

# Linux / macOS
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
./start_linux.sh          # or start_macos.sh

# Windows — run in PowerShell
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
.\start_windows.bat

The installer script creates a conda environment, installs PyTorch with the right CUDA version, and starts the server automatically.

Manual Installation

# Create a virtual environment
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate

# Install PyTorch with CUDA 12.4
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

# Install text-generation-webui
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
pip install -r requirements.txt

# Start the server
python server.py --api

Access the UI at http://localhost:7860.

---

Loading Models

GGUF Models (llama.cpp backend)

GGUF is the most common format and works on CPU or GPU. Download models from Hugging Face — look for repos by TheBloke, bartowski, or lmstudio-community.

# Download a model to the models directory
cd text-generation-webui
wget https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/resolve/main/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf \
  -P models/

In the WebUI:

1. Click the Model tab
2. Select llama.cpp as the loader
3. Choose your .gguf file from the dropdown
4. Set n-gpu-layers (e.g., 35 for 8B model on 8GB VRAM, 99 for full GPU)
5. Click Load

Transformers Models (Hugging Face Hub)

# Download directly from Hugging Face
python download-model.py mistralai/Mistral-7B-Instruct-v0.3

Or enter the model name in the UI’s Download model field.

GPTQ and AWQ Models

# Install GPTQ support
pip install auto-gptq

# Install AWQ support
pip install autoawq

Select AutoGPTQ or AutoAWQ as the loader in the Model tab.

---

GPU Layer Configuration

GPU VRAM	7B Model Layers	13B Model Layers
4 GB	20–25	10–15
6 GB	28–32	18–22
8 GB	35 (full)	25–28
12 GB	35 (full)	35 (full)
24 GB	35 (full)	40 (full)

Set n-gpu-layers to the values above. Any layers that don’t fit on GPU fall back to CPU automatically.

---

Chat Modes and Instruction Templates

Oobabooga’s chat mode requires the correct instruction template for your model. Using the wrong template causes garbled or low-quality responses.

Setting the Template

1. Go to the Chat tab
2. Click Instruction template
3. Select the correct template:
   • Llama-3: for Meta Llama 3.x models
   • Mistral: for Mistral/Mixtral models
   • ChatML: for Qwen, Phi-3, and many others
   • Alpaca: for older Alpaca-based fine-tunes

The WebUI usually auto-detects the template from the model name. If it doesn’t, check the model card on Hugging Face.

---

Character System

Oobabooga has a rich character system for building persistent AI personas.

Creating a Character

1. Go to Characters → New Character
2. Fill in:
   • Name: displayed in chat
   • Context: the system prompt defining personality
   • Greeting: the character’s opening message
   • Avatar: upload an image (JPG/PNG)
3. Save and select the character in chat

Example context for a security assistant:

You are CyberGuide, an expert in cybersecurity, penetration testing, and ethical hacking.
You explain complex security concepts clearly and provide practical examples.
You always emphasize responsible disclosure and legal authorization before testing.

Characters are saved as .yaml files in characters/ and can be shared.

---

Extensions

Extensions dramatically expand Oobabooga’s capabilities. Enable them via the Session tab → Extensions.

Popular Extensions

Extension	Function
openai	OpenAI-compatible API server
superbooga	Enhanced RAG with ChromaDB
silero_tts	Text-to-speech output
whisper_stt	Speech-to-text input
sd_api_pictures	Stable Diffusion image generation
multimodal	Vision support (LLaVA models)
google_translate	Auto-translate responses

Enabling Extensions via CLI

python server.py --extensions openai superbooga silero_tts

Installing Additional Extensions

# Most extensions install their own dependencies
pip install -r extensions/superbooga/requirements.txt

---

OpenAI-Compatible API

Enable the openai extension to expose a REST API:

python server.py --extensions openai --api

The API runs on port 5000 by default. Test it:

curl http://localhost:5000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is SQL injection?"}
    ],
    "temperature": 0.7,
    "max_tokens": 500
  }'

Point any OpenAI-compatible tool (LangChain, AnythingLLM, n8n, etc.) at http://localhost:5000 and it will work without modification.

---

Generation Parameters Explained

Parameter	Effect	Typical Range
Temperature	Randomness of output	0.1–1.0
Top-P (nucleus sampling)	Cumulative probability cutoff	0.85–0.95
Top-K	Limit token selection pool	40–100
Repetition Penalty	Discourages repeating tokens	1.05–1.3
Max New Tokens	Output length limit	256–4096
Context Length	Input window size	2048–32768

For factual and coding tasks: temperature 0.1–0.3, repetition penalty 1.1. For creative writing: temperature 0.7–0.9, top-p 0.95.

---

Useful Launch Flags

# Listen on all interfaces (for LAN access)
python server.py --listen

# Specific port
python server.py --port 7861

# Auto-load a model at startup
python server.py --model Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf --loader llama.cpp

# CPU only (no GPU)
python server.py --cpu

# Enable all extensions
python server.py --extensions openai superbooga silero_tts --api --listen

---

Troubleshooting

Model loads but output is garbage: Wrong instruction template. Check the model’s Hugging Face page.

Out of memory on GPU: Reduce n-gpu-layers or switch to a smaller quantization (Q3_K_M instead of Q4_K_M).

Slow on CPU: Normal — CPU inference is slow. Use Q3 or Q4 quants. Set --threads to your CPU core count.

API not accessible from other machines: Launch with --listen flag and check firewall allows port 5000.

---

Final Thoughts

Oobabooga Text Generation WebUI gives you more control than almost any other local LLM frontend. The extension system, character support, multi-backend flexibility, and OpenAI-compatible API make it ideal for power users building real applications on top of local models.

It has a steeper learning curve than GPT4All or LM Studio, but the payoff is a highly customizable local AI platform that can grow with your needs.