Build a secure, self-hosted MCP (Model Context Protocol) server on Ubuntu and connect it to Ollama to create a fully local AI Linux system assistant. This guide walks through installing the MCP Python SDK, connecting to Ollama’s API, exposing safe Linux automation tools, and generating AI-powered system summaries without using any cloud services.
Build a local MCP server on Ubuntu and connect it to Ollama to create a secure self-hosted AI assistant for Linux. Step-by-step tutorial for homelab automation, system monitoring, and local LLM integration without cloud dependencies.
In this tutorial you will:
This setup works for homelabs, self-hosted environments, and Linux system administrators.
Model Context Protocol (MCP) is a standardized way to connect AI models to structured tools and data sources.
Instead of building custom scripts for every integration, MCP allows you to:
In this guide, we build a local MCP server on Ubuntu and connect it to Ollama running on another server.
This tutorial assumes:
10.10.0.60:1143410.10.0.105:8000The MCP server exposes:
All AI processing happens locally using Ollama.
Update the system:
sudo apt update && sudo apt upgrade -y
Install required packages:
sudo apt install -y python3 python3-venv python3-pip
python3 -m venv homelab-ai
source homelab-ai/bin/activate
Install the MCP SDK and HTTP client:
pip install "mcp[cli]" httpx
Confirm Ollama is reachable from the MCP server:
curl http://10.10.0.60:11434/
List installed models:
curl http://10.10.0.60:11434/api/tags
Test model generation:
curl http://10.10.0.60:11434/api/generate \
-d '{"model":"llama3.2:latest","prompt":"Say hello in one sentence.","stream":false}'
To prevent slow first responses:
curl http://10.10.0.60:11434/api/generate \
-d '{"model":"phi4-mini:latest","prompt":"warm up","stream":false}'
Create the Python file:
nano homelab_mcp_server.py
Paste the full MCP server script from the video tutorial.
The script:
Set required environment variables:
export OLLAMA_BASE_URL="http://10.10.0.60:11434"
export OLLAMA_DEFAULT_MODEL="phi4-mini:latest"
export MCP_HOST="0.0.0.0"
export MCP_PORT="8000"
Optional configuration:
Enable Docker tools:
export ENABLE_DOCKER_TOOLS="true"
Enable system service restart (disabled by default):
export ENABLE_SERVICE_RESTART="true"
python3 homelab_mcp_server.py
The MCP endpoint will be:
http://<server-ip>:8000/mcp
Do not include a trailing slash.
Launch Inspector:
npx -y @modelcontextprotocol/inspector
Connect to:
http://10.10.0.105:8000/mcp
Transport type: streamable-http.
These return standard Linux command output.
These tools run system commands and pass the output into Ollama for summarization.
summarize_disk_usage
Identifies filesystems above 80 percent and provides a health overview.
summarize_memory_usage
Reports RAM usage, available memory, and swap status.
system_health_report
Combines disk, memory, and uptime into a concise AI-generated system summary.
If Docker is installed:
This allows AI-assisted container monitoring in your homelab.
This MCP server:
If you receive:
Invalid Host header
Add your server IP to the allowed_hosts configuration in the MCP script.
You can extend this setup by:
Always enforce strict security boundaries when expanding automation.
Using MCP with Ollama transforms a local LLM from a chatbot into:
This approach keeps full control in your environment with no cloud dependency.