Hundred Spires

AI powered ETL platform like unstructured.io

Basic setup using uv

brew install uv

# Create virtual env using uv
uv venv

# Install requirements
uv pip install -r requirements.txt

# Run commands in env
uv run python test_ollama.py

# or source env and run directly
source .venv/bin/activate
python test_ollama.py

LLM Models

From initial local testing it looks like gpt-oss:20b is too big for local testing. There is viable local alternative using smaller model like mistral:7b with ollama or using openrouter to test real gpt-oss even 120b to test initial performance.

Ollama

Browse available models at: https://ollama.com/search

brew install ollama

# List your models
ollama list

# Pull the model
ollama pull mistral:7b

# Serve the openai compatible endpoint
ollama serve

Use the model in code using openai client:

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="needed-for-compatibility",
)

Openrouter

Browse available models at: https://openrouter.ai/models

Create your token and use it with openai client.

client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key=os.getenv("OPENROUTER_API_KEY"),
)