Tutorial 3 — Background Jobs with Queue

Tutorial 3: Background Jobs con Queue

This tutorial shows how to process background jobs using PyGo’s native queue package — works with in-memory queue (dev) or Redis (production).

🎯 What You’ll Build

A newsletter signup app that:

Accepts email submissions via HTMX
Queues a “welcome email” job
Jobs are processed in the background

📁 Project Structure

myapp/
├── app/
│   ├── web/
│   │   └── newsletter.pgo  ← Handler + job processing
│   └── core/
│       └── jobs.py         ← Python job processors
└── pygo.toml

📝 Step 1: Define Handler & Queue

Create app/web/newsletter.pgo:

from queue import NewQueue
from mailer import Send

# Initialize queue (auto-switches to Redis if REDIS_URL set)
jobs = NewQueue()

model Subscriber:
  id: UUID
  email: Email
  confirmed: Boolean = false
  created: DateTime

# This runs in Go — handles the HTTP request
handler subscribe:
  subscribe(email: Email) -> String:
    # Save to database
    sub = Subscriber(email=email)
    sub.save()

    # Dispatch a background job
    jobs.dispatch("send_welcome_email", {
      "email": str(sub.email),
      "name": str(sub.email).split("@")[0],
    })

    # Return success immediately (HTMX swaps this in)
    return """
    <div class="alert alert-success">
      ✅ Check your email for a welcome message!
    </div>
    """

route POST /subscribe -> subscribe

📝 Step 2: Python Job Processor

Create app/core/jobs.py — this code runs in Python, processing the job:

import smtplib
from queue import job_handler

@job_handler("send_welcome_email")
def send_welcome_email(payload: dict):
    """Send welcome email to new subscriber."""
    email = payload["email"]
    name = payload["name"]

    # Use native mailer package
    from mailer import Send

    Send(
        to=[email],
        subject="Welcome to our newsletter!",
        html=f"<h1>Hello, {name}!</h1><p>Thanks for subscribing.</p>",
    )

    print(f"[worker] Sent welcome email to {email}")

📝 Step 3: HTML Form with HTMX

Create app/templates/newsletter.html:

<div x-data="{ submitting: false }">
  <form
    hx-post="/subscribe"
    hx-swap="outerHTML"
    @submit="submitting = true"
  >
    <input
      type="email"
      name="email"
      placeholder="tu@email.com"
      required
    />
    <button :disabled="submitting">
      <span x-show="!submitting">Subscribe</span>
      <span x-show="submitting">⏳ Sending...</span>
    </button>
  </form>
</div>

📝 Step 4: Run with Worker

# Terminal 1: Start the dev server
pygo dev

# Terminal 2: Start the job worker (Python)
pygo worker jobs.py

# Or in production with Redis:
export REDIS_URL=redis://localhost:6379
pygo dev
pygo worker jobs.py

Test the endpoint:

curl -X POST http://localhost:8080/subscribe \
  -d "email=test@example.com"

🏗️ How It Works

Handler runs in Go — fast, handles HTTP request
jobs.dispatch() queues the job — memory (dev) or Redis (prod)
pygo worker runs Python — executes @job_handler decorated functions
Mailer package — works in both Go (generated) and Python contexts

The queue package auto-detects REDIS_URL — if set, uses Redis as backend; if not, uses in-memory (perfect for development).

⚙️ Queue Configuration

[queue]
backend = "redis"      # or "memory"
redis_url = "redis://localhost:6379"
concurrency = 10       # worker goroutines per queue
max_retries = 3