GoQueue

A distributed message queue in Go.
Append-only log, its own coordination and leader election with no ZooKeeper or etcd, hierarchical timing wheel for O(1) delayed delivery, priority lanes, cooperative incremental rebalancing.

Get Started View on GitHub

What is GoQueue?

GoQueue is a distributed message queue whose feature set borrows from three different lineages:

Kafka-style log-based storage with partitions for ordering and parallelism
SQS-style visibility timeouts and dead letter queues for reliability
RabbitMQ-style priority queues and flexible routing

It is single-cluster and it is not a Kafka replacement. Kafka’s I/O path has had a decade of production tuning and a zero-copy fast path; no head-to-head has been run here and none is claimed. What has been measured, along with the harness that produced it, is on the Benchmarks page.

Key Features

📦

Topics & Partitions

Kafka-style log-based storage with configurable partitions for parallelism and ordering guarantees.

👥

Consumer Groups

Automatic partition assignment, rebalancing, and cooperative rebalancing (KIP-429 style).

✅

Message Reliability

ACK/NACK, visibility timeouts, automatic retries, and dead letter queues.

⚡

Priority Queues

5 priority levels with weighted fair queuing to prevent starvation.

⏰

Delayed Messages

Schedule messages for future delivery with second-precision timing.

📋

Schema Registry

JSON Schema validation with compatibility checking (Confluent API compatible).

🔄

Transactions

Exactly-once semantics with idempotent producers and atomic commits.

📊

Observability

Prometheus metrics, distributed tracing, and comprehensive health checks.

Quick Example

Publish a Message

curl -X POST http://localhost:8080/topics/orders/messages \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{
      "key": "user-123",
      "value": "{\"orderId\": \"12345\", \"amount\": 99.99}",
      "priority": "high"
    }]
  }'

Consume via Consumer Group

# Join group
curl -X POST http://localhost:8080/groups/order-processors/join \
  -H "Content-Type: application/json" \
  -d '{"client_id": "consumer-1", "topics": ["orders"]}'

# Poll for messages
curl "http://localhost:8080/groups/order-processors/poll?member_id=<member_id>&timeout=30s"

Using the Go Client

package main

import (
    "context"
    "log"
    
    "goqueue/pkg/client"
)

func main() {
    // Create client
    c, err := client.New(client.DefaultConfig("localhost:9000"))
    if err != nil {
        log.Fatal(err)
    }
    defer c.Close()

    // Publish
    resp, err := c.Publish(context.Background(), "orders", 
        []byte(`{"orderId": "12345"}`))
    if err != nil {
        log.Fatal(err)
    }
    log.Printf("Published to partition %d, offset %d", 
        resp.Partition, resp.Offset)
}

Why GoQueue?

Feature	Kafka	RabbitMQ	SQS	GoQueue
Deployment	Complex (JVM + ZK)	Medium	Managed	Simple (single binary)
Priority Queues	❌	✅	❌	✅
Delayed Messages	❌	✅ Plugin	✅	✅
Visibility Timeout	❌	❌	✅	✅
Partitioning	✅	❌	FIFO only	✅
Consumer Groups	✅	❌	❌	✅
Transactions	✅	✅	❌	✅
Schema Registry	Separate	❌	❌	Built-in

See full comparison →

Getting Started

Ready to try GoQueue? Follow our quickstart guide: