Tiny

A production-grade, distributed URL shortener built with Go microservices.

10,000+ lines of Go · 8 microservices · 6 data stores

Quick Start · API Docs · Architecture · Deep Dive

---

What is Tiny?

Tiny is a full-stack URL shortener designed as a real-world distributed systems project. It goes far beyond a simple redirect service -- it includes user authentication, custom aliases, QR code generation, real-time click analytics with geo/device enrichment, distributed tracing, full-text search, and a terminal UI client.

Every architectural decision maps to a production concern: read replicas for scale, Redis Streams for async event processing, ClickHouse materialized views for sub-second analytics, Snowflake IDs for conflict-free distributed ID generation, and distributed locks for custom alias reservation.

Key Features

---

Architecture

System Overview

Services

Redirect Flow (Hot Path)

Create URL Flow

Custom Alias Flow (with Distributed Locking)

Analytics Pipeline

---

Tech Stack

---

Quick Start

Prerequisites

• Go 1.25+
• Docker & Docker Compose
• Make (optional, for convenience commands)

1. Clone and configure

git clone https://github.com/Varun5711/tiny.git
cd tiny
cp .env.example .env

2. Start infrastructure

# Start PostgreSQL (primary + 3 replicas), Redis, and ClickHouse
docker compose -f deployments/docker/docker-compose.yml up -d \
  postgres-primary postgres-replica1 postgres-replica2 postgres-replica3 \
  redis clickhouse

3. Start services

# Option A: Run all services with Docker Compose
docker compose -f deployments/docker/docker-compose.yml up --build

# Option B: Run services locally (requires Go 1.25+)
go run ./cmd/url-service &
go run ./cmd/user-service &
go run ./cmd/redirect-service &
go run ./cmd/api-gateway &
go run ./cmd/pipeline-worker &
go run ./cmd/analytics-worker &
go run ./cmd/cleanup-worker &

4. Try it out

# Register a user
curl -s -X POST http://localhost:8080/api/auth/register \
  -H "Content-Type: application/json" \
  -d '{"email":"test@example.com","password":"secret123","name":"Test User"}'

# Save the token from the response
TOKEN="<token-from-response>"

# Shorten a URL
curl -s -X POST http://localhost:8080/api/urls \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"long_url":"https://github.com/Varun5711/tiny"}'

# Visit the short URL
curl -v http://localhost:8081/<short_code>
# → 302 redirect to https://github.com/Varun5711/tiny

5. Launch the TUI

go run ./cmd/tui

---

API Reference

Authentication

Register

POST /api/auth/register
Content-Type: application/json

{
  "email": "user@example.com",
  "password": "securepassword",
  "name": "John Doe"
}

Response 200 OK

{
  "user_id": "550e8400-e29b-41d4-a716-446655440000",
  "email": "user@example.com",
  "name": "John Doe",
  "token": "eyJhbGciOiJIUzI1NiIs..."
}

Login

POST /api/auth/login
Content-Type: application/json

{
  "email": "user@example.com",
  "password": "securepassword"
}

Get Profile

GET /api/auth/profile
Authorization: Bearer <token>

---

URLs

All URL endpoints require Authorization: Bearer <token> header.

Create Short URL

POST /api/urls
Content-Type: application/json

{
  "long_url": "https://example.com/very/long/path",
  "expires_at": 1735689600       // optional, unix timestamp
}

Response 201 Created

{
  "short_code": "7Bx9kL",
  "short_url": "http://localhost:8081/7Bx9kL",
  "long_url": "https://example.com/very/long/path",
  "created_at": 1704067200,
  "expires_at": 1735689600,
  "qr_code": "data:image/png;base64,iVBOR..."
}

Create Custom Alias

POST /api/urls/custom
Content-Type: application/json

{
  "alias": "my-brand",
  "long_url": "https://example.com",
  "expires_at": 1735689600       // optional
}

List URLs

GET /api/urls?limit=20&offset=0
Authorization: Bearer <token>

Response 200 OK

{
  "urls": [
    {
      "short_code": "7Bx9kL",
      "short_url": "http://localhost:8081/7Bx9kL",
      "long_url": "https://example.com",
      "clicks": 42,
      "created_at": 1704067200,
      "expires_at": 1735689600,
      "is_active": true
    }
  ],
  "total": 1,
  "has_more": false
}

Delete URL

DELETE /api/urls/{short_code}
Authorization: Bearer <token>

Redirect

GET http://localhost:8081/{short_code}
→ 302 Found (Location: https://original-url.com)

---

Analytics

Get URL Stats

GET /api/analytics/{short_code}/stats

Returns total clicks, unique visitors, last clicked timestamp.

Get Click Timeline

GET /api/analytics/{short_code}/timeline?period=7d

Returns hourly/daily click counts with unique visitor breakdowns.

Get Geo Stats

GET /api/analytics/{short_code}/geo

Returns click distribution by country with percentages.

Get Device Stats

GET /api/analytics/{short_code}/devices

Returns breakdown by device type, browser, and OS.

Get Top Referrers

GET /api/analytics/{short_code}/referrers

Returns ranked list of referrer URLs by click count.

Get Raw Click Events

GET /api/analytics/clicks?short_code={code}&limit=50&offset=0
Authorization: Bearer <token>

---

Search

Full-Text Search

GET /api/search?q=example&limit=10&offset=0

Searches across long URLs and short codes via Elasticsearch.

---

Health

GET /health
→ 200 OK    (PostgreSQL + Redis reachable)
→ 503        (dependency unavailable)

---

Configuration

All configuration is via environment variables (loaded from .env in development):

Database

Redis

ClickHouse

Services

Elasticsearch

Tracing

Rate Limiting

Cache

---

Project Structure

tiny/
├── cmd/                          # Service entry points
│   ├── api-gateway/              # HTTP REST API (Uber FX)
│   ├── redirect-service/         # Fast URL redirect (Uber FX)
│   ├── url-service/              # URL CRUD gRPC server (Uber FX)
│   ├── user-service/             # Auth gRPC server (Uber FX)
│   ├── analytics-worker/         # Redis Stream → PostgreSQL (Uber FX)
│   ├── pipeline-worker/          # Redis Stream → ClickHouse + ES (Uber FX)
│   ├── cleanup-worker/           # Expired URL deletion (Uber FX)
│   └── tui/                      # Terminal UI (Bubble Tea)
│
├── internal/                     # Private application packages
│   ├── analytics/                # Analytics aggregation service
│   ├── auth/                     # JWT manager + bcrypt passwords
│   ├── cache/                    # Multi-tier cache (LRU + Redis)
│   ├── clickhouse/               # ClickHouse client + analytics queries
│   ├── config/                   # Env-based configuration loader
│   ├── database/                 # PostgreSQL connection pool manager
│   ├── elasticsearch/            # ES client: URL index, click index, log shipping
│   ├── enrichment/               # GeoIP lookup + User-Agent parsing
│   ├── events/                   # Click event model + Redis Stream producer
│   ├── grpc/                     # gRPC client factory (with OTel instrumentation)
│   ├── handlers/                 # HTTP handlers (URL, Auth, Analytics, Swagger, Redirect)
│   ├── idgen/                    # Snowflake ID generator + Base62 encoder
│   ├── lock/                     # Redis-backed distributed lock (Lua script)
│   ├── logger/                   # Zap structured logging (JSON + ES syncer)
│   ├── middleware/               # CORS, rate limit, auth, recovery, tracing, request ID
│   ├── models/                   # Domain models (URL, User, errors)
│   ├── qrcode/                   # QR code PNG generation
│   ├── redis/                    # Redis client wrapper
│   ├── service/                  # Business logic (URL service, User service)
│   ├── storage/                  # PostgreSQL storage layer (CRUD, pagination, filters)
│   ├── tracing/                  # OpenTelemetry tracer provider setup
│   └── validation/               # Alias validation + alternative suggestions
│
├── proto/                        # Protobuf definitions
│   ├── url/                      # URL service (CreateURL, GetURL, ListURLs, DeleteURL, etc.)
│   ├── user/                     # User service (Register, Login, ValidateToken, etc.)
│   └── analytics/                # Analytics service
│
├── build/docker/                 # Multi-stage Dockerfiles (8 services)
├── deployments/
│   ├── docker/                   # docker-compose.yml (full stack)
│   ├── k8s/                      # Kubernetes manifests (base + overlays)
│   │   ├── base/                 # Deployments, Services, HPAs, NetworkPolicies
│   │   └── overlays/             # staging / production kustomizations
│   └── terraform/                # Infrastructure-as-code (placeholder)
│
├── scripts/
│   ├── databases/                # SQL schemas (PostgreSQL + ClickHouse)
│   ├── migrations/               # Database migration scripts
│   └── install.sh                # CLI installer (curl | bash)
│
├── docs/
│   ├── api/                      # gRPC API docs + examples
│   ├── architecture/             # System design + ADRs
│   └── deep-dive/                # 12-chapter technical deep dive
│
├── test/integration/             # End-to-end integration tests
├── api/openapi/                  # OpenAPI/Swagger specification
├── .github/workflows/ci.yaml    # CI pipeline (lint, test, build, Docker)
├── .env.example                  # Environment variable template
├── go.mod                        # Go module (30+ dependencies)
└── Makefile                      # Build automation

---

Database Schema

PostgreSQL

-- Users table
CREATE TABLE users (
    id            VARCHAR(50) PRIMARY KEY,
    email         VARCHAR(255) UNIQUE NOT NULL,
    name          VARCHAR(255) NOT NULL,
    password_hash TEXT NOT NULL,
    created_at    TIMESTAMPTZ DEFAULT NOW(),
    updated_at    TIMESTAMPTZ DEFAULT NOW()
);

-- URLs table
CREATE TABLE urls (
    short_code  VARCHAR(20) PRIMARY KEY,
    long_url    TEXT NOT NULL,
    user_id     VARCHAR(50),
    clicks      BIGINT DEFAULT 0,
    created_at  TIMESTAMPTZ DEFAULT NOW(),
    updated_at  TIMESTAMPTZ DEFAULT NOW(),
    expires_at  TIMESTAMPTZ,
    qr_code     TEXT
);

ClickHouse

CREATE TABLE analytics.click_events (
    event_id String, short_code String, original_url String,
    clicked_at DateTime64(3), ip_address String,
    country String, country_code String, region String, city String,
    latitude Float64, longitude Float64, timezone String,
    user_agent String, browser String, browser_version String,
    os String, os_version String, device_type String,
    device_brand String, device_model String,
    is_mobile UInt8, is_tablet UInt8, is_desktop UInt8, is_bot UInt8,
    referer String, query_params String
) ENGINE = MergeTree()
  PARTITION BY toYYYYMM(clicked_date)
  ORDER BY (short_code, clicked_at)
  TTL clicked_date + INTERVAL 180 DAY;

-- Pre-aggregated materialized views
-- daily_clicks_by_url, clicks_by_country, clicks_by_device, hourly_clicks

---

Deployment

Docker Compose (Development)

# Full stack: databases + all 7 services
docker compose -f deployments/docker/docker-compose.yml up --build

# Infrastructure only (bring your own services)
docker compose -f deployments/docker/docker-compose.yml up -d \
  postgres-primary postgres-replica1 postgres-replica2 postgres-replica3 \
  redis clickhouse

Kubernetes (Production)

# Apply base manifests
kubectl apply -k deployments/k8s/base/

# Or use overlays
kubectl apply -k deployments/k8s/overlays/production/

Includes:

• Deployments with security contexts (runAsNonRoot, readOnlyRootFilesystem, drop ALL)
• StatefulSets for PostgreSQL, Redis, ClickHouse, Elasticsearch
• Horizontal Pod Autoscalers for all services
• NetworkPolicies restricting traffic between services
• Pod Disruption Budgets (production overlay)
• CronJob for cleanup-worker
• Ingress with path-based routing

CI/CD

The GitHub Actions pipeline runs on every push to main and refactor:

---

Development

Build all services

go build ./cmd/...

Run tests

# Unit tests
go test ./...

# With race detector
go test -race ./...

# Integration tests (requires running infrastructure)
INTEGRATION_TEST=true go test ./test/integration/ -v

Lint

golangci-lint run ./...

Generate Protobuf

protoc --go_out=. --go-grpc_out=. proto/**/*.proto

---

Design Decisions

---

Documentation

The docs/deep-dive/ directory contains a 12-chapter technical walkthrough:

1. Big Picture -- System overview
2. Database Architecture -- PostgreSQL replication, ClickHouse schema
3. Messaging & Queuing -- Redis Streams pipeline
4. Caching Strategy -- Multi-tier cache design
5. Short Code Generation -- Snowflake + Base62
6. gRPC Communication -- Service-to-service calls
7. Authentication & JWT -- Auth flow
8. Rate Limiting -- Redis-based sliding window
9. Background Workers -- Pipeline, analytics, cleanup
10. Code Walkthrough: Create URL
11. Code Walkthrough: Redirect
12. Scaling Strategy -- Horizontal scaling plan

---

License

MIT License -- see LICENSE for details.

---

Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Run tests (go test -race ./...)
4. Run linter (golangci-lint run ./...)
5. Commit your changes
6. Push to the branch (git push origin feature/amazing-feature)
7. Open a Pull Request

---

Built by Varun Hotani