Build a Proxy Checker Tool in Go: Concurrent Validation

A proxy checker validates whether proxy servers are working, measures their performance, and detects their anonymity level. Go is ideal for this task because its goroutines and channels enable massive concurrent checking — testing thousands of proxies simultaneously with minimal resource usage.

This tutorial builds a complete proxy checker that validates connectivity, measures latency, checks anonymity level, and detects the proxy’s geographic location.

Project Structure

proxy-checker/
├── main.go
├── checker/
│   ├── checker.go      # Core checking logic
│   ├── anonymity.go    # Anonymity level detection
│   └── types.go        # Data types
├── output/
│   └── reporter.go     # Results formatting
├── go.mod
└── proxies.txt         # Input proxy list

Data Types

// checker/types.go
package checker

import "time"

type ProxyType string

const (
    HTTP   ProxyType = "http"
    HTTPS  ProxyType = "https"
    SOCKS5 ProxyType = "socks5"
)

type AnonymityLevel string

const (
    Transparent AnonymityLevel = "transparent"
    Anonymous   AnonymityLevel = "anonymous"
    Elite       AnonymityLevel = "elite"
)

type ProxyResult struct {
    Host          string         `json:"host"`
    Port          string         `json:"port"`
    Type          ProxyType      `json:"type"`
    Working       bool           `json:"working"`
    Latency       time.Duration  `json:"latency_ms"`
    Anonymity     AnonymityLevel `json:"anonymity"`
    Country       string         `json:"country"`
    ExternalIP    string         `json:"external_ip"`
    Error         string         `json:"error,omitempty"`
    CheckedAt     time.Time      `json:"checked_at"`
}

type CheckConfig struct {
    Timeout       time.Duration
    TestURL       string
    JudgeURL      string
    Workers       int
    RetryCount    int
}

func DefaultConfig() CheckConfig {
    return CheckConfig{
        Timeout:    10 * time.Second,
        TestURL:    "https://httpbin.org/ip",
        JudgeURL:   "https://httpbin.org/headers",
        Workers:    100,
        RetryCount: 1,
    }
}

Core Checker

// checker/checker.go
package checker

import (
    "context"
    "crypto/tls"
    "encoding/json"
    "fmt"
    "net"
    "net/http"
    "net/url"
    "strings"
    "sync"
    "time"
)

type ProxyChecker struct {
    config CheckConfig
}

func New(config CheckConfig) *ProxyChecker {
    return &ProxyChecker{config: config}
}

func (pc *ProxyChecker) CheckProxy(proxyStr string) ProxyResult {
    host, port, err := parseProxy(proxyStr)
    if err != nil {
        return ProxyResult{
            Host:    proxyStr,
            Working: false,
            Error:   fmt.Sprintf("parse error: %v", err),
        }
    }

    result := ProxyResult{
        Host:      host,
        Port:      port,
        CheckedAt: time.Now(),
    }

    // Try HTTPS first, then HTTP, then SOCKS5
    for _, proxyType := range []ProxyType{HTTPS, HTTP, SOCKS5} {
        r := pc.testProxy(host, port, proxyType)
        if r.Working {
            result = r
            result.Host = host
            result.Port = port
            result.CheckedAt = time.Now()

            // Check anonymity
            result.Anonymity = pc.checkAnonymity(host, port, proxyType)
            return result
        }
    }

    result.Working = false
    result.Error = "all protocols failed"
    return result
}

func (pc *ProxyChecker) testProxy(host, port string, proxyType ProxyType) ProxyResult {
    var proxyURL string
    switch proxyType {
    case SOCKS5:
        proxyURL = fmt.Sprintf("socks5://%s:%s", host, port)
    default:
        proxyURL = fmt.Sprintf("http://%s:%s", host, port)
    }

    parsedURL, err := url.Parse(proxyURL)
    if err != nil {
        return ProxyResult{Working: false, Error: err.Error()}
    }

    transport := &http.Transport{
        Proxy: http.ProxyURL(parsedURL),
        TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
        DialContext: (&net.Dialer{
            Timeout: pc.config.Timeout,
        }).DialContext,
    }

    client := &http.Client{
        Transport: transport,
        Timeout:   pc.config.Timeout,
    }

    start := time.Now()
    ctx, cancel := context.WithTimeout(context.Background(), pc.config.Timeout)
    defer cancel()

    req, _ := http.NewRequestWithContext(ctx, "GET", pc.config.TestURL, nil)
    resp, err := client.Do(req)
    latency := time.Since(start)

    if err != nil {
        return ProxyResult{Working: false, Type: proxyType, Error: err.Error()}
    }
    defer resp.Body.Close()

    if resp.StatusCode != 200 {
        return ProxyResult{
            Working: false,
            Type:    proxyType,
            Error:   fmt.Sprintf("status %d", resp.StatusCode),
        }
    }

    // Parse response to get external IP
    var ipResp struct {
        Origin string `json:"origin"`
    }
    json.NewDecoder(resp.Body).Decode(&ipResp)

    return ProxyResult{
        Working:    true,
        Type:       proxyType,
        Latency:    latency,
        ExternalIP: ipResp.Origin,
    }
}

func (pc *ProxyChecker) CheckBatch(proxies []string) []ProxyResult {
    results := make([]ProxyResult, len(proxies))
    var wg sync.WaitGroup
    sem := make(chan struct{}, pc.config.Workers)

    for i, proxy := range proxies {
        wg.Add(1)
        sem <- struct{}{}
        go func(idx int, p string) {
            defer wg.Done()
            defer func() { <-sem }()
            results[idx] = pc.CheckProxy(p)
        }(i, proxy)
    }

    wg.Wait()
    return results
}

func parseProxy(s string) (host, port string, err error) {
    s = strings.TrimSpace(s)
    // Handle format: host:port or ip:port
    parts := strings.Split(s, ":")
    if len(parts) < 2 {
        return "", "", fmt.Errorf("invalid proxy format: %s", s)
    }
    return parts[0], parts[1], nil
}

Anonymity Detection

// checker/anonymity.go
package checker

import (
    "encoding/json"
    "fmt"
    "net/http"
    "net/url"
    "strings"
)

func (pc *ProxyChecker) checkAnonymity(host, port string, proxyType ProxyType) AnonymityLevel {
    proxyURL := fmt.Sprintf("http://%s:%s", host, port)
    parsedURL, _ := url.Parse(proxyURL)

    client := &http.Client{
        Transport: &http.Transport{
            Proxy: http.ProxyURL(parsedURL),
        },
        Timeout: pc.config.Timeout,
    }

    resp, err := client.Get(pc.config.JudgeURL)
    if err != nil {
        return Anonymous // Default if we can't determine
    }
    defer resp.Body.Close()

    var headersResp struct {
        Headers map[string]string `json:"headers"`
    }
    json.NewDecoder(resp.Body).Decode(&headersResp)

    // Check for headers that reveal proxy usage
    revealingHeaders := []string{
        "X-Forwarded-For",
        "X-Real-Ip",
        "Via",
        "X-Proxy-Id",
        "Forwarded",
    }

    hasRevealingHeader := false
    hasRealIP := false

    for _, h := range revealingHeaders {
        if val, exists := headersResp.Headers[h]; exists {
            hasRevealingHeader = true
            // Check if our real IP is in the header
            if strings.Contains(val, host) {
                hasRealIP = true
            }
        }
    }

    if hasRealIP {
        return Transparent
    }
    if hasRevealingHeader {
        return Anonymous
    }
    return Elite
}

Main Program

// main.go
package main

import (
    "bufio"
    "encoding/json"
    "flag"
    "fmt"
    "os"
    "time"

    "proxy-checker/checker"
)

func main() {
    inputFile := flag.String("input", "proxies.txt", "Input proxy list file")
    outputFile := flag.String("output", "results.json", "Output results file")
    workers := flag.Int("workers", 100, "Number of concurrent workers")
    timeout := flag.Int("timeout", 10, "Timeout per proxy in seconds")
    flag.Parse()

    // Load proxies
    proxies, err := loadProxies(*inputFile)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Error loading proxies: %v\n", err)
        os.Exit(1)
    }
    fmt.Printf("Loaded %d proxies from %s\n", len(proxies), *inputFile)

    // Configure checker
    config := checker.DefaultConfig()
    config.Workers = *workers
    config.Timeout = time.Duration(*timeout) * time.Second

    pc := checker.New(config)

    // Run checks
    start := time.Now()
    fmt.Printf("Checking proxies with %d workers...\n", config.Workers)
    results := pc.CheckBatch(proxies)
    elapsed := time.Since(start)

    // Count results
    working := 0
    for _, r := range results {
        if r.Working {
            working++
            fmt.Printf("  OK: %s:%s [%s] %s %dms\n",
                r.Host, r.Port, r.Type, r.Anonymity, r.Latency.Milliseconds())
        }
    }

    fmt.Printf("\nResults: %d/%d working (%.1f%%) in %s\n",
        working, len(results),
        float64(working)/float64(len(results))*100,
        elapsed)

    // Save results
    saveResults(results, *outputFile)
    fmt.Printf("Results saved to %s\n", *outputFile)
}

func loadProxies(filename string) ([]string, error) {
    file, err := os.Open(filename)
    if err != nil {
        return nil, err
    }
    defer file.Close()

    var proxies []string
    scanner := bufio.NewScanner(file)
    for scanner.Scan() {
        line := scanner.Text()
        if line != "" && !strings.HasPrefix(line, "#") {
            proxies = append(proxies, line)
        }
    }
    return proxies, scanner.Err()
}

func saveResults(results []checker.ProxyResult, filename string) {
    file, _ := os.Create(filename)
    defer file.Close()
    encoder := json.NewEncoder(file)
    encoder.SetIndent("", "  ")
    encoder.Encode(results)
}

Building and Running

# Initialize module
go mod init proxy-checker
go mod tidy

# Build
go build -o proxy-checker .

# Run
./proxy-checker -input proxies.txt -output results.json -workers 200 -timeout 15

# Input format (proxies.txt):
# 192.168.1.1:8080
# 10.0.0.1:3128
# proxy.example.com:1080

Performance Optimization

Go’s goroutines make this checker extremely fast:

Workers	1,000 Proxies	10,000 Proxies
50	~30 seconds	~5 minutes
100	~15 seconds	~2.5 minutes
200	~8 seconds	~1.5 minutes
500	~4 seconds	~40 seconds

The bottleneck is typically network I/O and proxy response time, not CPU.

Extending the Checker

Add Geo-Location

func getGeoLocation(ip string) (string, error) {
    resp, err := http.Get(fmt.Sprintf("http://ip-api.com/json/%s", ip))
    if err != nil {
        return "", err
    }
    defer resp.Body.Close()

    var geo struct {
        Country     string `json:"country"`
        CountryCode string `json:"countryCode"`
        City        string `json:"city"`
    }
    json.NewDecoder(resp.Body).Decode(&geo)
    return fmt.Sprintf("%s (%s)", geo.Country, geo.City), nil
}

FAQ

How accurate is proxy anonymity detection?

The anonymity check inspects HTTP headers for proxy indicators. It is reliable for HTTP proxies but cannot fully assess SOCKS5 proxies which operate below the HTTP layer. For comprehensive testing, also check WebRTC leaks and DNS leak behavior.

Can this checker handle authenticated proxies?

Yes. Modify the parseProxy function to accept user:pass@host:port format and include credentials in the proxy URL. Our proxy authentication guide covers the different auth methods.

How do I check SOCKS5 proxies specifically?

The checker tests SOCKS5 automatically. For SOCKS5-only checking, modify the CheckProxy method to skip HTTP/HTTPS attempts. Use the golang.org/x/net/proxy package for advanced SOCKS5 handling.

Should I use this or a commercial proxy checker?

Build your own when you need custom checks, integration with your infrastructure, or high-speed validation of large proxy lists. Use commercial tools like the proxy checker at dataresearchtools.com for quick ad-hoc checks.

How often should I check my proxy pool?

For production scraping, check every 5-15 minutes. Dead proxies should be removed from rotation immediately and re-checked at longer intervals.