Proxy Integration with Java (OkHttp, HttpClient, Jsoup)

By /

Proxy Integration with Java (OkHttp, HttpClient, Jsoup)

Java remains one of the most widely used languages for enterprise data collection, web scraping, and API integration. Whether you are building a price monitoring system, aggregating product reviews, or collecting market intelligence, configuring proxies correctly in your Java application is essential. This guide covers proxy integration for three popular Java HTTP libraries: the built-in HttpClient, OkHttp, and Jsoup.

Java’s Built-in Proxy Support

Java has had proxy support since the early days through java.net.Proxy and ProxySelector. The modern java.net.http.HttpClient (introduced in Java 11) builds on these foundations.

HttpClient with HTTP Proxy

import java.net.InetSocketAddress;
import java.net.ProxySelector;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.Base64;

public class ProxyExample {

    public static void main(String[] args) throws Exception {
        String proxyHost = "gate.dataresearchtools.com";
        int proxyPort = 5432;
        String proxyUser = "your_username";
        String proxyPass = "your_password";

        HttpClient client = HttpClient.newBuilder()
            .proxy(ProxySelector.of(new InetSocketAddress(proxyHost, proxyPort)))
            .connectTimeout(Duration.ofSeconds(30))
            .build();

        // Proxy authentication via header
        String authString = proxyUser + ":" + proxyPass;
        String encodedAuth = Base64.getEncoder().encodeToString(authString.getBytes());

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create("https://httpbin.org/ip"))
            .header("Proxy-Authorization", "Basic " + encodedAuth)
            .header("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64)")
            .timeout(Duration.ofSeconds(30))
            .GET()
            .build();

        HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
        System.out.println("Status: " + response.statusCode());
        System.out.println("Body: " + response.body());
    }
}

System-Wide Proxy via Authenticator

Java supports a global Authenticator for proxy credentials:

import java.net.Authenticator;
import java.net.PasswordAuthentication;

public class ProxyAuthenticator extends Authenticator {

    private final String username;
    private final String password;

    public ProxyAuthenticator(String username, String password) {
        this.username = username;
        this.password = password;
    }

    @Override
    protected PasswordAuthentication getPasswordAuthentication() {
        if (getRequestorType() == RequestorType.PROXY) {
            return new PasswordAuthentication(username, password.toCharArray());
        }
        return null;
    }
}

// Usage
Authenticator.setDefault(new ProxyAuthenticator("your_username", "your_password"));

HttpClient client = HttpClient.newBuilder()
    .proxy(ProxySelector.of(new InetSocketAddress("gate.dataresearchtools.com", 5432)))
    .authenticator(new ProxyAuthenticator("your_username", "your_password"))
    .build();

Custom ProxySelector

For rotating proxies, implement a custom ProxySelector:

import java.io.IOException;
import java.net.*;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;

public class RotatingProxySelector extends ProxySelector {

    private final List<Proxy> proxies;
    private final AtomicInteger counter = new AtomicInteger(0);

    public RotatingProxySelector(List<InetSocketAddress> addresses) {
        this.proxies = addresses.stream()
            .map(addr -> new Proxy(Proxy.Type.HTTP, addr))
            .toList();
    }

    @Override
    public List<Proxy> select(URI uri) {
        int index = Math.abs(counter.getAndIncrement() % proxies.size());
        return Collections.singletonList(proxies.get(index));
    }

    @Override
    public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
        System.err.println("Proxy connection failed: " + sa + " - " + ioe.getMessage());
    }
}

// Usage
List<InetSocketAddress> proxyAddresses = List.of(
    new InetSocketAddress("gate.dataresearchtools.com", 5432),
    new InetSocketAddress("gate.dataresearchtools.com", 5433),
    new InetSocketAddress("gate.dataresearchtools.com", 5434)
);

HttpClient client = HttpClient.newBuilder()
    .proxy(new RotatingProxySelector(proxyAddresses))
    .build();

OkHttp Proxy Integration

OkHttp is the most popular third-party HTTP client for Java and Android. It provides cleaner proxy support with interceptors for advanced use cases.

Maven Dependency

<dependency>
    <groupId>com.squareup.okhttp3</groupId>
    <artifactId>okhttp</artifactId>
    <version>4.12.0</version>
</dependency>

Basic OkHttp Proxy

import okhttp3.*;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.io.IOException;
import java.util.concurrent.TimeUnit;

public class OkHttpProxyExample {

    public static void main(String[] args) throws IOException {
        Proxy proxy = new Proxy(Proxy.Type.HTTP,
            new InetSocketAddress("gate.dataresearchtools.com", 5432));

        OkHttpClient client = new OkHttpClient.Builder()
            .proxy(proxy)
            .proxyAuthenticator((route, response) -> {
                String credential = Credentials.basic("your_username", "your_password");
                return response.request().newBuilder()
                    .header("Proxy-Authorization", credential)
                    .build();
            })
            .connectTimeout(30, TimeUnit.SECONDS)
            .readTimeout(30, TimeUnit.SECONDS)
            .build();

        Request request = new Request.Builder()
            .url("https://httpbin.org/ip")
            .header("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64)")
            .build();

        try (Response response = client.newCall(request).execute()) {
            System.out.println("Status: " + response.code());
            System.out.println("Body: " + response.body().string());
        }
    }
}

OkHttp SOCKS5 Proxy

Proxy socksProxy = new Proxy(Proxy.Type.SOCKS,
    new InetSocketAddress("gate.dataresearchtools.com", 5433));

OkHttpClient client = new OkHttpClient.Builder()
    .proxy(socksProxy)
    .proxyAuthenticator((route, response) -> {
        String credential = Credentials.basic("your_username", "your_password");
        return response.request().newBuilder()
            .header("Proxy-Authorization", credential)
            .build();
    })
    .build();

OkHttp Interceptor for Logging and Retry

Interceptors are one of OkHttp’s most powerful features. Here is a retry interceptor:

public class RetryInterceptor implements Interceptor {

    private final int maxRetries;
    private final long retryDelayMs;

    public RetryInterceptor(int maxRetries, long retryDelayMs) {
        this.maxRetries = maxRetries;
        this.retryDelayMs = retryDelayMs;
    }

    @Override
    public Response intercept(Chain chain) throws IOException {
        Request request = chain.request();
        IOException lastException = null;

        for (int attempt = 0; attempt <= maxRetries; attempt++) {
            try {
                Response response = chain.proceed(request);

                if (response.isSuccessful() || attempt == maxRetries) {
                    return response;
                }

                // Retry on 429 (rate limit) or 5xx (server error)
                if (response.code() == 429 || response.code() >= 500) {
                    response.close();
                    Thread.sleep(retryDelayMs * (attempt + 1));
                    continue;
                }

                return response;
            } catch (IOException e) {
                lastException = e;
                if (attempt < maxRetries) {
                    try {
                        Thread.sleep(retryDelayMs * (attempt + 1));
                    } catch (InterruptedException ie) {
                        Thread.currentThread().interrupt();
                        throw new IOException("Retry interrupted", ie);
                    }
                }
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
                throw new IOException("Retry interrupted", e);
            }
        }

        throw lastException != null ? lastException : new IOException("Max retries exceeded");
    }
}

// Usage
OkHttpClient client = new OkHttpClient.Builder()
    .proxy(proxy)
    .proxyAuthenticator(authenticator)
    .addInterceptor(new RetryInterceptor(3, 2000))
    .build();

OkHttp Proxy Rotation

import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;

public class ProxyRotatingClient {

    private final List<ProxyConfig> proxies;
    private final AtomicInteger counter = new AtomicInteger(0);

    public record ProxyConfig(String host, int port, String username, String password) {}

    public ProxyRotatingClient(List<ProxyConfig> proxies) {
        this.proxies = proxies;
    }

    public OkHttpClient getClient() {
        int index = Math.abs(counter.getAndIncrement() % proxies.size());
        ProxyConfig config = proxies.get(index);

        Proxy proxy = new Proxy(Proxy.Type.HTTP,
            new InetSocketAddress(config.host(), config.port()));

        return new OkHttpClient.Builder()
            .proxy(proxy)
            .proxyAuthenticator((route, response) -> {
                String credential = Credentials.basic(config.username(), config.password());
                return response.request().newBuilder()
                    .header("Proxy-Authorization", credential)
                    .build();
            })
            .connectTimeout(30, TimeUnit.SECONDS)
            .readTimeout(30, TimeUnit.SECONDS)
            .build();
    }

    public String fetch(String url) throws IOException {
        OkHttpClient client = getClient();
        Request request = new Request.Builder().url(url).build();

        try (Response response = client.newCall(request).execute()) {
            return response.body().string();
        }
    }
}

Jsoup with Proxy

Jsoup is the go-to library for HTML parsing and web scraping in Java. It has built-in proxy support.

Maven Dependency

<dependency>
    <groupId>org.jsoup</groupId>
    <artifactId>jsoup</artifactId>
    <version>1.17.2</version>
</dependency>

Basic Jsoup Proxy

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import java.net.InetSocketAddress;
import java.net.Proxy;

public class JsoupProxyExample {

    public static void main(String[] args) throws Exception {
        Proxy proxy = new Proxy(Proxy.Type.HTTP,
            new InetSocketAddress("gate.dataresearchtools.com", 5432));

        // Set global authenticator for proxy auth
        java.net.Authenticator.setDefault(new java.net.Authenticator() {
            @Override
            protected java.net.PasswordAuthentication getPasswordAuthentication() {
                return new java.net.PasswordAuthentication(
                    "your_username", "your_password".toCharArray());
            }
        });

        Document doc = Jsoup.connect("https://example.com")
            .proxy(proxy)
            .userAgent("Mozilla/5.0 (Windows NT 10.0; Win64; x64)")
            .timeout(30000)
            .get();

        System.out.println("Title: " + doc.title());

        // Extract all links
        Elements links = doc.select("a[href]");
        for (Element link : links) {
            System.out.println(link.attr("href") + " - " + link.text());
        }
    }
}

Jsoup Scraping with Proxy Rotation

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;

public class JsoupScraper {

    private final List<Proxy> proxies = new ArrayList<>();
    private final AtomicInteger counter = new AtomicInteger(0);

    public JsoupScraper(List<InetSocketAddress> proxyAddresses) {
        for (InetSocketAddress addr : proxyAddresses) {
            proxies.add(new Proxy(Proxy.Type.HTTP, addr));
        }
    }

    private Proxy nextProxy() {
        int index = Math.abs(counter.getAndIncrement() % proxies.size());
        return proxies.get(index);
    }

    public Document scrape(String url) throws Exception {
        return Jsoup.connect(url)
            .proxy(nextProxy())
            .userAgent("Mozilla/5.0 (Windows NT 10.0; Win64; x64)")
            .timeout(30000)
            .followRedirects(true)
            .get();
    }

    public List<String> scrapeProductTitles(String url) throws Exception {
        Document doc = scrape(url);
        return doc.select(".product-title")
            .stream()
            .map(Element::text)
            .toList();
    }
}

Connection Pooling Best Practices

Java applications benefit from connection pooling. Here is how to configure OkHttp’s connection pool for proxy usage:

import okhttp3.ConnectionPool;

OkHttpClient client = new OkHttpClient.Builder()
    .proxy(proxy)
    .proxyAuthenticator(authenticator)
    .connectionPool(new ConnectionPool(20, 5, TimeUnit.MINUTES))
    .build();

For HttpClient, connection pooling is managed internally, but you can control it:

HttpClient client = HttpClient.newBuilder()
    .proxy(ProxySelector.of(proxyAddress))
    .executor(Executors.newFixedThreadPool(10))
    .build();

Thread-Safe Concurrent Scraping

For high-throughput scraping with Java’s virtual threads (Java 21+):

import java.util.concurrent.*;
import java.util.List;

public class ConcurrentScraper {

    private final ProxyRotatingClient proxyClient;

    public ConcurrentScraper(ProxyRotatingClient proxyClient) {
        this.proxyClient = proxyClient;
    }

    public List<String> scrapeAll(List<String> urls) throws InterruptedException {
        try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
            List<Future<String>> futures = urls.stream()
                .map(url -> executor.submit(() -> {
                    try {
                        return proxyClient.fetch(url);
                    } catch (Exception e) {
                        return "ERROR: " + e.getMessage();
                    }
                }))
                .toList();

            List<String> results = new ArrayList<>();
            for (Future<String> future : futures) {
                try {
                    results.add(future.get(60, TimeUnit.SECONDS));
                } catch (ExecutionException | TimeoutException e) {
                    results.add("ERROR: " + e.getMessage());
                }
            }
            return results;
        }
    }
}

System Properties for Global Proxy Configuration

Java supports system-wide proxy configuration through properties:

// Set programmatically
System.setProperty("http.proxyHost", "gate.dataresearchtools.com");
System.setProperty("http.proxyPort", "5432");
System.setProperty("https.proxyHost", "gate.dataresearchtools.com");
System.setProperty("https.proxyPort", "5432");

// Or via command line
// java -Dhttp.proxyHost=gate.dataresearchtools.com -Dhttp.proxyPort=5432 MyApp

This approach affects all HTTP connections in the JVM and is useful for applications where you want all traffic to go through a proxy.

Why Mobile Proxies for Java Scrapers

Java-based scraping systems often run at enterprise scale, making proxy quality critical. DataResearchTools mobile proxies provide several advantages for Java applications:

  • High success rates reduce wasted compute and network resources.
  • SEA coverage with mobile IPs from Singapore, Thailand, Indonesia, Philippines, and Malaysia.
  • Sticky sessions allow maintaining the same IP across multiple requests when needed.
  • Standard HTTP proxy protocol means no special SDK or adapter is required — all Java HTTP libraries work out of the box.

Conclusion

Java offers multiple approaches to proxy integration, from the built-in HttpClient and ProxySelector to third-party libraries like OkHttp and Jsoup. Each has its strengths: HttpClient for modern async workflows, OkHttp for its interceptor architecture, and Jsoup for HTML-focused scraping. Combine any of these with DataResearchTools mobile proxies for reliable, large-scale data collection that avoids the blocking issues common with datacenter proxies.

Related Reading

Scroll to Top