Foundit, formerly Monster India, is one of South Asia’s largest job boards with over 50 million registered job seekers and a rich dataset of employer hiring signals, salary benchmarks, and skills-in-demand across India, Singapore, Philippines, and the Gulf. scraping Foundit in 2026 is more involved than it looks: the site has layered bot detection, JavaScript-rendered listing pages, and API endpoints that rotate session tokens aggressively.
What Foundit’s infrastructure looks like
Foundit runs on a React SPA backed by GraphQL-style REST endpoints under https://www.foundit.in/srp/results and https://www.foundit.in/middleware/jobsearch/v2/search. listing cards on the search results page are server-side rendered for SEO but individual job detail pages load dynamically via XHR after the initial paint. this means a raw HTTP GET on a job URL gives you the shell, not the payload.
the authentication wall is partial: job title, company name, location, and posted date are visible without login. salary, contact details, and the full job description require a logged-in session. for most market research and hiring-signal use cases, the public fields are sufficient.
Cloudflare is the CDN and bot gateway. Foundit runs Cloudflare’s Bot Management tier (not just the free shield), which means TLS fingerprint checks, JavaScript challenge pages, and behavioral scoring are all active. you will get a 403 or a silent redirect to a CAPTCHA page if your request profile looks automated.
The two viable scraping approaches
Approach 1: direct HTTP with a residential proxy
If you only need the SRP (search results page) fields, direct HTTP works but requires residential IPs and careful header management. the key headers to mirror from a real browser session:
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
"Accept-Language": "en-IN,en;q=0.9",
"Accept-Encoding": "gzip, deflate, br",
"Referer": "https://www.foundit.in/",
"sec-ch-ua": '"Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"',
"sec-ch-ua-platform": '"Windows"',
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "same-origin",
}pair this with an Indian residential proxy (IN-geolocation matters because Foundit 302-redirects non-IN traffic on some listing categories). rotate IPs every 30-50 requests and keep a 2-4 second delay between pages. this approach handles about 500-800 listings per hour without triggering rate limits at low volume.
Approach 2: browser automation for full job detail
For full job descriptions and salary data you need a logged-in Playwright session. the scrape flow:
- launch Playwright with a persistent context so cookies survive across runs
- navigate to foundit.in and complete a one-time manual login
- save the auth state with
context.storage_state(path="foundit_auth.json") - on subsequent runs, load the saved state and go directly to search pages
the session stays valid for roughly 7 days before Foundit rotates the auth cookie. build in a re-login trigger when you get a 401 from the middleware API.
this is the same session-persistence pattern used on high-detection-risk targets. if you have scraped platforms like ZipRecruiter you will recognize the flow: grab the XHR endpoint from DevTools, replay it authenticated, parse JSON directly instead of parsing HTML.
Parsing the search results API
once you have a valid session, the middleware endpoint is easier to work with than the HTML. a search call looks like:
GET /middleware/jobsearch/v2/search?query=data+analyst&location=Mumbai&limit=30&offset=0the response is JSON with a jobDetails array. each object contains jobId, jobTitle, companyName, minSalary, maxSalary, experienceRange, skillDetails (array), and postedDate. page through with offset increments. the API caps results at offset 990 (33 pages of 30), which mirrors the same UX limit Foundit shows in the browser.
for skills trend analysis, the skillDetails field is the highest-value extraction target: it gives you a tagged skill list per posting that you can aggregate across thousands of jobs to build a real demand signal, not just keyword frequency from raw text.
Proxy and infrastructure setup
| Proxy type | Success rate (Foundit IN) | Cost per GB | Best for |
|---|---|---|---|
| Indian residential | 87-93% | $8-15 | Full scrape, salary data |
| Indian mobile | 91-96% | $15-25 | High-volume, low block rate |
| Datacenter (IN) | 12-30% | $0.5-2 | Not viable, blocked at CF |
| Global residential (non-IN) | 40-60% | $8-15 | Partial, geo-redirect issues |
Indian mobile proxies outperform residential here because Foundit’s Cloudflare profile scores mobile ASNs as lower-risk. if you are building a multi-market job data pipeline covering the Southeast Asian market, the approach overlaps with what we cover for JobStreet Southeast Asia, where residential proxy selection by country is equally important.
for a pure scraping infrastructure setup, keep these points in mind:
- run one IP per concurrent worker, never share across threads
- set a max of 80 requests per IP before rotating
- log HTTP status codes per IP and retire any address returning >15% 403s in a session
- use HTTPS proxies only (Foundit enforces HSTS)
Handling errors and blocks
Foundit’s blocking behavior follows a pattern you should instrument against:
- 403 with CF-RAY header: Cloudflare hard block, rotate IP immediately
- 200 with empty
jobDetailsarray: soft block or geo-mismatch, check proxy location - 302 to
/login: session expired, trigger re-auth flow - 429: rate limit, back off 60-90 seconds then resume on a different IP
similar error handling logic applies when scraping other structured job platforms. the Seek.com.au scraping guide goes into detail on the 200-but-empty pattern, which is common on Cloudflare-protected job boards specifically.
for financial and investment data pipelines that combine job posting signals with company-level data, the same proxy infrastructure that works for Foundit translates well to lower-volume targets. PitchBook public pages and Tracxn free tier pages both sit behind similar detection stacks, so your retry and rotation logic is reusable across the pipeline.
key fields to extract per job posting:
jobId(stable identifier for deduplication across runs)jobTitle,companyName,companyIdminSalary/maxSalary(INR monthly, often null for senior roles)experienceRange(years, structured)skillDetails[](tagged, structured, not free text)postedDate(Unix timestamp)jobLocation.city,jobLocation.stateindustryType,functionalArea
Bottom line
Foundit is scrapeable in 2026 if you use Indian residential or mobile proxies, mirror a real browser’s TLS and header fingerprint, and build a session-persistence layer for authenticated endpoints. the middleware JSON API is cleaner than HTML parsing and gives you structured salary and skills data that HTML scraping misses. dataresearchtools.com covers this class of high-detection-risk job board in depth, and the same techniques apply across the regional job board landscape.