How JobRadar Solved the “Black Hole” Job Search with an Edge-to-Kubernetes Ecosystem
Job hunting on major platforms often feels like sending resumes into a “black hole”. We architected JobRadar as an automated career scouting ecosystem delivering jobs from top companies to candidates within minutes of being posted. Today, JobRadar serves ~300 active CTOs and technical professionals.
Architecture Overview
The system is a decoupled, multi-service ecosystem leveraging a React/Vite frontend on Cloudflare Pages for global edge performance, and a high-throughput Python scraping engine running on Kubernetes.
┌─────────────────────────────────┐
│ External Clients │
│ 🌐 Web User 📱 Telegram User │
└─────────┬───────────────┬───────┘
│ │
┌─────────▼────────┐ │
│ Cloudflare Pages │ │
│ (React SPA Edge) │ │
└─────────┬────────┘ │
│ │
┌─────────▼─────────────────────────────────────┐
│ Kubernetes Cluster │
│ ┌──────────────┐ ┌─────────┐ ┌──────────────┐ │
│ │ Cron: │ │ Worker: │ │ Worker: │ │
│ │ Scheduler │ │ Finder │ │ Notifier ◄─┘
│ └──────┬───────┘ └────┬────┘ └──────┬───────┘ │
└────────┼──────────────┼─────────────┼─────────┘
│ │ │
┌────────▼──────────────▼─────────────▼─────────┐
│ Data Broker Layer │
│ ┌────────────────┐ ┌────────────────┐ │
│ │ Supabase (DB) │ │ Redis Broker │ │
│ └────────────────┘ └────────────────┘ │
└───────────────────────────────────────────────┘01
The Challenge
In the modern tech job market, speed is the ultimate competitive advantage. When a reputable company (e.g., Salesforce, EY, or Apple) posts a desirable role, standard LinkedIn job notifications are notoriously delayed. By the time a candidate receives the alert, the listing is already saturated with thousands of applications.
Third-Party API Rate Limits
Scraping job boards too aggressively leads to immediate IP bans (HTTP 429 errors).
Notification Bottlenecks
The Telegram Bot API enforces strict global limits (maximum 30 messages/second). Delivering a massive batch of new jobs would crash the gateway.
Duplicate Spam
Job boards frequently re-index or cross-post roles. Sending the same job to a user twice destroys the platform's value proposition.
02
Our Approach
To solve this, a system needed to continuously poll multiple job boards (LinkedIn, Indeed, Foundit, Naukri). We architected JobRadar as a highly decoupled, event-driven ecosystem.
React + Vite + TS
Fast, type-safe development.
Cloudflare Pages
Global CDN, zero-maintenance SSL.
Python
Excellent data scraping ecosystem.
Redis Queue (RQ)
Lightweight, robust background tasks.
Supabase (PostgreSQL)
RLS security, PostgREST direct access.
Kubernetes / Helm
Auto-scaling worker pools.
Playwright
Headless browser QA.
03
The Solution
JobRadar implements a microservices mindset utilizing best-in-class components for its scraping and delivery pipelines.
Edge-Optimized Frontend
The user-facing dashboard was built as an SPA using React 18, Vite 5, TypeScript 5, and Shadcn UI. Deployed to Cloudflare Pages for instant edge distribution, with authentication and data handled directly by Supabase via RLS.
High-Throughput Worker Pools
The scraping pipeline is orchestrated on Kubernetes. A CronJob schedules tasks, while scalable Python worker pods dequeue them via Redis Queue to execute the scrapes using python-jobspy.
Distributed Semaphores & Token Buckets
Implemented Redis distributed semaphores to coordinate multi-board scraping without IP bans, and a custom Lua token bucket script to perfectly respect Telegram's 30 messages/second limit.
Smart Deduplication
Maintained a sent_matches table in Supabase containing a composite key of user_id and job_url_hash. This ensures 100% idempotent alert delivery and prevents duplicate spam.
Technical Highlight — Distributed Semaphores
Before a Finder pod can scrape a board, it must acquire a Redis-backed TTL lease (e.g., semaphore:linkedin). This ensures the cluster never exceeds the maximum allowed concurrent requests per job board, preventing IP bans.
04
Results
The technical architecture directly translated into measurable career advantages for the platform's ~300 users. By decoupling the scraper and utilizing aggressive caching and smart rate-limiting, JobRadar delivers alerts fast enough to beat the crowd.
Daily/Weekly
Standard Platforms
Minutes
JobRadar
Real-time Access
Massive competitive advantage
1,000+
Late Arrival
0-10
Early Access
High Visibility
Near-zero competition upon arrival
Email Inbox
Prone to Spam
Telegram Push
Instant Mobile
Direct Attention
Instant read rates
Real-World Alert Speed vs. Applications
Users frequently receive alerts for highly desirable roles when the applicant count is at or near zero.
Facctum - Senior DevOps Engineer
Alerted 44 minutes after posting: 0 Applications
Salesforce - Forward Deployed Engineer
Alerted 22 minutes after posting: 1 Application
EY - SAP Consultant
Alerted 1 hour after posting: 32 Applications
Hakkoda - Application Developer
Alerted 3 hours after posting: 3 Applications
Yash Technologies - Sr Software Engineer
Alerted 6 hours after posting: 4 Applications
05
Key Takeaways
Decouple Heavy Workloads from the Edge
Serving the frontend via Cloudflare Pages while handling Python scraping inside Kubernetes allows both systems to scale independently based on their unique resource profiles.
Redis is More Than a Cache
By utilizing Redis for Distributed Semaphores and Lua Token Buckets, JobRadar solved complex distributed rate-limiting problems without introducing heavy third-party rate-limiting services.
Idempotency is Non-Negotiable in Event-Driven Systems
Using cryptographic hashes of job URLs mapped to user IDs in a relational database ensures that worker pod crashes or queue retries never result in spamming the user.
Ready to automate your workflows?
Let's discuss how custom platform engineering and high-throughput automation can transform your business processes.