<h2>Our Mission</h2>Join us in shaping the future of paid search intelligence.Adthena is an award-winning search intelligence platform powered by patented AI technology and a team of dedicated experts. We help brands, marketers, and agencies dominate their competitive landscapes with unparalleled insights into the paid search market. Our Whole Market View, Smart Monitor, and Local View solutions give clients the clarity to optimize spend, increase ROI, and stay ahead of the competition.Trusted by global brands like Citibank, L’Oréal, and Volvo, and backed by Updata Partners, we’ve been redefining search intelligence since 2012. Ready to make an impact in an industry-leading company? Let’s do it together.In 2024, Adthena continued its winning streak, earning 9 major search industry awards. Highlights include Best AI Search Software Solution at the Global Search Awards, Best Use of AI for Data at the US Search Awards, and Best PPC Management Software Suite at the UK Search Awards.<h2>Why Join Us?</h2>We offer the best of both worlds: the maturity and stability of a well-established company combined with the energy, agility, and impact of a scaling tech organisation.If you’ve been hesitant about joining a typical tech company due to concerns about chaos, fragility, or volatility, Adthena is your answer.<ul><li>We’re smaller than large-scale enterprises, offering more autonomy, less red tape, and accelerated growth.</li></ul>Here’s what we offer:<ul><li>Startup Engineering culture</li><li>Strong Culture: Our quarterly surveys show a 93% average approval rating for company culture, with an overall eNPS of 27.</li><li>Award-Winning Product: Best AI Search Software Solution, Global Search Award 2024</li><li>Stock Options: Our employees do incredible work, so as we grow, everyone shares the rewards.</li><li>Trust-Based Vacation: Take as much time off as you need, when you need it.</li><li>Remote-First: 52% of Adthenian’s work fully remote, with others choosing Hybrid Work at our 3 Geohubs (London, Austin, Sydney).</li><li>Flexible Work: Work how and where you do your best, with full autonomy over your day.</li><li>Career Growth: Bespoke training and career development via Sherpa plans” to guide your growth.</li><li>Monthly Hackdays &amp; Training Days</li><li>Private Medical Insurance: Fully covered health care.</li><li>Mental Health Support: Employee Assistance Program offering 24/7 confidential counselling, with access to ‘healthier living’ services such as ‘HeadSpace’.</li><li>Family Care Package: Up to 6 months fully paid maternity leave, and 2 months paternity leave.</li><li>Pension: Competitive UK pension scheme, available from day one.</li><li>Home Office Stipend: $200 for your ideal remote setup.</li><li>Swag Welcome Gift: $50 credit to grab some merch.</li><li>Birthday Day Off: Celebrate your special day.</li><li>Regular Socials: Summer &amp; Christmas parties, annual and quarterly offsites, monthly meet-ups. See here for more info.</li><li>Volunteer Days: Make a difference with paid volunteer time each quarter. See here for more info.</li></ul>About the RoleReporting to: Vasanth John, VP of EngineeringLocation: United Kingdom Start Date: ASAPMore info: See FAQ below, our candidate hub, or reach out to <a target="_blank" href="http://adthena.com/">recruitment@adthena.com</a>About your TeamAdthena is a market leader in developing complex ETL and machine learning solutions. With published authors and award winning data scientists we are a friendly, passionate group of engineers making a career out of building great software for our customers.Adthena’s scraping team is at the heart of how we capture the web and AI ecosystem. Every day, our systems pull in massive volumes of search and advertising data, alongside emerging sources like Google’s AI Overviews, Perplexity, and ChatGPT, at a scale of hundreds of millions of data points. These scrapers run around the clock, orchestrating thousands of processes, working across distributed systems, and feeding the insights that make our products indispensable.We are a team that values curiosity and collaboration. Knowledge is openly shared, new approaches are encouraged, and we support one another in solving the kinds of problems that few companies get to tackle. It’s a place where engineers grow by taking ownership, experimenting with new tools, and building systems that need to withstand constant change.What you’ll do at AdthenaAs a Web Scrape Engineer, you’ll be the go-to person for web and LLM scraping at Adthena. You’ll design and operate resilient pipelines that can outsmart anti-bot defenses, adapt to fast-moving platforms, and deliver clean data into our core systems. You’ll work side by side with Data Engineering and DevOps, ensuring scrapers are reliable, observable, and cost-efficient in the cloud. This role is perfect for someone who thrives on independence, loves solving difficult problems at scale, and wants their work to have immediate, visible impact for global customers.Responsibilities<ul><li>Lead design, build, deploy and maintain scalable, resilient web / AI / LLM scraping systems, ensuring optimal performance and productivity.</li><li>Innovate anti-bot / evasion strategies: proxies, headless browsers, fingerprinting, CAPTCHAs.</li><li>Integrate scrape outputs into Adthena’s pipeline: cleaning, deduplication, schema, delivery to our warehouse.</li><li>Ensure resilience: monitoring, alerting, retries, failure modes, fallback strategies.</li><li>Work with our cloud infra (containers, orchestration), collaborating with DevOps for cost, reliability, deployment.</li><li>Proactively adapt to upstream changes (site changes, new anti-bot defenses).</li><li>Maintain strong documentation, best practices, code hygiene; communicate clearly across time zones and teams.</li></ul><h2>Who we look for</h2>Research shows men apply if they meet ~60% of the criteria, while women and underrepresented groups apply only if they meet all requirements. If you have the skills but dont meet every requirement, reach out – wed love to explore how you could be a great fit!Preferred Experience:<ul><li>Bachelor degree in Computer Science, similar technical field of study or equivalent practical experience.</li><li>Minimum 5 years of commercial experience building and running production web scraping systems that operate at scale.</li><li>Deep Python skills; experience with frameworks such as Scrapy, Playwright, or Selenium.</li><li>Practical anti-bot experience: proxies, CAPTCHAs, request fingerprinting, JavaScript rendering, and related evasion strategies.</li><li>Experience in distributed systems and pipelines (Spark, Airflow) with integration into data warehouses like Redshift and PostgreSQL.</li><li>Commercial experience scraping LLM/AI sources such as Perplexity, ChatGPT, Claude, or Gemini.</li><li>Experience deploying and managing scraping workloads in cloud environments (AWS/GCP) using Docker and Kubernetes.</li><li>Proven ability to build observable and resilient systems with monitoring stacks such as Grafana, Prometheus, and ELK.</li><li>Strong debugging, performance optimization, reliability focus.</li><li>Independent self-starter with excellent communication who owns outcomes end-to-end with minimal oversight.</li></ul>Bonus Points If You Have:<ul><li>Awareness of legal, ethical, and compliance considerations in scraping (e.g., GDPR, terms of service).</li><li>Familiarity with messaging/queuing systems such as Kafka or RabbitMQ.</li><li>Knowledge of infrastructure as code (Terraform, Ansible) and CI/CD pipelines (Jenkins, GitHub Actions, TeamCity).</li><li>Experience in cloud cost optimisation and scaling strategies for large-scale scraping.</li></ul><h2>Our Core Values</h2>All Adthenians are expected to align closely with our ‘Core Values’.What it means...<ul><li>Get Shit Done: Were big on taking action, owning our challenges, and finding solutions.</li><li>Freedom with Responsibility: We take charge of results to recognition. We know that our hustle not only moves the needle for the company but also sets us up for personal success.</li><li>Deliver Remarkable Quality: We care about exceeding expectations in the work we do and aim to always do our best work as individuals and as a team.</li><li>Smart &amp; Always Improving: We strive daily to expand our knowledge and become better at what we do.</li></ul><h2>Meet your teammates</h2>You’ll work closely with: Paul Felby Title: Chief Product &amp; Tech Officer LinkedIn: <a target="_blank" href="https://www.linkedin.com/in/paul-felby-1a475a18a/">https://www.linkedin.com/in/paul-felby-1a475a18a/</a>Vasanth John Title: VP of Engineering LinkedIn: <a target="_blank" href="https://www.linkedin.com/in/vasanthjohn/">https://www.linkedin.com/in/vasanthjohn/</a>Ivan Nikolov Title: Lead Technical Architect LinkedIn: <a target="_blank" href="https://www.linkedin.com/in/ivan-nikolov-475b2040/">https://www.linkedin.com/in/ivan-nikolov-475b2040/</a>

Adthena

Web Scraping Engineer (Python)

Adthena

Similar Remote Jobs