Web Scraping Engineer II
YipitData
full-time
Posted on:
Location Type: Remote
Location: India
Visit company websiteExplore more
About the role
- Refactor and Maintain Web Scrapers
- Overhaul existing scraping scripts to improve reliability, maintainability, and efficiency.
- Implement best coding practices (clean code, modular architecture, code reviews, etc.) to ensure quality and sustainability.
- Implement Advanced Scraping Techniques
- Utilize sophisticated fingerprinting methods (cookies, headers, user-agent rotation, proxies) to avoid detection and blocking.
- Handle dynamic content, navigate complex DOM structures, and manage session/cookie lifecycles effectively.
- Collaborate with Cross-Functional Teams
- Work closely with analysts and other stakeholders to gather requirements, align on targets, and ensure data quality.
- Provide support, documentation, and best practices to internal stakeholders to ensure effective use of our web scraped data in critical reporting workflows.
- Monitor and Troubleshoot
- Develop robust monitoring solutions, alerting frameworks to quickly identify and address failures.
- Continuously evaluate scraper performance, proactively diagnosing bottlenecks and scaling issues.
- Drive Continuous Improvement
- Propose new tooling, methodologies, and technologies to enhance our scraping capabilities and processes.
- Stay up to date with industry trends, evolving bot-detection tactics, and novel approaches to web data extraction.
Requirements
- 3+ years of experience with web scraping frameworks (e.g., Selenium, Playwright, or Puppeteer).
- Strong understanding of HTTP, RESTful APIs, HTML parsing, browser rendering, and TLS/SSL mechanics.
- Expertise in advanced fingerprinting and evasion strategies (e.g., browser fingerprint spoofing, request signature manipulation).
- Deep experience managing cookies, headers, session states, and proxy rotations, including the deployment of both residential and data center proxies.
- Experience with logging, metrics, and alerting to ensure high availability.
- Troubleshooting skills to optimize scraper performance for efficiency, reliability, and scalability.
Benefits
- We care about your personal life and we mean it. We offer vacation time, parental leave, team events, learning reimbursement, and more!
- Your growth at YipitData is determined by the impact that you are making, not by tenure, unnecessary facetime, or office politics. Everyone at YipitData is empowered to learn, self-improve, and master their skills in an environment focused on ownership, respect, and trust.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
web scrapingSeleniumPlaywrightPuppeteerHTTPRESTful APIsHTML parsingTLSSSLfingerprinting
Soft Skills
collaborationcommunicationtroubleshootingproblem-solvingdocumentationcontinuous improvementanalytical skillsattention to detailstakeholder managementadaptability