Recorded Future

Senior Data Engineer

Recorded Future

full-time

Posted on:

Origin:  • 🇺🇸 United States

Visit company website
✨ AI Apply
Manual Apply

Salary

💰 $107,000 - $145,000 per year

Job Level

Senior

Tech Stack

AWSCloudCyber SecurityElasticSearchMongoDBPython

About the role

  • Build, maintain, monitor and collect data from hard-to-reach dark web sites and ransomware blogs.
  • Produce a constant stream of fresh, high-fidelity analytics derived from unreliable dark web and underground data sources to enable clients to track threat actors and take security actions.
  • Own the reliability of the automatic processing of structured raw data; investigate and resolve data processing problems and improve data processing code to handle similar scenarios automatically.
  • Maintain and improve data ingestion pipeline that processes and indexes metadata from large files, including analytic-generation services that use NLP to extract high-security-value content.
  • Lead by example when architecting, collaborating and delivering new and improved solutions.

Requirements

  • 5+ years experience with Python working in a professional software engineering environment that involves thorough code testing and review processes
  • Strong problem solving capabilities and experience troubleshooting data issues within your own processes and those of your peers.
  • 5+ years experience maintaining production-ready, efficient data engineering pipelines and systems with extensive monitoring to accommodate complex data ingestion problems
  • Very flexible and outcome-oriented. Comfortable in a work environment where urgent issues arise often, and where quickly resolving issues has immediate security value for our end customers
  • A solid understanding of cybersecurity, along with an interest in analyzing dark web activities related to computer and network security.
  • Experience with NLP, as well as MongoDB, ElasticSearch, and AWS cloud services is highly desirable.