Principal Engineer
Oportun
full-time
Posted on:
Location Type: Remote
Location: Remote • 🇮🇳 India
Visit company websiteJob Level
Lead
Tech Stack
AWSAzureCloudGoogle Cloud PlatformHadoopPySparkPython
About the role
- Set the strategic vision and lead the implementation of a cutting-edge data infrastructure roadmap, encompassing all facets as highlighted above.
- Provide exceptional technical leadership, mentoring, and guidance to a team of data engineers and analytic engineers, fostering a culture of continuous learning and innovation.
- Collaborate closely with data scientists to translate intricate model requirements into optimized data pipelines, ensuring impeccable data quality, processing, and integration.
- Spearhead the establishment of best practices for enabling data platform to do experimentation across the company and enable efficient decision making.
- Engineer automated CI/CD pipelines that facilitate seamless deployment, monitoring, and continuous optimization for code and configurations in data engineering.
- Define and refine performance benchmarks, and optimize data infrastructure to achieve peak correctness, availability, cost efficiency, scalability, and robustness.
- Highly motivated self-starter who loves ownership and responsibility while working in a collaborative and interdependent team environment.
- Work with multiple teams of data engineers to design, develop, and test major software and data systems components using an agile, scrum methodology.
- Drive strong data engineering practices around product development execution, operational excellence in observability, quality, reliability, and developer efficiency.
- Remain at the forefront of industry trends and emerging technologies, expertly integrating the latest advancements into our data ecosystem.
Requirements
- Requires 14+ years of related experience in data engineering, with a Bachelor's degree in Computer Science; or a Master's degree with an equivalent combination of education and experience.
- Extensive experience orchestrating the development of end-to-end data engineering pipelines including data analytics for intricate and large-scale applications.
- Proven record of transformative leadership, guiding technical teams to achieve remarkable outcomes and innovation.
- Profound mastery of data engineering architecture and frameworks across batch and stream processing of data, such as Hadoop ecosystem, Medallion architecture, Databricks or equivalent data warehouse / data lake platforms, coupled with Python / PySpark programming.
- Thorough comprehension of software engineering principles, data governance and collaborative development workflows.
- Adeptness with cloud platforms (AWS / Azure / GCP) and utilization of cloud-native services for crafting robust data engineering infrastructure.
- Track record of successfully integrating DevOps practices, continuous integration, and continuous deployment (CI/CD) pipelines.
- Superior problem-solving acumen and ability to navigate intricate technical challenges with dexterity.
- Exceptional communication aptitude, capable of fostering effective collaboration across diverse teams and stakeholders.
Benefits
- Health insurance
- Retirement plans
- Paid time off
- Professional development
- Employee resource groups
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
data engineeringdata pipelinesdata analyticsdata engineering architecturebatch processingstream processingPythonPySparkDevOpsCI/CD
Soft skills
technical leadershipmentoringcollaborationproblem-solvingcommunicationownershipresponsibilityinnovationoperational excellenceteamwork
Certifications
Bachelor's degree in Computer ScienceMaster's degree in related field