Streamline

Senior Data Engineer

Streamline

contract

Posted on:

Location Type: Remote

Location: Remote • 🇺🇸 United States

Visit company website
✨ AI Apply
Apply

Job Level

Senior

Tech Stack

AzureCassandraCloudETLKafkaMongoDBPythonSparkSQL

About the role

  • Design, build, and maintain robust, scalable, and efficient ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks (using Python).
  • Develop and optimize data models in various storage solutions, including Azure Data Explorer (ADX) and No-SQL databases.
  • Tune and improve the performance of data pipelines and data storage solutions for optimal data retrieval and processing speeds.
  • Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet business needs.
  • Ensure data is accurately structured and prepared for visualization tools, primarily Power BI.
  • Implement best practices for data quality, security, and governance across all data platforms.

Requirements

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
  • Minimum 10+ years of professional, hands-on experience across the following technologies:
  • Expert-level proficiency in Python for data manipulation, scripting, and pipeline development.
  • Deep knowledge and experience with the Azure cloud platform and its core data services.
  • Extensive experience in building and managing complex, orchestrated data pipelines with Azure Data Factory (ADF).
  • Proven experience with ingesting, querying, and optimizing data within Azure Data Explorer (ADX) (Kusto).
  • Strong understanding and practical experience with No-SQL databases (e.g., Cosmos DB, MongoDB, Cassandra).
  • Hands-on experience with Databricks environments, Spark, and developing scalable data processing solutions.
  • Experience preparing data specifically for, and a strong understanding of, how data is consumed by Power BI reports and dashboards.
  • Experience with stream processing technologies (e.g., Azure Event Hubs, Kafka).
  • Azure certification (e.g., Azure Data Engineer Associate - DP-203).
  • Solid understanding of modern data warehousing concepts, including Kimball and Inmon methodologies.
  • Experience with version control systems (e.g., Git).
  • Familiarity with CI/CD processes for data infrastructure deployment.
Benefits
  • A challenging and rewarding role in a dynamic and international environment.
  • Opportunity to be part of a growing company with a strong commitment to innovation and excellence.
  • A supportive and collaborative team culture that values personal growth and development.
  • Competitive compensation and benefits package.

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard skills
PythonETLELTAzure Data FactoryDatabricksAzure Data ExplorerNo-SQL databasesSparkPower BIstream processing
Soft skills
collaborationcommunicationproblem-solvingdata qualitydata governance
Certifications
Azure Data Engineer Associate - DP-203