Callista

Senior Data Engineer, Databricks, Azure

Callista

full-time

Posted on:

Location Type: Remote

Location: Remote • 🇷🇸 Serbia

Visit company website
✨ AI Apply
Apply

Job Level

Senior

Tech Stack

AzureCloudETLPySparkPythonScalaSQL

About the role

  • Collaborate with the Product Owner, Business Analyst and other team members to understand domain-specific data requirements and design scalable data pipelines and architectures
  • Transform and enrich data from the Bronze layer (raw data) into the Silver layer by performing data quality checks, cleansing, validation, standardization, and modeling activities
  • Build and maintain data products and data marts in the Gold layer that are curated, business-ready, and optimized for analytics and reporting purposes
  • Develop efficient ETL/ELT workflows to extract, transform, and load data from various sources through the medallion architecture layers
  • Optimize and fine-tune data pipelines for performance, reliability, and scalability using PySpark and Databricks
  • Implement comprehensive data quality checks and monitoring to ensure data accuracy, consistency, and integrity across all layers
  • Develop Power BI reports and dashboards that leverage the Gold layer data products to deliver business insights
  • Work with BI developers and data analysts to provide them with the necessary data infrastructure and tools for analysis and reporting
  • Troubleshoot and resolve data-related issues, including performance bottlenecks and data inconsistencies
  • Stay up to date with the latest trends and technologies in data engineering and recommend improvements to existing systems and processes
  • Document data engineering processes, data flows, and system configurations

Requirements

  • Bachelor's degree in Computer Science, Engineering, or a related field
  • 5+ years of experience as a Data Engineer on Databricks and Azure, with a focus on designing and building data pipelines within a medallion architecture (Bronze, Silver, Gold layers)
  • Strong problem-solving and analytical skills
  • Effective communication in English and collaboration in agile environments
  • Experience working in global, multicultural teams
  • Advanced proficiency in PySpark and SparkSQL for large-scale data transformations and processing
  • Hands-on experience with Databricks workspace, Delta Lake, workflows, and notebooks
  • Proven experience developing interactive reports and dashboards in Power BI
  • Strong programming skills in SQL and Python (Scala knowledge is a plus)
  • Hands-on experience with Azure services such as Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), Azure SQL, Synapse Analytics is an advantage
  • Proficiency in data modeling, database design, and SQL optimization
  • Solid understanding of data integration patterns, ETL/ELT best practices, cloud computing, and security principles
  • You should have demonstrable experience in at least one of the following domains: Sales, Finance, Project Business
Benefits
  • Workload: 100%
  • Working model: Remote
  • Long-term project with option to extend

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard skills
data engineeringETLELTdata modelingSQLPythonPySparkSparkSQLPower BIdata quality checks
Soft skills
problem-solvinganalytical skillseffective communicationcollaborationagile environments
Certifications
Bachelor's degree in Computer ScienceBachelor's degree in Engineering