MagicOrange

Data Engineer

• Design, develop, and maintain robust ETL / ELT pipelines across cloud-based data platforms.
• Build and optimise data lakes and lakehouse architectures (Medallion approach) using Azure and Databricks.
• Develop scalable, reusable, and metadata-driven data processing frameworks.
• Ensure data pipelines are reliable, maintainable, and production-ready across dev, test, and production environments.
• Design and implement dimensional and analytical data models to support reporting, analytics, and AI use cases.
• Optimise data processing and query performance across large-scale datasets.
• Implement indexing, partitioning, and distribution strategies to ensure high-performance data access.
• Continuously monitor and improve data platform efficiency and cost usage.
• Implement best practices for data quality, validation, and observability.
• Support data governance initiatives, including metadata management, lineage, and data compliance.
• Ensure data platforms align with security, privacy, and regulatory requirements (including GDPR).
• Collaborate with DevOps and security teams to ensure secure and compliant data environments.
• Work closely with data science and AI teams to enable model training, deployment, and experimentation.
• Support Databricks-based analytics, streaming, and AI workloads.
• Translate business and product requirements into scalable data solutions.
• Provide technical guidance and mentorship to other data engineers.

Senior Data Engineer – Architect