Data Engineer

• Design and implement a comprehensive data lake architecture using modern cloud-native technologies
• Build scalable ETL/ELT pipelines for real-time and batch data processing across all data sources
• Establish data ingestion frameworks to collect data from application APIs, third-party services, and databases
• Architect automated data quality monitoring, validation, and alerting systems
• Create robust data warehousing solutions optimized for analytics and business intelligence
• Implement DataOps practices with automated testing and deployment pipelines (CI/CD for data)
• Develop and maintain Python-based data processing frameworks and utilities
• Build an automated data pipeline orchestration using Apache Airflow or similar tools
• Create streaming data processing solutions using Apache Kafka, Kinesis, or Pub/Sub
• Implement infrastructure as code for all data platform components (Terraform, CloudFormation)
• Establish feature stores and data models that support both operational and analytical workloads
• Optimize data storage costs and query performance across the entire platform
• Collaborate with product and business teams to identify key metrics, KPIs, and analytical requirements
• Build automated reporting dashboards and self-service business intelligence tools
• Support predictive modeling initiatives and A/B testing frameworks

Way

Senior Data Engineer