Data Engineer

• Guide team to migrate from on Prem Cloudera to Azure cloud environment.
• Designed and implemented scalable data lake solutions using Snowflake and Databricks.
• Developed and optimized data pipelines for ingestion, transformation, and storage.
• Managed data governance, quality, and security across cloud environments.
• Implemented performance tuning, automation, and CI/CD for data workflows.
• Collaborated with cross-functional teams to support cloud migration activities.
• Install, configure, manage, and monitor Cloudera Hadoop clusters, ensuring high availability, performance, and security.
• Tune Hadoop, Hive, and Spark jobs and configurations for optimal performance, efficiency, and resource utilization.
• Diagnose and resolve issues related to Linux servers, networks, cluster health, job failures, and performance bottlenecks.
• Implement and manage security measures within the Cloudera environment.
• Migrate Hadoop, Hive, and Spark data and applications to Azure cloud services.
• Develop scripts (e.g., shell, Ansible, Python) for automating administrative tasks, deployments, and monitoring.
• Create and maintain documentation for system configurations, operational procedures, and troubleshooting knowledge bases.
• Work closely with the Cloudera vendor to stay current with the latest releases, perform upgrades, and address vulnerabilities.

Senior Data Engineering Specialist

Junior Data Engineer

Junior Data Engineer

ETL Data Engineer, Databricks

Senior Data Engineer