Senior Data Engineer

Optima Partners

Senior Data Engineer building scalable pipelines and architectures for Optima Partners, an Edinburgh data and business consultancy. Delivering advanced solutions for leading consumer brands.

Posted 9/3/2026full-timeEdinburgh • 🇬🇧 United KingdomSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and implementing scalable data pipelines and architectures, with proficiency in Python, Java, and cloud ecosystems. Capable of mentoring teams and delivering impactful data solutions while ensuring data quality and performance.

Highest-signal resume keywords

Data Pipeline DesignETL/ELT WorkflowsCloud Ecosystems (AWS, Azure, GCP)Data Warehousing (Snowflake, Redshift, BigQuery)Apache Spark

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

PythonJavaScalaSQLPostgresApache SparkHadoopData WarehousingETL/ELTSchema Design

Soft Skills

Problem-SolvingAnalytical SkillsCommunicationCollaborationAdaptability

Tools & Technologies

DatabricksAirflowApache NiFiTalendGitDockerCI/CD

Industry Keywords

Data EngineeringData ArchitectureData QualityAgileScrum

Tech Stack

Tools & technologies

AirflowAmazon RedshiftApacheAWSAzureBigQueryCloudDockerETLGoogle Cloud PlatformHadoopJavaPostgresPythonScalaSparkSQL

About the role

Key responsibilities & impact

Design and implement robust, scalable, and efficient data pipelines to collect, transform, and integrate data from various sources
Develop and optimise data architectures, including data warehouses, data lakes, and other storage solutions
Create and maintain reliable ETL/ELT workflows to ensure data quality and accessibility
Partner with data scientists, analysts, and client stakeholders to understand requirements and deliver impactful solutions
Monitor and enhance data-system performance to ensure minimal downtime and rapid query responses
Identify automation opportunities and recommend tools to improve data-engineering workflows
Maintain detailed technical documentation for solutions and processes
Mentor junior engineers and lead technical initiatives within cross-functional teams

Requirements

What you’ll need

Proficiency in Python, Java, Scala, or similar languages for data processing
Hands-on experience with Databricks, Apache Spark, and Hadoop
Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks
Expertise in relational databases such as Postgres and SQL Server
Knowledge of Airflow, Apache NiFi, or Talend
Experience with data warehousing tools such as Snowflake, Redshift, or BigQuery and schema design
Familiarity with Git, Docker, and CI/CD pipelines
5+ years of experience in data engineering or a related role
Proven track record delivering advanced data solutions in consulting or client-facing environments
Experience with Agile or Scrum practices is beneficial
Strong problem-solving and analytical skills
Excellent communication abilities to explain complex technical concepts to non-technical stakeholders
Collaborative, adaptable, and team-oriented mindset

Benefits

Comp & perks

Inclusive environment fostering continuous learning, knowledge sharing, and clear career development
Opportunity to collaborate with experts across fields and gain exposure to varied business challenges
Hybrid working arrangement