Senior Data Engineer

DKSH Portugal, Unipessoal, Lda.

Senior Data Engineer at Opplane designing, building, and maintaining high-performance data platforms. Collaborating with teams for accurate, reliable, and secure analytics and machine learning workflows.

Posted 7/20/2026full-timeSanta Clara • California • 🇺🇸 United StatesSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and maintaining scalable data processing pipelines using frameworks like Apache Spark and PySpark, with a strong focus on data integrity and performance optimization. Proven ability to lead data platform migrations and collaborate effectively with cross-functional teams to support analytics and machine learning workflows.

Highest-signal resume keywords

Apache SparkPySparkData Platform MigrationPython Software EngineeringDistributed Systems

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data Processing PipelinesMicroservices DevelopmentCI/CD PipelinesData IntegrationData ValidationPerformance OptimizationData SecurityAccess ControlCloud-Native PlatformsBig Data Frameworks

Soft Skills

CollaborationProblem-SolvingCommunication

Tools & Technologies

Apache BeamClouderaSnowflakeDatabricks

Industry Keywords

Data ArchitectureData QualityData VersioningAnalytics WorkflowsMachine Learning

Tech Stack

Tools & technologies

ApacheCloudDistributed SystemsMicroservicesPySparkPythonSpark

About the role

Key responsibilities & impact

Design, develop, and maintain scalable data processing pipelines and workflows using frameworks such as Apache Spark, PySpark, and Apache Beam.
Build and maintain microservices in Python that serve data-driven features in production.
Develop internal tools to support CI/CD pipelines, experiment tracking, and data versioning.
Collect, process, and integrate large datasets from multiple sources, including databases, file systems, and APIs.
Ensure data integrity, consistency, and quality through robust validation and monitoring processes.
Optimize data systems for performance, scalability, and high availability.
Implement best practices for data security, access control, and privacy.
Collaborate with data scientists, analysts, and engineers to support analytics and ML workflows.
Lead complex migration initiatives involving the transition from on-premise Cloudera environments to cloud-native platforms, ensuring zero data loss and minimal downtime.

Requirements

What you’ll need

Strong understanding of distributed systems and modern data architectures.
Proven, hands-on experience with large-scale data platform migrations, specifically transitioning from Cloudera (CDH/HDP) to either Snowflake or Databricks.
Deep technical expertise in building and orchestrating a high-performance data platform from scratch.
5+ years of professional experience in software engineering or data engineering.
Strong software engineering skills with Python in large-scale, high-performance production environments.
Hands-on experience with Spark/PySpark and other big data frameworks.

Benefits

Comp & perks

Global & Multicultural – Diverse perspectives, global collaboration (US, Portugal, India and Singapore offices)
Startup Energy – Fast-moving, impact-driven environment
Ownership Mindset – Engineers own what they build
Collaborative & Friendly – Open, curious, and supportive culture