Pro Talent

Real-time Data Streaming Engineer, Kafka, Flink, Spark Streaming

Pro Talent

full-time

Posted on:

Location Type: Remote

Location: Remote • 🇪🇺 Anywhere in Europe

Visit company website
✨ AI Apply
Apply

Job Level

Mid-LevelSenior

Tech Stack

ApacheAWSAzureCassandraCloudDistributed SystemsDockerGoogle Cloud PlatformGrafanaIoTJavaKafkaKubernetesMicroservicesMongoDBNoSQLPostgresPrometheusPulsarPythonRabbitMQScalaSparkSQL

About the role

  • Design, develop, and maintain real-time data streaming pipelines using Kafka, Flink, and Spark Streaming.
  • Architect event-driven and microservices-based solutions for real-time analytics and processing.
  • Implement data ingestion, transformation, and enrichment workflows across distributed systems.
  • Optimize performance and scalability of streaming jobs for high-throughput, low-latency environments.
  • Ensure data quality, governance, and fault tolerance within the streaming infrastructure.
  • Integrate streaming solutions with data warehouses, data lakes, and cloud platforms (AWS, Azure, GCP).
  • Collaborate with data engineers, data scientists, and application teams to deliver business-critical real-time insights.
  • Monitor, troubleshoot, and improve the reliability and resilience of streaming systems.
  • Participate in system design, code reviews, and best practices development.

Requirements

  • 5+ years of experience in data engineering with at least 3+ years focused on real-time streaming.
  • Strong expertise in Apache Kafka (producers, consumers, Connect, Streams, schema registry).
  • Hands-on experience with Apache Flink or Spark Streaming for real-time data processing.
  • Solid understanding of event-driven architectures, pub/sub systems, and distributed computing.
  • Strong programming skills in Java, Scala, or Python.
  • Proficiency in SQL and experience with databases (relational and NoSQL: PostgreSQL, Cassandra, MongoDB, etc.).
  • Familiarity with cloud-native streaming solutions (AWS Kinesis, Azure Event Hubs, GCP Pub/Sub).
  • Knowledge of CI/CD, containerization (Docker, Kubernetes), and monitoring tools (Prometheus, Grafana, ELK).
  • Strong problem-solving and debugging skills, with experience in large-scale production environments.
  • Language Requirements: Fluent English.
  • Nice to Have: Knowledge of data lakehouse architectures (Delta Lake, Iceberg, Hudi).
  • Nice to Have: Experience with machine learning pipelines on streaming data.
  • Nice to Have: Familiarity with message brokers (RabbitMQ, Pulsar, ActiveMQ).
  • Nice to Have: Background in industries like fintech, telecom, IoT, or e-commerce where real-time data is critical.
  • Nice to Have: Contributions to open-source streaming projects.

ATS Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard skills
data engineeringreal-time streamingApache KafkaApache FlinkSpark StreamingJavaScalaPythonSQLCI/CD
Soft skills
problem-solvingdebuggingcollaborationcommunicationreliabilityresilience
Grainger

Senior Data Engineer

Grainger
Seniorfull-time$111k–$184k / yearIllinois · 🇺🇸 United States
Posted: 30 days agoSource: jobs.grainger.com
ApacheAWSAzureCloudDistributed SystemsDockerETLGoogle Cloud PlatformKafkaKubernetesOpenShiftPostgres+5 more
Twilio

Senior Software Engineer (L3)

Twilio
Seniorfull-time$139k–$204k / year🇺🇸 United States
Posted: 41 days agoSource: boards.greenhouse.io
AWSCloudDistributed SystemsDockerDynamoDBGoGrafanaJavaKafkaKubernetesLinuxPostgres+4 more
Fortive

Principal Software Engineer

Fortive
Leadfull-time🇺🇸 United States
Posted: 11 days agoSource: ejta.fa.us6.oraclecloud.com
AWSAzureCloudDistributed SystemsETLGoogle Cloud PlatformJavaKafkaNoSQLPythonSparkSQL
Paradigm Health

Staff SW Engineer, DevOps/MLOps

Paradigm Health
Leadfull-time$180k–$220k / year🇺🇸 United States
Posted: 37 days agoSource: boards.greenhouse.io
AirflowApacheAWSCloudDistributed SystemsDockerETLGoogle Cloud PlatformGrafanaKubernetesPrometheusPython+5 more
CrowdStrike

Senior SDET – Cloud, Endpoint

CrowdStrike
Seniorfull-time$140k–$215k / year🇺🇸 United States
Posted: 22 days agoSource: crowdstrike.wd5.myworkdayjobs.com
AWSAzureCassandraCloudCyber SecurityDistributed SystemsDockerGoGoogle Cloud PlatformGrafanaGRPCJava+12 more