BT Group

Data Integration Engineering Specialist

BT Group

full-time

Posted on:

Location Type: Office

Location: Bengaluru • 🇮🇳 India

Visit company website
✨ AI Apply
Apply

Job Level

Mid-LevelSenior

Tech Stack

AnsibleApacheAWSBigQueryCloudDockerGoogle Cloud PlatformGrafanaJavaKafkaKubernetesPrometheusPythonRabbitMQScalaSparkTerraform

About the role

  • Design and implement Kafka-based streaming solutions for high availability and fault tolerance.
  • Define topic structure, partitioning, replication, retention, and compaction policies.
  • Architect stream processing pipelines using Apache Flink for real-time analytics.
  • Implement disaster recovery, geo-replication, and security (TLS, SASL, RBAC, ACLs).
  • Hands on experience in setting up confluent kafka on On-prem and cloud platforms
  • Optimize cloud resources for cost efficiency and performance.
  • Implement Dynatrace for end-to-end monitoring of Kafka clusters and streaming jobs.
  • Configure dashboards, alerts, and performance metrics for proactive issue detection.
  • Build producers, consumers, and connectors using Kafka Connect.
  • Integrate with data lakes, databases, and APIs across hybrid environments.
  • Build POC's for different solutions.
  • Work with DevOps teams for infrastructure automation (Terraform, Ansible).
  • Mentor teams on Kafka best practice and streaming architecture.

Requirements

  • Core Technologies: Confluent/Apache Kafka, Kafka Streams, Kafka Connect, ksqlDB, Schema Registry, Ab initio,
  • Stream Processing: Apache Flink, Spark Streaming (optional)
  • Cloud Platforms: AWS (MSK, Lambda, S3), GCP (Pub/Sub, Dataflow, BigQuery).
  • Monitoring Tools: Dynatrace, Prometheus, Grafana.
  • Containerization: Kubernetes, Docker, Helm
  • Programming: Java, Scala, Python.
  • Infrastructure: Terraform, Ansible.
  • Security: TLS, SASL, RBAC, ACLs.
  • Data Integration & Messaging: Ab Initio (Continuous Process Integration), RabbitMQ, IBM MQ, Ab Initio Queue
  • Preferred: Confluent certifications, experience with CDC patterns and event sourcing.
  • Soft Skills: - Strong problem-solving and communication skills - Ability to lead architecture discussions and mentor teams - Experience in cross-functional collaboration
Benefits
  • Flexible working hours

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard skills
KafkaKafka StreamsKafka ConnectksqlDBApache FlinkJavaScalaPythonTerraformAnsible
Soft skills
problem-solvingcommunicationleadershipmentoringcross-functional collaboration
Certifications
Confluent certifications