FREE ACCESS
5,000–10,000 jobs/day
See all jobs on JobTailor
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Hadoop Developer
Bright Vision TechnologiesHadoop Developer designing and operating big-data pipelines at Bright Vision Technologies. Contributing to the transformation of business processes through innovative technology solutions.
Tech Stack
Tools & technologiesAirflowApacheDistributed SystemsETLHadoopHBaseHDFSJavaKafkaMapReduceNoSQLPythonScalaSparkSQL
About the role
Key responsibilities & impact- Design, develop, and operate end-to-end big-data pipelines on Hadoop, ingesting data from various sources.
- Build robust ETL/ELT workflows using Apache Spark, Hive, Pig, and Sqoop.
- Develop high-throughput streaming data pipelines using Kafka, Spark Streaming, or Flink.
- Optimize Spark and MapReduce jobs to meet SLAs at minimal cost.
- Design and maintain data models and storage layouts on HDFS and related formats.
- Implement data governance and quality controls.
- Build robust monitoring and logging strategies for big-data pipelines.
- Partner with data scientists to deliver reliable datasets.
- Automate pipeline orchestration using Airflow or Oozie.
- Continuously evaluate and adopt new technologies in the big-data ecosystem.
- Mentor junior engineers and contribute to the team’s engineering standards.
Requirements
What you’ll need- 5+ years of professional experience designing and operating big-data pipelines on Hadoop.
- Strong hands-on expertise with Apache Spark (Scala, Python, or Java) in production environments.
- Solid experience with Hive, HDFS, Sqoop, HBase, and the broader Hadoop ecosystem.
- Hands-on experience with streaming data platforms such as Kafka, Spark Streaming, or Flink.
- Strong SQL skills and experience working with both relational and NoSQL data stores.
- Experience with workflow orchestration tools such as Airflow or Oozie.
- Solid understanding of distributed systems concepts, including partitioning, replication, and fault tolerance.
- Strong scripting skills in Python or Shell.
- Excellent troubleshooting, debugging, and documentation skills.
Benefits
Comp & perks- Comprehensive benefits
- Competitive compensation packages
- Supportive work-life balance
ATS Keywords
✓ Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
HadoopApache SparkETLELTKafkaSpark StreamingFlinkSQLPythonShell
Soft Skills
troubleshootingdebuggingdocumentationmentoring