Oscilar

Senior Data Engineer

Oscilar

full-time

Posted on:

Origin:  • 🇺🇸 United States

Visit company website
✨ AI Apply
Manual Apply

Job Level

Senior

Tech Stack

AirflowCloudDynamoDBETLGrafanaJavaKafkaPostgresPrometheusPythonRedisSQLTerraform

About the role

  • Architect and implement scalable ETL and data pipelines spanning ClickHouse, Postgres, Athena, and diverse cloud-native sources to support real-time risk management and advanced analytics
  • Design, develop, and optimize distributed data storage solutions for high performance and reliability at scale serving mission-critical ML models
  • Drive schema evolution, data modeling, sharding, partitioning, and pipeline orchestration (batch, streaming, CDC)
  • Own end-to-end data flow: integrate multiple internal and external data sources, enforce data validation and lineage, automate and monitor workflow reliability (CI/CD, anomaly detection)
  • Collaborate cross-functionally with engineers, product managers, and data scientists to enable fast experimentation and operationalization of ML/AI models
  • Champion radical ownership: identify opportunities, propose improvements, and implement technical and process solutions
  • Mentor and upskill team members and contribute to a collaborative, mission-oriented culture

Requirements

  • 5+ years in data engineering (or equivalent), including architecting and operating production ETL/ELT pipelines for real-time, high-volume analytic platforms
  • Deep proficiency with ClickHouse, Postgres, Athena
  • Experience with distributed data systems (Kafka, cloud-native stores)
  • Proven experience with both batch and streaming pipeline design
  • Advanced programming in Python and SQL; bonus points for Java
  • Expertise in workflow orchestration (Airflow, Step Functions), CI/CD, and automated testing for data
  • Experience in high-scale, low-latency environments
  • Understanding of security, privacy, and compliance requirements for financial-grade platforms
  • Strong communication, business alignment, and documentation abilities
  • Alignment with Oscilar’s values: customer obsession, radical ownership, bold vision, efficient growth, unified teamwork
  • Nice-to-have: Experience integrating Kafka with ClickHouse
  • Nice-to-have: Knowledge of event-driven architecture and streaming patterns (CQRS, event sourcing)
  • Nice-to-have: Hands-on experience with monitoring tools (Prometheus, Grafana, Kafka Manager)
  • Nice-to-have: Experience automating infrastructure with Terraform or CloudFormation
  • Nice-to-have: Proficiency with Redis, ClickHouse, Postgres, DynamoDB; data modeling and query optimization
  • Nice-to-have: Familiarity with encryption, role-based access control, and secure API development