Staff Data Engineer
Abbott
Amazon RedshiftAWSCloudGoKafkaPySparkPythonSpark
Cloud Data Engineer
AlphaPoint
full-time
Posted on:
Location Type: Remote
Location: Remote • 🇺🇸 United States
Visit company websiteJob Level
SeniorLead
Tech Stack
AirflowApacheAWSAzureCassandraCloudDynamoDBElasticSearchETLGoogle Cloud PlatformJavaJavaScriptKafkaMicroservicesMongoDBNode.jsNoSQLPythonRabbitMQRedisSpark
About the role
- Build a scalable and highly performant infrastructure to process batch and real-time workloads
- Work with the AI engineering team and external engineering teams to monitor and extract data from a vast array of data sources
- Implement ETL data pipelines
- Architect backend data solutions to support various microservices
- Develop third-party integrations with large-scale legacy systems
Requirements
- Bachelor’s degree in computer science or similar discipline
- 7-10 years experience in software development
- Proficient in Python, Node.js, and/or Java
- Familiarity with the basic principles of distributed computing and data modeling
- Experience building ETL pipelines using Apache Airflow and Spark, Databricks, or other pipeline orchestration tools
- Experience with NoSQL databases such as MongoDB, Cassandra, DynamoDB, or CosmosDB
- Experience with real-time stream processing systems like Kafka, AWS Kinesis, GCP Data Flow
- Experience with Redis, Elasticsearch, Solr
- Experience with messaging systems like RabbitMQ, AWS SQS, GCP Cloud Tasks
- Ability to find creative ways to harvest data in unstructured formats by scraping, modeling, and ingesting data into semantic databases and graphs
- Familiarity with Delta Lake and Parquet files
- Familiarity with one or more cloud providers: AWS, GCP, or Azure
- Proficiency with Test Driven Development (TDD)
- Proficiency with Git using services such as Github or Bitbucket
Benefits
- 100% Remote Work Environment
- Competitive compensation
- Equity or stock options (if applicable)
- A culture of autonomy, experimentation, and learning
- Opportunity to make a real impact on company trajectory
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
PythonNode.jsJavaETL pipelinesApache AirflowSparkDatabricksNoSQL databasesRedisTest Driven Development
Soft skills
creativity
