Principal Data Engineer
ATPCO
Amazon RedshiftApacheAWSCloudJavaKafkaPythonScalaSQL
Staff Data Engineer
Abbott
full-time
Posted on:
Location Type: Remote
Location: Remote • 🇺🇸 United States
Visit company websiteSalary
💰 $97,300 - $194,700 per year
Job Level
Lead
Tech Stack
Amazon RedshiftAWSCloudGoKafkaPySparkPythonSpark
About the role
- Design and implement data pipelines to be processed and visualized across a variety of projects and initiatives
- Develop and maintain optimal data pipeline architecture by designing and implementing data ingestion solutions on AWS using AWS native services.
- Design and optimize data models on AWS Cloud using Databricks and AWS data stores such as Redshift, RDS, S3
- Integrate and assemble large, complex data sets that meet a broad range of business requirements
- Read, extract, transform, stage and load data to selected tools and frameworks as required and requested
- Customizing and managing integration tools, databases, warehouses, and analytical systems
- Process unstructured data into a form suitable for analysis and assist in analysis of the processed data
- Working directly with the technology and engineering teams to integrate data processing and business objectives
- Monitoring and optimizing data performance, uptime, and scale; Maintaining high standards of code quality and thoughtful design
- Create software architecture and design documentation for the supported solutions and overall best practices and patterns
- Support team with technical planning, design, and code reviews including peer code reviews
- Provide Architecture and Technical Knowledge training and support for the solution groups
- Develop good working relations with the other solution teams and groups, such as Engineering, Marketing, Product, Test, QA.
- Stay current with emerging trends, making recommendations as needed to help the organization innovate
- Proactively planning complex projects from scope/timeline development through technical design and execution.
- Demonstrate leadership through mentoring other team members.
Requirements
- Bachelors Degree in Computer Science, Information Technology or other relevant field
- At least 5 to 10 years of recent experience in Software Engineering, Data Engineering or Big Data
- Ability to work effectively within a team in a fast-paced changing environment
- Knowledge of or direct experience with Databricks and/or Spark.
- Software development experience, ideally in Python, PySpark, Kafka or Go, and a willingness to learn new software development languages to meet goals and objectives.
- Knowledge of strategies for processing large amounts of structured and unstructured data, including integrating data from multiple sources
- Knowledge of data cleaning, wrangling, visualization and reporting
- Ability to explore new alternatives or options to solve data mining issues, and utilize a combination of industry best practices, data innovations and experience
- Familiarity of databases, BI applications, data quality and performance tuning
- Excellent written, verbal and listening communication skills
- Comfortable working asynchronously with a distributed team
Benefits
- Career development with an international company where you can grow the career you dream of.
- Employees can qualify for free medical coverage in our Health Investment Plan (HIP) PPO medical plan in the next calendar year.
- An excellent retirement savings plan with high employer contribution.
- Tuition reimbursement, the Freedom 2 Save student debt program and FreeU education benefit - an affordable and convenient path to getting a bachelor’s degree.
- A company recognized as a great place to work in dozens of countries around the world and named one of the most admired companies in the world by Fortune.
- A company that is recognized as one of the best big companies to work for as well as a best place to work for diversity, working mothers, female executives, and scientists.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
data pipeline designdata ingestion solutionsdata modelingAWSDatabricksRedshiftRDSS3PythonPySpark
Soft skills
team collaborationleadershipmentoringcommunicationproblem-solvingproject planningadaptabilitytechnical designcode reviewsrelationship building
Certifications
Bachelor's Degree in Computer ScienceBachelor's Degree in Information Technology
