Senior Data Engineer – AWS
Rockcruit
full-time
Posted on:
Location Type: Remote
Location: Remote • 🇦🇷 Argentina
Visit company websiteJob Level
Senior
Tech Stack
AirflowAmazon RedshiftAWSDockerETLGrafanaKubernetesNumpyPandasPrometheusPythonSQLTerraform
About the role
- Design, build, and maintain a secure, scalable, and high-performance data platform on AWS.
- Migrate and modernize infrastructure across AWS accounts while ensuring minimal downtime and data integrity.
- Develop ETL/ELT pipelines, including those that incorporate LLM-based transformations, semantic enrichment, or generative AI workflows.
- Apply prompt engineering best practices—including prompt templates, few-shot learning, and validation strategies—to optimize LLM performance.
- Leverage AI-assisted development tools such as GitHub Copilot to enhance productivity and maintain code quality.
- Build standardized, reusable datasets to serve diverse analytical and modeling use cases.
- Integrate and manage data ingestion from varied sources: APIs, SaaS platforms, databases, logs, and event streams.
- Implement DevOps and MLOps best practices, including CI/CD, observability, and infrastructure-as-code (Terraform, Kubernetes, Docker).
- Establish data quality automation frameworks (e.g., Great Expectations) and monitor data drift, schema evolution, and anomalies.
- Collaborate with Product, Quant, and Engineering teams to align infrastructure design with analytical and business objectives.
- Monitor and optimize pipeline performance, reliability, and cost, with clearly defined SLOs/SLAs and observability metrics.
- Ensure security, governance, and compliance across all data layers (IAM, encryption, RBAC, data lineage, GDPR/PII safeguards).
- Contribute to technical architecture, code reviews, and documentation, setting standards for scalability and maintainability.
Requirements
- 4-6 years of experience as a Data Engineer, Software Engineer, or Data Platform Engineer in production-grade data systems.
- Proven expertise in AWS services (Lambda, Glue, ECS/EKS, S3, Redshift/Snowflake, Step Functions, Airflow).
- Strong proficiency in Python and SQL; hands-on experience with Pandas, Numpy, or similar data frameworks.
- Experience building LLM-integrated data pipelines, including using APIs or frameworks for inference, enrichment, or automated reasoning.
- Solid understanding of prompt engineering concepts and tools (few-shot, chain-of-thought, prompt tuning, etc.).
- Experience using GitHub Copilot or other AI-assisted coding tools in production environments.
- Expertise in infrastructure-as-code (Terraform), container orchestration (Docker, Kubernetes), and CI/CD pipelines.
- Familiarity with data observability and monitoring (Prometheus, Grafana, Datadog, OpenTelemetry).
- Understanding of metadata management, data lineage, and data governance practices.
- Experience with data warehousing and MLOps workflows for deploying and maintaining models in production.
- Awareness of cost optimization strategies, autoscaling, and performance tuning for large-scale systems.
- Strong testing and validation mindset—automated schema testing, anomaly detection, and quality checks.
- Excellent communication and collaboration skills; comfortable working cross-functionally with analysts, quants, and product managers.
- Experience in fintech or financial data environments preferred.
- Familiarity with private equity or venture capital markets a plus—or the curiosity to learn fast.
Benefits
- Be part of a small, high-impact team building the data foundation for a fintech leader in private market trading.
- Work on cutting-edge AI and data infrastructure, integrating LLMs and automation into real financial data products.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
AWSPythonSQLETLELTprompt engineeringinfrastructure-as-codedata warehousingMLOpsdata quality automation
Soft skills
communicationcollaborationtesting mindsetvalidation mindsetcross-functional teamwork