Lead Data Engineer, Python, AWS, Spark, Kafka, SQL, Snowflake, Databricks, GenAI

Capital One

Lead Data Engineer building Python, Spark, Kafka, and cloud data platforms for Capital One’s personalized marketing systems. Delivering scalable omnichannel messaging solutions.

Posted 9/3/2026full-timeMcLean • New York, Virginia • 🇺🇸 United StatesSenior💰 $179,400 - $245,600 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in application development with a strong focus on data movement, cloud computing, and big data technologies. Proficient in Python, SQL, and data warehousing solutions, while effectively leading teams and optimizing systems for enhanced user experience.

Highest-signal resume keywords

Python ProgrammingSQL Database ManagementCloud Computing (AWS, Azure, Google Cloud)Big Data Technologies (Spark, Hadoop, Kafka)Data Warehousing (Snowflake, Redshift)

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Application DevelopmentData MovementDistributed ComputingETL ToolsData ModelingReal-Time Data ProcessingNoSQL ImplementationUnit TestingCode ReviewPerformance Tuning

Soft Skills

CollaborationMentoringAgile Practices

Tools & Technologies

SnowflakeDatabricksAWS GlueUNIX/LinuxMongoDBCassandraMapReduceHiveEMR

Industry Keywords

Cloud-Based SolutionsOmnichannel MarketingAdtechFinancial Empowerment

Tech Stack

Tools & technologies

Amazon RedshiftAWSAzureCassandraCloudETLHadoopKafkaLinuxMapReduceMySQLNoSQLPythonRDBMSShell ScriptingSparkSQLUnix

About the role

Key responsibilities & impact

Collaborate across Agile teams to design, develop, test, implement, and support technical solutions using data movement tools and technologies
Lead a team of developers experienced in data movement, distributed computing, and full-stack systems
Use Python, SQL, open-source RDBMS and NoSQL databases, Snowflake, and Databricks
Optimize information systems for end users and downstream application consumers through sound data design practices
Stay current with technology trends, experiment with and learn new technologies, participate in internal and external technology communities, and mentor engineering community members
Collaborate with digital product managers to deliver robust cloud-based solutions supporting millions of Americans’ financial empowerment
Perform unit tests and conduct code reviews to ensure rigorous design, elegant coding, and performance tuning
Build scalable platforms for hyper-personalized omnichannel marketing and messaging experiences across owned and paid Adtech channels

Requirements

What you’ll need

Bachelor’s Degree
At least 4 years of experience in application development (internship experience does not apply)
At least 2 years of experience in big data technologies
At least 1 year of experience with cloud computing (AWS, Microsoft Azure, Google Cloud)
No employment authorization sponsorship or immigration-related support for new applicants
Preferred: Master’s Degree
Preferred: 7+ years of application development experience including Python, SQL, Spark, ETL tools, or AWS Glue
Preferred: 4+ years of experience with a public cloud
Preferred: 4+ years with distributed data/computing tools such as MapReduce, Hadoop, Hive, EMR, Kafka, Spark, or MySQL
Preferred: 4+ years working on real-time data and streaming applications
Preferred: 4+ years of NoSQL implementation experience with Mongo or Cassandra
Preferred: 4+ years of data warehousing experience with Redshift or Snowflake
Preferred: 4+ years of UNIX/Linux experience, including basic commands and shell scripting
Preferred: 4+ years of data modeling for data warehousing
Preferred: 2+ years of Agile engineering practices
Preferred: Experience leveraging interactive AI tooling such as Claude Code or GitHub Copilot

Benefits

Comp & perks

Performance-based incentive compensation, which may include cash bonuses and/or long-term incentives (LTI)
Comprehensive, competitive, and inclusive health, financial, and other benefits supporting total well-being
Reasonable accommodations for applicants who require them