Senior Software Engineer, Data Engineering
LMArena
full-time
Posted on:
Location Type: Hybrid
Location: Bay Area • California • 🇺🇸 United States
Visit company websiteJob Level
Senior
Tech Stack
ApachePythonRayScalaSparkSQL
About the role
- Design and build robust data pipelines to ingest, process, and transform user vote data to features essential for model performance evaluation.
- Collaborate with researchers and product leadership to understand product goals and necessary data.
- Design and implement solutions to generate result dashboards and reports, providing useful information for the public, model providers, and researchers.
- Ensure the integrity, data quality, and reliability of the pipelines.
- Scale our data infrastructure to accommodate increasing data volumes and evolving analytical needs.
Requirements
- 5+ years of experience in software engineering, with a dedicated focus on data engineering and big data technologies
- Proficiency in SQL and at least one programming language commonly used for data analysis (Python (preferred), Scala, R).
- Hands-on experience with data processing and pipeline frameworks (Apache Spark, Ray Data, etc.) and at least one popular big data analytics platform (Databricks, Snowflake).
- Demonstrated experience in designing, implementing, optimizing, and debugging production data pipelines.
Benefits
- Comprehensive health and wellness benefits, including medical, dental, vision, and additional support programs.
- The opportunity to work on cutting-edge AI with a small, mission-driven team.
- A culture that values transparency, trust, and community impact.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
data pipelinesSQLPythonScalaRApache SparkRay DataDatabricksSnowflakedata processing
Soft skills
collaborationcommunicationproblem-solvingattention to detailorganizational skills