FREE ACCESS
5,000–10,000 jobs/day
See all jobs on JobTailor
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Senior AI Researcher – Pre-training Data
Aleph Alpha. Stay at the bleeding edge of foundation model research.
Tech Stack
Tools & technologiesPythonPyTorch
About the role
Key responsibilities & impact- Stay at the bleeding edge of foundation model research. You will identify, implement, and iterate on novel approaches to estimating data quality, synthetic data generation, curriculum learning, and advanced curation techniques.
- Design and lead rigorous ablation studies across various scales. You will systematically analyse how changes in data composition, deduplication strategies, heuristic and model-based curation, and scaling laws affect training dynamics and target model and system capabilities.
- Move beyond basic perplexity filtering. Research and build advanced algorithms to score and select data, such as influence functions, gradient-based matching, or using smaller models to curate data for larger ones.
- Partner closely with a diverse team to scale your research from prototypes to trillions-of-tokens-scale pipelines, and work with the post-training team to ensure pre-training distributions effectively support targeted fine-tuning and customer-alignment.
Requirements
What you’ll need- A deep understanding of machine learning theory, specifically regarding foundation model training dynamics, scaling laws, and data-centric AI.
- Experience designing and evaluating complex ML experiments related to data composition, curriculum learning, or data quality on language model training.
- Familiarity with statistical methods for evaluation and experiment design.
- Ability to reason about the information-theoretic properties of a dataset and its predictive power for evaluated tasks: not just processing data, but understanding its signal.
- Strong Python skills and comfort with ML tooling and deep learning frameworks (especially PyTorch).
- Willingness to relocate to Heidelberg or travel at least fortnightly.
- PhD in machine learning, NLP, or equivalent research experience focusing on large-scale language modeling or data curation (preferred).
- A history of contributions to top-tier venues (NeurIPS, ICML, ICLR, ACL, etc.) specifically regarding data curation, scaling laws, synthetic data, or LLM pre-training (preferred).
Benefits
Comp & perks- 30 days of paid vacation
- Access to a variety of fitness & wellness offerings via Wellhub
- Mental health support through nilo.health
- Substantially subsidized company pension plan for your future security
- Subsidized Germany-wide transportation ticket
- Budget for additional technical equipment
- Flexible working hours for better work-life balance and hybrid working model
- Virtual Stock Option Plan
- JobRad® Bike Lease
ATS Keywords
✓ Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
machine learning theoryfoundation model training dynamicsscaling lawsdata-centric AIML experimentsdata compositioncurriculum learningdata qualityPythondeep learning frameworks
Soft Skills
analytical skillscollaborationproblem-solvingcommunication
Certifications
PhD in machine learningPhD in NLP