FREE ACCESS
5,000–10,000 jobs/day

See all jobs on JobTailor
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.

Senior Data Engineer, AI Ingestion Platform
Software MindSenior Data Engineer building AI ingestion pipelines for the real estate sector. Joining a multicultural team at Software Mind focused on innovative technology solutions.
Tech Stack
Tools & technologiesAWSDynamoDBETLPython
About the role
Key responsibilities & impact- Build and own the historical email ingestion pipeline via Microsoft Graph API
- Implement SharePoint / OneDrive document ingestion pipeline with scoped folder access
- Design and implement the PII minimisation pre-processing layer
- Build the vector store indexing workflow (OpenSearch/Pinecone) with per-tenant data isolation
- Define and implement the data processing schema; produce and maintain schema documentation
- Build the OCR routing orchestrator and integrate OCR service for scanned documents
- Implement the raw text / content extraction layer for all supported document types
- Define and prototype push vs. pull ingestion strategy, from one-time PoC through to incremental nightly pipeline
- Ensure data lineage and audit traceability are built into pipeline outputs from the outset
Requirements
What you’ll need- 6+ years in data engineering; strong pipeline and ETL/ELT experience required
- Proficiency in Python for data pipeline development
- Experience with Microsoft Graph API or similar enterprise email/document APIs (M365, Exchange Online)
- AWS data services: S3, DynamoDB, Glue, and/or Lambda-based event-driven processing
- Familiarity with PII detection and data minimisation techniques (regex-based, NER-based, or purpose-built libraries)
- Experience with vector store indexing or semantic search pipeline construction
Benefits
Comp & perks- Flexible work arrangements
- Professional development opportunities
- Work in a multicultural team
ATS Keywords
✓ Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
data engineeringpipeline developmentETLELTPythonMicrosoft Graph APIAWS S3AWS DynamoDBAWS Gluevector store indexing