Explore more
About the role
- Provision all cloud infrastructure via Terraform: object storage, vector databases, event streaming, Kubernetes, time-series databases, authentication. All reproducible for new tenants.
- Design multi-tenant storage: shared vector indices for public data, per-tenant indices for proprietary data. Row-level security or schema-level isolation.
- Design per-tenant storage structure with bucket policies enforcing isolation.
- Build market data storage pipeline: exchange feeds → event bus → time-series database.
- Build monitoring dashboards for data pipeline health across all data sources.
- Design feedback data storage: per-tenant schema for feedback events and training data candidates.
- Build data archival pipelines for cost-efficient long-term storage.
- Automate tenant provisioning: a script that creates a new tenant’s storage, network policies, and service accounts.
Requirements
- 4+ years data engineering; strong SQL, Python, and cloud infrastructure.
- Experience designing multi-tenant data architectures with isolation requirements.
- Infrastructure as Code: Terraform or Pulumi — mandatory.
- PostgreSQL experience (vector extensions, partitioning, row-level security a plus).
- Kafka consumer/producer development.
- Time-series data storage and querying experience.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
SQLPythonTerraformPulumiPostgreSQLKafkatime-series databasesdata engineeringdata architecturedata pipeline