Tech Stack
DjangoOpen SourcePythonSQL
About the role
- Maintain and grow our data pipeline that enables users to import data from API and database sources
- Expand our source library and optimize streaming data from ClickHouse to object storage using Arrow
- Build well-architected, well-tested code and ship fast
- Design core interfaces to expand source library; debug memory issues in data pipeline; implement granular schema control; build a graph traverser; instrument usage tracking
Requirements
- Experience with Python and Django. Our core application backend and data pipeline services are built with Python and Django
- Hands-on experience with the Arrow data format. We stream data from ClickHouse to object storage with Arrow as the intermediary format
- Strong skills in designing, architecting, and building data systems from the ground up
- While frontend may not be your primary focus, you’re not afraid to dive in when needed