Agility

Data Engineer – Temporary Position

Agility

contract

Posted on:

Location Type: Remote

Location: India

Visit company website

Explore more

✨ AI Apply
Apply

Tech Stack

ElasticSearchETLLogstashPandasPostgresPython

About the role

  • Build and maintain ETL pipelines that ingest daily Excel/CSV trade files, transform and validate the data, and persist it to PostgreSQL
  • Design and evolve database schemas using SQLAlchemy ORM and Alembic migrations
  • Implement business logic services: shareholder balance tracking, sharebook reconciliation, movement detection, classification rules
  • Maintain and extend Elasticsearch sync services (incremental and full resync workflows)
  • Support business stakeholders with data questions using the ELK stack — writing and running queries, interpreting results, and surfacing relevant data from Kibana
  • Parse and process structured financial data files (Excel, CSV) using Pandas, including data validation and deduplication
  • Write robust, type-safe Python following project standards (mypy strict mode, Black, isort)
  • Write and maintain automated tests with pytest — unit tests, repository mocks, snapshot tests
  • Collaborate with the CTO on data flow design and review migration strategies before deployment

Requirements

  • 3–5 years of professional software/data engineering experience
  • At least 2 years working with Python in a production environment
  • **Python 3.8+** — 3+ years of production experience; comfortable with type hints, async/await, and clean OOP design
  • **PostgreSQL** — schema design, indexing strategies, query optimization, migrations
  • **SQLAlchemy 2.0** — declarative ORM models, session management, Alembic for migrations
  • **Pandas** — data transformation, Excel/CSV parsing, validation pipelines
  • **FastAPI** — building and maintaining REST APIs with dependency injection and async handlers
  • **Pytest** — writing unit and integration tests, using fixtures and mocking patterns
  • **ELK Stack (Elasticsearch, Kibana)** — working knowledge of Elasticsearch indexing and querying; able to write queries and navigate Kibana to answer business questions (dashboard building not required; Logstash not used)
  • **Code quality discipline** — strong typing, Black formatting, readable and maintainable code
  • **Nice to Have**
  • **Web scraping** — familiarity with Playwright or similar browser automation tools
  • **Financial data domain** — understanding of stock trading concepts (OHLCV data, shareholder registers, portfolio performance metrics such as XIRR or TWR)
  • **Financial APIs** — experience with yfinance or similar market data sources
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
Python 3.8+PostgreSQLSQLAlchemy 2.0PandasFastAPIPytestETL pipelinesdata validationtype hintsasync/await
Soft Skills
collaborationcommunicationproblem-solvingattention to detailcode quality discipline