Vytwo Technologies Inc

Data Engineer

• Design, develop, and maintain scalable data pipelines using Apache Spark (PySpark and/or Scala)
• Build and optimize data workflows on Databricks , including Delta Lake, notebooks, and scheduled jobs
• Ingest, transform, and curate large-scale structured and semi-structured datasets
• Perform performance tuning and cost optimization of Spark workloads and Databricks clusters
• Implement data quality checks, monitoring, and error handling
• Collaborate with analytics and business stakeholders to deliver well-modeled, analytics-ready data
• Support batch processing and, where applicable, streaming data pipelines
• Follow best practices for testing, documentation, security, and version control

Senior Data Engineer

Data Scientist

• Build, deploy, and optimize ML models for predictive analytics, forecasting, classification, and regression
• Perform large-scale feature engineering using PySpark and Big Data tools
• Work on batch pipelines, model versioning, and experiment tracking
• Develop cost estimation and risk/likelihood models using statistical and ML techniques
• Build NLP pipelines using deep learning frameworks such as PyTorch, TensorFlow, or similar
• Develop real‑time, low‑latency inference systems for text classification, embeddings, semantic search, summarization, and retrieval
• Create prompts, context graphs, and agentic workflows for LLM-based systems
• Apply knowledge of prompt engineering, context engineering, and autonomous agent frameworks to production systems
• Work in Databricks for ETL, feature engineering, ML training, and orchestration
• Use Azure services for model deployment, data pipelines, and infrastructure
• Collaborate using Git-based workflows; leverage tools like GitHub Copilot, Claude Code, etc.
• Implement model monitoring, observability, drift detection, and performance tracking

AI Engineer

• Design and build end-to-end data and AI pipelines using Azure Databricks.
• Develop robust ETL/ELT workflows using Python (PySpark) and SQL.
• Implement CI/CD pipelines for Databricks deployments (jobs, notebooks, workflows).
• Integrate Databricks with Azure services (Data Lake, Blob Storage, Key Vault, Azure OpenAI, Azure Functions, etc.).
• Optimize jobs for performance, cost, and reliability.
• Build reusable, modular code.
• Collaborate with data scientists and platform teams to move models from experimentation to production.
• Implement logging, monitoring, and error handling for production pipelines.
• Develop and deploy ML and Generative AI models (LLMs, embeddings, RAG pipelines) for NLP, computer vision, and predictive analytics.
• Fine-tune LLMs using LoRA/QLoRA and integrate with Azure OpenAI or Hugging Face models.
• Implement vector search and retrieval pipelines using FAISS or Azure Cognitive Search.
• Ensure responsible AI practices, including bias detection and model governance.

Senior Data Engineer

Data Scientist

Lead AI Engineer