C the Signs

Data Engineer

• Collaborate with data scientists and machine learning engineers to understand data requirements for LLM and machine learning model fine-tuning.
• Design, build, and maintain scalable data pipelines to ingest, process, and store massive and diverse healthcare datasets.
• Implement robust data validation and monitoring to ensure the integrity, accuracy, and consistency of all training datasets.
• Implement robust data cleaning, validation, and transformation processes to ensure data quality and integrity.
• Develop and optimize data structures and schemas for efficient access and utilization by LLMs and machine learning models.
• Work with the team to identify and acquire new data sources, ensuring compliance with relevant healthcare regulations (e.g., HIPAA).
• Monitor data pipeline performance, troubleshoot issues, and implement optimizations to improve efficiency and reliability.
• Document data engineering processes, data models, and data dictionaries.
• Stay up-to-date with the latest advancements in data engineering, big data technologies, and machine learning.

AI Data Engineer