Data Engineering Lead

Data Engineering Lead, RhythmX AI Location: India, Bangalore Classification: Full Time Associate About RhythmX AI RhythmX AI is a generative AI-native health company driving a paradigm shift in hyper-personalized care. RhythmX AI’s precision care platform helps physicians pioneer a new era of whole-person care through generative and predictive AI-powered copilots. An SAIGroup company, RhythmX AI leverages SAIGroup assets, including the advanced Eureka AI platform and longitudinal data related to 300 million patients, more than 4.4 billion total annual claims, and more than 1.8 million healthcare professionals at more than 300 thousand facilities. RhythmX AI comprises healthcare and technology experts, operators, and the industry’s leading clinical advisors. https://rhythmx.ai/ Job Overview We are seeking an experienced Data Engineering Lead with a solid foundation in Python, MongoDB, PostgreSQL, Apache Airflow and other data management tools to join our innovative team. This role is critical for building and managing robust data pipelines that extract and process data from health system EHR systems, supporting our AI-driven healthcare applications. Key Responsibilities: AI Infused Data Pipeline Construction: Develop and optimize data pipelines for the efficient extraction, transformation, loading and management of large-scale healthcare and clinical guidelines data. Healthcare Data Handling: Manage sensitive healthcare data adhering to FHIR, HL7 standards, as well as the version and variability of Epic Clarity extracts, Quality Measures, Drug Formularies and Clinical Guidelines ensuring compliance with data regulations and client needs. Data Orchestration: Use Airflow, Temporal or other similar AI-native next generation tools to manage complex data and knowledge workflows essential for processing healthcare data at scale. This includes RAG, GraphRAG, Tool calling, MCP and other AI enabled Data Orchestration, Search, Retrieval and Aggregation. Assessments and Collaboration: Support data scientists and backend developers by integrating and maintaining data systems across the organization. As well as in assessment and validation of large-scale Gen-AI based systems using tools such as RAGAs, Opik etc. Innovative Problem-Solving: Apply technical skills to address unique challenges in the healthcare sector, contributing to solutions that enhance patient care. Minimum Qualifications Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field. Minimum of 12 years of experience in data engineering, with proficiency in Python, PostgreSQL or other variant, MongoDB and Milvus Vector DB. Experience building and managing data pipelines using healthcare data from hospital EHR systems and other clinical data sources such as HIEs and clinical standards bodies (CMS, NCQA etc). Demonstrated experience with data lakes, ensuring robust and scalable data storage solutions. Highly Preferred Qualifications Azure technology stack expertise to improve data processing and storage capabilities. Experience with clinical data quality management and validation to ensure the accuracy and reliability of data solutions. Knowledge of anomaly and outlier detection techniques in large datasets. Proficiency in querying massive datasets using database queries to drive insights and decisions. Experience with databases/OLAP systems such as Clickhouse and Snowflake as well as data platforms such as Starburst. Experience in building and maintaining RESTful Web Services to support data integration and accessibility. Experience with Large Scale Recommendation Systems and managing the lifecycle of recommended items that receive end-user clinical feedback. Experience with learning systems (Reinforcement Learning) that use feedback as an input to Machine Learning. Is familiar with LLMs, RAG and fine-tuning architectures. Detailed Skills: Must-have skills: In-depth knowledge of LLM, Prompt Engineering, Embedding Techniques In-depth knowledge of Prompt Optimization Knowledge of Various Chunking strategies Performance and Scalability of GenAI Solution Hands-on experience in Python and application to Data Engineering via Airflow Hands-on Azure OpenAI stack for GPT 4, 4o, o3/o4 models Hands-on experience with Weaviate/Milvus Vector DB Should be able to work with Onshore/Offshore team In-depth knowledge of NLP Guide the team for problem solving In-depth knowledge in Prompt Tuning and Context Management Experience in maximizing accuracy, minimizing latency, and enhancing performance of GenAI solutions Should have taken at least one GenAI implementation into Production Hands-on experience with tools and frameworks like LangChain, LlamaIndex, or similar Should have implemented at least one solution in RAG implementation Strong exposure to Healthcare domain data workflows Hands-on experience with FHIR (Fast Healthcare Interoperability Resources) and HL7 standards Experience in Healthcare data engineering including ETL, normalization, validation, and secure data exchange Ability to integrate clinical data sources (EHR/EMR systems) into AI and Analytics pipelines Knowledge of tool calling, MCP, RAG/GraphRAG and other LLM based retrieval and search. Nice to have skills: Expertise in Model Monitoring and Debugging Expertise in CI/CD of GenAI Solution Familiarity with HIPAA-compliant data handling and PHI/PII data governance in Healthcare AI/ML systems Experience in fine tuning for Data Applications, as well as applications healthcare scenarios (such as fined tuned language models for medication terminologies) Benefits Competitive salary and performance-based bonuses. Opportunities for professional development and advancement within a rapidly growing company. Collaborative and inclusive work environment. Cutting-edge technology projects with real-world impact. Company History and Leadership RhythmX AI is a new company chaired and invested by Romesh Wadhwani (Chairman of SymphonyAI Group) and is fully owned by the SAIGroup. SAIGroup is one of the largest, fastest-growing players in enterprise AI. Our portfolio includes SymphonyAI and ConcertAI, both founded in 2017 and both are working with several hundred of the Fortune 1000, generating hundreds of millions in revenues, and employing 4000 employees globally. ConcertAI is the leader in real-world data and enterprise AI for life sciences and healthcare and recently at $1.9 billion valuation. SymphonyAI is the leader in enterprise AI for key vertical sectors, including retail, financial services, manufacturing, and IT operations, and has hundreds of millions of dollars in revenue. Now, for his third pillar under the SAIGroup, Dr. Romesh Wadhwani is starting a company called RhythmX AI and fully anticipates it, too, will become a unicorn- joining his first two successful companies SymphonyAI and ConcertAI. Deepthi Bathina , formerly a Chief Product Officer at Humana, and Senior Executive & GM of Operations at Nuance Communications and COO of Wolter Kluwers HealthTech division, has joined as CEO, bringing years of healthcare, AI and business experience to the team. Prior to founding SAIGroup, Dr. Wadhwani was the founder and CEO of Symphony Technology Group, a strategic private equity firm that builds great software and technology-enabled services companies. Dr. Wadhwani founded STG in 2002, growing it from startup to $ 2.5 billion in combined revenue and 15,000 employees. Dr. Wadhwani was the founder and CEO of Aspect Development Inc., a B2B software firm. Aspect Development was acquired in 2000 for $9.3 billion, the largest software acquisition at the time. TIME Magazine recently recognized our Chairman Romesh as one of the Top 100 Most Influential AI leaders in the world. Under the highly engaged leadership of successful Silicon Valley serial entrepreneur Dr. Romesh Wadhwani who has committed $1B of his personal capital to SAIGroup’s growth and the success of our clients, we are launching RhythmX AI to market with a clear path to a minimum $1+ billion value outcome. We need passionate and purpose-driven leaders who want to change the trajectory of healthcare for generations to come to be part of our leadership team. Some articles/links about our firm - Symphony AI Group (SAIGroup): SAIGroup commits to $1 Billion capital, an advanced AI platform that currently processes 300M+ patients, and 4000+ global employee base to solve enterprise AI and high priority healthcare problems. Our website - SAIGroup - Growing companies with advanced AI RhythmX AI launch and funding: New Generative AI-Native Health Company RhythmX AI Announces Precision Care Platform for Doctors to Deliver Hyper-Personalized Care to the Right Patient at the Right Time (prnewswire.com) Bio of our chairman Dr. Romesh Wadhwani: Team - SAIGroup (Informal at Romesh Wadhwani - Wikipedia ) TIME Magazine recently recognized our Chairman Romesh as one of the Top 100 AI leaders in the world - Romesh and Sunil Wadhwani: The 100 Most Influential People in AI 2023 | TIME $1 Billion investment from our chairman into AI: https://www.cnbc.com/2023/12/08/75-year-old-tech-mogul-betting-1-billion-of-his-fortune-on-ai-future.html