As data ecosystems become more diverse and time-critical, traditional monolithic ETL pipelines face challenges to meet the demands of modern data engineering workloads in terms of scalability, adaptability, and operational resilience. In this paper, we introduce an event-driven microservices approach to orchestrate and deploy AI-based ETL (ETL = Extraction, Transformation, and Loading) pipelines in a Kubernetes-managed environment that includes the following components: asynchronous orchestration using Apache Kafka, hybrid anomaly detection, adaptive schema inference, and predictive load balancing. The proposed architecture breaks the ETL processing into loosely coupled services, which can b... More >