Data Engineering Project

Fundamentals of data engineering. The role of data engineer. Data engineering lifecycle. Data generation, ingestion, transformation, storage, serving Artificial Intelligence and Machine Leaning (AI/ML), visualization and business analytics. Automation and task orchestration. Data governance, quality and and the role of data provenance. Data systems. E(xtract)T(ransform)L(oad) data. Build, test, maintain data pipelines. Data lakes. Real-world problems with emphasis on end-to-end engineering solutions. Cloud services and open-source data engines/platforms. Engineering portfolio. Master of Information Technology (MIT) students only. Pre: Graduate standing.