AI Data Pipeline Engineer

A Leading Banking & Financial Services Group is seeking a delivery-focused AI Data Pipeline Engineer to modernize enterprise data pipelines within its Enterprise Data & Data Capabilities division. In this hands-on engineering position, you will design, develop, test, and optimize AI-assisted data ingestion and transformation pipelines on Snowflake. Furthermore, you will apply AI-assisted development tools to accelerate Python-based data preparation, pipeline orchestration, and model-ready dataset generation. Consequently, this role is essential for delivering trusted data for analytics and Generative AI workflows.

The AI Data Pipeline Engineer must combine advanced Python and SQL programming capabilities with extensive experience in cloud data warehousing and dimensional modeling. Collaborating closely with architects, product owners, and data scientists, you will implement automated data quality checks, reconciliation routines, and pipeline monitoring. Therefore, the organization is looking for an engineer who embeds data privacy, governance, and auditability standards seamlessly into daily delivery. If you want to engineer next-generation cloud pipelines, this role offers an ideal track.

Key Responsibilities

• Design, build, optimize, and monitor scalable ETL/ELT data ingestion pipelines delivering structured and unstructured data into Snowflake.
• Apply AI-assisted development approaches to accelerate Python code generation, transformation logic, and dataset preparation.
• Cleanse, validate, enrich, and transform high-volume data streams to create model-ready datasets for Generative AI and analytics.
• Develop advanced SQL queries, automated data profiling routines, exception handling rules, and reconciliation controls.
• Maintain comprehensive technical documentation covering data mappings, transformation rules, lineage, and operational runbooks.
• Embed data governance, metadata management, quality assurance, privacy, and auditability standards into pipeline architectures.
• Manage batch workflow orchestration, job scheduling, and production triage using tools like Airflow or Autosys.

Requirements and Qualifications

• Bachelor’s or Master’s degree in Computer Science, Data Analytics, Engineering, Information Systems, or a related quantitative field.
• 8-10 Years of overall IT experience, with 5+ recent years specializing in Data Engineering, Data Warehousing, or Analytics Delivery.
• Deep hands-on technical proficiency in SQL, Python scripting, relational database architectures, and dimensional modeling.
• Proven track record implementing ETL/ELT pipelines in Snowflake and AWS cloud environments using PySpark and APIs.
• Hands-on experience with AI technologies (LLMs, prompt engineering, vector databases, RAG concepts) and workflow tools (Airflow).