Principal Artificial Intelligence LLM Engineer

Prestigious Financial Institution is currently seeking a Principal Artificial Intelligence Engineer with strong LLM experience Candidate will be a technical leader who will lead through expertise and ensure the technical success of the team's work. The position requires a seasoned engineer with strong software engineering experience, technical leadership skills, and foundational knowledge of AI / LLMs with a keen interest in applying these technologies to financial infrastructure. You will play a key role in executing the AI roadmap while mentoring a team of talented junior engineers in driving organizational transformation. As a team leaning into emerging technology, we also support continuous growth and learning by providing time for experimentation with AI tools and technologies.

Responsibilities :

Partner with the Executive Director, AI Engineering to help define and execute on the AI technical roadmap

Lead and partner on the architecture of scalable systems incorporating LLMs and AI into organizational processes and infrastructure

Design and build production applications in support of organizational use cases

Provide technical mentorship to junior engineers on engineering best practices and system design (no direct reports)

Conduct code / architectural reviews ensuring production readiness, safety, and adherence to high security and compliance standards

Implement testing, evaluation, and monitoring frameworks for AI systems including hallucination detection and bias assessment

Establish safety guardrails and responsible AI practices for LLM applications in a regulated environment

Connect AI agents to organizational systems and workflows

Evaluate emerging technologies and recommend adoption strategies

Foster continuous learning culture in a rapidly evolving AI landscape

Qualifications :

Bachelor's or Master's in Computer Science or related technical field

10+ years software engineering / systems architecture experience with strong technical leadership

5+ years as senior technical contributor on complex production systems

Expert in Python and proficient in SQL

System design expertise : monolithic and microservice architectures, distributed systems, event-driven architectures, APIs

Hands-on experience in data (e.g., data engineering, data pipelines, data transformation)

Foundational experience with LLMs and familiarity with multiple frontier lab models (e.g., Gemini, Claude, GPT)

Familiarity of risk vectors in AI applications including hallucinations, bias, prompt injection, data privacy

Experience mentoring and reviewing the work of junior engineers

Exceptional communication skills explaining complex technical concepts

Strong interest in AI / LLM space with demonstrated commitment to continuous learning

Preferred Experience :

Production experience with AI applications or LLM-powered systems

Experience with RAG architectures and context engineering

Familiarity with diverse LLM provider APIs and experience connecting agents to systems

Experience tinkering with frontier lab tools and products

Cloud platforms / technologies : AWS, Docker, Kubernetes, CI / CD

Infrastructure as code : Terraform

Experience in financial market infrastructure or derivatives industry

Background in regulated environments