Lead AI Engineer (Agentic Systems)

ABOUT THE ROLE:

Grade Level (for internal use):

11

Lead AI Engineer (Agentic Systems)

Role Summary

As the Lead AI Engineer (Agentic Systems), you will help architect and build the organization’s next generation of autonomous AI workflows. This is a multidisciplinary technical role operating at the intersection of Software Engineering, Data Engineering, and Machine Learning Engineering. You will move beyond simple "chatbots" to design production-grade Agentic Systems: intelligent applications capable of reasoning, planning, and executing complex tasks autonomously.

Responsibilities

Agentic Systems Architecture & Core Engineering

* Architect & Build Multi-Agent Workflows: Lead the hands-on design and coding of stateful, production-grade agentic systems using Python and orchestration frameworks like LangGraph, CrewAI, or AutoGen.

* Agent-to-Agent (A2A) Communication: Design and implement robust A2A protocols enabling autonomous agents to collaborate, hand off sub-tasks, and negotiate execution paths dynamically within multi-agent environments.

* State Management & Orchestration: Engineer robust control flows for non-deterministic agents; implement complex message passing, memory persistence, and interruptible state handling to support long-running autonomous tasks.

* Tool Interface Design (MCP): Implement and standardize the Model Context Protocol (MCP) to create universal interfaces between agents, data sources, and operational tools, ensuring modularity and scalability.

* Model Integration & Optimization: Utilize proxy services (i.e. LiteLLM) to manage model routing and fallback strategies; optimize context windows and inference costs across proprietary and open-source models.

* Production Deployment: Containerize agentic workloads using Docker and orchestrate deployments on Kubernetes; leverage AWS AgentCore or similar cloud-native services for scalable infrastructure.

Data Engineering & Operational Real-Time Integration

* Build Agent Data Pipelines: Write and maintain high-throughput ingestion pipelines (using Databricks or Python-based ETL) that transform raw operational signals into structured context for agents.

* Real-Time Context Injection: Ensure agents have access to "operational real-time" data (seconds/minutes latency) by optimizing retrieval architectures and vector store performance.

* Cross-Functional Engineering: Act as the technical bridge between Data Engineering and AI teams; translate complex agent requirements into concrete data schemas and pipeline specifications, while stepping in to resolve hands-on bottlenecks in data availability.

Observability, Governance & Human-in-the-Loop

* LLMOps & Tracing: Implement comprehensive observability using tools like Langfuse to trace agent reasoning steps, monitor token usage, and debug latency issues in production.

* Safety & Control Frameworks: Design hybrid execution modes ranging from Human-in-the-Loop (HITL) for sensitive operations to fully autonomous execution; build "break-glass" mechanisms and guardrails for automated decision-making.

* Evaluation & Reliability: Establish technical standards for testing non-deterministic outputs; automate evaluation pipelines to measure agent accuracy, hallucination rates, and drift before deployment.

Technical Leadership & Strategy

* Technical Roadmap Definition: Partner with Product and Engineering leadership to scope feasibility for autonomous projects; define the "Agentic Architecture" roadmap.

* Mentorship & Standards: Define code quality standards, architectural patterns, and PR review processes for the AI engineering team; upskill team members on the latest agentic frameworks and methodologies.

* Innovation: Proactively prototype with emerging tools (e.g., new reasoning models, graph-based RAG) to solve high-value business problems, moving successful experiments into the production roadmap.

Qualifications

Required

* Experience: 7+ years of total technical experience in Software Engineering, Data Engineering, or Machine Learning.

* GenAI Specialization: 2+ years of specific experience building and deploying LLM-based applications or Agentic Systems in production.

* Database & Lakehouse Mastery: Experience architecting storage layers for AI, including Vector Databases (e.g., Pinecone, Weaviate, Qdrant), NoSQL/Relational Databases (PostgreSQL, DynamoDB), and modern Data Lakehouses (specifically Databricks or Snowflake).

* Cloud & Infrastructure: Expertise in cloud architecture and container orchestration (AWS, GCP, or Azure) using Kubernetes and Docker. You must be comfortable deploying and scaling your own applications.

* LLM Ecosystem: Familiarity with common LLM frameworks and orchestration libraries (e.g., LangGraph, LangChain, CrewAI, AutoGen). You understand the mechanics of RAG, embeddings, and context window management.

* Hybrid Engineering Skillset: A unique blend of Data Science (understanding model behavior, probability, and prompting) and Software Engineering (CI/CD, API design, asynchronous programming, and system reliability).

* Language Proficiency: Advanced proficiency in Python for systems engineering, capable of writing modular, testable, and maintainable production code.

* Education: Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related technical field.

Preferred

* Advanced Education: Master’s degree or PhD in Computer Science, Artificial Intelligence, or a related quantitative field.

* NLP Expertise: 5+ years of hands-on experience in Natural Language Processing (NLP), ranging from foundational techniques (e.g., text processing, embeddings, classification) to modern architectures.

* Graph Technologies: Experience with Knowledge Graphs (e.g., Neo4j, AWS Neptune), Graph Databases, and GraphML (Graph Machine Learning) to support complex reasoning and relationship modeling.

* Agentic Tooling: Specific experience with LangGraph, LiteLLM, Langfuse, AWS AgentCore, or implementing the Model Context Protocol (MCP).

* Advanced Architectures: Proven track record of implementing Agent-to-Agent (A2A) communication, swarm intelligence, or multi-modal agent workflows.

* Real-Time Operations: Experience working in environments requiring operational real-time processing (e.g., FinTech, Energy, Logistics).

Why This Role Matters

You won't just be building chatbots here; you will be architecting the organization’s "central nervous system." As the Lead AI Engineer for Agentic Systems, you are bridging the gap between static data models and active decision-making. The autonomous workflows you design—capable of planning, collaborating (A2A), and executing tasks—will fundamentally change how we operate, moving us from human-dependent processes to self-healing, intelligent systems. This is a rare opportunity to define the standards for Agentic AI in a production environment, working with a stack that represents the absolute cutting edge of the industry.

About S&P Global Energy
At S&P Global Energy, our comprehensive view of global energy and commodities markets enables our customers to make superior decisions and create long-term, sustainable value. Our four core capabilities are: Platts for news and pricing; CERA for research and advisory; Horizons for energy expansion and sustainability solutions; and Events for industry collaboration.

S&P Global Energy is a division of S&P Global (NYSE: SPGI). S&P Global enables businesses, governments, and individuals with trusted data, expertise, and technology to make decisions with conviction. We are Advancing Essential Intelligence through world-leading benchmarks, data, and insights that customers need in order to plan confidently, act decisively, and thrive economically in a rapidly changing global landscape. Learn more at www.spglobal.com/energy.

What’s In It For You?

Our Mission:

Advancing Essential Intelligence.

Our People:

We're more than 35,000 strong worldwide—so we're able to understand nuances while having a broad perspective

View and apply on HireChats