Course Overview
In this course, you will use your knowledge of developing agents using the Agent Development Kit to operationalize agents using Agent Operations (AgentOps) on Google Cloud. After reviewing the challenges of managing production agents and deployment targets, you will build CI/CD pipelines for agents and leverage a governed artifact management ecosystem. You will implement evaluation systems using the GenAI Evaluation Service and ADK and apply observability solutions for debugging with Cloud Logging and Cloud Trace. You will integrate security guardrails against agent-specific threats using Model Armor and Sensitive Data Protection. Finally, you will apply FinOps strategies to understand and manage agent costs.
Who should attend
Application Developers, DevOps Engineers, ML Engineers and anyone deploying agentic applications on Google Cloud
Prerequisites
- Completion of Deploy Multi-Agent Systems with Agent Development Kit and Agent Engine (DMASADKAE) or equivalent knowledge
- Python
- Prompt engineering
- Agent Development Kit
Course Objectives
- Optimize efficiency and consistency of agent deployments with CI/CD
- Ensure agent quality by implementing and operating evaluation systems
- Leverage observability solutions for debugging and continuous improvement
- Establish guardrails against agent-specific threats
- Build and use a governed artifact management ecosystem
- Apply FinOps strategies to understand and manage agent costs
Outline: Agent Operations on Google Cloud (AOPGC)
Module 1 - Introduction to AgentOps on Google Cloud
Topics:
- Challenges of managing production agents
- Core principles of AgentOps
- AgentOps on Google Cloud
Objectives:
- Navigate the challenges of managing production agents
- Define the core principles of AgentOps
- Architect agent operations on Google Cloud
Module 2 - CI/CD for Agent Deployments
Topics:
- CI/CD review
- Agentic deployment targets
- CI/CD tooling and patterns on Google Cloud
- Cloud Build automation
Objectives:
- Leverage CI/CD tooling and patterns for agentic solutions on Google Cloud
- Select the deployment target for agents on Google Cloud
- Build a complete CI/CD pipeline for an agent
Activities:
- Lab: CI/CD for Agents on Google Cloud
Module 3 - Observability for Debugging and Improvement
Topics:
- Observability review
- Logging with agent callback logging
- Logging and tracing with OpenTelemetry
Objectives:
- Identify challenges addressed by observability
- Instrument an ADK agent with structured logs
- Enable OpenTelemetry tracing on Agent Engine and Cloud Run
- Leverage BigQuery and Looker Studio for visualization
Activities:
- Lab: Instrument and Debug Agents with Cloud Logging, and Cloud Trace
Module 4 - Agent Evaluation and Quality Assurance
Topics:
- Testing generative AI model responses
- Evaluating model responses
Objectives:
- Perform validation on model responses
- Evaluate agent behavior, tool usage, and trajectory correctness
- Create and manage evalsets using ADK Web UI
- Evaluate evalsets with ADK UI, CLI, or code
- Use the Vertex AI Generative AI Evaluation Service
Activities:
- Lab: Evaluating Agents with ADK
Module 5 - Security and Governance
Topics:
- Model and context security
- Agent access
Objectives:
- Secure model inputs and outputs with Model Armor
- Protect sensitive data using Sensitive Data Protection with Model Armor
- Secure the connection between a user and an agent
Activities:
- Lab: Enhancing AI Security with Model Armor and Sensitive Data Protection
Module 6 - Applying FinOps to Agent Costs
Topics:
- Primary cost drivers for AI Agents
- Cost-efficienct agentic systems
- FinOps on Google Cloud
Objectives:
- Identify the primary cost drivers of AI agents
- Reduce token and model costs
- Architect cost-efficient agent systems
- Implement a measurable AI FinOps loop on Google Cloud