Product Manager, AI Agents & MCP Tools

What You'll Do:

• Own the end-to-end lifecycle of AI agents — design, build, evaluation, customer validation, and deployment.
• Build agents that orchestrate LLMs and tools against real intelligence use cases, selecting the right model for each task based on capability, latency, and cost tradeoffs.
• Own and refine the MCP tool surface — writing clear, effective tool descriptions, identifying gaps in coverage, and improving how tools expose Recorded Future intelligence to agents and customers.
• Analyze MCP tool usage patterns to understand what customers and agents actually invoke, where tools fail or underperform, and where new tools are needed.
• Write, maintain, and expand evaluation suites to measure agent and tool quality, catch regressions, and guide iteration; update agents and tools as models, data, and customer needs evolve.
• Test agents and tools directly with customers, gathering feedback and confirming that outputs meet their workflows and expectations before and after launch.
• Work hands-on in the codebase (GitHub) alongside engineers — reviewing changes, prototyping, and contributing to agent logic and tool definitions where appropriate.
• Maintain a working understanding of the LLM landscape, tracking the strengths, weaknesses, and cost profiles of available models to make informed build decisions.
• Define and track agent and tool performance metrics — accuracy, task completion, tool invocation success, latency, cost per task, and customer satisfaction.
• Prioritize the agent and tool roadmap, focusing effort on the capabilities that deliver the most customer value.
• Partner with intelligence, engineering, and design teams to ensure agents and tools integrate cleanly into the broader platform and customer experience.
• Establish repeatable practices for building, evaluating, and shipping agents and tools reliably and safely.

What You'll Bring:

• Agent Builder with hands-on experience building LLM-powered agents or workflows — or the technical aptitude to ramp up quickly.
• Model-Literate, with working knowledge of major LLMs and a practical sense of their pros, cons, and cost tradeoffs.
• Tool-Design Sense, able to write clear tool descriptions, reason about how agents select and invoke tools, and spot coverage gaps. Familiarity with MCP (Model Context Protocol) or similar tool-integration frameworks is a plus.
• Technically Comfortable, able to work in GitHub, read and reason about code, and engage credibly with engineers on agent and tool design and evaluation.
• Evaluation-Minded, understanding how to define quality, write evals, and use them to drive iteration rather than relying on vibes.
• Data-Informed, comfortable analyzing usage patterns to guide decisions about what to refine, build, or retire.
• Customer-Oriented, skilled at working directly with users to validate that what's built actually solves their problem.
• Strong PM or Business Analyst Skills, able to prioritize, define requirements, and connect technical work to business outcomes.
• Pragmatic and Outcome-Driven, comfortable shipping, measuring, and improving in fast iteration cycles.
• Cybersecurity experience is a plus but not required — provided you can ramp up quickly on the domain.
• 3–5 years in product management, technical program management, or a hands-on technical role building AI/LLM-powered products, agents, or tool integrations