Overview
We are looking for a sharp, curious, and execution-focused AI Product Manager to join the Ryt AI team. This team powers intelligent banking experiences through conversational agents and AI features. You will begin by owning the chatbot's payment capabilities and expand into areas like transaction insights, deposit automation, fraud detection, and internal operational tools. This role involves managing the unpredictable nature of AI outputs, designing evaluation frameworks, maintaining high performance standards, and collaborating with multiple stakeholders to ship safe and valuable AI products. You will also work closely with other product teams to uncover and develop new use cases where AI can improve customer experience or operational efficiency.
Key Responsibilities
Responsibilities
- AI Agent Management and Output Quality
Own the performance and output quality of Ryt AI, including intent accuracy, hallucination risk, completion rate, and user satisfaction.
Define clear benchmarks to measure agent success across various scenarios, such as task execution, fallback response, and coverage.Develop evaluation plans and collaborate with AI engineers and compliance teams to ensure output is aligned with business goals, regulatory expectations, and customer needs.Implement structured review cycles and test plans to address the non-deterministic nature of LLM-based agents.Performance Monitoring and Feedback LoopBuild dashboards and monitoring systems that track agent behavior, user outcomes, and interaction friction.
Analyze conversation data and identify areas for improvement through qualitative review and quantitative trends.Create automated feedback loops for improving prompts, adjusting guardrails, and surfacing failure patterns for retraining.Feature Development and Use Case ExpansionDefine user stories, edge cases, and requirements for AI-related features.
Support end-to-end development including prioritization, execution, rollout planning, and quality assurance.Work closely with other product pods such as Payments, Deposits, Cards, and Operations to identify surfaces where AI can create new value or remove friction.Coordinate with design and engineering to build human-centered AI flows that are safe and reliable.Risk, Governance, and Compliance CollaborationPartner with internal risk, compliance, governance, and legal teams to ensure AI deployments meet internal standards and external regulations.
Support documentation, explainability, and approval processes for AI features.Identify areas where additional controls, fallback mechanisms, or transparency are needed to protect users and the bank.Continuous Learning and ExperimentationStay updated on advances in LLM architecture, evaluation techniques, and AI toolchains such as retrieval augmentation and agent orchestration.
Run experiments to validate improvements in response quality, engagement, or safety.Contribute to prompt tuning, error categorization, and improvement playbooks that support long-term agent growth.Required Qualifications
Experience
Two to four years of experience in product management or a similar role within technology, fintech, or AI domains.Exposure to AI-driven products such as chatbots, recommendation systems, or retrieval-based assistants.Experience managing cross-functional initiatives with product, engineering, and data teams.Skills and Mindset
Familiarity with large language models, vector databases, prompt design, and retrieval techniques.Strong analytical mindset with experience using tools like Amplitude, Looker, or Metabase.Comfortable reviewing user interaction logs and surfacing actionable product improvements.Detail-oriented, curious, and motivated to solve ambiguous problems using data and experimentation.Strong communication and collaboration skills with the ability to align multiple teams.Bonus Qualifications
Familiarity with risk and compliance considerations in regulated environments such as banking or insurance.Understanding of AI evaluation methods for output quality, including qualitative rubrics and automated scoring.Seniority level
AssociateEmployment type
Full-timeJob function
Product ManagementIndustries
Banking#J-18808-Ljbffr
