1 day ago
Sr. Product Mgr. Tech , Catalog Experimentation and Impact Measurement
US, WA, Seattle
Do you want to shape Amazon's experimentation capabilities in the era of Generative AI? As we navigate the rapidly changing landscape of e-commerce, particularly with the advent of Generative AI, your role will be crucial in shaping how Amazon teams, sellers, and vendors optimize catalog listings for superior customer experience. Join us on the mission to create best product listings to facilitate well-informed and fast buying decisions for Amazon's customers. We are building feedback loops for Large Language Models and Amazon teams enriching product data through experimentation at Amazon scale.
In this role, you will lead the conceptualization, delivery, and worldwide adoption of Catalog Experimentation Platform (CEP). Through experiments, we bring voice of the customer to our business stakeholders. You collaborate with the economists to help users design the experiments correctly and also guide the users on interpreting outcomes, which are generated through a statistical analysis. You advocate for decisions in the best interest of our customers even when the outcomes might be unpleasant for the experimenters. Impact measurement techniques often rely on experimentation.
Measurement of the impact of each initiative that intends to improve product data quality is crucial for prioritization decisions and to fine tune each initiative's strategy. You collaborate with the economists to identify and apply the best possible causal inference methodology for each initiative. You have an opportunity to extend the scope of CEP and make it a go-to destination in Amazon for impact measurements of Catalog improvement programs.
Finally, to make experimentation and impact measurement work at Amazon scale without compromising on the reliability, you collaborate with your engineering counterparts to build systems for automation, self service mechanisms, and guardrails.
Key job responsibilities
1. Work backwards from customer experience to define strategy and build a product roadmap for Catalog Experimentation Platform (CEP). CEP is used to run A/B experiments on Catalog listings.
2. Launch new products/features by partnering with CEP engineering and science teams.
3. Drive adoption by partnering with teams building large language models for the Catalog and business stakeholders across different product categories.
4. Partner with economists and other PMs in the team to develop frameworks and shape the thinking of business stakeholders and leadership on impact measurement methodologies.
4. Represent CEP in Amazon wide forums on experimentation and impact measurements.
A day in the life
Learn about the strengths and challenges of experimentation platforms used industry wide. Develop product strategy (positioning, GTM, competitive advantage) in collaboration with your engineering and science counterparts.
Track key product metrics, communicate wins/misses/learnings to the key stakeholders. Conduct office hours to guide the experimentation users.
Communicate strategy and key updates to org leadership through monthly flashes and business review forums.
About the team
Amazon’s customers rely on the completeness, consistency and correctness of Amazon's product data to make well-informed purchase decisions. Improving the quality of product data is a continuous process. Amazon Selection Catalog Systems (ASCS) is building Catalog Experimentation and Impact Measurement Platform (CEM). CEM enables teams across Amazon to make data driven decisions on what product data changes to simplify and improve the Customers’ experience. It empowers its users to remove personal bias and let data guide decision-making by testing their hypothesis against a representative sample of a broader population. Last but not the least, CEM helps LLMs to draw causal inferences between catalog quality improvement actions and customer experience metrics.
In this role, you will lead the conceptualization, delivery, and worldwide adoption of Catalog Experimentation Platform (CEP). Through experiments, we bring voice of the customer to our business stakeholders. You collaborate with the economists to help users design the experiments correctly and also guide the users on interpreting outcomes, which are generated through a statistical analysis. You advocate for decisions in the best interest of our customers even when the outcomes might be unpleasant for the experimenters. Impact measurement techniques often rely on experimentation.
Measurement of the impact of each initiative that intends to improve product data quality is crucial for prioritization decisions and to fine tune each initiative's strategy. You collaborate with the economists to identify and apply the best possible causal inference methodology for each initiative. You have an opportunity to extend the scope of CEP and make it a go-to destination in Amazon for impact measurements of Catalog improvement programs.
Finally, to make experimentation and impact measurement work at Amazon scale without compromising on the reliability, you collaborate with your engineering counterparts to build systems for automation, self service mechanisms, and guardrails.
Key job responsibilities
1. Work backwards from customer experience to define strategy and build a product roadmap for Catalog Experimentation Platform (CEP). CEP is used to run A/B experiments on Catalog listings.
2. Launch new products/features by partnering with CEP engineering and science teams.
3. Drive adoption by partnering with teams building large language models for the Catalog and business stakeholders across different product categories.
4. Partner with economists and other PMs in the team to develop frameworks and shape the thinking of business stakeholders and leadership on impact measurement methodologies.
4. Represent CEP in Amazon wide forums on experimentation and impact measurements.
A day in the life
Learn about the strengths and challenges of experimentation platforms used industry wide. Develop product strategy (positioning, GTM, competitive advantage) in collaboration with your engineering and science counterparts.
Track key product metrics, communicate wins/misses/learnings to the key stakeholders. Conduct office hours to guide the experimentation users.
Communicate strategy and key updates to org leadership through monthly flashes and business review forums.
About the team
Amazon’s customers rely on the completeness, consistency and correctness of Amazon's product data to make well-informed purchase decisions. Improving the quality of product data is a continuous process. Amazon Selection Catalog Systems (ASCS) is building Catalog Experimentation and Impact Measurement Platform (CEM). CEM enables teams across Amazon to make data driven decisions on what product data changes to simplify and improve the Customers’ experience. It empowers its users to remove personal bias and let data guide decision-making by testing their hypothesis against a representative sample of a broader population. Last but not the least, CEM helps LLMs to draw causal inferences between catalog quality improvement actions and customer experience metrics.