Project Overview
Type: Internal GenAI Insight tool
Team: Fay Cai, Iris Bierlein, Engineer team.
Role: AI Product Strategist
Tools: Google Sheets + Apps Script + OpenAI API
Timeline: 02/2025 — 04/2025
A lightweight internal GenAI insight system for NYU Libraries UX team that transformed unstructured user feedback into semantically and behaviorally organized intelligence. It reduced manual tagging time by 90%, uncovered trends across cohorts, and enabled two major UX interventions—all without introducing a separate product UI.
Impact & Outcome
• 90%+ reduction in manual tagging
• 2 new UX interventions in progress
• Shifted UX team’s workflow from reactive to proactive
• Recurring themes surfaced across time
• Callout quote by UX Director Lisa Gayhart:
“This system made it so persuasive and powerful for us to advocate for UX changes to the leadership team.”
Key Takeaways
• AI is most powerful when it augments existing habits, not replaces them.
• Low-friction adoption mattered more than UI overhaul.
• UX strategy isn’t just about screens, it’s also about mental models and workflows.
• Lightweight tools can create deep change when aligned with user behavior.
• Insight design goes beyond visualizing data—it interprets it, prioritizes it, and transforms it into clear, actionable understanding—turns feedback from noise into navigation—helping UX teams prioritize interventions and advocate for systemic change with ease.
What's the Problem?
The UX team collected large volumes of user feedback from different cohort each quater—but relied on:
• Manual categorization (inconsistent)
• Basic sentiment scoring (often inaccurate or vague)
• No system for recurrence detection (hard to spot chronic vs. emerging issues)
Design Goal
To guide a large language model (LLM) to infer intent, emotion, and behavioral patter from unstructured user feedback using principles from cognitive and behavioral science, and to design a lightweight, maintainable internal tooling system that extracts:
• Recurring unmet needs
• Suggestions and support patterns
• Latent behavioral insights
—all without introducing a new product interface or dashboard, but instead leveraging
existing workflows (Google Sheets + Apps Script) to empower the UX team with actionable,
psychologically-aware intelligence.
User
Story
"As a UX manager, I need a simple, private, low-maintenance system that helps me understand what users are consistently struggling with—without reading every single feedback—so I can prioritize fixes and advocate for changes more efficient and consistently within the organization."
Solution: GenAI-Powered Insight System (Inside Google Sheets)
Input: A raw feedback sheet with timestamped, anonymous user entries across multiple cohorts.
Automation Pipeline:
1. Trigger & Connection: Google Apps Script monitors new entries and connects to the OpenAI API through a custom add-on menu—enabling non-technical users to initiate analysis on demand.
2. LLM-Powered Processing: Custom-engineered prompts guide the LLM (GPT-4) to return structured outputs
• Semantic Category (e.g., Complaint, Suggestion, Positive Feedback)
• Precise Topic (e.g., “bathroom,” “wifi,” “staff,” etc.)
• Behavioral Intent & Psychological Cue (e.g., loss of control, unmet need, motivation blocker)
• Insight Summary for immediate pattern recognition
3. Data Parsing & Output:
Responses are parsed and written directly back into structured columns—transforming the raw Sheet into a live, intelligent insight dashboard.
Output & Interface: The Sheet becomes a lightweight yet powerful UX intelligence system with:
• Filterable, tagged feedback for fast qualitative review
• Topic-level clustering to map recurring themes
• Cohort-based analysis (e.g., quarter-over-quarter segmentation)
• Dynamic heatmaps to visualize frequency and persistence
• Trend pivots to detect chronic vs. emerging pain points
Why this Approach Works:
• Zero Adoption Friction: No new platform—integrates directly into an existing toolchain (Google Sheets)
• AI-Native Intelligence: Every feedback entry is semantically and behaviorally analyzed for deeper insight
• Action-Oriented Design: Insight is immediately usable for UX prioritization and stakeholder reporting
• Cost-Conscious Optimization: Batching logic and token-aware prompt design reduce OpenAI API costs while maintaining output quality
Key Design Principles
• Behavior-First Prompting: Aligns LLM reasoning with human affect models
• Workflow-Native Design: No new tools, just smart layers on top of existing ones
• Time-Sensitive Insight: Uses cohort mapping to distinguish chronic vs. transient issues
• Low-Friction Collaboration: Designed for non-technical UX stakeholders
Add-on Menu through Google App Script Allow non-technical team member easy to operate the system
Stack Overview
Component
Role
Google Sheets
Central UI and database for feedback
Apps Script
Connects Sheets to GenAI via API, handles automation
OpenAI API (GPT-4)
Processes each comment and returns: semantic category, precise topic, behavioral intent, etc.
Custom Prompt Engineering
Structured prompts instruct GPT to output structured insights (not open-ended summaries)
Advanced Prompt Engineering:
Cognitive Design Layer— Prompt as Behavioral Scaffold
To move beyond sentiment and extract actionable insight, I embedded behavioral psychology principles directly into the prompt. This allowed LLM to:
• Detect user frustration signals
• Identify latent needs (not just surface complaints)
• Categorize intent through a behavioral lens
Behavioral Dimensions Used in the Prompt:
Behavioral_Intent: Based on intent inference from behavioral economics and affective computing.
Psychological_Cue: Inspired by self-determination theory, BJ Fogg’s behavior model, and affective UX research.
Insight_Summary: Designed to reflect latent needs, not just surface issues (e.g., “seeking control over environment”).
Example Insights Generated by the AI-powered System
Information Architecture & System Flow (Automated)
User Journey: Before vs. After
Future Vision for the System
While the current system is built inside Google Sheets for speed and adoption, its architecture and insight model can be extended to other platforms and scaled as a more robust internal AI product.
Expansion Opportunities:
• Jira/Monday Integration
Auto-create UX tickets from GenAI-tagged insights, assign priority based on frequency or severity, and track resolution loops.
• Notion or Confluence Integration
Push insight summaries into team knowledge bases for context-aware documentation and product rituals.
• Multi-model LLM Layer
Use additional models (Claude, Mistral, etc.) for cross-validation, bias detection, or multilingual feedback support.
• Insight Feedback Loop
Allow UX managers to give feedback on insight quality, retrain prompt design dynamically, and improve accuracy over time.