# Criticism Into Creative Direction > How sentiment, LLM topic analysis, and human review turned scattered criticism into a simple “do this, not that” brief, then checked the lesson against a later campaign with 86.8% positive sentiment. - Author: Paul Yu - Role: Senior Analytics Engineer - Employer: Viral Nation - Year: 2025 to 2026 - Theme: Insight - Human article: [https://paul-yu.com/case-studies/ai-audience-insight-loop](https://paul-yu.com/case-studies/ai-audience-insight-loop) - Structured data: [https://paul-yu.com/agent/case-studies/ai-audience-insight-loop.json](https://paul-yu.com/agent/case-studies/ai-audience-insight-loop.json) ## Evidence | Value | Measure | Evidence level | | --- | --- | --- | | 86.8% | positive sentiment in the later campaign | Measured | | 1,664 | later comments coded | Measured | | 84.0% vs 62.6% | human-story vs problem-first sentiment | Measured | | 4.52% vs 1.36% | human-story vs problem-first engagement | Measured | ## Situation Select creator posts were drawing heavy negative reaction. A sentiment score could show that humans were upset, but the client still did not know why or what to change. ## Task Find the real causes inside a large, noisy comment stream and turn them into choices a creative team could use. ## Action I led a comment intelligence workflow that combined sentiment classification, LLM-assisted topic discovery, a frozen theme taxonomy, and human review. It separated product concerns from ad fatigue, creator criticism, harassment, and unrelated negativity, then translated the useful themes into a plain creative playbook. ## Result The client received a clear “do this, not that” direction instead of a negative score. A later related campaign reached 86.8% positive sentiment across 1,664 captured comments. Human-story openings reached 4.52% engagement versus 1.36% for creation-problem openings. ## System flow Audience comments → Sentiment + theme analysis → Creative playbook → Later campaign check ## Technical notes ### Sentiment and stance Sentiment says how a comment feels. Stance says what position it takes and what the feeling is aimed at. A hostile reply may defend the campaign, while a polite comment may reject the product. ### Frozen topic taxonomy An LLM discovered repeated ideas, but it could not invent new labels during final coding. Every useful comment had to map to a reviewed list, which kept totals stable and auditable. ### From comment to brief Each repeated concern became a choice the creative team could make: lead with the human need, keep AI in a supporting role, show user control, and make sponsorship feel natural. ### Concept comparison I grouped later posts by how they opened, then compared engagement and captured-comment sentiment. The comparison is directional because the concepts were not assigned at random. ## Ownership I built the sentiment analysis, read the audience themes, wrote the creative learning, and measured the later campaign. ## Limitations These were two related campaigns, not a controlled experiment. The later result fits the lesson, but it does not prove that the brief caused the change. Facebook comment coverage was incomplete, and one creator community supplied 43.7% of the comments. A positive comment was not always direct support for the company or AI. ## Takeaway The useful output was not the sentiment score. It was a plain creative rule grounded in audience language, followed by an honest check of what happened next. ## Tools Python, XLM-RoBERTa, LLM topic analysis, Human review, Campaign measurement