Understanding what your users truly think about your product, brand, or service has never been more critical. With millions of conversations happening daily across social media, review platforms, and community forums, businesses that harness sentiment analysis gain a significant competitive advantage. According to Grand View Research, the sentiment analytics market is projected to reach $6.1 billion by 2030, growing at a CAGR of 14.2%.
In this comprehensive guide, we'll build production-ready sentiment analysis systems using modern NLP techniques. From transformer-based models like BERT and RoBERTa to aspect-based sentiment extraction, real-time social media monitoring, and multilingual support, you'll learn practical implementations that help web platforms understand and respond to user sentiment at scale.
Understanding Sentiment Analysis
Sentiment analysis, also known as opinion mining, uses natural language processing (NLP) to identify and extract subjective information from text. Modern approaches leverage deep learning models that understand context, sarcasm, and implicit sentiment far better than traditional rule-based systems. Stanford's Sentiment Analysis research has been instrumental in advancing this field.
Types of Sentiment Analysis
Different use cases require different levels of sentiment granularity:
// sentiment-types.ts
// 1. Document-Level Sentiment
// Overall sentiment of an entire review/document
interface DocumentSentiment {
text: string;
sentiment: 'positive' | 'negative' | 'neutral';
confidence: number;
score: number; // -1 to 1 scale
}
// 2. Sentence-Level Sentiment
// Sentiment for each sentence in a document
interface SentenceSentiment {
sentence: string;
startIndex: number;
endIndex: number;
sentiment: 'positive' | 'negative' | 'neutral';
confidence: number;
}
// 3. Aspect-Based Sentiment Analysis (ABSA)
// Sentiment toward specific aspects/features
interface AspectSentiment {
aspect: string;
category: string;
sentiment: 'positive' | 'negative' | 'neutral' | 'mixed';
confidence: number;
mentions: AspectMention[];
}
interface AspectMention {
text: string;
startIndex: number;
endIndex: number;
implicit: boolean; // Aspect not explicitly mentioned
}
// 4. Emotion Detection
// Fine-grained emotion classification
interface EmotionAnalysis {
text: string;
emotions: {
joy: number;
sadness: number;
anger: number;
fear: number;
surprise: number;
disgust: number;
trust: number;
anticipation: number;
};
dominantEmotion: string;
}
// 5. Intent Classification
// Understanding user intent alongside sentiment
interface IntentSentiment {
text: string;
sentiment: DocumentSentiment;
intent: 'complaint' | 'praise' | 'question' | 'suggestion' | 'comparison';
urgency: 'low' | 'medium' | 'high' | 'critical';
}Building Transformer-Based Sentiment Analysis
Modern sentiment analysis leverages transformer architectures that understand contextual relationships in text. BERT (Bidirectional Encoder Representations from Transformers) revolutionized NLP by enabling bidirectional context understanding. Here's a complete implementation using Hugging Face transformers:
// sentiment-analyzer.ts
import { pipeline, Pipeline } from '@xenova/transformers';
interface SentimentResult {
label: 'POSITIVE' | 'NEGATIVE' | 'NEUTRAL';
score: number;
}
interface AnalysisResult {
text: string;
sentiment: SentimentResult;
aspects: AspectSentiment[];
emotions: EmotionScores;
processedAt: Date;
}
interface EmotionScores {
joy: number;
sadness: number;
anger: number;
fear: number;
surprise: number;
}
class SentimentAnalyzer {
private sentimentPipeline: Pipeline | null = null;
private emotionPipeline: Pipeline | null = null;
private nerPipeline: Pipeline | null = null;
private isInitialized: boolean = false;
async initialize(): Promise {
console.log('Initializing sentiment analysis models...');
// Load sentiment classification model
this.sentimentPipeline = await pipeline(
'sentiment-analysis',
'Xenova/distilbert-base-uncased-finetuned-sst-2-english'
);
// Load emotion detection model
this.emotionPipeline = await pipeline(
'text-classification',
'Xenova/bert-base-uncased-go-emotions-merged'
);
// Load NER for aspect extraction
this.nerPipeline = await pipeline(
'token-classification',
'Xenova/bert-base-NER'
);
this.isInitialized = true;
console.log('Models initialized successfully');
}
async analyzeSentiment(text: string): Promise {
if (!this.sentimentPipeline) {
throw new Error('Sentiment pipeline not initialized');
}
const result = await this.sentimentPipeline(text);
return {
label: result[0].label as 'POSITIVE' | 'NEGATIVE' | 'NEUTRAL',
score: result[0].score
};
}
async analyzeEmotions(text: string): Promise {
if (!this.emotionPipeline) {
throw new Error('Emotion pipeline not initialized');
}
const results = await this.emotionPipeline(text, {
topk: 5
});
const emotions: EmotionScores = {
joy: 0,
sadness: 0,
anger: 0,
fear: 0,
surprise: 0
};
results.forEach((result: { label: string; score: number }) => {
const emotion = result.label.toLowerCase();
if (emotion in emotions) {
emotions[emotion as keyof EmotionScores] = result.score;
}
});
return emotions;
}
async extractAspects(text: string): Promise {
if (!this.nerPipeline) {
throw new Error('NER pipeline not initialized');
}
const entities = await this.nerPipeline(text);
const aspects: string[] = [];
entities.forEach((entity: { word: string; entity_group: string }) => {
if (['PRODUCT', 'ORG', 'MISC'].includes(entity.entity_group)) {
aspects.push(entity.word);
}
});
return [...new Set(aspects)]; // Remove duplicates
}
async fullAnalysis(text: string): Promise {
if (!this.isInitialized) {
await this.initialize();
}
const [sentiment, emotions, aspectTerms] = await Promise.all([
this.analyzeSentiment(text),
this.analyzeEmotions(text),
this.extractAspects(text)
]);
// Analyze sentiment for each aspect
const aspects = await this.analyzeAspectSentiments(text, aspectTerms);
return {
text,
sentiment,
aspects,
emotions,
processedAt: new Date()
};
}
private async analyzeAspectSentiments(
text: string,
aspects: string[]
): Promise {
const results: AspectSentiment[] = [];
for (const aspect of aspects) {
// Find sentences containing the aspect
const sentences = text.split(/[.!?]+/).filter(
s => s.toLowerCase().includes(aspect.toLowerCase())
);
if (sentences.length > 0) {
const sentimentScores = await Promise.all(
sentences.map(s => this.analyzeSentiment(s))
);
const avgScore = sentimentScores.reduce(
(sum, s) => sum + (s.label === 'POSITIVE' ? s.score : -s.score),
0
) / sentimentScores.length;
results.push({
aspect,
category: this.categorizeAspect(aspect),
sentiment: avgScore > 0.2 ? 'positive' :
avgScore < -0.2 ? 'negative' : 'neutral',
confidence: Math.abs(avgScore),
mentions: sentences.map(s => ({
text: s.trim(),
startIndex: text.indexOf(s),
endIndex: text.indexOf(s) + s.length,
implicit: false
}))
});
}
}
return results;
}
private categorizeAspect(aspect: string): string {
const categories: Record = {
'product_quality': ['quality', 'build', 'material', 'durability'],
'customer_service': ['service', 'support', 'staff', 'help'],
'price': ['price', 'cost', 'value', 'expensive', 'cheap'],
'delivery': ['shipping', 'delivery', 'arrival', 'packaging'],
'usability': ['easy', 'interface', 'user-friendly', 'intuitive']
};
for (const [category, keywords] of Object.entries(categories)) {
if (keywords.some(k => aspect.toLowerCase().includes(k))) {
return category;
}
}
return 'general';
}
}
// Singleton instance for reuse
let analyzerInstance: SentimentAnalyzer | null = null;
export async function getSentimentAnalyzer(): Promise {
if (!analyzerInstance) {
analyzerInstance = new SentimentAnalyzer();
await analyzerInstance.initialize();
}
return analyzerInstance;
}
// React hook for sentiment analysis
export function useSentimentAnalysis() {
const [analyzer, setAnalyzer] = useState(null);
const [isLoading, setIsLoading] = useState(true);
useEffect(() => {
getSentimentAnalyzer()
.then(setAnalyzer)
.finally(() => setIsLoading(false));
}, []);
const analyze = useCallback(async (text: string) => {
if (!analyzer) return null;
return analyzer.fullAnalysis(text);
}, [analyzer]);
return { analyze, isLoading };
} Aspect-Based Sentiment Analysis (ABSA)
Aspect-based sentiment analysis provides granular insights by identifying specific features or aspects mentioned in text and determining sentiment toward each. This is essential for understanding nuanced feedback like "The food was amazing but the service was slow." Here's a comprehensive ABSA implementation:
// aspect-sentiment.ts
interface AspectExtractionResult {
aspects: ExtractedAspect[];
sentenceAspects: SentenceAspectMap[];
}
interface ExtractedAspect {
term: string;
category: AspectCategory;
frequency: number;
averageSentiment: number;
sentimentDistribution: {
positive: number;
negative: number;
neutral: number;
};
}
type AspectCategory =
| 'product_feature'
| 'service_quality'
| 'price_value'
| 'user_experience'
| 'performance'
| 'design'
| 'support'
| 'other';
interface SentenceAspectMap {
sentence: string;
aspects: {
term: string;
sentiment: number;
opinionWords: string[];
}[];
}
class AspectBasedSentimentAnalyzer {
private aspectPatterns: Map;
private opinionLexicon: Map;
private negationWords: Set;
private intensifiers: Map;
constructor() {
this.aspectPatterns = this.loadAspectPatterns();
this.opinionLexicon = this.loadOpinionLexicon();
this.negationWords = new Set([
'not', 'no', 'never', 'neither', 'nobody', 'nothing',
"don't", "doesn't", "didn't", "won't", "wouldn't",
"can't", "couldn't", "shouldn't", "isn't", "aren't"
]);
this.intensifiers = new Map([
['very', 1.5], ['extremely', 2.0], ['incredibly', 2.0],
['really', 1.3], ['absolutely', 2.0], ['totally', 1.8],
['slightly', 0.5], ['somewhat', 0.7], ['fairly', 0.8]
]);
}
private loadAspectPatterns(): Map {
return new Map([
['product_feature', [
/\b(feature|function|capability|option|setting)\b/gi,
/\b(screen|display|camera|battery|storage)\b/gi
]],
['service_quality', [
/\b(service|support|staff|team|representative)\b/gi,
/\b(response|assistance|help|communication)\b/gi
]],
['price_value', [
/\b(price|cost|value|money|expensive|cheap|affordable)\b/gi,
/\b(worth|overpriced|budget|deal)\b/gi
]],
['user_experience', [
/\b(experience|interface|navigation|usability)\b/gi,
/\b(easy|intuitive|confusing|seamless)\b/gi
]],
['performance', [
/\b(performance|speed|fast|slow|lag|responsive)\b/gi,
/\b(efficient|reliable|stable|crash)\b/gi
]],
['design', [
/\b(design|look|appearance|style|aesthetic)\b/gi,
/\b(beautiful|ugly|modern|sleek|clean)\b/gi
]],
['support', [
/\b(support|documentation|tutorial|guide)\b/gi,
/\b(helpful|confusing|clear|comprehensive)\b/gi
]]
]);
}
private loadOpinionLexicon(): Map {
// Simplified opinion lexicon (in production, use AFINN or similar)
return new Map([
// Positive words
['amazing', 0.9], ['excellent', 0.95], ['great', 0.8],
['good', 0.6], ['nice', 0.5], ['love', 0.9],
['fantastic', 0.9], ['wonderful', 0.85], ['perfect', 1.0],
['best', 0.9], ['awesome', 0.85], ['outstanding', 0.9],
['impressive', 0.8], ['satisfied', 0.7], ['happy', 0.75],
['reliable', 0.6], ['efficient', 0.65], ['fast', 0.6],
['easy', 0.5], ['intuitive', 0.7], ['seamless', 0.75],
// Negative words
['terrible', -0.95], ['horrible', -0.9], ['awful', -0.85],
['bad', -0.6], ['poor', -0.7], ['hate', -0.9],
['disappointing', -0.7], ['frustrating', -0.75], ['annoying', -0.6],
['worst', -0.95], ['useless', -0.8], ['broken', -0.85],
['slow', -0.5], ['confusing', -0.6], ['difficult', -0.5],
['expensive', -0.4], ['overpriced', -0.6], ['buggy', -0.7],
['crash', -0.8], ['lag', -0.6], ['unreliable', -0.7]
]);
}
extractAspects(text: string): ExtractedAspect[] {
const aspectMentions: Map = new Map();
const sentences = this.splitIntoSentences(text);
for (const sentence of sentences) {
for (const [category, patterns] of this.aspectPatterns) {
for (const pattern of patterns) {
const matches = sentence.match(pattern);
if (matches) {
for (const match of matches) {
const normalizedTerm = match.toLowerCase();
const sentiment = this.calculateSentenceSentiment(
sentence, normalizedTerm
);
if (!aspectMentions.has(normalizedTerm)) {
aspectMentions.set(normalizedTerm, {
category,
sentiments: []
});
}
aspectMentions.get(normalizedTerm)!.sentiments.push(sentiment);
}
}
}
}
}
return Array.from(aspectMentions.entries()).map(([term, data]) => ({
term,
category: data.category,
frequency: data.sentiments.length,
averageSentiment: this.average(data.sentiments),
sentimentDistribution: this.calculateDistribution(data.sentiments)
}));
}
private calculateSentenceSentiment(sentence: string, aspect: string): number {
const words = sentence.toLowerCase().split(/\s+/);
const aspectIndex = words.findIndex(w => w.includes(aspect));
if (aspectIndex === -1) return 0;
let sentiment = 0;
let count = 0;
let negationActive = false;
let intensifier = 1;
// Analyze words around the aspect (window of 5 words)
const windowStart = Math.max(0, aspectIndex - 5);
const windowEnd = Math.min(words.length, aspectIndex + 6);
for (let i = windowStart; i < windowEnd; i++) {
const word = words[i].replace(/[^\w]/g, '');
// Check for negation
if (this.negationWords.has(word)) {
negationActive = true;
continue;
}
// Check for intensifiers
if (this.intensifiers.has(word)) {
intensifier = this.intensifiers.get(word)!;
continue;
}
// Check for opinion words
if (this.opinionLexicon.has(word)) {
let opinionScore = this.opinionLexicon.get(word)! * intensifier;
if (negationActive) {
opinionScore *= -0.8; // Negation flips sentiment
}
sentiment += opinionScore;
count++;
negationActive = false;
intensifier = 1;
}
}
return count > 0 ? sentiment / count : 0;
}
analyzeReview(text: string): SentenceAspectMap[] {
const sentences = this.splitIntoSentences(text);
const results: SentenceAspectMap[] = [];
for (const sentence of sentences) {
const aspects: SentenceAspectMap['aspects'] = [];
for (const [category, patterns] of this.aspectPatterns) {
for (const pattern of patterns) {
const matches = sentence.match(pattern);
if (matches) {
for (const match of matches) {
const term = match.toLowerCase();
const sentiment = this.calculateSentenceSentiment(sentence, term);
const opinionWords = this.findOpinionWords(sentence, term);
aspects.push({ term, sentiment, opinionWords });
}
}
}
}
if (aspects.length > 0) {
results.push({ sentence, aspects });
}
}
return results;
}
private findOpinionWords(sentence: string, aspect: string): string[] {
const words = sentence.toLowerCase().split(/\s+/);
const opinionWords: string[] = [];
for (const word of words) {
const cleanWord = word.replace(/[^\w]/g, '');
if (this.opinionLexicon.has(cleanWord)) {
opinionWords.push(cleanWord);
}
}
return opinionWords;
}
private splitIntoSentences(text: string): string[] {
return text
.split(/(?<=[.!?])\s+/)
.filter(s => s.trim().length > 0);
}
private average(numbers: number[]): number {
if (numbers.length === 0) return 0;
return numbers.reduce((a, b) => a + b, 0) / numbers.length;
}
private calculateDistribution(sentiments: number[]): {
positive: number;
negative: number;
neutral: number;
} {
const distribution = { positive: 0, negative: 0, neutral: 0 };
for (const s of sentiments) {
if (s > 0.1) distribution.positive++;
else if (s < -0.1) distribution.negative++;
else distribution.neutral++;
}
const total = sentiments.length || 1;
return {
positive: distribution.positive / total,
negative: distribution.negative / total,
neutral: distribution.neutral / total
};
}
}
export { AspectBasedSentimentAnalyzer }; Social Media Monitoring and Brand Listening
Social listening involves monitoring social media platforms, forums, and review sites to track brand mentions and sentiment trends. According to Sprout Social, 57% of consumers follow brands on social media to learn about new products. Here's a comprehensive social listening implementation:
// social-listener.ts
interface SocialMention {
id: string;
platform: SocialPlatform;
text: string;
author: AuthorInfo;
timestamp: Date;
engagement: EngagementMetrics;
sentiment: SentimentResult;
entities: ExtractedEntity[];
isRepost: boolean;
replyToId?: string;
url: string;
}
type SocialPlatform = 'twitter' | 'reddit' | 'facebook' | 'instagram' |
'linkedin' | 'youtube' | 'tiktok' | 'news' | 'blog';
interface AuthorInfo {
id: string;
username: string;
displayName: string;
followers: number;
verified: boolean;
influenceScore: number;
}
interface EngagementMetrics {
likes: number;
shares: number;
comments: number;
reach: number;
impressions: number;
}
interface ExtractedEntity {
text: string;
type: 'brand' | 'product' | 'competitor' | 'person' | 'location';
sentiment: number;
}
interface AlertRule {
id: string;
name: string;
conditions: AlertCondition[];
actions: AlertAction[];
isActive: boolean;
}
interface AlertCondition {
field: 'sentiment' | 'volume' | 'influencer' | 'keyword';
operator: 'lt' | 'gt' | 'eq' | 'contains';
value: any;
timeWindow?: number; // minutes
}
interface AlertAction {
type: 'email' | 'slack' | 'webhook' | 'dashboard';
config: Record;
}
class SocialListeningPlatform {
private mentions: Map = new Map();
private alertRules: AlertRule[] = [];
private sentimentAnalyzer: SentimentAnalyzer;
private websocketClients: Set = new Set();
constructor(sentimentAnalyzer: SentimentAnalyzer) {
this.sentimentAnalyzer = sentimentAnalyzer;
}
// Process incoming social mention
async processMention(rawMention: RawMention): Promise {
// Analyze sentiment
const sentimentResult = await this.sentimentAnalyzer.analyzeSentiment(
rawMention.text
);
// Extract entities
const entities = await this.extractEntities(rawMention.text);
// Calculate influence score
const influenceScore = this.calculateInfluenceScore(rawMention.author);
const mention: SocialMention = {
id: rawMention.id,
platform: rawMention.platform,
text: rawMention.text,
author: {
...rawMention.author,
influenceScore
},
timestamp: new Date(rawMention.timestamp),
engagement: rawMention.engagement,
sentiment: sentimentResult,
entities,
isRepost: rawMention.isRepost || false,
replyToId: rawMention.replyToId,
url: rawMention.url
};
this.mentions.set(mention.id, mention);
// Check alert rules
await this.checkAlerts(mention);
// Broadcast to real-time clients
this.broadcastMention(mention);
return mention;
}
private async extractEntities(text: string): Promise {
// In production, use NER model
const entities: ExtractedEntity[] = [];
const brandPatterns = [
{ pattern: /\b@\w+/g, type: 'brand' as const },
{ pattern: /#\w+/g, type: 'product' as const }
];
for (const { pattern, type } of brandPatterns) {
const matches = text.match(pattern) || [];
for (const match of matches) {
const sentiment = await this.sentimentAnalyzer
.analyzeSentiment(text)
.then(r => r.label === 'POSITIVE' ? r.score : -r.score);
entities.push({
text: match,
type,
sentiment
});
}
}
return entities;
}
private calculateInfluenceScore(author: RawAuthor): number {
// Weighted score based on follower count and verification
const followerWeight = Math.log10(author.followers + 1) / 7; // Normalize to 0-1
const verifiedBonus = author.verified ? 0.2 : 0;
const engagementRate = author.avgEngagement || 0;
return Math.min(1, followerWeight * 0.5 + verifiedBonus + engagementRate * 0.3);
}
// Add alert rule for monitoring
addAlertRule(rule: AlertRule): void {
this.alertRules.push(rule);
}
private async checkAlerts(mention: SocialMention): Promise {
for (const rule of this.alertRules.filter(r => r.isActive)) {
const triggered = rule.conditions.every(condition =>
this.evaluateCondition(condition, mention)
);
if (triggered) {
await this.executeAlertActions(rule, mention);
}
}
}
private evaluateCondition(
condition: AlertCondition,
mention: SocialMention
): boolean {
switch (condition.field) {
case 'sentiment':
const sentimentScore = mention.sentiment.label === 'NEGATIVE'
? -mention.sentiment.score
: mention.sentiment.score;
return this.compareValues(sentimentScore, condition.operator, condition.value);
case 'volume':
const recentCount = this.getRecentMentionCount(condition.timeWindow || 60);
return this.compareValues(recentCount, condition.operator, condition.value);
case 'influencer':
return this.compareValues(
mention.author.influenceScore,
condition.operator,
condition.value
);
case 'keyword':
return mention.text.toLowerCase().includes(condition.value.toLowerCase());
default:
return false;
}
}
private compareValues(actual: number, operator: string, expected: number): boolean {
switch (operator) {
case 'lt': return actual < expected;
case 'gt': return actual > expected;
case 'eq': return actual === expected;
default: return false;
}
}
private getRecentMentionCount(windowMinutes: number): number {
const cutoff = Date.now() - windowMinutes * 60 * 1000;
let count = 0;
this.mentions.forEach(mention => {
if (mention.timestamp.getTime() > cutoff) {
count++;
}
});
return count;
}
private async executeAlertActions(
rule: AlertRule,
mention: SocialMention
): Promise {
for (const action of rule.actions) {
switch (action.type) {
case 'slack':
await this.sendSlackAlert(action.config, rule, mention);
break;
case 'email':
await this.sendEmailAlert(action.config, rule, mention);
break;
case 'webhook':
await this.sendWebhook(action.config, rule, mention);
break;
}
}
}
private async sendSlackAlert(
config: Record,
rule: AlertRule,
mention: SocialMention
): Promise {
const message = {
blocks: [
{
type: 'header',
text: {
type: 'plain_text',
text: `Alert: ${rule.name}`
}
},
{
type: 'section',
fields: [
{
type: 'mrkdwn',
text: `*Platform:*\n${mention.platform}`
},
{
type: 'mrkdwn',
text: `*Sentiment:*\n${mention.sentiment.label} (${mention.sentiment.score.toFixed(2)})`
},
{
type: 'mrkdwn',
text: `*Author:*\n@${mention.author.username}`
},
{
type: 'mrkdwn',
text: `*Influence:*\n${(mention.author.influenceScore * 100).toFixed(0)}%`
}
]
},
{
type: 'section',
text: {
type: 'mrkdwn',
text: `"${mention.text.substring(0, 200)}..."`
}
},
{
type: 'actions',
elements: [
{
type: 'button',
text: { type: 'plain_text', text: 'View Post' },
url: mention.url
},
{
type: 'button',
text: { type: 'plain_text', text: 'Respond' },
action_id: `respond_${mention.id}`
}
]
}
]
};
await fetch(config.webhookUrl, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(message)
});
}
// Get aggregated sentiment trends
getSentimentTrends(
timeRange: { start: Date; end: Date },
granularity: 'hour' | 'day' | 'week'
): SentimentTrend[] {
const trends: Map = new Map();
this.mentions.forEach(mention => {
if (mention.timestamp >= timeRange.start && mention.timestamp <= timeRange.end) {
const bucket = this.getTimeBucket(mention.timestamp, granularity);
if (!trends.has(bucket)) {
trends.set(bucket, { positive: 0, negative: 0, neutral: 0, total: 0 });
}
const data = trends.get(bucket)!;
data.total++;
if (mention.sentiment.label === 'POSITIVE') data.positive++;
else if (mention.sentiment.label === 'NEGATIVE') data.negative++;
else data.neutral++;
}
});
return Array.from(trends.entries())
.map(([timestamp, data]) => ({
timestamp,
...data,
sentimentScore: (data.positive - data.negative) / data.total
}))
.sort((a, b) => a.timestamp.localeCompare(b.timestamp));
}
private getTimeBucket(date: Date, granularity: string): string {
const d = new Date(date);
switch (granularity) {
case 'hour':
d.setMinutes(0, 0, 0);
break;
case 'day':
d.setHours(0, 0, 0, 0);
break;
case 'week':
d.setDate(d.getDate() - d.getDay());
d.setHours(0, 0, 0, 0);
break;
}
return d.toISOString();
}
// WebSocket for real-time updates
private broadcastMention(mention: SocialMention): void {
const message = JSON.stringify({
type: 'new_mention',
data: mention
});
this.websocketClients.forEach(client => {
if (client.readyState === WebSocket.OPEN) {
client.send(message);
}
});
}
addWebSocketClient(client: WebSocket): void {
this.websocketClients.add(client);
client.on('close', () => this.websocketClients.delete(client));
}
} Customer Feedback Categorization and Prioritization
Efficiently processing customer feedback requires automatic categorization and prioritization. This enables support teams to focus on urgent issues while tracking sentiment across product areas. Zendesk research shows that 52% of consumers expect a response within an hour:
// feedback-processor.ts
interface CustomerFeedback {
id: string;
source: FeedbackSource;
text: string;
customerId?: string;
timestamp: Date;
metadata: Record;
}
type FeedbackSource = 'support_ticket' | 'review' | 'survey' | 'social' |
'chat' | 'email' | 'in_app';
interface ProcessedFeedback extends CustomerFeedback {
sentiment: SentimentResult;
category: FeedbackCategory;
subcategory: string;
priority: Priority;
suggestedActions: SuggestedAction[];
relatedFeedback: string[];
tags: string[];
}
type FeedbackCategory = 'bug_report' | 'feature_request' | 'complaint' |
'praise' | 'question' | 'churn_risk' | 'general';
type Priority = 'critical' | 'high' | 'medium' | 'low';
interface SuggestedAction {
type: 'respond' | 'escalate' | 'route' | 'automate';
description: string;
confidence: number;
template?: string;
}
interface FeedbackClassificationModel {
categories: Map;
priorityRules: PriorityRule[];
}
interface PriorityRule {
conditions: RuleCondition[];
priority: Priority;
reason: string;
}
interface RuleCondition {
field: string;
operator: 'contains' | 'gt' | 'lt' | 'eq' | 'matches';
value: any;
}
class FeedbackProcessor {
private sentimentAnalyzer: SentimentAnalyzer;
private classificationModel: FeedbackClassificationModel;
private responseTemplates: Map;
constructor(sentimentAnalyzer: SentimentAnalyzer) {
this.sentimentAnalyzer = sentimentAnalyzer;
this.classificationModel = this.loadClassificationModel();
this.responseTemplates = this.loadResponseTemplates();
}
private loadClassificationModel(): FeedbackClassificationModel {
return {
categories: new Map([
['bug_report', [
'bug', 'error', 'crash', 'broken', 'not working', 'fails',
'issue', 'problem', 'glitch', 'doesn\'t work', 'stopped working'
]],
['feature_request', [
'would be nice', 'should add', 'feature request', 'wish',
'please add', 'suggestion', 'could you', 'it would help'
]],
['complaint', [
'disappointed', 'frustrated', 'terrible', 'worst', 'hate',
'unacceptable', 'ridiculous', 'waste', 'refund', 'cancel'
]],
['praise', [
'love', 'amazing', 'great', 'excellent', 'best', 'awesome',
'fantastic', 'perfect', 'thank you', 'impressed'
]],
['question', [
'how do i', 'how can i', 'where is', 'what is', 'why does',
'can i', 'is it possible', 'help me', '?'
]],
['churn_risk', [
'cancel', 'cancellation', 'leaving', 'switching', 'competitor',
'alternative', 'done with', 'last chance', 'final warning'
]]
]),
priorityRules: [
{
conditions: [
{ field: 'category', operator: 'eq', value: 'churn_risk' }
],
priority: 'critical',
reason: 'Customer showing signs of churn'
},
{
conditions: [
{ field: 'sentiment.score', operator: 'lt', value: -0.8 },
{ field: 'customer.tier', operator: 'eq', value: 'enterprise' }
],
priority: 'critical',
reason: 'Very negative sentiment from enterprise customer'
},
{
conditions: [
{ field: 'category', operator: 'eq', value: 'bug_report' },
{ field: 'text', operator: 'contains', value: 'data loss' }
],
priority: 'critical',
reason: 'Potential data loss issue reported'
},
{
conditions: [
{ field: 'category', operator: 'eq', value: 'complaint' },
{ field: 'sentiment.score', operator: 'lt', value: -0.5 }
],
priority: 'high',
reason: 'Negative complaint requiring attention'
},
{
conditions: [
{ field: 'category', operator: 'eq', value: 'feature_request' }
],
priority: 'medium',
reason: 'Feature request for product consideration'
}
]
};
}
private loadResponseTemplates(): Map {
return new Map([
['bug_acknowledged', `Hi {customer_name},
Thank you for reporting this issue. We apologize for the inconvenience you're experiencing.
Our engineering team has been notified and is investigating the {issue_summary}.
We'll keep you updated on our progress.
In the meantime, {workaround_if_available}
Best regards,
{agent_name}`],
['churn_prevention', `Hi {customer_name},
I noticed you mentioned {concern}. As a valued customer, your satisfaction is our top priority.
I'd love to personally help resolve this. Would you be available for a quick call to discuss how we can make things right?
{special_offer_if_applicable}
Looking forward to hearing from you.
Best,
{agent_name}`],
['praise_response', `Hi {customer_name},
Thank you so much for your kind words! We're thrilled to hear that {positive_aspect} has been helpful for you.
Your feedback means a lot to our team and motivates us to keep improving.
If there's anything else we can do to enhance your experience, please don't hesitate to reach out!
Best,
{agent_name}`]
]);
}
async processFeedback(feedback: CustomerFeedback): Promise {
// Analyze sentiment
const sentiment = await this.sentimentAnalyzer.analyzeSentiment(feedback.text);
// Classify category
const { category, subcategory } = this.classifyFeedback(feedback.text);
// Determine priority
const priority = this.calculatePriority(feedback, sentiment, category);
// Generate suggested actions
const suggestedActions = this.generateSuggestedActions(
feedback, sentiment, category, priority
);
// Extract tags
const tags = this.extractTags(feedback.text);
// Find related feedback
const relatedFeedback = await this.findRelatedFeedback(feedback);
return {
...feedback,
sentiment,
category,
subcategory,
priority,
suggestedActions,
relatedFeedback,
tags
};
}
private classifyFeedback(text: string): { category: FeedbackCategory; subcategory: string } {
const normalizedText = text.toLowerCase();
let bestMatch: { category: FeedbackCategory; score: number; keyword: string } = {
category: 'general',
score: 0,
keyword: ''
};
for (const [category, keywords] of this.classificationModel.categories) {
for (const keyword of keywords) {
if (normalizedText.includes(keyword)) {
const score = keyword.length / normalizedText.length;
if (score > bestMatch.score) {
bestMatch = { category, score, keyword };
}
}
}
}
return {
category: bestMatch.category,
subcategory: bestMatch.keyword || 'general'
};
}
private calculatePriority(
feedback: CustomerFeedback,
sentiment: SentimentResult,
category: FeedbackCategory
): Priority {
const context = {
category,
sentiment,
text: feedback.text.toLowerCase(),
customer: feedback.metadata?.customer || {}
};
for (const rule of this.classificationModel.priorityRules) {
const matches = rule.conditions.every(condition =>
this.evaluateCondition(condition, context)
);
if (matches) {
return rule.priority;
}
}
return 'low';
}
private evaluateCondition(
condition: RuleCondition,
context: Record
): boolean {
const value = this.getNestedValue(context, condition.field);
switch (condition.operator) {
case 'contains':
return String(value).includes(condition.value);
case 'gt':
return Number(value) > condition.value;
case 'lt':
return Number(value) < condition.value;
case 'eq':
return value === condition.value;
case 'matches':
return new RegExp(condition.value).test(String(value));
default:
return false;
}
}
private getNestedValue(obj: Record, path: string): any {
return path.split('.').reduce((current, key) => current?.[key], obj);
}
private generateSuggestedActions(
feedback: CustomerFeedback,
sentiment: SentimentResult,
category: FeedbackCategory,
priority: Priority
): SuggestedAction[] {
const actions: SuggestedAction[] = [];
// Auto-response for simple cases
if (category === 'question' && sentiment.label !== 'NEGATIVE') {
actions.push({
type: 'automate',
description: 'Send knowledge base article',
confidence: 0.7
});
}
// Escalation for critical issues
if (priority === 'critical') {
actions.push({
type: 'escalate',
description: 'Escalate to senior support',
confidence: 0.95
});
}
// Response templates
if (category === 'churn_risk') {
actions.push({
type: 'respond',
description: 'Send churn prevention response',
confidence: 0.85,
template: this.responseTemplates.get('churn_prevention')
});
}
if (category === 'praise') {
actions.push({
type: 'respond',
description: 'Thank customer for feedback',
confidence: 0.9,
template: this.responseTemplates.get('praise_response')
});
}
// Route to appropriate team
if (category === 'bug_report') {
actions.push({
type: 'route',
description: 'Route to engineering team',
confidence: 0.9
});
}
return actions.sort((a, b) => b.confidence - a.confidence);
}
private extractTags(text: string): string[] {
const tags: string[] = [];
const tagPatterns: Record = {
'mobile': /\b(mobile|ios|android|app)\b/i,
'web': /\b(website|browser|chrome|firefox|safari)\b/i,
'billing': /\b(billing|payment|invoice|charge|subscription)\b/i,
'performance': /\b(slow|fast|speed|lag|performance)\b/i,
'security': /\b(security|password|login|authentication|2fa)\b/i,
'integration': /\b(integration|api|webhook|connect)\b/i
};
for (const [tag, pattern] of Object.entries(tagPatterns)) {
if (pattern.test(text)) {
tags.push(tag);
}
}
return tags;
}
private async findRelatedFeedback(feedback: CustomerFeedback): Promise {
// In production, use vector similarity search
// This is a simplified keyword-based approach
return [];
}
}
export { FeedbackProcessor }; Multilingual Sentiment Analysis
Global platforms need sentiment analysis across multiple languages. Multilingual transformer models enable sentiment analysis without language-specific models:
// multilingual-sentiment.ts
import { pipeline, Pipeline } from '@xenova/transformers';
interface MultilingualAnalysisResult {
text: string;
detectedLanguage: string;
languageConfidence: number;
sentiment: SentimentResult;
translatedText?: string;
}
class MultilingualSentimentAnalyzer {
private multilingualPipeline: Pipeline | null = null;
private languageDetector: Pipeline | null = null;
private translationPipelines: Map = new Map();
private supportedLanguages = [
'en', 'es', 'fr', 'de', 'it', 'pt', 'nl', 'ru',
'ja', 'ko', 'zh', 'ar', 'hi', 'tr', 'pl'
];
async initialize(): Promise {
// Load multilingual sentiment model
this.multilingualPipeline = await pipeline(
'sentiment-analysis',
'Xenova/bert-base-multilingual-uncased-sentiment'
);
console.log('Multilingual sentiment analyzer initialized');
}
async detectLanguage(text: string): Promise<{ language: string; confidence: number }> {
// Simple language detection using character patterns
// In production, use a proper language detection model
const languagePatterns: Record = {
'ja': /[\u3040-\u309F\u30A0-\u30FF]/,
'ko': /[\uAC00-\uD7AF]/,
'zh': /[\u4E00-\u9FFF]/,
'ar': /[\u0600-\u06FF]/,
'ru': /[\u0400-\u04FF]/,
'hi': /[\u0900-\u097F]/
};
for (const [lang, pattern] of Object.entries(languagePatterns)) {
if (pattern.test(text)) {
return { language: lang, confidence: 0.9 };
}
}
// Default to English for Latin scripts
return { language: 'en', confidence: 0.8 };
}
async analyze(text: string): Promise {
if (!this.multilingualPipeline) {
await this.initialize();
}
const { language, confidence: languageConfidence } = await this.detectLanguage(text);
const result = await this.multilingualPipeline!(text);
// Map model output to standard format
const sentiment: SentimentResult = {
label: this.mapSentimentLabel(result[0].label),
score: result[0].score
};
return {
text,
detectedLanguage: language,
languageConfidence,
sentiment
};
}
private mapSentimentLabel(label: string): 'POSITIVE' | 'NEGATIVE' | 'NEUTRAL' {
// Multilingual model uses 1-5 star ratings
const starRating = parseInt(label.replace(' stars', ''));
if (starRating >= 4) return 'POSITIVE';
if (starRating <= 2) return 'NEGATIVE';
return 'NEUTRAL';
}
async analyzeMultiple(texts: { text: string; id: string }[]): Promise> {
const results = new Map();
// Batch by language for efficiency
const languageBatches = new Map();
for (const item of texts) {
const { language } = await this.detectLanguage(item.text);
if (!languageBatches.has(language)) {
languageBatches.set(language, []);
}
languageBatches.get(language)!.push(item);
}
// Process each language batch
for (const [language, batch] of languageBatches) {
const batchResults = await Promise.all(
batch.map(item => this.analyze(item.text))
);
batch.forEach((item, index) => {
results.set(item.id, batchResults[index]);
});
}
return results;
}
// Get sentiment distribution by language
async getSentimentByLanguage(
mentions: SocialMention[]
): Promise> {
const distribution = new Map();
for (const mention of mentions) {
const result = await this.analyze(mention.text);
const lang = result.detectedLanguage;
if (!distribution.has(lang)) {
distribution.set(lang, { positive: 0, negative: 0, neutral: 0 });
}
const langDist = distribution.get(lang)!;
if (result.sentiment.label === 'POSITIVE') langDist.positive++;
else if (result.sentiment.label === 'NEGATIVE') langDist.negative++;
else langDist.neutral++;
}
return distribution;
}
}
export { MultilingualSentimentAnalyzer }; Real-Time Processing at Scale
Processing high volumes of social mentions and feedback requires efficient streaming architecture. Here's a scalable implementation using Redis Streams and worker pools:
// realtime-processor.ts
import Redis from 'ioredis';
interface StreamMessage {
id: string;
text: string;
source: string;
timestamp: number;
metadata: Record;
}
interface ProcessingMetrics {
messagesProcessed: number;
averageLatency: number;
errorRate: number;
queueDepth: number;
}
class RealtimeSentimentProcessor {
private redis: Redis;
private sentimentAnalyzer: SentimentAnalyzer;
private streamKey: string = 'sentiment:stream';
private consumerGroup: string = 'sentiment-processors';
private consumerId: string;
private isProcessing: boolean = false;
private metrics: ProcessingMetrics;
constructor(redisUrl: string, sentimentAnalyzer: SentimentAnalyzer) {
this.redis = new Redis(redisUrl);
this.sentimentAnalyzer = sentimentAnalyzer;
this.consumerId = `processor-${process.pid}-${Date.now()}`;
this.metrics = {
messagesProcessed: 0,
averageLatency: 0,
errorRate: 0,
queueDepth: 0
};
}
async initialize(): Promise {
// Create consumer group if not exists
try {
await this.redis.xgroup(
'CREATE',
this.streamKey,
this.consumerGroup,
'0',
'MKSTREAM'
);
} catch (error: any) {
if (!error.message.includes('BUSYGROUP')) {
throw error;
}
}
console.log(`Initialized consumer ${this.consumerId}`);
}
async addToStream(message: StreamMessage): Promise {
const id = await this.redis.xadd(
this.streamKey,
'*',
'id', message.id,
'text', message.text,
'source', message.source,
'timestamp', message.timestamp.toString(),
'metadata', JSON.stringify(message.metadata)
);
return id;
}
async startProcessing(batchSize: number = 10): Promise {
this.isProcessing = true;
while (this.isProcessing) {
try {
// Read messages from stream
const messages = await this.redis.xreadgroup(
'GROUP', this.consumerGroup, this.consumerId,
'COUNT', batchSize,
'BLOCK', 5000,
'STREAMS', this.streamKey, '>'
);
if (messages) {
await this.processBatch(messages[0][1]);
}
// Process pending messages (retry failed)
await this.processPending();
} catch (error) {
console.error('Processing error:', error);
await this.sleep(1000);
}
}
}
private async processBatch(messages: any[]): Promise {
const startTime = Date.now();
const processingPromises = messages.map(async ([streamId, fields]) => {
try {
const message = this.parseMessage(fields);
const result = await this.processMessage(message);
// Store result
await this.storeResult(message.id, result);
// Acknowledge message
await this.redis.xack(this.streamKey, this.consumerGroup, streamId);
this.metrics.messagesProcessed++;
} catch (error) {
console.error(`Failed to process message ${streamId}:`, error);
this.metrics.errorRate =
(this.metrics.errorRate * this.metrics.messagesProcessed + 1) /
(this.metrics.messagesProcessed + 1);
}
});
await Promise.all(processingPromises);
const latency = Date.now() - startTime;
this.updateLatencyMetric(latency / messages.length);
}
private parseMessage(fields: string[]): StreamMessage {
const obj: Record = {};
for (let i = 0; i < fields.length; i += 2) {
obj[fields[i]] = fields[i + 1];
}
return {
id: obj.id,
text: obj.text,
source: obj.source,
timestamp: parseInt(obj.timestamp),
metadata: JSON.parse(obj.metadata || '{}')
};
}
private async processMessage(message: StreamMessage): Promise {
return this.sentimentAnalyzer.fullAnalysis(message.text);
}
private async storeResult(messageId: string, result: AnalysisResult): Promise {
const key = `sentiment:result:${messageId}`;
await this.redis.set(key, JSON.stringify(result), 'EX', 86400); // 24h TTL
// Update aggregates
await this.updateAggregates(result);
// Publish for real-time subscribers
await this.redis.publish('sentiment:updates', JSON.stringify({
messageId,
sentiment: result.sentiment,
timestamp: Date.now()
}));
}
private async updateAggregates(result: AnalysisResult): Promise {
const hour = new Date().toISOString().slice(0, 13);
const key = `sentiment:hourly:${hour}`;
await this.redis.hincrby(key, 'total', 1);
await this.redis.hincrby(key, result.sentiment.label.toLowerCase(), 1);
await this.redis.hincrbyfloat(key, 'score_sum',
result.sentiment.label === 'POSITIVE' ? result.sentiment.score : -result.sentiment.score
);
await this.redis.expire(key, 86400 * 7); // 7 days
}
private async processPending(): Promise {
// Get pending messages older than 1 minute
const pending = await this.redis.xpending(
this.streamKey,
this.consumerGroup,
'-', '+', 10,
this.consumerId
);
for (const [streamId, , idleTime] of pending) {
if (idleTime > 60000) {
// Claim and reprocess
await this.redis.xclaim(
this.streamKey,
this.consumerGroup,
this.consumerId,
60000,
streamId
);
}
}
}
private updateLatencyMetric(latency: number): void {
const alpha = 0.1; // Exponential moving average factor
this.metrics.averageLatency =
this.metrics.averageLatency * (1 - alpha) + latency * alpha;
}
async getMetrics(): Promise {
const info = await this.redis.xinfo('GROUPS', this.streamKey);
this.metrics.queueDepth = info[0]?.pending || 0;
return { ...this.metrics };
}
stopProcessing(): void {
this.isProcessing = false;
}
private sleep(ms: number): Promise {
return new Promise(resolve => setTimeout(resolve, ms));
}
}
export { RealtimeSentimentProcessor }; Sentiment Dashboard and Visualization
Presenting sentiment insights effectively requires interactive dashboards. Here's a React component for visualizing sentiment trends:
// sentiment-dashboard.tsx
import React, { useState, useEffect } from 'react';
import { LineChart, Line, XAxis, YAxis, CartesianGrid, Tooltip,
ResponsiveContainer, PieChart, Pie, Cell, BarChart, Bar } from 'recharts';
interface DashboardProps {
dateRange: { start: Date; end: Date };
}
interface SentimentData {
trends: TrendDataPoint[];
distribution: { name: string; value: number }[];
topAspects: AspectData[];
alertCount: { critical: number; high: number; medium: number };
volumeBySource: { source: string; count: number }[];
}
interface TrendDataPoint {
timestamp: string;
positive: number;
negative: number;
neutral: number;
score: number;
}
interface AspectData {
aspect: string;
sentiment: number;
mentions: number;
}
const COLORS = {
positive: '#22c55e',
negative: '#ef4444',
neutral: '#6b7280'
};
function SentimentDashboard({ dateRange }: DashboardProps) {
const [data, setData] = useState(null);
const [isLoading, setIsLoading] = useState(true);
const [selectedMetric, setSelectedMetric] = useState<'volume' | 'sentiment'>('sentiment');
useEffect(() => {
fetchDashboardData(dateRange)
.then(setData)
.finally(() => setIsLoading(false));
}, [dateRange]);
if (isLoading || !data) {
return Loading sentiment data...;
}
return (
{/* Summary Cards */}
sum + d.positive + d.negative + d.neutral, 0)}
subtitle="Last 7 days"
icon="message-circle"
/>
0 ? 'danger' : 'success'}
icon="alert-triangle"
/>
{/* Sentiment Trend Chart */}
Sentiment Trend
new Date(v).toLocaleDateString()}
/>
new Date(v).toLocaleString()}
formatter={(value: number, name: string) => [
selectedMetric === 'sentiment'
? value.toFixed(2)
: value,
name.charAt(0).toUpperCase() + name.slice(1)
]}
/>
{selectedMetric === 'sentiment' ? (
{/* Sentiment Distribution */}
Sentiment Distribution
{data.distribution.map((entry, index) => (
|
))}
{data.distribution.map((item) => (
{item.name}: {item.value}%
))}
{/* Top Aspects */}
Aspect Sentiment
{data.topAspects.map((entry, index) => (
| = 0 ? COLORS.positive : COLORS.negative}
/>
))}
|
{/* Volume by Source */}
Mentions by Source
);
}
function SummaryCard({ title, value, trend, subtitle, status, icon }: {
title: string;
value: string | number;
trend?: { direction: 'up' | 'down'; percentage: number };
subtitle?: string;
status?: 'success' | 'danger' | 'warning';
icon: string;
}) {
return (
{title}
{value}
{trend && (
{trend.direction === 'up' ? '↑' : '↓'} {trend.percentage}%
)}
{subtitle && {subtitle}}
);
}
export { SentimentDashboard }; Key Takeaways
Remember These Points
- Use transformer models for accuracy: BERT-based models achieve 90%+ accuracy on sentiment classification, significantly outperforming rule-based approaches
- Implement aspect-based analysis: Understanding sentiment toward specific features provides actionable insights for product improvement
- Build real-time monitoring: Social listening requires streaming architecture to detect and respond to issues quickly
- Support multiple languages: Multilingual models enable global sentiment analysis without language-specific implementations
- Prioritize customer feedback: Automatic categorization and priority assignment ensures urgent issues receive immediate attention
- Create alerting rules: Define conditions for negative sentiment spikes, influencer mentions, and churn signals
- Visualize trends effectively: Dashboards should surface both real-time metrics and historical trends
Conclusion
Sentiment analysis and social listening have become essential capabilities for modern web platforms. The implementations covered in this guide provide a foundation for understanding user sentiment at scale, from individual reviews to millions of social media mentions.
Start with transformer-based sentiment classification for high accuracy, then add aspect-based analysis for granular insights. Implement real-time processing to catch issues early, and build comprehensive dashboards to make sentiment data actionable for your teams.
For further learning, explore the Hugging Face documentation on text classification, NLTK for traditional NLP techniques, and spaCy for production NLP pipelines. The key to successful sentiment analysis is continuous refinement based on your specific domain and user feedback patterns.
