virality-scoring.md · jhustle44/jcr-guide-pro

Code Commits Issues Pulls Cloud Agent CI/CD

master

copilot/add-firebase copilot/change-app-name-to-jcrguru copilot/create-zip-file copilot/create-zip-file-again copilot/merge-any-updates main master

md 1,212 lines 27.2 KB

Rendered Raw

1	# Virality Scoring Pipeline
2
3	Complete implementation guide for AI-powered video analysis and auto-trimming based on virality potential.
4
5	## Table of Contents
6
7	1. [Overview](#overview)
8	2. [Step 1: Scene Detection](#step-1-scene-detection)
9
10	3. [Step 2: Frame Extraction](#step-2-frame-extraction)
11	4. [Step 3: AI Analysis](#step-3-ai-analysis)
12
13	5. [Step 4: Scoring and Ranking](#step-4-scoring-and-ranking)
14	6. [Step 5: Output Assembly](#step-5-output-assembly)
15
16	7. [Complete Pipeline Example](#complete-pipeline-example)
17
18	---
19
20	## Overview
21
22	The virality scoring pipeline takes a video file as input, breaks it into segments, analyzes each segment using AI vision, scores them on engagement potential, and outputs the best content in the user's preferred format.
23
24	```text
25
26	Input Video
27
28	↓
29
30	Scene Detection (FFmpeg)
31
32	↓
33
34	Frame Extraction (FFmpeg) — 2-3 keyframes per segment
35
36	↓
37
38	AI Vision Analysis (OpenAI gpt-5.2 or Gemini gemini-2.5-flash)
39
40	↓
41
42	Score & Rank Segments
43
44	↓
45
46	Ask User: Best clip / Multiple clips / Highlight reel
47
48	↓
49
50	Trim & Assemble (FFmpeg)
51
52	↓
53
54	Output File(s)
55
56	```
57
58	---
59
60	## Step 1: Scene Detection
61
62	Use FFmpeg's scene detection filter to find natural cut points in the video. This splits the video into segments based on visual changes.
63
64	Important: The `ffprobe -f lavfi`approach does not work reliably in Replit's environment. Instead, use`ffmpeg`with the`select`and`showinfo`filters, then parse`pts_time` values from stderr output. This is the tested, working approach.
65
66	```typescript
67
68	import { execFile } from 'child_process';
69
70	import { promisify } from 'util';
71
72	const execFileAsync = promisify(execFile);
73
74	interface Segment {
75
76	index: number;
77
78	startTime: number;
79
80	endTime: number;
81
82	duration: number;
83
84	}
85
86	async function detectScenes(videoPath: string, threshold = 0.3): Promise<Segment[]> {
87
88	const { stderr } = await execFileAsync('ffmpeg', [
89
90	'-i', videoPath,
91
92	'-filter:v', `select='gt(scene,${threshold})',showinfo`,
93
94	'-f', 'null', '-'
95
96	], { maxBuffer: 10 * 1024 * 1024 });
97
98	const sceneChanges: number[] = [0];
99
100	const lines = stderr.split('\n');
101
102	for (const line of lines) {
103
104	const match = line.match(/pts_time:([0-9.]+)/);
105
106	if (match) {
107
108	const pts = parseFloat(match[1]);
109
110	if (pts > 0) sceneChanges.push(pts);
111
112	}
113
114	}
115
116	const { stdout: probeOut } = await execFileAsync('ffprobe', [
117
118	'-v', 'quiet',
119
120	'-print_format', 'json',
121
122	'-show_format',
123
124	videoPath
125
126	]);
127
128	const totalDuration = parseFloat(JSON.parse(probeOut).format.duration);
129
130	sceneChanges.push(totalDuration);
131
132	const segments: Segment[] = [];
133
134	for (let i = 0; i < sceneChanges.length - 1; i++) {
135
136	const start = sceneChanges[i];
137
138	const end = sceneChanges[i + 1];
139
140	const duration = end - start;
141
142	if (duration >= 2.0) {
143
144	segments.push({ index: segments.length, startTime: start, endTime: end, duration });
145
146	}
147
148	}
149
150	return segments;
151
152	}
153
154	```
155
156	Note on segment indexing: Use `index: segments.length`(not`index: i`) when building segments. When short segments are filtered out, `i` no longer matches the array position, which causes index mismatches during the export step.
157
158	### Alternative: Fixed-interval segmentation
159
160	For videos without clear scene changes (talking head, screencasts), split into fixed-length chunks:
161
162	```typescript
163
164	async function splitFixed(videoPath: string, chunkSeconds = 10): Promise<Segment[]> {
165
166	const { stdout } = await execFileAsync('ffprobe', [
167
168	'-v', 'quiet', '-print_format', 'json', '-show_format', videoPath
169
170	]);
171
172	const totalDuration = parseFloat(JSON.parse(stdout).format.duration);
173
174	const segments: Segment[] = [];
175
176	let start = 0;
177
178	let index = 0;
179
180	while (start < totalDuration) {
181
182	const end = Math.min(start + chunkSeconds, totalDuration);
183
184	segments.push({ index, startTime: start, endTime: end, duration: end - start });
185
186	start = end;
187
188	index++;
189
190	}
191
192	return segments;
193
194	}
195
196	```
197
198	### Choosing the right approach
199
200	- Use scene detection for edited content, vlogs, multi-shot videos
201	- Use fixed-interval for single-camera footage, lectures, screencasts, livestreams
202
203	- If scene detection produces too many or too few segments, adjust the `threshold` (lower = more sensitive, higher = fewer scenes). Default 0.3 works well for most content.
204
205	---
206
207	## Step 2: Frame Extraction
208
209	Extract representative frames for AI analysis. The number of frames should scale with the content duration:
210
211	- Short segments (2-10s): 3 frames, evenly spread
212	- Medium clips (10-30s): 5 frames, including the first and last frame
213
214	- Long clips (30-60s): 8 frames, including the first and last frame
215
216	Including the first and last frame is important for clip-level scoring — the AI needs to evaluate the hook (opening) and payoff (ending).
217
218	```typescript
219
220	import * as fs from 'fs';
221
222	import * as path from 'path';
223
224	function getFrameCount(durationSeconds: number): number {
225
226	if (durationSeconds <= 10) return 3;
227
228	if (durationSeconds <= 30) return 5;
229
230	return 8;
231
232	}
233
234	async function extractFrames(
235
236	videoPath: string,
237
238	segment: Segment,
239
240	outputDir: string,
241
242	framesPerSegment?: number
243
244	): Promise<string[]> {
245
246	await fs.promises.mkdir(outputDir, { recursive: true });
247
248	const count = framesPerSegment ?? getFrameCount(segment.duration);
249
250	const framePaths: string[] = [];
251
252	for (let i = 0; i < count; i++) {
253
254	const fraction = count === 1 ? 0.5 : i / (count - 1);
255
256	const timestamp = Math.min(segment.startTime + fraction * segment.duration, segment.endTime - 0.1);
257
258	const framePath = path.join(outputDir, `segment_${segment.index}_frame_${i + 1}.jpg`);
259
260	await execFileAsync('ffmpeg', [
261
262	'-y',
263
264	'-ss', timestamp.toFixed(3),
265
266	'-i', videoPath,
267
268	'-frames:v', '1',
269
270	'-q:v', '2',
271
272	framePath
273
274	]);
275
276	framePaths.push(framePath);
277
278	}
279
280	return framePaths;
281
282	}
283
284	```
285
286	### Notes
287
288	- The frame extraction uses `i / (count - 1)`for timestamp calculation, which ensures the first frame is at the very start and the last frame is at the very end of the segment. This is critical for clip-level scoring where the AI evaluates the opening hook and closing payoff.
289	- The timestamp is clamped to`segment.endTime - 0.1`to prevent seeking past the end of the video. Without this, the last clip in a video can fail because`startTime + 1.0 * duration` may exceed the actual file duration due to floating point arithmetic.
290
291	---
292
293	## Step 3: AI Analysis
294
295	Send extracted frames to an AI vision model to analyze each segment's virality potential. The AI evaluates visual dynamism, emotional impact, hook quality, pacing, and shareability.
296
297	### Content Safety Handling
298
299	OpenAI's vision API may reject certain frames that trigger content safety filters (e.g., medical imagery, sensitive content involving minors or elderly people). Always wrap AI calls in try/catch and skip segments that fail rather than crashing the entire pipeline:
300
301	```typescript
302
303	try {
304
305	const score = await analyzeSegment(framePaths, segment, videoContext);
306
307	scores.push(score);
308
309	} catch (err: any) {
310
311	if (err?.code === 'content_policy_violation') {
312
313	console.log(`Segment ${segment.index + 1}: Skipped (content safety filter)`);
314
315	} else {
316
317	console.log(`Segment ${segment.index + 1}: Error — ${err?.message}`);
318
319	}
320
321	}
322
323	```
324
325	This is especially common with documentary, medical, or news content. The pipeline should continue and rank whatever segments it can analyze.
326
327	### OpenAI Client Setup
328
329	For script-based usage (not using the full `@workspace/integrations-openai-ai-server`library), install`openai` directly and create the client with the AI Integrations environment variables:
330
331	```typescript
332
333	import OpenAI from 'openai';
334
335	const openai = new OpenAI({
336
337	baseURL: process.env.AI_INTEGRATIONS_OPENAI_BASE_URL,
338
339	apiKey: process.env.AI_INTEGRATIONS_OPENAI_API_KEY,
340
341	});
342
343	```
344
345	Install with: `pnpm add -w openai`
346
347	If using the full workspace integration, import from `@workspace/integrations-openai-ai-server`instead (see the`ai-integrations-openai` skill for setup).
348
349	### Using OpenAI (gpt-5.2)
350
351	```typescript
352
353	import OpenAI from 'openai';
354
355	import * as fs from 'fs';
356
357	const openai = new OpenAI({
358
359	baseURL: process.env.AI_INTEGRATIONS_OPENAI_BASE_URL,
360
361	apiKey: process.env.AI_INTEGRATIONS_OPENAI_API_KEY,
362
363	});
364
365	interface ViralityScore {
366
367	segmentIndex: number;
368
369	overallScore: number;
370
371	factors: {
372
373	visualDynamism: number;
374
375	emotionalImpact: number;
376
377	hookPotential: number;
378
379	pacingEnergy: number;
380
381	uniqueness: number;
382
383	shareability: number;
384
385	};
386
387	reasoning: string;
388
389	suggestedCaption: string;
390
391	}
392
393	async function analyzeSegment(
394
395	framePaths: string[],
396
397	segment: Segment,
398
399	videoContext: string
400
401	): Promise<ViralityScore> {
402
403	const imageContents = framePaths.map(fp => ({
404
405	type: 'image_url' as const,
406
407	image_url: {
408
409	url: `data:image/jpeg;base64,${fs.readFileSync(fp).toString('base64')}`,
410
411	detail: 'low' as const
412
413	}
414
415	}));
416
417	const response = await openai.chat.completions.create({
418
419	model: 'gpt-5.2',
420
421	max_completion_tokens: 1024,
422
423	messages: [
424
425	{
426
427	role: 'system',
428
429	content: getScoringPrompt(segment.duration)
430
431	},
432
433	{
434
435	role: 'user',
436
437	content: [
438
439	{
440
441	type: 'text',
442
443	text: `Analyze this video segment for virality potential.
444
445	Video context: ${videoContext}
446
447	Segment: ${segment.index + 1}
448
449	Timestamp: ${formatTime(segment.startTime)} - ${formatTime(segment.endTime)}
450
451	Duration: ${segment.duration.toFixed(1)}s
452
453	These ${framePaths.length} frames are sampled evenly across the segment. Score this segment's virality potential.`
454
455	},
456
457	...imageContents
458
459	]
460
461	}
462
463	],
464
465	response_format: { type: 'json_object' }
466
467	});
468
469	const result = JSON.parse(response.choices[0]?.message?.content ?? '{}');
470
471	return {
472
473	segmentIndex: segment.index,
474
475	overallScore: result.overallScore,
476
477	factors: result.factors,
478
479	reasoning: result.reasoning,
480
481	suggestedCaption: result.suggestedCaption
482
483	};
484
485	}
486
487	function formatTime(seconds: number): string {
488
489	const m = Math.floor(seconds / 60);
490
491	const s = Math.floor(seconds % 60);
492
493	return `${m}:${s.toString().padStart(2, '0')}`;
494
495	}
496
497	```
498
499	### Using Gemini (gemini-2.5-flash)
500
501	```typescript
502
503	import { ai } from '@workspace/integrations-gemini-ai';
504
505	import * as fs from 'fs';
506
507	async function analyzeSegmentGemini(
508
509	framePaths: string[],
510
511	segment: Segment,
512
513	videoContext: string
514
515	): Promise<ViralityScore> {
516
517	const imageParts = framePaths.map(fp => ({
518
519	inlineData: {
520
521	mimeType: 'image/jpeg',
522
523	data: fs.readFileSync(fp).toString('base64')
524
525	}
526
527	}));
528
529	const response = await ai.models.generateContent({
530
531	model: 'gemini-2.5-flash',
532
533	contents: [{
534
535	role: 'user',
536
537	parts: [
538
539	{ text: `${getScoringPrompt(segment.duration)}\n\nAnalyze this video clip for virality potential.\n\nVideo context: ${videoContext}\nClip: ${segment.index + 1}\nTimestamp: ${formatTime(segment.startTime)} - ${formatTime(segment.endTime)}\nDuration: ${segment.duration.toFixed(1)}s\n\nThese ${framePaths.length} frames are sampled across the clip (first frame = opening hook, last frame = ending). Score this clip's virality potential. Respond in JSON format.` },
540
541	...imageParts
542
543	]
544
545	}],
546
547	config: { responseMimeType: 'application/json' }
548
549	});
550
551	const result = JSON.parse(response.text ?? '{}');
552
553	return {
554
555	segmentIndex: segment.index,
556
557	overallScore: result.overallScore,
558
559	factors: result.factors,
560
561	reasoning: result.reasoning,
562
563	suggestedCaption: result.suggestedCaption
564
565	};
566
567	}
568
569	```
570
571	### Scoring Prompts
572
573	There are two prompts — one for scoring short segments (Mode 1) and one for scoring complete clips (Mode 2). Use the clip-level prompt when clips are 10+ seconds.
574
575	#### Segment Scoring Prompt (Mode 1 — finding best moments)
576
577	Use this when scoring raw scene-detected segments (typically 2-10 seconds). This is the original prompt.
578
579	```typescript
580
581	const SEGMENT_SCORING_PROMPT = `You are a social media virality analyst. Your job is to score video segments on their potential to go viral on platforms like TikTok, Instagram Reels, YouTube Shorts, and X/Twitter.
582
583	You will receive 2-3 frames from a video segment. Analyze them and score the segment on these factors (each 1-10):
584
585	1. Visual Dynamism (weight: 20%) — How visually interesting and dynamic is the content? Look for: action/movement implied between frames, striking compositions, dramatic lighting, vibrant colors, visual contrast. Low scores: static shots, bland compositions, poor lighting.
586	2. Emotional Impact (weight: 25%) — Does this segment evoke a strong emotional response? Look for: facial expressions, dramatic moments, humor potential, surprise elements, awe-inspiring visuals, relatable situations. This is the strongest predictor of shares.
587
588	3. Hook Potential (weight: 20%) — Would this segment work as the opening of a viral clip? Could it stop someone mid-scroll in the first 1-2 seconds? Look for: immediate visual intrigue, curiosity gap, unexpected elements, bold visuals.
589	4. Pacing & Energy (weight: 15%) — Does the visual pacing feel energetic and engaging? Look for: variety between frames (different angles, scales, subjects), implied motion, dynamic framing changes. Monotonous sameness across frames scores low.
590
591	5. Uniqueness (weight: 10%) — Is this something unusual, novel, or different from typical content? Everyday/generic scenes score low. Unusual situations, rare moments, or distinctive aesthetics score high.
592	6. Shareability (weight: 10%) — Would someone send this to a friend? Look for: "you have to see this" factor, relatable moments, impressive skills/feats, beautiful or shocking visuals.
593
594	Respond with JSON in this exact format:
595
596	{
597
598	"factors": {
599
600	"visualDynamism": <1-10>,
601
602	"emotionalImpact": <1-10>,
603
604	"hookPotential": <1-10>,
605
606	"pacingEnergy": <1-10>,
607
608	"uniqueness": <1-10>,
609
610	"shareability": <1-10>
611
612	},
613
614	"overallScore": <weighted average, 1-10, one decimal>,
615
616	"reasoning": "<2-3 sentences explaining why this segment scored the way it did>",
617
618	"suggestedCaption": "<a short, punchy caption that could accompany this clip on social media>"
619
620	}
621
622	Be honest and critical. Most segments in a typical video are NOT viral-worthy (score 3-5). Reserve scores above 7 for genuinely compelling moments. A score of 9-10 means this could realistically trend on social media.`;
623
624	```
625
626	#### Clip Scoring Prompt (Mode 2 — ranking ready-to-post clips)
627
628	Use this when scoring complete clips (10-60 seconds) that have been chunked from a longer video. This prompt adds narrative arc evaluation and is the preferred prompt for the chunking pipeline.
629
630	```typescript
631
632	const CLIP_SCORING_PROMPT = `You are a social media content strategist and virality analyst. You are evaluating complete video clips (10-60 seconds) that are ready to be posted on TikTok, Instagram Reels, YouTube Shorts, or X/Twitter.
633
634	You will receive 5-8 frames sampled across the entire clip, including the very first and very last frame. The first frame represents the hook (what viewers see first in their feed), and the last frame represents the payoff/ending.
635
636	Score this clip on these factors (each 1-10):
637
638	### Visual & Production Quality (35% total)
639
640	1. Visual Dynamism (weight: 13%) — How visually interesting and dynamic is the content across the full clip? Look for: variety of shots, movement, striking compositions, dynamic lighting, color. Score the overall visual journey, not just one frame.
641	2. Pacing & Energy (weight: 12%) — Does the clip maintain engaging pacing throughout? Look for: variety between frames (different angles, scales, subjects), good rhythm. Does it drag anywhere or rush past key moments?
642
643	3. Uniqueness (weight: 10%) — Is this clip distinctive? Would it stand out in a crowded feed? Everyday/generic content scores low. Fresh angles, unusual moments, or distinctive aesthetics score high.
644
645	### Emotional & Engagement Power (35% total)
646
647	4. Emotional Impact (weight: 20%) — Does this clip evoke a strong emotional response? This is the \#1 predictor of shares. Look for: relatable moments, humor, surprise, awe, empathy, or tension. A clip that makes you feel something deeply scores high.
648	5. Shareability (weight: 15%) — Would someone send this to a friend or repost it? Look for: "you have to see this" factor, quotable moments, impressive feats, beautiful or shocking visuals, relatable situations.
649
650	### Clip Structure (30% total)
651
652	6. Hook Strength (weight: 12%) — Look at the FIRST frame specifically. Would this stop someone mid-scroll in the first 1-2 seconds? Does the clip open with intrigue, a bold visual, or an attention-grabbing moment? A weak opening kills a clip regardless of how good the rest is.
653	7. Narrative Completeness (weight: 10%) — Does this clip tell a complete micro-story? Does it have a clear arc — a setup, development, and resolution (or intentional cliffhanger)? Would a viewer feel satisfied, intrigued, or moved at the end? Clips that end abruptly mid-thought or feel like random fragments score low.
654
655	8. Standalone Quality (weight: 8%) — Would this clip make sense to someone who has never seen the full video? Can it be posted without any additional context or explanation? Does it work as an independent piece of content?
656
657	Respond with JSON in this exact format:
658
659	{
660
661	"factors": {
662
663	"visualDynamism": <1-10>,
664
665	"pacingEnergy": <1-10>,
666
667	"uniqueness": <1-10>,
668
669	"emotionalImpact": <1-10>,
670
671	"shareability": <1-10>,
672
673	"hookStrength": <1-10>,
674
675	"narrativeCompleteness": <1-10>,
676
677	"standaloneQuality": <1-10>
678
679	},
680
681	"overallScore": <weighted average using the weights above, 1-10, one decimal>,
682
683	"reasoning": "<2-3 sentences explaining this clip's strengths and weaknesses as a standalone social media post>",
684
685	"suggestedCaption": "<a short, punchy caption optimized for the platform>",
686
687	"postingRecommendation": "<one of: 'post as-is', 'post with minor edits', 'consider skipping'>"
688
689	}
690
691	Be honest and critical. A mediocre clip is worse than no post at all — it hurts the algorithm. Most clips from a chunked video will score 3-5. Scores above 7 mean this clip could genuinely perform well. Scores 8+ mean this is strong enough to lead a content calendar.`;
692
693	```
694
695	#### Choosing the Right Prompt
696
697	```typescript
698
699	function getScoringPrompt(durationSeconds: number): string {
700
701	return durationSeconds >= 10 ? CLIP_SCORING_PROMPT : SEGMENT_SCORING_PROMPT;
702
703	}
704
705	```
706
707	Use `SEGMENT_SCORING_PROMPT`for scene-detected segments under 10 seconds (Mode 1: finding best moments). Use`CLIP_SCORING_PROMPT`for clips 10 seconds or longer (Mode 2: ranking chunks for posting).
708
709	---
710
711	## Step 4: Scoring and Ranking
712
713	After all segments are analyzed, rank them by overall score.
714
715	Important: Each segment requires an AI API call with image attachments. For a 90-second video with ~13 segments, expect the analysis to take 1-2 minutes. For longer videos (5+ minutes), this can take much longer. Always warn the user that analysis will take a moment.
716
717	If the`@workspace/integrations-openai-ai-server/batch` utilities are available, use them for rate limiting. Otherwise, a simple sequential loop with try/catch works well:
718
719	```typescript
720
721	async function scoreAllSegments(
722
723	videoPath: string,
724
725	segments: Segment[],
726
727	videoContext: string,
728
729	tmpDir: string
730
731	): Promise<ViralityScore[]> {
732
733	const scores: ViralityScore[] = [];
734
735	for (const segment of segments) {
736
737	const framePaths = await extractFrames(videoPath, segment, tmpDir);
738
739	try {
740
741	const score = await analyzeSegment(framePaths, segment, videoContext);
742
743	scores.push(score);
744
745	} catch (err: any) {
746
747	if (err?.code === 'content_policy_violation') {
748
749	console.log(`Segment ${segment.index + 1}: Skipped (content safety filter)`);
750
751	} else {
752
753	console.log(`Segment ${segment.index + 1}: Error — ${err?.message}`);
754
755	}
756
757	}
758
759	for (const fp of framePaths) {
760
761	await fs.promises.unlink(fp).catch(() => {});
762
763	}
764
765	}
766
767	return scores.sort((a, b) => b.overallScore - a.overallScore);
768
769	}
770
771	```
772
773	---
774
775	## Step 5: Output Assembly
776
777	Based on the user's choice, produce one of three outputs.
778
779	### Option A: Best Clip (single highest-scoring segment)
780
781	```typescript
782
783	async function exportBestClip(
784
785	videoPath: string,
786
787	segments: Segment[],
788
789	scores: ViralityScore[],
790
791	outputPath: string
792
793	): Promise<{ path: string; score: ViralityScore }> {
794
795	const best = scores[0];
796
797	const segment = segments[best.segmentIndex];
798
799	await execFileAsync('ffmpeg', [
800
801	'-y',
802
803	'-i', videoPath,
804
805	'-ss', segment.startTime.toFixed(3),
806
807	'-to', segment.endTime.toFixed(3),
808
809	'-c:v', 'libx264', '-crf', '23', '-preset', 'medium',
810
811	'-c:a', 'aac', '-b:a', '128k',
812
813	outputPath
814
815	]);
816
817	return { path: outputPath, score: best };
818
819	}
820
821	```
822
823	### Option B: Multiple Clips (ranked by score)
824
825	```typescript
826
827	async function exportTopClips(
828
829	videoPath: string,
830
831	segments: Segment[],
832
833	scores: ViralityScore[],
834
835	outputDir: string,
836
837	count = 5
838
839	): Promise<Array<{ path: string; score: ViralityScore }>> {
840
841	await fs.promises.mkdir(outputDir, { recursive: true });
842
843	const topScores = scores.slice(0, count);
844
845	const results: Array<{ path: string; score: ViralityScore }> = [];
846
847	for (let i = 0; i < topScores.length; i++) {
848
849	const score = topScores[i];
850
851	const segment = segments[score.segmentIndex];
852
853	const outputPath = path.join(outputDir, `clip_${i + 1}_score_${score.overallScore.toFixed(1)}.mp4`);
854
855	await execFileAsync('ffmpeg', [
856
857	'-y',
858
859	'-i', videoPath,
860
861	'-ss', segment.startTime.toFixed(3),
862
863	'-to', segment.endTime.toFixed(3),
864
865	'-c:v', 'libx264', '-crf', '23', '-preset', 'medium',
866
867	'-c:a', 'aac', '-b:a', '128k',
868
869	outputPath
870
871	]);
872
873	results.push({ path: outputPath, score });
874
875	}
876
877	return results;
878
879	}
880
881	```
882
883	### Option C: Highlight Reel (top moments stitched together)
884
885	```typescript
886
887	async function exportHighlightReel(
888
889	videoPath: string,
890
891	segments: Segment[],
892
893	scores: ViralityScore[],
894
895	outputPath: string,
896
897	topCount = 5,
898
899	transitionDuration = 0.5
900
901	): Promise<string> {
902
903	const topScores = scores.slice(0, topCount);
904
905	const sortedByTime = [...topScores].sort(
906
907	(a, b) => segments[a.segmentIndex].startTime - segments[b.segmentIndex].startTime
908
909	);
910
911	const tmpDir = path.join(path.dirname(outputPath), 'tmp_highlight');
912
913	await fs.promises.mkdir(tmpDir, { recursive: true });
914
915	const clipPaths: string[] = [];
916
917	for (let i = 0; i < sortedByTime.length; i++) {
918
919	const segment = segments[sortedByTime[i].segmentIndex];
920
921	const clipPath = path.join(tmpDir, `clip_${i}.mp4`);
922
923	await execFileAsync('ffmpeg', [
924
925	'-y',
926
927	'-i', videoPath,
928
929	'-ss', segment.startTime.toFixed(3),
930
931	'-to', segment.endTime.toFixed(3),
932
933	'-c:v', 'libx264', '-crf', '23', '-preset', 'medium',
934
935	'-c:a', 'aac', '-b:a', '128k',
936
937	clipPath
938
939	]);
940
941	clipPaths.push(clipPath);
942
943	}
944
945	if (clipPaths.length === 1) {
946
947	await fs.promises.copyFile(clipPaths[0], outputPath);
948
949	} else {
950
951	const filterParts: string[] = [];
952
953	let lastVideo = '';
954
955	let lastAudio = '';
956
957	for (let i = 0; i < clipPaths.length; i++) {
958
959	filterParts.push(`[${i}:v]setpts=PTS-STARTPTS[v${i}];`);
960
961	filterParts.push(`[${i}:a]asetpts=PTS-STARTPTS[a${i}];`);
962
963	}
964
965	// Chain xfade transitions between clips
966
967	if (clipPaths.length === 2) {
968
969	filterParts.push(
970
971	`[v0][v1]xfade=transition=fade:duration=${transitionDuration}:offset=auto[outv];`
972
973	);
974
975	filterParts.push(
976
977	`[a0][a1]acrossfade=d=${transitionDuration}[outa]`
978
979	);
980
981	lastVideo = 'outv';
982
983	lastAudio = 'outa';
984
985	} else {
986
987	// For 3+ clips, use concat with crossfade between each pair
988
989	const concatList = path.join(tmpDir, 'concat.txt');
990
991	const lines = clipPaths.map(p => `file '${p}'`).join('\n');
992
993	await fs.promises.writeFile(concatList, lines);
994
995	await execFileAsync('ffmpeg', [
996
997	'-y',
998
999	'-f', 'concat', '-safe', '0',
1000
1001	'-i', concatList,
1002
1003	'-c:v', 'libx264', '-crf', '23', '-preset', 'medium',
1004
1005	'-c:a', 'aac', '-b:a', '128k',
1006
1007	outputPath
1008
1009	]);
1010
1011	// Clean up
1012
1013	for (const cp of clipPaths) await fs.promises.unlink(cp).catch(() => {});
1014
1015	await fs.promises.rm(tmpDir, { recursive: true, force: true });
1016
1017	return outputPath;
1018
1019	}
1020
1021	const inputs = clipPaths.flatMap(p => ['-i', p]);
1022
1023	await execFileAsync('ffmpeg', [
1024
1025	'-y',
1026
1027	...inputs,
1028
1029	'-filter_complex', filterParts.join(''),
1030
1031	'-map', `[${lastVideo}]`,
1032
1033	'-map', `[${lastAudio}]`,
1034
1035	'-c:v', 'libx264', '-crf', '23', '-preset', 'medium',
1036
1037	'-c:a', 'aac', '-b:a', '128k',
1038
1039	outputPath
1040
1041	]);
1042
1043	}
1044
1045	// Clean up
1046
1047	for (const cp of clipPaths) await fs.promises.unlink(cp).catch(() => {});
1048
1049	await fs.promises.rm(tmpDir, { recursive: true, force: true });
1050
1051	return outputPath;
1052
1053	}
1054
1055	```
1056
1057	---
1058
1059	## Complete Pipeline Example
1060
1061	Here's how to tie everything together in a script:
1062
1063	```typescript
1064
1065	import * as path from 'path';
1066
1067	import * as os from 'os';
1068
1069	async function viralityAutoTrim(
1070
1071	videoPath: string,
1072
1073	outputMode: 'best' \| 'multiple' \| 'highlight',
1074
1075	videoContext = ''
1076
1077	) {
1078
1079	const tmpDir = path.join(os.tmpdir(), `virality_${Date.now()}`);
1080
1081	console.log('Step 1: Detecting scenes...');
1082
1083	let segments = await detectScenes(videoPath);
1084
1085	// Fall back to fixed intervals if scene detection yields too few segments
1086
1087	if (segments.length < 3) {
1088
1089	console.log('Few scenes detected, using fixed-interval segmentation...');
1090
1091	segments = await splitFixed(videoPath, 10);
1092
1093	}
1094
1095	console.log(`Found ${segments.length} segments`);
1096
1097	console.log('Step 2: Analyzing segments with AI...');
1098
1099	const scores = await scoreAllSegments(videoPath, segments, videoContext, tmpDir);
1100
1101	console.log('\nSegment Rankings:');
1102
1103	for (const score of scores) {
1104
1105	const seg = segments[score.segmentIndex];
1106
1107	console.log(
1108
1109	`\#${score.segmentIndex + 1} [${formatTime(seg.startTime)}-${formatTime(seg.endTime)}]` +
1110
1111	`Score: ${score.overallScore.toFixed(1)}/10 — ${score.reasoning}`
1112
1113	);
1114
1115	}
1116
1117	console.log(`\nStep 3: Generating output (mode: ${outputMode})...`);
1118
1119	const outputDir = path.dirname(videoPath);
1120
1121	switch (outputMode) {
1122
1123	case 'best': {
1124
1125	const { path: outPath, score } = await exportBestClip(
1126
1127	videoPath, segments, scores,
1128
1129	path.join(outputDir, 'best_clip.mp4')
1130
1131	);
1132
1133	console.log(`Best clip exported: ${outPath} (score: ${score.overallScore.toFixed(1)})`);
1134
1135	console.log(`Suggested caption: ${score.suggestedCaption}`);
1136
1137	break;
1138
1139	}
1140
1141	case 'multiple': {
1142
1143	const clips = await exportTopClips(
1144
1145	videoPath, segments, scores,
1146
1147	path.join(outputDir, 'top_clips')
1148
1149	);
1150
1151	console.log(`Exported ${clips.length} clips to ${path.join(outputDir, 'top_clips')}/`);
1152
1153	for (const clip of clips) {
1154
1155	console.log(` ${path.basename(clip.path)} — ${clip.score.reasoning}`);
1156
1157	}
1158
1159	break;
1160
1161	}
1162
1163	case 'highlight': {
1164
1165	const reelPath = await exportHighlightReel(
1166
1167	videoPath, segments, scores,
1168
1169	path.join(outputDir, 'highlight_reel.mp4')
1170
1171	);
1172
1173	console.log(`Highlight reel exported: ${reelPath}`);
1174
1175	break;
1176
1177	}
1178
1179	}
1180
1181	// Clean up temp directory
1182
1183	await fs.promises.rm(tmpDir, { recursive: true, force: true });
1184
1185	console.log('Done!');
1186
1187	}
1188
1189	```
1190
1191	## Scoring Calibration Notes
1192
1193	- Most content scores 3-5. This is normal. Everyday footage, filler shots, and transitional moments are not viral.
1194	- Scores 6-7 indicate genuinely interesting moments worth keeping.
1195
1196	- Scores 8+ are rare and represent truly compelling, share-worthy content.
1197	- If every segment scores above 7, the entire video is likely high-quality content — suggest keeping more segments or using a longer highlight reel.
1198
1199	- If no segment scores above 4, be honest with the user — the video may not have strong viral moments. Suggest what would make it more engaging (better lighting, more action, stronger hook, etc.).
1200
1201	## Performance Tips
1202
1203	- Timeout awareness — For a 90-second video with ~13 segments, expect analysis to take 1-2 minutes. A 5-minute video with fixed 10s segments = 30 API calls = 3-5 minutes. Warn the user and set generous script timeouts.
1204	- For long videos (30+ minutes), use fixed-interval segmentation with larger chunks (15-20s) to keep the number of AI calls manageable.
1205
1206	- Use `concurrency: 2` in batch processing to avoid rate limits while maintaining speed.
1207	- Extract frames at reduced quality (`-q:v 5`) if API costs are a concern — the AI can still analyze lower-quality frames effectively.
1208
1209	- Use `detail: 'low'` for OpenAI vision to reduce token usage per frame.
1210	- Content safety skips — Expect some segments to be skipped due to content safety filters, especially with documentary, medical, or news content. The pipeline should gracefully continue.
1211
1212	- Minimum segment duration — Use 2.0s as the minimum segment threshold (not 1.0s). Very short segments (< 2s) don't produce useful virality scores and waste API calls.