SKILL.md · jhustle44/jcr-guide-pro

Code Commits Issues Pulls Cloud Agent CI/CD

copilot/create-zip-file-again

copilot/add-firebase copilot/change-app-name-to-jcrguru copilot/create-zip-file copilot/create-zip-file-again copilot/merge-any-updates main master

md 705 lines 26 KB

Rendered Raw

1	---
2	name: video-editing
3	description: Edit video files with FFmpeg — trim, merge, add subtitles, effects, voiceovers, and reformat.
4	---
5
6	# Video Editing
7
8	Edit existing video files server-side using FFmpeg. This skill covers the full editing workflow: trimming, splicing, concatenating, transitions, text overlays, subtitles, audio mixing, effects, and format conversion.
9
10	## When to Use
11
12	### Trimming & Cutting
13
14	- "Trim this video to 0:30-1:15" / "Cut out the first 10 seconds"
15	- "Extract a clip from 2:00 to 3:30"
16
17	- "Remove the intro" / "Cut the ending"
18
19	#### Merging & Compositing
20
21	- "Merge these clips together" / "Concatenate these videos"
22	- "Put two videos side by side" / "Split screen"
23
24	- "Picture-in-picture"
25
26	##### Text, Titles & Subtitles
27
28	- "Add subtitles" / "Add captions"
29	- "Put a title card at the beginning"
30
31	- "Add text overlay"
32
33	###### Audio
34
35	- "Add background music" / "Mix in this audio track"
36	- "Add a voiceover" / "Narrate this video" (uses ElevenLabs)
37
38	- "Mute this video" / "Remove the audio"
39	- "Replace the audio with this track"
40
41	###### Speed & Motion
42
43	- "Speed up this video 2x" / "Slow motion"
44	- "Make this part slow-mo"
45
46	- "Reverse this clip"
47	- "Loop this clip 3 times"
48
49	###### Visual Effects & Fixes
50
51	- "Rotate this video 90 degrees" / "Flip horizontally"
52	- "Stabilize this shaky footage"
53
54	- "Crop out the watermark" / "Remove the watermark"
55	- "Add my logo to this video" (image overlay / branding)
56
57	- "Color correct" / "Make it brighter" / "Adjust contrast"
58
59	###### Format & Size
60
61	- "Convert this MP4 to WebM" / "Change format"
62	- "Compress this video" / "Make this file smaller"
63
64	- "Resize to 720p" / "Downscale" / "Change resolution"
65	- "Extract the audio as MP3"
66
67	- "Create a GIF from this clip"
68	- "Make a thumbnail from this video" (frame extraction as image)
69
70	###### Social Media & Platform
71
72	- "Make TikTok clips from this" / "Break this into Reels"
73	- "Make this vertical for Shorts"
74
75	- "Reframe for Instagram" / "LinkedIn" / "Pinterest" / "Facebook" / "Snapchat"
76	- "Convert this to square for Instagram feed"
77
78	###### AI-Powered Analysis
79
80	- "Find the best moments" / "Auto-trim for virality"
81	- "Extract the highlights" / "Score these clips"
82
83	- "Which parts are most engaging?"
84
85	###### Cleanup & Optimization
86
87	- "Remove dead space" / "Remove silence"
88	- "Shorten this by cutting out boring parts"
89
90	- "Tighten up this video"
91
92	###### Chunking & Splitting
93
94	- "Split this into short clips" / "Chunk this for TikTok"
95	- "Break this talk into Reels-length videos"
96
97	- "Make 15-second clips from this"
98
99	###### No Direction (user just uploads a video)
100
101	- Probe the file, report what it is, and ask what they want to do
102
103	## When NOT to Use
104
105	- User wants to create an animated video from scratch using code (use the `video-js` skill instead)
106	- User wants a browser-based video editor UI (build as a `react-vite` artifact with this skill powering the backend)
107
108	## User Interaction — ALWAYS Ask Before Acting
109
110	When a user uploads a video without specific instructions, you MUST ask what they want to do with it. Do NOT assume a workflow or start processing automatically.
111
112	Probe the video first (to report basic info), then ask the user what they'd like. Here's a good pattern:
113
114	1. Probe the file and tell the user what they uploaded (duration, resolution, format)
115	2. Ask what they want to do. Offer common options like:
116
117	- Trim or cut specific sections
118	- Find the best/most viral moments
119
120	- Break it into short clips for a specific platform (TikTok, Reels, Shorts, X)
121	- Remove dead space / silence
122
123	- Add text, subtitles, or a voiceover
124	- Reframe for a different aspect ratio
125
126	- Convert to a different format
127	- Something else
128
129	When the user specifies a platform, the full pipeline includes reframing to that platform's native format. Do not just chunk the video and leave it in the original aspect ratio. Always deliver platform-ready output — correct aspect ratio, resolution, and duration range.
130
131	### Platform Specifications
132
133	\| Platform \| Aspect Ratio \| Resolution \| Duration Range \| Ideal Length \| Max File Size \|
134
135	\|----------\|-------------\|------------\|---------------\|-------------\|--------------\|
136
137	\| TikTok \| 9:16 vertical \| 1080×1920 \| 10–45s \| 25s \| 287 MB \|
138
139	\| Instagram Reels \| 9:16 vertical \| 1080×1920 \| 10–30s \| 20s \| 250 MB \|
140
141	\| Instagram Stories \| 9:16 vertical \| 1080×1920 \| 1–60s \| 15s \| 250 MB \|
142
143	\| Instagram Feed \| 1:1 square or 4:5 portrait \| 1080×1080 or 1080×1350 \| 3–60s \| 30s \| 250 MB \|
144
145	\| YouTube Shorts \| 9:16 vertical \| 1080×1920 \| 15–60s \| 40s \| 256 MB \|
146
147	\| YouTube (standard) \| 16:9 landscape \| 1920×1080 \| any \| any \| 256 GB \|
148
149	\| X / Twitter \| 16:9 landscape \| 1280×720 \| 10–45s \| 25s \| 512 MB \|
150
151	\| X / Twitter (square) \| 1:1 square \| 720×720 \| 10–45s \| 25s \| 512 MB \|
152
153	\| Facebook Reels \| 9:16 vertical \| 1080×1920 \| 10–30s \| 20s \| 250 MB \|
154
155	\| Facebook Feed \| 16:9 or 1:1 \| 1920×1080 or 1080×1080 \| any \| 15–60s \| 4 GB \|
156
157	\| LinkedIn \| 16:9 landscape or 1:1 \| 1920×1080 or 1080×1080 \| 10–60s \| 30s \| 5 GB \|
158
159	\| Pinterest \| 9:16 or 2:3 vertical \| 1080×1920 or 1000×1500 \| 6–60s \| 15–30s \| 2 GB \|
160
161	\| Snapchat Spotlight \| 9:16 vertical \| 1080×1920 \| 5–60s \| 15–30s \| 250 MB \|
162
163	When the user says a platform name, apply both the correct duration targets AND the correct aspect ratio/resolution. For example:
164
165	- "Make TikTok clips" → chunk to 10-45s AND reframe to 9:16 (1080×1920)
166	- "Instagram feed posts" → chunk to 3-60s AND reframe to 1:1 (1080×1080) or 4:5 (1080×1350)
167
168	- "YouTube Shorts" → chunk to 15-60s AND reframe to 9:16 (1080×1920)
169	- "Clips for X" → chunk to 10-45s, keep 16:9 (1280×720)
170
171	When the user asks for clips or chunking without specifying a platform, ask which platform they're targeting so you can apply the right duration targets and reframing.
172
173	When you deliver results, always present the output files so the user can download and preview them. Never just describe what was done — show the files.
174
175	## Environment
176
177	FFmpeg 6.1.2 is pre-installed in Replit with full codec support:
178
179	- Video codecs: H.264 (libx264), H.265 (libx265), VP8/VP9 (libvpx), AV1 (libaom, libsvtav1, librav1e), Theora
180	- Audio codecs: AAC, MP3 (libmp3lame), Opus, Vorbis, FLAC, WAV
181
182	- Image formats: PNG, JPEG, WebP, JPEG XL (libjxl)
183	- Subtitle rendering: libass (for styled subtitles), libfreetype + libfontconfig (for drawtext)
184
185	- Filters: All standard filters available — scale, crop, overlay, drawtext, concat, fade, crossfade, etc.
186	- Tools: `ffmpeg`,`ffprobe` (both available on PATH)
187
188	No additional installation is needed for FFmpeg itself.
189
190	## Architecture Decisions
191
192	### Script-based editing (simple tasks)
193
194	For one-off or single-operation tasks (trim a clip, convert a format, extract audio), write a Node.js script in `scripts/src/`using`fluent-ffmpeg`or direct`child_process.execFile` calls to FFmpeg.
195
196	```bash
197
198	pnpm --filter @workspace/scripts add fluent-ffmpeg
199
200	pnpm --filter @workspace/scripts add -D @types/fluent-ffmpeg
201
202	```
203
204	### API-based editing (complex workflows or UI)
205
206	For multi-step editing workflows or when building a video editor UI, add routes to the API server (`artifacts/api-server/`) that accept video files, process them with FFmpeg, and return results.
207
208	```bash
209
210	pnpm --filter @workspace/api-server add fluent-ffmpeg multer
211
212	pnpm --filter @workspace/api-server add -D @types/fluent-ffmpeg @types/multer
213
214	```
215
216	### Choosing the right approach
217
218	\| Scenario \| Approach \|
219
220	\|----------\|----------\|
221
222	\| "Trim this video to 0:30-1:15" \| Script in `scripts/src/` \|
223
224	\| "Convert this MP4 to WebM" \| Script in `scripts/src/` \|
225
226	\| "Build me a video editor app" \| React-vite artifact + API routes \|
227
228	\| "Add subtitles to all my videos" \| Script in `scripts/src/` \|
229
230	\| "Merge these 5 clips with transitions" \| Script in `scripts/src/` (or API if repeated) \|
231
232	## Instructions
233
234	### 1. Probe Before Processing
235
236	Always probe the input file first to understand its properties. This prevents errors from wrong assumptions about codecs, resolution, frame rate, or duration.
237
238	```typescript
239
240	import { execFile } from 'child_process';
241
242	import { promisify } from 'util';
243
244	const execFileAsync = promisify(execFile);
245
246	async function probeVideo(filePath: string) {
247
248	const { stdout } = await execFileAsync('ffprobe', [
249
250	'-v', 'quiet',
251
252	'-print_format', 'json',
253
254	'-show_format',
255
256	'-show_streams',
257
258	filePath
259
260	]);
261
262	return JSON.parse(stdout);
263
264	}
265
266	```
267
268	### 2. Use fluent-ffmpeg for Complex Operations
269
270	fluent-ffmpeg provides a chainable API that is easier to maintain than raw command strings. Use it for multi-step operations.
271
272	```typescript
273
274	import ffmpeg from 'fluent-ffmpeg';
275
276	function trimVideo(input: string, output: string, start: string, duration: string): Promise<void> {
277
278	return new Promise((resolve, reject) => {
279
280	ffmpeg(input)
281
282	.setStartTime(start)
283
284	.setDuration(duration)
285
286	.output(output)
287
288	.on('end', resolve)
289
290	.on('error', reject)
291
292	.run();
293
294	});
295
296	}
297
298	```
299
300	### 3. Use execFile for Simple One-Liners
301
302	For straightforward operations where the FFmpeg command is well-known, calling FFmpeg directly is cleaner.
303
304	```typescript
305
306	await execFileAsync('ffmpeg', [
307
308	'-i', inputPath,
309
310	'-vf', 'scale=1280:720',
311
312	'-c:a', 'copy',
313
314	outputPath
315
316	]);
317
318	```
319
320	### 4. File Management
321
322	- Store uploaded/input files in a `tmp/`or`uploads/` directory
323	- Store output files where the user can access them — either `public/` for web serving or a known output directory
324
325	- Clean up temporary files after processing
326	- For large files, stream the output rather than buffering in memory
327
328	### 5. Progress Reporting
329
330	FFmpeg outputs progress to stderr. Parse it for progress reporting in longer operations:
331
332	```typescript
333
334	ffmpeg(input)
335
336	.output(output)
337
338	.on('progress', (progress) => {
339
340	console.log(`Processing: ${progress.percent?.toFixed(1)}% done`);
341
342	})
343
344	.on('end', () => console.log('Done'))
345
346	.run();
347
348	```
349
350	### 6. Error Handling
351
352	FFmpeg errors are often cryptic. Always:
353
354	- Log the full stderr output on failure
355	- Validate input files exist before processing
356
357	- Check that output codecs are compatible with the output container
358	- Use `-y` flag to overwrite output files without prompting
359
360	## AI-Powered Virality Scoring
361
362	Use AI vision to analyze video content and score it on viral potential. This feature has two modes depending on the workflow.
363
364	This feature requires an AI integration (OpenAI or Gemini) to be set up. Set up the AI integration following the `ai-integrations-openai`or`ai-integrations-gemini` skill before proceeding. OpenAI is preferred because gpt-5.2 handles image analysis well and is the default for most tasks. Gemini (gemini-2.5-flash) is a good alternative, especially for high-volume analysis since it supports video/image input natively.
365
366	### Two Scoring Modes
367
368	Mode 1: Segment scoring (find the best moments) — For when the user says "find the best clips" or "auto-trim for virality." Detects scenes, scores small segments, and exports the top moments. Good for finding highlights in raw footage.
369
370	#### Mode 2: Clip scoring (rank ready-to-post clips)— For when clips already exist (from chunking, manual trimming, etc.) and the user wants to know which ones to post first. Scores each complete clip with a richer evaluation including narrative arc and standalone quality.This is the preferred mode when used with the chunking pipeline
371
372	### Recommended Pipeline
373
374	When the user has a long video and wants social-ready clips, use this order:
375
376	```text
377
378	Long Video → Dead Space Removal → Chunking (10-60s clips) → Clip-Level Scoring → Reframe for Platform → Output
379
380	```
381
382	This way the AI scores complete, self-contained clips rather than tiny scene fragments. The scores are much more meaningful because each clip is what will actually be posted.
383
384	For "find the best moment in this video" requests (no chunking), use the original segment-level pipeline:
385
386	```text
387
388	Video → Scene Detection → Frame Extraction → Segment Scoring → Rank → Trim/Export
389
390	```
391
392	### When the user asks for auto-trim, ask which output they want
393
394	Present these three options and let the user choose each time. Do not assume a default:
395
396	- Best clip — The single highest-scoring segment/clip, trimmed and exported
397	- Multiple clips — Several top segments/clips exported as separate files, ranked by score
398
399	- Highlight reel — Top moments stitched together with crossfade transitions into one video
400
401	### Scoring Criteria
402
403	Segment scoring evaluates: visual dynamism, emotional impact, hook potential, pacing/energy, uniqueness, and shareability. Each factor gets a 1-10 score, weighted to produce a final virality score.
404
405	Clip scoring adds three additional criteria for complete clips:
406
407	- Narrative completeness (weight: 15%) — Does the clip tell a complete micro-story? Does it have a clear beginning, middle, and end? Would a viewer feel satisfied or intrigued, not confused?
408	- Hook-to-payoff flow (weight: 10%) — Does the clip open with something that grabs attention and deliver on that promise? Or does it start slow and meander?
409
410	- Standalone quality (weight: 10%) — Would this clip make sense to someone who hasn't seen the full video? Can it be posted without context?
411
412	These three criteria reduce the weights of the original six factors proportionally so the total still sums to 100%.
413
414	### Frame Extraction
415
416	- Segments (2-10s): Extract 3 frames evenly spread across the segment.
417	- Clips (10-60s): Extract 5-8 frames to capture the full arc. Always include the first frame (hook evaluation) and last frame (payoff evaluation).
418
419	### Workflow
420
421	For the complete step-by-step implementation with code examples, see:
422
423	- `virality-scoring.md` — Full virality analysis pipeline, AI prompts, scoring criteria, and output assembly
424
425	### Key Gotchas (learned from testing)
426
427	- Scene detection: Use `ffmpeg`with`select`+`showinfo`filters and parse stderr — the`ffprobe -f lavfi` approach doesn't work reliably in Replit.
428	- Content safety: OpenAI vision may reject frames from documentary/medical/news content. Always wrap AI calls in try/catch and skip failed segments gracefully.
429
430	- Segment indexing: Use `index: segments.length`when filtering short segments, not the loop counter`i`.
431	- Package setup: For scripts, install `openai`directly (`pnpm add -w openai`) and create the client with`AI_INTEGRATIONS_OPENAI_BASE_URL`/`AI_INTEGRATIONS_OPENAI_API_KEY` env vars. No need for the full workspace integration library.
432
433	- Timeouts: A 90-second video with ~13 segments takes 1-2 minutes for AI analysis. Warn the user and set generous timeouts.
434	- Minimum segment duration: Use 2.0s minimum — segments under 2 seconds waste API calls and don't produce useful scores.
435
436	- Prefer clip-level scoring: When the video has already been chunked, always score the chunks rather than re-running segment detection. The clips are what will actually be posted.
437
438	## Dead Space Removal
439
440	Automatically detect and remove silence, dead air, and filler from a video to produce a tighter, more engaging cut. This is especially useful for talks, interviews, podcasts, tutorials, and raw footage.
441
442	### How It Works
443
444	1. Silence detection — Use FFmpeg's `silencedetect` filter to find all silent intervals (configurable threshold and minimum duration)
445	2. Segment extraction — Extract all non-silent segments as individual clips
446
447	3. Reassembly — Concatenate the non-silent segments back together with optional brief crossfade transitions
448	4. Optional: AI-assisted filler removal — For more aggressive editing (removing "ums", filler words, repetitive sections), combine silence detection with AI transcription analysis
449
450	### When the user asks to remove dead space
451
452	Ask these clarifying questions if not clear from context:
453
454	- How aggressive? Light (remove only true silence, > 1s gaps) vs. aggressive (remove short pauses > 0.3s for a fast-paced edit)
455	- Transitions? Hard cuts between segments (default) or brief crossfades (0.2-0.5s) for smoother flow
456
457	### Configuration
458
459	\| Preset \| Silence Threshold \| Min Silence Duration \| Use Case \|
460
461	\|--------\|------------------\|---------------------\|----------\|
462
463	\| Light \| -40dB \| 1.0s \| Talks, interviews — remove obvious dead air \|
464
465	\| Medium \| -35dB \| 0.5s \| Podcasts, tutorials — tighter pacing \|
466
467	\| Aggressive \| -30dB \| 0.3s \| Fast-paced edits, social media — maximum tightness \|
468
469	### Implementation
470
471	For the complete FFmpeg commands and Node.js pipeline, see:
472
473	- `dead-space-and-chunking.md` — See the "Dead Space Removal" section
474
475	### Quick Summary
476
477	```text
478
479	Video → Silence Detection (FFmpeg silencedetect) → Identify Non-Silent Ranges → Extract Segments → Concatenate → Output
480
481	```
482
483	## Social Media Chunking
484
485	Break a longer video (ad, promo, talk, presentation, livestream) into self-contained clips suitable for TikTok, Reels, and Shorts. Each clip should be 10-60 seconds and feel complete on its own.
486
487	### How It Works (2)
488
489	1. Scene detection + silence detection — Find natural break points using both visual scene changes and audio silence gaps
490	2. Smart boundary selection — Merge adjacent segments into clips targeting 10-60 seconds, preferring to break at silence or scene changes rather than mid-sentence
491
492	3. AI content analysis (optional) — Score each potential clip with AI to identify which ones are worth posting, suggest captions, and recommend clip order
493	4. Export — Output each clip as a separate file, numbered and ranked
494
495	### Clip Length Guidelines
496
497	\| Platform \| Ideal Length \| Max Length \| Notes \|
498
499	\|----------\|-------------\|------------\|-------\|
500
501	\| TikTok \| 15-45s \| 10 min \| 15-30s performs best for new accounts \|
502
503	\| Instagram Reels \| 15-30s \| 90s \| Under 30s gets more reach \|
504
505	\| YouTube Shorts \| 30-60s \| 60s \| Hard cap at 60 seconds \|
506
507	\| X / Twitter \| 15-45s \| 2m 20s \| Shorter gets more engagement \|
508
509	### When the user asks to chunk a video
510
511	Ask which platform they're targeting (or default to TikTok at 15-45s). The chunking algorithm should:
512
513	- Never cut mid-sentence — Always break at silence gaps or natural pauses
514	- Prefer scene boundaries — Break where the visual content changes
515
516	- Ensure each clip stands alone — Each clip should have a clear beginning, not start mid-thought
517	- Target the platform's sweet spot — Not just "under 60s" but actually the ideal range for that platform
518
519	### Implementation (2)
520
521	For the complete pipeline with code examples, see:
522
523	- `dead-space-and-chunking.md` — See the "Social Media Chunking" section
524
525	### Quick Summary (2)
526
527	```text
528
529	Video → Scene Detection + Silence Detection → Merge into 10-60s Clips at Natural Boundaries → (Optional) AI Scoring → Export Individual Clips
530
531	```
532
533	### Combining with Other Features
534
535	These features work well together in a pipeline:
536
537	```text
538
539	Long Video → Dead Space Removal → Chunking into Clips → Virality Scoring → Reframe for TikTok → Output
540
541	```
542
543	This takes a raw long-form video and produces polished, platform-ready social clips end to end.
544
545	## Voiceovers with ElevenLabs
546
547	Add AI-generated voiceovers to videos using ElevenLabs text-to-speech, then mix the audio into the video with FFmpeg.
548
549	### Prerequisites
550
551	ElevenLabs is available as a Replit integration (connector). The user needs to connect their ElevenLabs account through Replit's integration system — no manual API keys needed. Search for the integration and propose it to the user:
552
553	```javascript
554
555	const results = await searchIntegrations("elevenlabs");
556
557	// Then propose the connector so the user can authorize
558
559	await proposeIntegration("connector:ccfg_elevenlabs_...");
560
561	```
562
563	After authorization, use `addIntegration`to wire it to the project, then use`listConnections('elevenlabs')` in the code execution sandbox to get the credentials.
564
565	### Workflow (2)
566
567	1. Generate voiceover audio — Send text to ElevenLabs TTS API, receive audio file
568	2. Mix into video — Use FFmpeg to add the voiceover as a new audio track, optionally ducking (lowering volume of) existing audio
569
570	### Voiceover Modes
571
572	When the user asks for a voiceover, they may want one of these:
573
574	- Replace audio — Remove existing audio entirely, use only the voiceover
575	- Mix over — Layer voiceover on top of existing audio (with existing audio volume reduced)
576
577	- Add as track — Keep existing audio at full volume, add voiceover on top
578
579	Ask which mode the user prefers if not clear from context.
580
581	### Implementation (3)
582
583	For the complete step-by-step implementation with code examples, see:
584
585	- `voiceover.md` — ElevenLabs TTS integration, voice selection, audio mixing with FFmpeg, and timed voiceover segments
586
587	### Quick Summary (3)
588
589	```text
590
591	Text Script → ElevenLabs TTS → Audio File → FFmpeg Mix with Video → Output
592
593	```
594
595	Key capabilities:
596
597	- Choose from ElevenLabs' voice library or cloned voices
598	- Control voice settings (stability, similarity boost, style)
599
600	- Generate voiceover for the full video or specific time segments
601	- Duck existing audio under voiceover sections
602
603	- Support for multiple languages
604
605	## Social Media Reframing
606
607	Automatically reframe videos for different social media platforms. Each platform has a preferred aspect ratio and resolution — this feature handles the conversion intelligently.
608
609	### Platform Specs
610
611	\| Platform \| Aspect Ratio \| Resolution \| Notes \|
612
613	\|----------\|-------------\|------------\|-------\|
614
615	\| YouTube \| 16:9 \| 1920x1080 \| Standard landscape, also supports 4K (3840x2160) \|
616
617	\| TikTok / Reels \| 9:16 \| 1080x1920 \| Vertical video, max 60s for Reels, 10min for TikTok \|
618
619	\| Instagram Feed \| 1:1 \| 1080x1080 \| Square format \|
620
621	\| X / Twitter \| 16:9 or 1:1 \| 1280x720 or 720x720 \| Supports both, 2min 20s max \|
622
623	### Reframing Strategies
624
625	When converting between aspect ratios, there are three strategies:
626
627	1. Center crop (default) — Crop from the center to fill the target ratio. Fast, works well for most content. May cut off edges.
628	2. Letterbox/pillarbox — Add black bars (or blurred background) to fit without cropping. Preserves all content but adds empty space.
629
630	3. Blurred fill — Use a blurred, scaled-up version of the video as the background behind the original. Looks much better than black bars, especially for vertical reframing. This is the recommended approach for going from 16:9 to 9:16.
631
632	When the user asks to reframe, ask which platform they're targeting. Use blurred fillby default for aspect ratio changes that would lose significant content (e.g., landscape to portrait). Usecenter cropwhen the content is center-focused. Useletterbox only if the user explicitly asks for it.
633
634	### Implementation (4)
635
636	For the complete FFmpeg commands for each reframing strategy, see:
637
638	- `operations.md` — See the "Social Media Reframing" section
639
640	### Multi-platform Export
641
642	When the user wants to export for multiple platforms at once, generate all versions in a batch:
643
644	```text
645
646	Input (16:9) → YouTube (copy) + TikTok (9:16 blurred fill) + Instagram (1:1 center crop) + X (1280x720 copy)
647
648	```
649
650	This is a common workflow — the user shoots in landscape and needs versions for every platform.
651
652	## Common Operations Reference
653
654	For detailed FFmpeg commands and patterns for each operation type, see:
655
656	- `operations.md` — Complete command reference for all supported editing operations
657	- `virality-scoring.md` — AI-powered virality analysis and auto-trim pipeline
658
659	- `voiceover.md` — ElevenLabs voiceover generation and audio mixing
660	- `dead-space-and-chunking.md` — Dead space removal and social media chunking pipelines
661
662	## Building a Video Editor UI
663
664	If the user wants a visual editor interface:
665
666	1. Create a `react-vite` artifact for the frontend
667	2. Add API routes to `artifacts/api-server/` for video processing
668
669	3. Use `multer` for file uploads to the API
670	4. Process videos with FFmpeg on the server
671
672	5. Return processed files for download or preview
673	6. Use the `object-storage` skill for persisting uploaded and processed files if needed
674
675	Key frontend considerations:
676
677	- Use `<video>` element for preview playback
678	- Show a timeline UI for trim/cut operations
679
680	- Display progress during processing (poll an API status endpoint or use Server-Sent Events)
681	- Provide download links for processed output
682
683	## Tested Bugs & Caveats (Do Not Regress)
684
685	These issues were discovered during real-world testing and are documented here to prevent regressions:
686
687	- Scene detection: The `ffprobe -f lavfi`approach does NOT work in Replit. Use`ffmpeg`with`select='gt(scene,threshold)',showinfo`and parse`pts_time` from stderr.
688	- Segment indexing: Use `index: segments.length`when building segments, NOT the loop counter`i` — filtered segments cause index mismatches.
689
690	- Content safety: OpenAI vision may reject frames (medical/documentary content). Always wrap AI calls in try/catch and skip gracefully.
691	- Frame extraction timestamp clamping: Clamp the last frame timestamp to `segment.endTime - 0.1`. Without this, the last clip in a video fails because`startTime + 1.0 * duration` can exceed the actual file duration due to floating point arithmetic.
692
693	- `silenceremove` filter: Only strips audio silence, NOT video frames. Must use segmented extract+concat approach for proper dead space removal.
694	- Dead space with many segments: Videos with lots of short dialogue (50+ speaking segments) can be very slow to process with re-encoding. Use `-preset ultrafast` for testing, or skip dead space removal for dialogue-heavy content where there's little dead space anyway.
695
696	- OpenAI for scripts: Install with `pnpm add -w openai`, create client with`AI_INTEGRATIONS_OPENAI_BASE_URL`and`AI_INTEGRATIONS_OPENAI_API_KEY` env vars.
697	- AI scoring duration: A 90-second video with ~13 segments takes 1-2 minutes for AI analysis. A 3.5-minute video with 9 clips takes ~3 minutes. Always warn the user.
698
699	## Limitations
700
701	- No GPU acceleration — Replit containers do not have GPU access, so hardware encoding (NVENC, VAAPI) is unavailable. Use software encoding (libx264, libx265, libvpx-vp9).
702	- Memory and CPU — Very large files or high-resolution encoding (4K+) may be slow or hit memory limits. For large files, prefer stream-copy (`-c copy`) where possible and avoid unnecessary re-encoding.
703
704	- Disk space — Video files are large. Clean up temporary files after processing. Monitor disk usage for batch operations.
705	- No real-time preview — FFmpeg processes offline. Users cannot preview effects in real-time during editing (unlike desktop NLEs). Generate short previews of segments instead.