|
1
|
# Audio Generation |
|
2
|
|
|
3
|
## Available Functions |
|
4
|
|
|
5
|
### generateMusic({prompt, ...}) |
|
6
|
|
|
7
|
Generate original music from a text prompt and save it as an audio file. |
|
8
|
|
|
9
|
**Parameters:** |
|
10
|
|
|
11
|
- `prompt` (str, required): Description of the desired music |
|
12
|
- `outputPath` (str, optional): Must end in `.mp3` for MP3 output or `.ogg` for Opus output. Defaults to `attached_assets/generated_audio/music_{summary}.mp3` |
|
13
|
- `summary` (str, default "generated_music"): Short description for the filename |
|
14
|
- `durationSeconds` (int, optional): 3-600 seconds. If omitted, the model chooses a length |
|
15
|
- `forceInstrumental` (bool, default True): Generate instrumental music |
|
16
|
- `outputFormat` (str, default "mp3_44100_128"): Audio format |
|
17
|
- Options are alaw_8000, mp3_22050_32, mp3_24000_48, mp3_44100_128, mp3_44100_192, mp3_44100_32, mp3_44100_64, mp3_44100_96, opus_48000_128, opus_48000_192, opus_48000_32, opus_48000_64, opus_48000_96, pcm_16000, pcm_22050, pcm_24000, pcm_32000, pcm_44100, pcm_48000, pcm_8000, ulaw_8000, wav_16000, wav_22050, wav_24000, wav_32000, wav_44100, wav_48000, wav_8000 |
|
18
|
- `overwrite` (bool, default True): Whether to overwrite existing files |
|
19
|
|
|
20
|
**Returns:** Dict with `filePath` and `description` keys |
|
21
|
|
|
22
|
**Example:** |
|
23
|
|
|
24
|
```javascript |
|
25
|
const result = await generateMusic({ |
|
26
|
prompt: "Upbeat electronic background music with warm synths and a steady pulse", |
|
27
|
summary: "upbeat synth bed", |
|
28
|
durationSeconds: 30, |
|
29
|
forceInstrumental: true |
|
30
|
}); |
|
31
|
console.log(`Music saved to: ${result.filePath}`); |
|
32
|
``` |
|
33
|
|
|
34
|
### generateSoundEffect({prompt, ...}) |
|
35
|
|
|
36
|
Generate a short sound effect from a text prompt and save it as an audio file. |
|
37
|
|
|
38
|
**Parameters:** |
|
39
|
|
|
40
|
- `prompt` (str, required): Description of the sound effect |
|
41
|
- `outputPath` (str, optional): Must end in `.mp3` for MP3 output or `.ogg` for Opus output. Defaults to `attached_assets/generated_audio/sound_effect_{summary}.mp3` |
|
42
|
- `summary` (str, default "sound_effect"): Short description for the filename |
|
43
|
- `durationSeconds` (number, optional): 0.5-30 seconds. If omitted, the model chooses a length |
|
44
|
- `loop` (bool, default False): Create a smoothly looping sound effect |
|
45
|
- `promptInfluence` (number, default 0.3): 0-1, higher follows the prompt more closely |
|
46
|
- `outputFormat` (str, default "mp3_44100_128"): Audio format |
|
47
|
- `overwrite` (bool, default True): Whether to overwrite existing files |
|
48
|
|
|
49
|
**Returns:** Dict with `filePath` and `description` keys |
|
50
|
|
|
51
|
**Example:** |
|
52
|
|
|
53
|
```javascript |
|
54
|
const result = await generateSoundEffect({ |
|
55
|
prompt: "Soft futuristic UI confirmation chime, clean and pleasant", |
|
56
|
summary: "ui confirmation chime", |
|
57
|
durationSeconds: 1.5 |
|
58
|
}); |
|
59
|
console.log(`Sound effect saved to: ${result.filePath}`); |
|
60
|
``` |
|
61
|
|
|
62
|
### searchVoices({search, ...}) |
|
63
|
|
|
64
|
Search available voices for text-to-speech. Use this before `textToSpeech` when you do not know the `voiceId`. |
|
65
|
|
|
66
|
**Parameters:** |
|
67
|
|
|
68
|
- `search` (str, optional): Search by name, description, or labels |
|
69
|
- `pageSize` (int, default 10): Number of voices to return, 1-100 |
|
70
|
- `voiceType` (str, optional): "personal", "community", "default", "workspace", "non-default", "non-community", or "saved" |
|
71
|
- `category` (str, optional): "premade", "cloned", "generated", or "professional" |
|
72
|
- `sort` (str, optional): "created_at_unix" or "name" |
|
73
|
- `sortDirection` (str, optional): "asc" or "desc" |
|
74
|
- `nextPageToken` (str, optional): Token for pagination |
|
75
|
|
|
76
|
**Returns:** Dict containing `voices`, `has_more`, and pagination fields. |
|
77
|
|
|
78
|
### textToSpeech({text, voiceId, ...}) |
|
79
|
|
|
80
|
Convert text into speech using a selected voice and save it as an audio file. |
|
81
|
|
|
82
|
**Parameters:** |
|
83
|
|
|
84
|
- `text` (str, required): Text to convert into speech |
|
85
|
- `voiceId` (str, required): Voice ID from `searchVoices` |
|
86
|
- `outputPath` (str, optional): Must end in `.mp3` for MP3 output or `.ogg` for Opus output. Defaults to `attached_assets/generated_audio/speech_{summary}.mp3` |
|
87
|
- `summary` (str, default "speech"): Short description for the filename |
|
88
|
- `modelId` (str, default "eleven_multilingual_v2"): Text-to-speech model ID |
|
89
|
- `outputFormat` (str, default "mp3_44100_128"): Audio format |
|
90
|
- `voiceSettings` (object, optional): Per-request voice settings such as `stability`, `similarity_boost`, `style`, `use_speaker_boost`, and `speed` |
|
91
|
- `overwrite` (bool, default True): Whether to overwrite existing files |
|
92
|
|
|
93
|
**Example:** |
|
94
|
|
|
95
|
```javascript |
|
96
|
const voices = await searchVoices({ search: "warm narrator", pageSize: 5 }); |
|
97
|
const voiceId = voices.voices[0].voice_id; |
|
98
|
const result = await textToSpeech({ |
|
99
|
text: "Welcome to a faster way to build.", |
|
100
|
voiceId, |
|
101
|
summary: "welcome voiceover" |
|
102
|
}); |
|
103
|
console.log(`Speech saved to: ${result.filePath}`); |
|
104
|
``` |
|
105
|
|
|
106
|
## When to Use Each Function |
|
107
|
|
|
108
|
### generateMusic |
|
109
|
|
|
110
|
- Original background music, loops, jingles, and music beds |
|
111
|
- Instrumental tracks for videos, games, presentations, or apps |
|
112
|
|
|
113
|
### generateSoundEffect |
|
114
|
|
|
115
|
- UI sounds, notification tones, transitions, ambience, and game SFX |
|
116
|
- Short audio moments for motion graphics or interactive experiences |
|
117
|
|
|
118
|
### searchVoices / textToSpeech |
|
119
|
|
|
120
|
- Voiceovers, narration, read-aloud content, and character speech |
|
121
|
- Use `searchVoices` first when the voice ID is not known |
|
122
|
|
|
123
|
## Best Practices |
|
124
|
|
|
125
|
1. **Write detailed prompts**: Include genre, mood, instrumentation, pacing, duration, and intended use |
|
126
|
|
|
127
|
## Output Locations |
|
128
|
|
|
129
|
- Generated audio: `attached_assets/generated_audio/` |
|
130
|
|
|
131
|
## Limitations |
|
132
|
|
|
133
|
- Generated sound effects: 30 seconds maximum |
|
134
|
- Generated music: 600 seconds maximum |
|
135
|
|
|
136
|
## Copyright |
|
137
|
|
|
138
|
- Generated music, sound effects, and speech are created for your use |