@splitfireai / demucs-swift
Code Commits Issues Pulls Cloud Agent CI/CD
main
main
md 106 lines 3.48 KB
Rendered Raw
1 # Demucs for Swift
2
3 On-device music source separation for Swift apps. Give it a song; get back
4 drums, bass, vocals and the rest as separate tracks. It runs Meta's HTDemucs v4
5 through [demucs-rs](https://github.com/SplitFireAI/demucs-rs) on the GPU via
6 Metal, entirely on the device.
7
8 Supports iOS 16, macOS 14 (Apple silicon), tvOS 16 and visionOS 1.
9
10 ## Installation
11
12 In Xcode, choose **File > Add Package Dependencies** and enter:
13
14 ```text
15 https://github.com/SplitFireAI/demucs-swift
16 ```
17
18 Or in `Package.swift`:
19
20 ```swift
21 dependencies: [
22 .package(url: "https://github.com/SplitFireAI/demucs-swift", from: "0.1.0"),
23 ]
24 ```
25
26 Then add the `Demucs` product to your target.
27
28 ## Quick start
29
30 ```swift
31 import Demucs
32
33 // Once at launch: keep the weights inside the app container.
34 let caches = FileManager.default.urls(for: .cachesDirectory, in: .userDomainMask)[0]
35 DemucsEngine.configureCacheDirectory(caches.appendingPathComponent("demucs"))
36
37 // Download the weights once (84–333 MB from Hugging Face, cached on disk).
38 try await DemucsEngine.download(.fourStem) { progress in
39 print("download \(Int(progress.fraction * 100))%")
40 }
41
42 // Load the model. Warmup compiles the GPU shaders ahead of the first song.
43 let engine = DemucsEngine()
44 try await engine.load(.fourStem)
45 try await engine.warmup()
46
47 // Separate. Input is per-channel Float PCM at any sample rate (decode files
48 // with AVFoundation); each stem comes back at the same rate and length.
49 let stems = try await engine.separate(left: left, right: right, sampleRate: 44_100) { progress in
50 print("separating \(Int(progress.fraction * 100))%")
51 }
52 for stem in stems {
53 print(stem.kind, stem.left.count)
54 }
55 ```
56
57 Cancelling the calling task cancels a download immediately and a separation at
58 the next ~7.8 second chunk boundary. Progress handlers run on a background
59 thread.
60
61 See [`Examples/DemucsExample`](Examples/DemucsExample) for a SwiftUI app that
62 decodes a file with AVFoundation, separates it and shares the stems as WAV.
63
64 ## Models
65
66 | Model | Case | Stems | Size |
67 |-------|------|-------|------|
68 | htdemucs | `.fourStem` | drums, bass, other, vocals | ~84 MB |
69 | htdemucs_6s | `.sixStem` | + guitar, piano | ~84 MB |
70 | htdemucs_ft | `.fineTuned(stems:)` | up to 4, best quality | ~333 MB |
71
72 The fine-tuned model runs one sub-model per stem, so
73 `.fineTuned(stems: [.vocals])` is about four times faster than
74 `.fineTunedAll`. All selections share one download.
75
76 `DemucsEngine.models`, `isDownloaded(_:)`, `downloadedFileURL(for:)` and
77 `deleteDownload(_:)` manage the files.
78
79 ## How this package is built
80
81 `Sources/Demucs/demucs_ffi.swift` is generated by UniFFI from the `demucs-ffi`
82 crate in demucs-rs, and `DemucsFramework` is the matching static library. On
83 every `swift-vX.Y.Z` tag, demucs-rs's **Release SDK Swift** workflow uploads the
84 XCFramework, then rewrites the release URL and checksum in `Package.swift` and
85 the generated file here, and tags this repository. Do not edit those by hand;
86 the hand-written API lives in `Sources/Demucs/DemucsEngine.swift`.
87
88 ## Local development
89
90 Keep `demucs-rs` and `demucs-swift` next to each other, then build a local
91 framework before running tests:
92
93 ```sh
94 make macos
95 swift test
96 ```
97
98 Use `make ios`, `make tvos` or `make visionos` for another platform (tvOS and
99 visionOS need a nightly toolchain with `rust-src`). Set `DEMUCS_REPO` when the
100 repositories are not siblings. The package uses `DemucsFramework.xcframework`
101 automatically when that local file exists; without it, SwiftPM downloads the
102 release asset.
103
104 ## License
105
106 Apache-2.0, same as demucs-rs.