| 1 | # Design: File Management (`ls`, `rm`, `upload`, `download`, `edit`) |
| 2 | |
| 3 | ## Overview |
| 4 | File management on the Colab VM will be implemented using the Jupyter Contents API. |
| 5 | |
| 6 | ## Approach |
| 7 | |
| 8 | ### 1. Listing Files (`colab ls`) |
| 9 | - **API**: `GET /api/contents/<path>` (as seen in HAR L68181). |
| 10 | - **Parameters**: |
| 11 | - `authuser`: 0 |
| 12 | - `colab-runtime-proxy-token`: <session_token> |
| 13 | - **Response**: JSON with `content` field containing an array of directory entries. |
| 14 | - **Display**: Pretty-print the list (similar to `ls -F` or a formatted table). |
| 15 | |
| 16 | ### 2. Uploading Files (`colab upload`) |
| 17 | - **API**: `PUT /api/contents/<remote_path>` (as seen in HAR). |
| 18 | - **Payload**: JSON body: |
| 19 | ```json |
| 20 | { |
| 21 | "name": "filename.txt", |
| 22 | "path": "path/filename.txt", |
| 23 | "type": "file", |
| 24 | "format": "text", |
| 25 | "content": "..." |
| 26 | } |
| 27 | ``` |
| 28 | - **Base64 Encoding**: Use `format: base64` for binary files. |
| 29 | - **Progress**: Implement a simple progress bar for large uploads by chunking or providing status updates. |
| 30 | |
| 31 | ### 3. Downloading Files (`colab download`) |
| 32 | - **API**: `GET /api/contents/<remote_path>?content=1` (as seen in HAR). |
| 33 | - **Response**: JSON with `content` field. |
| 34 | - **Handling**: Decodes content based on `format` (text or base64) and saves it locally. |
| 35 | |
| 36 | ### 4. Deleting Files (`colab rm`) |
| 37 | - **API**: `DELETE /api/contents/<remote_path>`. |
| 38 | |
| 39 | ### 5. Editing Files (`colab edit`) |
| 40 | - **Approach**: Combines downloading the remote file, opening it in the user's `$EDITOR` locally, and subsequently uploading the changed file if modifications were made. |
| 41 | - **State tracking**: Uses a SHA-256 hash to track file changes securely and deterministically between before and after the editor is invoked. |
| 42 | - **Fallbacks**: Creates an empty local temporary file if the target file on the Colab runtime doesn't exist yet, essentially acting like `touch`. |
| 43 | |
| 44 | ## Implementation Details |
| 45 | - **Base URL**: The backend URL obtained during session assignment. |
| 46 | - **Proxy Token**: The `colab-runtime-proxy-token` is required for each request. |
| 47 | - **Error Handling**: Handle 404 (not found) and 403 (unauthorized). |
| 48 | - **Large Files**: The Contents API might have limitations for very large files. If so, we'll implement a fallback via the kernel (streaming chunks). |
| 49 | |
| 50 | ## Testing Strategy |
| 51 | TDD is mandatory for all file management features. |
| 52 | |
| 53 | ### 1. Mock Contents API |
| 54 | - **Test Case**: Verify `colab ls` correctly parses a Jupyter `contents` JSON response with `type: directory` and `type: file`. |
| 55 | - **Test Case**: Verify `colab upload` correctly base64-encodes a binary local file for the `PUT` payload. |
| 56 | - **Test Case**: Verify `colab download` correctly decodes the `content` field from the `GET` response and saves it locally. |
| 57 | - **Test Case**: Verify `colab edit` safely handles when a file is or isn't modified. |
| 58 | - **Test Case**: Verify `colab edit` securely opens a system editor safely through mocks without hanging the testing environment. |
| 59 | |
| 60 | ### 2. Error Cases |
| 61 | - **Test Case**: Verify 404 responses are correctly caught and presented as a "File not found" error to the user. |
| 62 | - **Test Case**: Verify correct handling of large file uploads exceeding API limits via kernel streaming. |