b301e35 · samitouri/QOSami-colab-cli

@@ -1,104 +1,181 @@

1 # Colab CLI

2

3 -A command-line interface for Google Colab. Create sessions, run code, manage files, and capture work — all without leaving your terminal.

3 +A command-line interface for Google Colab. Provision high-performance CPU, GPU, and TPU runtimes, execute local code, manage remote files, and orchestrate automated cloud pipelines — directly from your terminal.

4

5 -> Why? The agents are coming.

5 +Designed to support seamless developer productivity, headless automation, and AI agent integrations.

6

7 -## Install

7 +---

8

9 -With uv

9 +## Key Features

10

11 -```bash

12 -uv tool install -U google-colab-cli

13 -```

11 +* **Instant VM Provisioning:** Spin up CPU, GPU (T4, L4, G4, H100, A100), or TPU (v5e1, v6e1) runtimes in seconds.

12 +* **Robust Code Execution:** Run local Python scripts, Jupyter Notebooks (`.ipynb`), or piped `stdin` code; launch interactive REPLs or raw TTY console shells.

13 +* **Ephemeral Job Runner (`colab run`):** Provision a fresh VM, execute a local script with forwarded arguments, retrieve output files, and automatically tear down the runtime in a single command.

14 +* **Automatic Keep-Alive:** Built-in background daemon automatically prevents idle VM termination, keeping resource allocations active without requiring open browser tabs.

15 +* **Seamless Workspace Automation:** Mount Google Drive, authenticate Google Cloud Platform (GCP) credentials, and install dependencies with high-performance `uv` package management.

16 +* **State & Log Archival:** Inspect local session states or export interactive history logs to standard Jupyter Notebooks, Markdown, or structured JSONL.

17 +

18 +---

19

15 -or with pip

20 +## Installation

21 +

22 +Install the package using `uv` (recommended) or standard `pip`:

23

24 ```bash

18 -pip install -U google-colab-cli

19 -```

25 +# Using uv (recommended)

26 +uv tool install google-colab-cli

27

28 +# Using pip

29 +pip install google-colab-cli

30 +```

31

22 -Note: If you have a non-standard default package index (Googlers), you may also need to add `--index https://pypi.org/simple`.

32 +---

33

34 +## Quick Start

35

25 -## Quick start

36 +Run a CPU-based VM runtime, execute some code, and clean up:

37

38 ```bash

28 -colab new # provision a CPU session

29 -echo "print('hello')" | colab exec # run code

30 -colab stop # release the VM

39 +# 1. Provision a new session

40 +colab new

41 +

42 +# 2. Execute code from stdin

43 +echo "print('Hello from Google Colab!')" | colab exec

44 +

45 +# 3. Stop and release the VM resource

46 +colab stop

47 ```

48

33 -When only one session is active you can omit `-s <session>`; the CLI selects it automatically.

49 +> [!NOTE]

50 +> When only one session is active, you can omit the `-s, --session` option;

51 +> the CLI automatically knows it.

52 +

53 +

54 +---

55

35 -## Commands

56 +## Command Index

57

37 -### Sessions

58 +Run `colab <command> --help` to view specific options, defaults, and detailed help.

59 +

60 +### Session Management

61 | Command | Description |

62 | --- | --- |

40 -| `colab new [-s NAME] [--gpu T4\|L4\|A100\|H100] [--tpu v5e1\|v6e1]` | Provision a new session (CPU by default) |

41 -| `colab sessions` | List all active sessions on the backend |

42 -| `colab status [-s NAME]` | Show one session, or all locally-known sessions |

43 -| `colab stop -s NAME` | Terminate a session |

44 -| `colab url [-s NAME] [--open]` | Print a browser URL that opens the session in Colab |

63 +| `colab new [-s NAME] [--gpu GPU] [--tpu TPU]` | Allocate a new CPU, GPU, or TPU VM runtime |

64 +| `colab sessions` | List all active sessions currently active on the backend |

65 +| `colab status [-s NAME]` | Display hardware, status, and local metadata for active sessions |

66 +| `colab restart-kernel [-s NAME]` | Restart the active session's Jupyter kernel |

67 +| `colab stop [-s NAME]` | Terminate a session VM and tear down its keep-alive daemon |

68 +| `colab url [-s NAME] [--open]` | Print or open a browser URL connecting to the active session |

69

70 ### Execution

71 | Command | Description |

72 | --- | --- |

49 -| `colab exec [-s NAME] [-f FILE] [--output-image PATH]` | Run Python from stdin, a `.py` file, or a `.ipynb` notebook |

50 -| `colab repl [-s NAME] [--output-image PATH]` | Interactive Python REPL (or one-shot if stdin is piped) |

51 -| `colab console [-s NAME]` | Raw TTY shell on the VM (or one-shot if stdin is piped) |

73 +| `colab run [--gpu GPU] [--tpu TPU] [--keep] SCRIPT [ARGS...]` | Run a local script on a fresh VM, forwarding arguments, then release it |

74 +| `colab exec [-s NAME] [-f FILE] [--output-image PATH]` | Execute Python code from stdin, a local `.py` file, or a `.ipynb` notebook |

75 +| `colab repl [-s NAME] [--output-image PATH]` | Start an interactive Python REPL on the VM (exits cleanly on piped EOF) |

76 +| `colab console [-s NAME]` | Connect to a raw interactive TTY shell (tmux) on the remote VM |

77

53 -### Files

78 +### File Operations

79 | Command | Description |

80 | --- | --- |

56 -| `colab ls [-s NAME] [PATH]` | List remote files |

57 -| `colab upload -s NAME LOCAL REMOTE` | Upload a file |

58 -| `colab download -s NAME REMOTE LOCAL` | Download a file |

59 -| `colab rm -s NAME PATH` | Delete a remote file |

60 -| `colab edit -s NAME PATH` | Edit a remote file in `$EDITOR` |

81 +| `colab ls [-s NAME] [PATH]` | List remote files on the VM |

82 +| `colab upload [-s NAME] LOCAL REMOTE` | Upload a local file to the VM filesystem |

83 +| `colab download [-s NAME] REMOTE LOCAL` | Download a remote file from the VM filesystem |

84 +| `colab rm [-s NAME] PATH` | Delete a remote file on the VM filesystem |

85 +| `colab edit [-s NAME] PATH` | Edit a remote file in-place using your local `$EDITOR` |

86

62 -### Automation & utility

87 +### Automation & Utilities

88 | Command | Description |

89 | --- | --- |

65 -| `colab auth -s NAME` | Authenticate the VM for GCP services |

66 -| `colab drivemount -s NAME [PATH]` | Mount Google Drive (default `/content/drive`) |

67 -| `colab install -s NAME [-r requirements.txt \| pkg ...]` | Install packages with `uv` (falls back to `pip`) |

68 -| `colab log [-s NAME] [-n N] [-o FILE]` | View or export session history (`.ipynb`/`.md`/`.txt`/`.jsonl`) |

69 -| `colab pay` | Open the Colab signup page |

70 -| `colab version` | Print the installed version |

71 -| `colab update [--install]` | Check for a newer release (and optionally install it) |

72 -| `colab help` | Show usage |

73 -

74 -### Global options

75 -- `--auth {oauth2,adc}` — authentication strategy (default `oauth2`)

76 -- `-c, --client-oauth-config PATH` — OAuth client config (default `~/.colab-cli-oauth-config.json`)

77 -- `--config PATH` — session state file (default `~/.config/colab-cli/sessions.json`)

78 -- `--logtostderr` — send all output to stderr

79 -

80 -## Examples

90 +| `colab auth [-s NAME]` | Authenticate the VM for GCP services (BigQuery, GCS, etc.) |

91 +| `colab drivemount [-s NAME] [PATH]` | Mount Google Drive on the VM (default: `/content/drive`) |

92 +| `colab install [-s NAME] [-r FILE \| PKG...]` | Install packages on the VM using `uv` (falls back to `pip`) |

93 +| `colab log [-s NAME] [-n N] [-o FILE]` | View or export session history (`.ipynb`, `.md`, `.txt`, `.jsonl`) |

94 +| `colab pay` | Open the Colab subscription page to manage compute units |

95 +| `colab version` | Print the installed version of the CLI |

96 +| `colab update [--install]` | Check for a newer release (and optionally upgrade the CLI in place) |

97 +

98 +### Global Options

99 +* `--auth {oauth2,adc}` — Authentication strategy for the Colab API (default: `adc`).

100 +* `-c, --client-oauth-config PATH` — Path to public OAuth client credentials configuration (default: `~/.colab-cli-oauth-config.json`).

101 +* `--config PATH` — Path to local session metadata storage (default: `~/.config/colab-cli/sessions.json`).

102 +* `--logtostderr` — Direct debug logging output to stderr.

103 +

104 +---

105 +

106 +## Practical Examples

107 +

108 +### Accelerator Training with Checkpoint Retrieval

109 +

110 +Provision an A100 GPU, install requirements, run a local training script, retrieve the resulting model weights, and terminate the VM:

111

112 ```bash

83 -# Train a model on an A100, save the checkpoint locally

113 colab new -s trainer --gpu A100

114 colab install -s trainer torch transformers

115 colab exec -s trainer -f train.py

116 colab download -s trainer checkpoints/model.bin ./model.bin

117 colab stop -s trainer

118 +```

119 +

120 +### Workspace Notebook Execution with Drive Integration

121

90 -# Mount Drive and analyze a notebook

122 +Mount Google Drive, run a local notebook against the VM kernel (outputs are written back into `report_output.ipynb`), export a Markdown log of the execution, and clean up:

123 +

124 +```bash

125 colab new -s analysis

126 colab drivemount -s analysis

93 -colab exec -s analysis -f analysis.ipynb # writes analysis_output.ipynb

94 -colab log -s analysis -o report.ipynb

127 +colab exec -s analysis -f report.ipynb

128 +colab log -s analysis -o execution_log.md

129 colab stop -s analysis

130 ```

131

98 -## Notes

132 +---

133 +

134 +## Usage Notes

135 +

136 +* **TTY Requirements:** The interactive commands `repl` and `console` require a local TTY. When running inside automated scripts or pipelines, make sure to pipe stdin (e.g., `echo "print(1)" | colab repl`) to trigger non-interactive execution modes.

137 +* **Transparent Code Execution:** When calling `colab exec -f file.py`, the CLI reads the file locally and transmits its content to the remote kernel. You do not need to manually upload files before execution.

138 +* **Storage & State Paths:** Session tokens and metadata are stored at `~/.config/colab-cli/sessions.json`. Global CLI settings are located at `~/.config/colab-cli/settings.json`. These can be customized or isolated via the global `--config` flag.

139 +

140 +### Ephemeral Accelerator Jobs

141 +

142 +Use `colab run` to run a local script on dedicated hardware without manual session lifecycle management. The CLI handles provisioning, script execution, and immediate VM teardown automatically:

143 +

144 +```bash

145 +# Run train.py on a T4 GPU and release the VM on completion

146 +colab run --gpu T4 train.py

147 +```

148 +

149 +### Shebang Execution Support

150 +

151 +To execute a local file directly on a remote accelerator, place the `colab run` interpreter in the shebang line:

152 +

153 +```python

154 +#!/usr/bin/env -S colab run --gpu L4 --keep

155 +import torch

156 +

157 +print("L4 GPU Available:", torch.cuda.is_available())

158 +print("Device Name:", torch.cuda.get_device_name(0))

159 +```

160 +

161 +Make the script executable (`chmod +x script.py`) and run it: `./script.py`. The `--keep` option tells the CLI to preserve the session VM on completion so you can re-execute or inspect logs.

162 +

163 +---

164 +

165 +## Deep Dive Documentation

166 +

167 +For comprehensive architectural overviews and deep-dives into specific CLI sub-systems, refer to the detailed documentation:

168 +

169 +* [Session Management & Keep-Alive Architecture](docs/01_session_management.md)

170 +* [Interactive & Non-Interactive Execution Design](docs/02_execution_and_interactive.md)

171 +* [File Management & Jupyter Contents API](docs/03_file_management.md)

172 +* [Authentication Providers & VM Automation](docs/04_automation_and_utility.md)

173 +* [Ephemeral Job Runner Design](docs/05_run_command.md)

174 +

175 +To view interactive walkthroughs of eleven real-world automated scenarios, check out the [Demo Walkthroughs](docs/demos.md).

176 +

177 +---

178

100 -- `repl` and `console` require a TTY when run interactively. Pipe stdin to use them in scripts.

101 -- `exec` reads files locally and ships their contents to the VM — local edits don't require uploading.

102 -- Session metadata is stored at `~/.config/colab-cli/sessions.json`. Settings (auto-update etc.) live at `~/.config/colab-cli/settings.json`.

179 +## Contributing

180

104 -See [`CONTRIBUTING.md`](./CONTRIBUTING.md) for how to file feedback.

181 +Feedback and contributions are welcome! Please read [`CONTRIBUTING.md`](./CONTRIBUTING.md) for details.

docs: Pass on README.md (#39)