A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[misc] nsys profile output kernel classifier and visualizer (#22971)
Signed-off-by: Grace Ho <grho@nvidia.com>
Grace Ho committed
Aug 15, 2025 at 19:52 UTC
1f83e7d849ccb03990bb896f49df20343a2828b9
5 files changed
+601
tools/profiler/nsys_profile_tools/README.md
new
+175
@@ -0,0 +1,175 @@
1
+# gputrc2graph.py
2
+
3
+This script processes NVIDIA Nsight Systems (`nsys`) GPU trace files
4
+(`.nsys-rep`) with -t cuda tracing enabled, and generates kernel-level
5
+summaries and visualizations of GPU and non-GPU time. It is useful for
6
+profiling and analyzing nsys profile output.
7
+
8
+## Usage
9
+
10
+### Command-line Arguments
11
+
12
+- `--in_file`
13
+ **(required)**
14
+ List of input files and their metadata. Each entry should be in the format:
15
+ `<nsys-rep>,<engine>,<model>,<elapsed_nonprofiled_sec>`
16
+ - `nsys-rep`: Path to the `.nsys-rep` file.
17
+ - `engine`: Engine name (e.g., `vllm`).
18
+ - `model`: Model name (e.g., `llama`, `gpt-oss`, `ds`).
19
+ - `elapsed_nonprofiled_sec`: Wall-clock runtime (in seconds) without
20
+ profiling. Specify `0` to use the elapsed time from the nsys-rep file
21
+ (this may inflate non-GPU time if actual runtime without profiling is
22
+ less). Multiple entries can be provided, separated by spaces.
23
+
24
+- `--out_dir`
25
+ Output directory for the generated CSV and HTML files.
26
+ If not specified, results are saved in the current directory.
27
+
28
+- `--title`
29
+ Title for the HTML chart/visualization.
30
+
31
+- `--nsys_cmd`
32
+ Path to the `nsys` command.
33
+ Default: `nsys` (assumes it is in your PATH).
34
+ Use this if `nsys` is not in your system PATH.
35
+
36
+## Notes
37
+
38
+- Make sure you have pandas installed.
39
+- Make sure nsys is installed, and specify the path to the `nsys` command with
40
+ `--nsys_cmd` if it is not in your PATH.
41
+- For more details on available engines and models, see the help string in
42
+ the script or run:
43
+
44
+```bash
45
+python3 gputrc2graph.py --help
46
+```
47
+
48
+## Example 1: analyze a single profile
49
+
50
+To analyze the GPU cycles for say, gpt-oss model with vLLM engine:
51
+
52
+1. Run the following command to collect nsys profile, for vllm serve config.
53
+
54
+ ```bash
55
+ nsys profile -t cuda -o run1 -f true --trace-fork-before-exec=true \
56
+ --cuda-graph-trace=node --delay <DELAY> --duration <DURATION> \
57
+ vllm serve openai/gpt-oss-120b ...
58
+ ```
59
+
60
+ where:
61
+
62
+ - DELAY: how many seconds to delay nsys from collecting profiles, needed so
63
+ that profiles aren't captured till vllm server has come up and load
64
+ generation starts.
65
+ - DURATION: how many seconds for nsys profile to run before generating the
66
+ profile. This should be > the duration of the run.
67
+
68
+2. Run again, this time without collecting the profile, and get the total run
69
+ time in seconds. This value will be used by the script to calculate the
70
+ CPU(non-GPU) seconds for the analysis.
71
+
72
+3. Say the run elapsed time is 306 seconds, from step #2. Run script to
73
+ analyze:
74
+
75
+ ```bash
76
+ python3 gputrc2graph.py \
77
+ --in_file run1.nsys-rep,vllm,gpt-oss,306 \
78
+ --title "vLLM-gpt-oss profile"
79
+ ```
80
+
81
+The command will produce 2 files for analysis:
82
+
83
+- result.html: this categorizes kernel names into different categories in a
84
+ stacked bar chart.
85
+- result.csv: shows how the kernel names are mapped to the different
86
+ categories.
87
+
88
+### HTML visualization with result.html
89
+
90
+The html file shows the number of elapsed seconds due to different GPU
91
+Substages or categories, which consist of moe_gemm (Mixture of Experts GEMM)
92
+kernels the biggest category, at 148 seconds, followed by "attn" or attention
93
+kernels. This lets the user prioritize the kernels to focus on for performance
94
+optimizations.
95
+
96
+
97
+
98
+There's also an appended data table underneath the bar chart for copying out to other post-processing tools.
99
+
100
+
101
+
102
+### Kernel to category mapping with result.csv
103
+
104
+Suppose the user would like to focus on improving triton kernels. It's not the
105
+biggest consumer of cycles at 9.74 sec but perhaps it hasn't been optimized.
106
+The next step is to use the result.csv to dive into what the kernels are which
107
+compose the triton kernel GPU cycles. The following image shows that
108
+triton_poi_fused__to_copy_add_addmm_cat_.. kernel to be the biggest
109
+contributor to GPU cycles.
110
+
111
+
112
+
113
+## Example 2: analyze multiple profiles
114
+
115
+Suppose the user has multiple nsys trace files, captured for different models,
116
+say llama and gpt-oss in this case, and wish to compare their GPU/non-GPU
117
+time, something like the following command can be used.
118
+
119
+```bash
120
+python3 gputrc2graph.py \
121
+--in_file run1.nsys-rep,vllm,llama,100 run2.nsys-rep,vllm,gpt-oss,102 \
122
+--out_dir results \
123
+--title "Comparison of vLLM Models"
124
+```
125
+
126
+The analysis process is similar to example 1 but now there will be multiple
127
+stack bar charts that can be compared. The categories for the different
128
+kernels will remain the same, so that it's easy to compare the GPU cycles for
129
+the same categories.
130
+
131
+Once a category is shown to have more cycles for one configuration than
132
+another, the next step would be to use the csv file to see what kernels are
133
+mapped into that category, and which kernels are taking the largest amount of
134
+time which would cause a difference for the overall category.
135
+
136
+## Example 3: add new classification for a new model
137
+
138
+Suppose there's a new model ABC that is available for engine DEF, and say there
139
+are 4 kernels to be classified into "gemm" and "attn", where the gemm kernels
140
+have names with "*H*" or "*I*" in them, and attn kernels have names with "*J*"
141
+or "*K*" in them, add a new entry like so:
142
+
143
+```python
144
+engine_model = {
145
+ 'DEF': {
146
+ 'ABC': {
147
+ 'layer_anno': {
148
+ 'Stage': {
149
+ '.*': 'layer',
150
+ },
151
+ 'Substage': {
152
+ 'H|I': 'gemm',
153
+ 'J|K': 'attn',
154
+ 'CUDA mem': 'non-gpu-H_D_memops',
155
+ '.*': 'misc'
156
+ }
157
+ }
158
+ },
159
+ }
160
+ 'vllm': {...}
161
+```
162
+
163
+Basically Substage is a dictionary with a list of key/value pairs, where the
164
+keys are regex's of the kernel names to be classified, and values are the
165
+classification bins which one wishes to compare across engines/models.
166
+
167
+The last 2 entries are common for all engine/models, consisting of CUDA memory
168
+operations and a 'misc' for anything that's leftover and can't be classified.
169
+
170
+When invoking gputrc2graph.py, specify a trace file with this new model/engine
171
+like the following:
172
+
173
+```bash
174
+--infile new.nsys-rep,DEF,ABC,<runtime>
175
+```
tools/profiler/nsys_profile_tools/gputrc2graph.py
new
+426
@@ -0,0 +1,426 @@
1
+# SPDX-License-Identifier: Apache-2.0
2
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
3
+"""
4
+ This generates gpu kernel analysis output from nsys rep. Will call nsys
5
+ stats -r cuda_gpu_kern_trace, get non-overlapped gpu cycles, then generate
6
+ csv and html output for analysis
7
+"""
8
+import argparse
9
+import logging
10
+import os
11
+
12
+import regex as re
13
+
14
+logger = logging.getLogger(__name__)
15
+
16
+
17
+# helper data class for annotating kernels
18
+class EngineModelData:
19
+ # engine + model mappings
20
+ engine_model = {
21
+ 'vllm': {
22
+ 'llama': {
23
+ 'layer_anno': {
24
+ 'Stage': {
25
+ '.*': 'layer',
26
+ },
27
+ 'Substage': {
28
+ 'gemm': 'gemm',
29
+ 'fused_moe_kernel|GroupProblemShape|group_gemm_starts':
30
+ 'moe_gemm', #llama4
31
+ 'moe|sigmoid': 'moe', #llama4
32
+ 'CatArrayBatched|prepare_inputs': 'prepare_next',
33
+ 'flash': 'attn',
34
+ 'ncclDevKernel|cross_device_reduce':
35
+ 'nccl_and_custom_ar',
36
+ '_norm_': 'norm',
37
+ 'act_and_mul_': 'silu',
38
+ 'rotary_embedding_kernel': 'rope',
39
+ 'SoftMax': 'softmax',
40
+ 'elementwise': 'elementwise',
41
+ 'fp8_quant': 'quantize',
42
+ 'reduce_kernel': 'reduce',
43
+ 'triton': 'triton_kernel',
44
+ 'CUDA mem': 'non-gpu-H_D_memops',
45
+ '.*': 'misc'
46
+ }
47
+ }
48
+ },
49
+ 'ds': {
50
+ 'layer_anno': {
51
+ 'Stage': {
52
+ '.*': 'layer',
53
+ },
54
+ 'Substage': {
55
+ 'block_fp8|gemm_fp8_blockwise':
56
+ 'block_fp8_gemm',
57
+ 'fused_moe_kernel|_group_gemm|GroupProblemShape|GemmUniversal':
58
+ 'moe_gemm',
59
+ 'gemm|matmul|nvjet':
60
+ 'gemm',
61
+ 'moe|sigmoid|expert':
62
+ 'moe',
63
+ '_fwd_|FlashAttn|_mla_|_attn_':
64
+ 'attn',
65
+ 'CatArrayBatched':
66
+ 'prepare_next',
67
+ 'ncclDevKernel|cross_device_reduce':
68
+ 'nccl_and_custom_ar',
69
+ 'Norm|_norm_':
70
+ 'norm',
71
+ 'sbtopk':
72
+ 'topk',
73
+ 'act_and_mul_':
74
+ 'activation',
75
+ 'compute_position_kernel':
76
+ 'rope',
77
+ 'elementwise':
78
+ 'elementwise',
79
+ 'fp8_quant|quant_fp8|cvt_fp16_to_fp4':
80
+ 'quantize',
81
+ 'reduce':
82
+ 'reduce',
83
+ 'SoftMax':
84
+ 'softmax',
85
+ 'triton':
86
+ 'triton_kernel',
87
+ 'CUDA mem':
88
+ 'non-gpu-H_D_memops',
89
+ '.*':
90
+ 'misc'
91
+ }
92
+ }
93
+ },
94
+ 'gpt-oss': {
95
+ 'layer_anno': {
96
+ 'Stage': {
97
+ '.*': 'layer',
98
+ },
99
+ 'Substage': {
100
+ 'block_fp8|gemm_fp8_blockwise':
101
+ 'block_fp8_gemm',
102
+ 'fused_moe_kernel|_group_gemm|GroupProblemShape|GemmUniversal|bmm_'
103
+ # this section is triton_moe_gemm
104
+ '|matmul_ogs_|_topk_forward|_combined_routing'
105
+ '|_sum_bitmatrix_rows|_compute_writeback_idx':
106
+ 'moe_gemm',
107
+ 'gemm|matmul|nvjet':
108
+ 'gemm',
109
+ 'moe|sigmoid|expert|splitKreduce':
110
+ 'moe',
111
+ '_fwd_|FlashAttn|_mla_|_attn_|_flash_|flash::prepare_varlen|fmha':
112
+ 'attn',
113
+ 'CatArrayBatched':
114
+ 'prepare_next',
115
+ 'ncclDevKernel|cross_device_reduce':
116
+ 'nccl_and_custom_ar',
117
+ 'Norm|_norm_':
118
+ 'norm',
119
+ 'sbtopk':
120
+ 'topk',
121
+ 'act_and_mul_':
122
+ 'activation',
123
+ 'compute_position_kernel':
124
+ 'rope',
125
+ 'elementwise':
126
+ 'elementwise',
127
+ 'fp8_quant|quant_fp8|cvt_fp16_to_fp4|quantize':
128
+ 'quantize',
129
+ 'reduce':
130
+ 'reduce',
131
+ 'SoftMax':
132
+ 'softmax',
133
+ 'triton':
134
+ 'triton_kernel',
135
+ 'CUDA mem':
136
+ 'non-gpu-H_D_memops',
137
+ '.*':
138
+ 'misc'
139
+ }
140
+ }
141
+ }
142
+ },
143
+ }
144
+
145
+
146
+class GPUTrace2Graph:
147
+ """
148
+ Parses output of nsys report, generates csv and bar chart output
149
+ """
150
+
151
+ def __init__(self, nsys_cmd):
152
+ self.nsys_cmd = nsys_cmd
153
+ import pandas as pd # avoid importing till needed
154
+ self.pd = pd
155
+ self.pd.options.mode.copy_on_write = True
156
+
157
+ # helper functions for generating trace->summary csvs
158
+ def gen_nonoverlapped_sum_from_gputrace(self, in_file, out_file):
159
+ logger.info('loading %s', in_file)
160
+ df = self.pd.read_csv(
161
+ in_file,
162
+ usecols=['Start (ns)', 'Duration (ns)', 'Device', 'Strm', 'Name'])
163
+ df['End (ns)'] = df['Start (ns)'] + df['Duration (ns)']
164
+ df = self.sum_non_overlapping_intervals(df)
165
+ # get ready to print table with elapsed times per kernel
166
+ df['Instances'] = 1
167
+ df_sum = df.groupby('Name', as_index=False).agg({
168
+ 'Elapsed Time (ns)': 'sum',
169
+ 'Duration (ns)': 'sum',
170
+ 'Instances': 'size'
171
+ })
172
+
173
+ # generate csv
174
+ df_sum['Total Time (sec)'] = df_sum['Duration (ns)'] / 1e9
175
+ df_sum['Elapsed Time (sec)'] = df_sum['Elapsed Time (ns)'] / 1e9
176
+ df_sum = df_sum.sort_values(by='Elapsed Time (sec)', ascending=False)
177
+ df_sum[['Elapsed Time (sec)', 'Total Time (sec)', 'Instances',
178
+ 'Name']].to_csv(out_file, index=False)
179
+
180
+ def sum_non_overlapping_intervals(self, df):
181
+ """
182
+ returns new sorted df with Elapsed Time (ns) column using
183
+ vectorized operations
184
+ """
185
+ logger.info("sorting %s trace records by start time", str(df.shape))
186
+
187
+ # Sort by start time and reset index
188
+ df = df.sort_values(by='Start (ns)').reset_index(drop=True)
189
+
190
+ # Initialize elapsed time as duration
191
+ df['Elapsed Time (ns)'] = df['Duration (ns)']
192
+
193
+ # Get numpy arrays for faster operations
194
+ starts = df['Start (ns)'].values
195
+ ends = df['End (ns)'].values
196
+
197
+ # Keep track of current interval end
198
+ current_end = ends[0]
199
+ display_units = int(len(df) / 100)
200
+ # Update current_end for overlapping intervals
201
+ for i in range(1, len(df)):
202
+ if i % display_units == 0:
203
+ print(f'processing trace: {int(i/len(df) * 100)} %', end="\r")
204
+ if starts[i] <= current_end:
205
+ if ends[i] > current_end:
206
+ # Partial overlap
207
+ df.iloc[i, df.columns.get_loc('Elapsed Time (ns)'
208
+ )] = ends[i] - current_end
209
+ current_end = ends[i]
210
+ else:
211
+ # Complete overlap
212
+ df.iloc[i, df.columns.get_loc('Elapsed Time (ns)')] = 0
213
+ else:
214
+ # No overlap
215
+ current_end = ends[i]
216
+
217
+ return df
218
+
219
+ # functions for generating html files
220
+ def make_html(self, df, output_dir, title):
221
+ """ make html graph from df """
222
+ import plotly.express as px
223
+ if df.empty:
224
+ return
225
+ output_name = output_dir + '/result'
226
+ if not title:
227
+ title = 'Model_Engine'
228
+ x = 'Model_Engine'
229
+ y = 'Elapsed Time (sec)'
230
+ color = 'Substage'
231
+ """ generate kernel mapping table """
232
+ # Sort Model_Engine categories by last field after underscore
233
+ df['Model_Engine'] = self.pd.Categorical(
234
+ df['Model_Engine'],
235
+ sorted(df['Model_Engine'].unique(),
236
+ key=lambda x: x.split('_')[-1]))
237
+ df[['Model_Engine', color, 'Instances', 'Name',
238
+ y]].sort_values(by=color).to_csv(f'{output_name}.csv', index=False)
239
+ graph = px.histogram(df.round(2),
240
+ x=x,
241
+ y=y,
242
+ title=(f'{y} for {title}'),
243
+ color=color,
244
+ text_auto=True)
245
+ # wrap x axis labels
246
+ graph.update_xaxes(automargin=True)
247
+ graph.write_html(f'{output_name}.html')
248
+ """
249
+ Generate data table with columns per Model_Engine into result.html
250
+ """
251
+ pivot_df = df.pivot_table(values='Elapsed Time (sec)',
252
+ index='Substage',
253
+ columns='Model_Engine',
254
+ aggfunc='sum',
255
+ observed=False).round(2)
256
+ # Add sum row at bottom
257
+ pivot_df.loc['total_elapsed_sec'] = pivot_df.sum()
258
+ pivot_df.fillna('').to_html('temp.html')
259
+ print('got')
260
+ with (open(f'{output_name}.html', 'a', encoding='utf-8') as
261
+ outfile, open('temp.html', encoding='utf-8') as infile):
262
+ outfile.write(infile.read())
263
+ os.remove('temp.html')
264
+
265
+ print(f'Finished generating: \n'
266
+ f' {output_name}.html for stack bar chart \n'
267
+ f' {output_name}.csv for Kernel-Substage mapping')
268
+
269
+ def anno_gpu_kernname(self, df, mapping):
270
+ """ add "stage" and "substage" columns """
271
+
272
+ def anno_gpu_kernname_helper(name, stage):
273
+ for kern_name, val in mapping['layer_anno'][stage].items():
274
+ if re.search(kern_name, name):
275
+ return val
276
+
277
+ for stage in ['Stage', 'Substage']:
278
+ df[stage] = df['Name'].apply(anno_gpu_kernname_helper, stage=stage)
279
+
280
+ def make_nongpu_row(self, df, nongpu_sec):
281
+ """ this will append non-gpu time entry at end of df """
282
+ nongpu_row = self.pd.DataFrame([df.iloc[-1]])
283
+ nongpu_row['Substage'] = nongpu_row['Name'] = 'CPU(non-GPU)'
284
+ nongpu_row['Instances'] = 1
285
+ nongpu_row['Elapsed Time (sec)'] = nongpu_sec
286
+ return (nongpu_row)
287
+
288
+ def is_valid_file(self, base_file):
289
+ """ asserts if base_file is non-existent or is empty """
290
+ assert os.path.isfile(base_file) and os.path.getsize(base_file) > 0, \
291
+ f"{base_file} doesn't exist or is empty"
292
+
293
+ def should_gen_file(self, new_file, base_file):
294
+ """ figure out if new file should be generated from base_file """
295
+ self.is_valid_file(base_file)
296
+ if (os.path.exists(new_file)
297
+ and (os.path.getmtime(new_file) > os.path.getmtime(base_file))
298
+ and (os.path.getsize(base_file) > 0)):
299
+ logger.info('reusing %s', new_file)
300
+ return False
301
+ else:
302
+ logger.info('generating %s', new_file)
303
+ return True
304
+
305
+ def gen_sum_file(self, file):
306
+ """
307
+ generates sum file from nsys trace with times per kernel and
308
+ returns the name of the sum file
309
+ """
310
+ import subprocess
311
+ file_dir = os.path.dirname(file)
312
+ file_name = os.path.basename(file)
313
+
314
+ if not file_dir:
315
+ file_dir = '.'
316
+ # Walk through trace and get the total non-overlapped time
317
+ nsys_stats_file = f'{file_dir}/{file_name}_cuda_gpu_trace.csv'
318
+ sum_file = f'{file_dir}/{file_name}_cuda_gpu_kernel_tracesum.csv'
319
+ if self.should_gen_file(nsys_stats_file, file):
320
+ cmd = [
321
+ self.nsys_cmd, 'stats', '-r', 'cuda_gpu_trace', file, '-o',
322
+ f'{file_dir}/{file_name}'
323
+ ]
324
+ cmd_str = ' '.join(cmd)
325
+ logger.info('+ %s', cmd_str)
326
+ try:
327
+ subprocess.run(cmd)
328
+ except Exception:
329
+ logger.error(
330
+ "%s failed, specify --nsys_cmd for correct nsys path",
331
+ cmd_str)
332
+ exit(1)
333
+ logger.info('generating non-overalapped sum %s', sum_file)
334
+ self.gen_nonoverlapped_sum_from_gputrace(nsys_stats_file, sum_file)
335
+ self.is_valid_file(sum_file)
336
+ logger.info('Finished generating %s', sum_file)
337
+ return sum_file
338
+
339
+ def gen_graph(self, in_file, out_dir, title):
340
+ """ generates graph and csv file from in_file into out_dir """
341
+ # Initialize an empty DataFrame to store combined data
342
+ combined_df = self.pd.DataFrame()
343
+ for idx, (file, engine, model, total_sec) in enumerate(in_file):
344
+ file_dir = os.path.dirname(file)
345
+ file_name = os.path.basename(file)
346
+ if not file_dir:
347
+ file_dir = '.'
348
+ sum_file = self.gen_sum_file(file)
349
+ # read kernel summary file
350
+ df = self.pd.read_csv(sum_file)
351
+ # annotate kernel to their categories
352
+ assert EngineModelData.engine_model.get(engine)
353
+ assert EngineModelData.engine_model[engine].get(model)
354
+ # remove nsys-rep from file_name for shorter x-label
355
+ file_name = file_name.replace('.nsys-rep', '')
356
+ df['Model_Engine'] = f'{model}_{engine}_{file_name}_{idx}'
357
+ self.anno_gpu_kernname(df,
358
+ EngineModelData.engine_model[engine][model])
359
+ # patch in non-gpu time
360
+ gpu_sec = round(df['Elapsed Time (sec)'].sum(), 1)
361
+ total_sec = round(float(total_sec), 1)
362
+ if total_sec < gpu_sec:
363
+ logger.warning(
364
+ "Elapsed sec %.2f < GPU sec %.2f resetting Elapsed sec ",
365
+ total_sec,
366
+ gpu_sec,
367
+ )
368
+ total_sec = gpu_sec
369
+ nongpu_row = self.make_nongpu_row(df, total_sec - gpu_sec)
370
+ df = self.pd.concat([df, nongpu_row], ignore_index=True)
371
+ combined_df = self.pd.concat([combined_df, df], ignore_index=True)
372
+ if out_dir is None:
373
+ out_dir = '.'
374
+ else:
375
+ os.makedirs(out_dir, exist_ok=True)
376
+ # generate html file
377
+ self.make_html(combined_df, out_dir, title)
378
+
379
+
380
+def parse_tuple(s):
381
+ return tuple(s.split(','))
382
+
383
+
384
+def main():
385
+ logging.basicConfig(format=('%(asctime)s - %(levelname)s - %(message)s'),
386
+ level=logging.INFO)
387
+ parser = argparse.ArgumentParser(
388
+ description=(
389
+ 'Process nsys rep and generate kernel non-overlapped cycles. \n'
390
+ 'Example:\n'
391
+ "gputrc2graph.py --in_file d1.nsys-rep,vllm,llama,100 \n"
392
+ "d2.nsys-rep,vllm,gpt-oss,102 "
393
+ "--out_dir results/ --title \"Model=gpt-oss vLLM chart\""),
394
+ formatter_class=argparse.RawDescriptionHelpFormatter)
395
+
396
+ # Build help string showing available engine/model combinations
397
+ engine_model_help = []
398
+ for engine, models in EngineModelData.engine_model.items():
399
+ model_list = list(models.keys())
400
+ engine_model_help.append(f"{engine}:[{','.join(model_list)}]")
401
+ engine_model_str = ' '.join(engine_model_help)
402
+ parser.add_argument(
403
+ '--in_file',
404
+ type=parse_tuple,
405
+ nargs='+',
406
+ help=(
407
+ 'list of (nsys-rep, engine, model, elapsed_nonprofiled_sec) '
408
+ 'separated by space. Elapsed_nonprofiled_sec is runtime without '
409
+ 'profiling used to calculate non-gpu time. Specify 0 to use '
410
+ 'elapsed time from nsys-rep but that might inflate non-gpu time. '
411
+ f'Available engine:[model] are: {engine_model_str} '
412
+ f'Example: --infile d1.nsys-rep,vllm,llama,100 '
413
+ 'd2.nsys-rep,vllm,gpt-oss,102'),
414
+ required=True)
415
+ parser.add_argument('--out_dir', help=('output dir for result.csv/html'))
416
+ parser.add_argument('--title', help=('title for html chart'))
417
+ parser.add_argument('--nsys_cmd',
418
+ help=('nsys cmd, e.g. /usr/bin/nsys, Default: nsys'),
419
+ default="nsys")
420
+ args = parser.parse_args()
421
+ gputrace = GPUTrace2Graph(args.nsys_cmd)
422
+ gputrace.gen_graph(args.in_file, args.out_dir, args.title)
423
+
424
+
425
+if __name__ == '__main__':
426
+ main()
tools/profiler/nsys_profile_tools/images/csv1.png
Binary files /dev/null and b/tools/profiler/nsys_profile_tools/images/csv1.png differ
tools/profiler/nsys_profile_tools/images/html.png
Binary files /dev/null and b/tools/profiler/nsys_profile_tools/images/html.png differ
tools/profiler/nsys_profile_tools/images/html_tbl.png
Binary files /dev/null and b/tools/profiler/nsys_profile_tools/images/html_tbl.png differ