@matheusbsbs / matheus-bagliano / commits / 779a206632

gitweb: use highlight's shebang detection

The "highlight" binary can, in some cases, determine the language type by the means of file contents, for example the shebang in the first line for some scripting languages. Make use of this autodetection for files which syntax is not known by gitweb. In that case, pass the blob contents to "highlight --force"; the parameter is needed to make it always generate HTML output (which includes HTML-escaping). Although we now run highlight on files which do not end up highlighted, performance is virtually unaffected because when we call highlight, it is used for escaping HTML. In the case that highlight is used, gitweb calls sanitize() instead of esc_html(), and the latter is significantly slower (it does more, being roughly a superset of sanitize()). Simple benchmark comparing performance of 'blob' view of files without syntax highlighting in gitweb before and after this change indicates ±1% difference in request time for all file types. Benchmark was performed on local instance on Debian, using Apache/2.4.23 web server and CGI. Document the feature and improve syntax highlight documentation, add test to ensure gitweb doesn't crash when language detection is used. Signed-off-by: Ian Kelling <ian@iankelling.org> Acked-by: Jakub Narębski <jnareb@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Ian Kelling committed Sep 24, 2016 at 15:32 UTC 779a20663230ab068dcbc3c5bc53d44a5e37b0aa

3 files changed +27 -12

Documentation/gitweb.conf.txt

+14 -7

@@ -246,13 +246,20 @@ $highlight_bin::

246 Note that 'highlight' feature must be set for gitweb to actually

247 use syntax highlighting.

248 +

249 -*NOTE*: if you want to add support for new file type (supported by

250 -"highlight" but not used by gitweb), you need to modify `%highlight_ext`

251 -or `%highlight_basename`, depending on whether you detect type of file

252 -based on extension (for example "sh") or on its basename (for example

253 -"Makefile"). The keys of these hashes are extension and basename,

254 -respectively, and value for given key is name of syntax to be passed via

255 -`--syntax <syntax>` to highlighter.

249 +*NOTE*: for a file to be highlighted, its syntax type must be detected

250 +and that syntax must be supported by "highlight". The default syntax

251 +detection is minimal, and there are many supported syntax types with no

252 +detection by default. There are three options for adding syntax

253 +detection. The first and second priority are `%highlight_basename` and

254 +`%highlight_ext`, which detect based on basename (the full filename, for

255 +example "Makefile") and extension (for example "sh"). The keys of these

256 +hashes are the basename and extension, respectively, and the value for a

257 +given key is the name of the syntax to be passed via `--syntax <syntax>`

258 +to "highlight". The last priority is the "highlight" configuration of

259 +`Shebang` regular expressions to detect the language based on the first

260 +line in the file, (for example, matching the line "#!/bin/bash"). See

261 +the highlight documentation and the default config at

262 +/etc/highlight/filetypes.conf for more details.

263 +

264 For example if repositories you are hosting use "phtml" extension for

265 PHP files, and you want to have correct syntax-highlighting for those

gitweb/gitweb.perl

+5 -5

@@ -3931,15 +3931,16 @@ sub guess_file_syntax {

3931 # or return original FD if no highlighting

3932 sub run_highlighter {

3933 my ($fd, $highlight, $syntax) = @_;

3934 - return $fd unless ($highlight && defined $syntax);

3934 + return $fd unless ($highlight);

3935

3936 close $fd;

3937 + my $syntax_arg = (defined $syntax) ? "--syntax $syntax" : "--force";

3938 open $fd, quote_command(git_cmd(), "cat-file", "blob", $hash)." | ".

3939 quote_command($^X, '-CO', '-MEncode=decode,FB_DEFAULT', '-pse',

3940 '$_ = decode($fe, $_, FB_DEFAULT) if !utf8::decode($_);',

3941 '--', "-fe=$fallback_encoding")." | ".

3942 quote_command($highlight_bin).

3942 - " --replace-tabs=8 --fragment --syntax $syntax |"

3943 + " --replace-tabs=8 --fragment $syntax_arg |"

3944 or die_error(500, "Couldn't open file or run syntax highlighter");

3945 return $fd;

3946 }

@@ -7063,8 +7064,7 @@ sub git_blob {

7064

7065 my $highlight = gitweb_check_feature('highlight');

7066 my $syntax = guess_file_syntax($highlight, $file_name);

7066 - $fd = run_highlighter($fd, $highlight, $syntax)

7067 - if $syntax;

7067 + $fd = run_highlighter($fd, $highlight, $syntax);

7068

7069 git_header_html(undef, $expires);

7070 my $formats_nav = '';

@@ -7117,7 +7117,7 @@ sub git_blob {

7117 $line = untabify($line);

7118 printf qq!<div class="pre"><a id="l%i" href="%s#l%i" class="linenr">%4i</a> %s</div>\n!,

7119 $nr, esc_attr(href(-replay => 1)), $nr, $nr,

7120 - $syntax ? sanitize($line) : esc_html($line, -nbsp=>1);

7120 + $highlight ? sanitize($line) : esc_html($line, -nbsp=>1);

7121 }

7122 }

7123 close $fd

t/t9500-gitweb-standalone-no-errors.sh

@@ -709,6 +709,14 @@ test_expect_success HIGHLIGHT \

709 git commit -m "Add test.sh" &&

710 gitweb_run "p=.git;a=blob;f=test.sh"'

711

712 +test_expect_success HIGHLIGHT \

713 + 'syntax highlighting (highlighter language autodetection)' \

714 + 'git config gitweb.highlight yes &&

715 + echo "#!/usr/bin/perl" > test &&

716 + git add test &&

717 + git commit -m "Add test" &&

718 + gitweb_run "p=.git;a=blob;f=test"'

719 +

720 # ----------------------------------------------------------------------

721 # forks of projects

722