gitweb: Load checking
This changes slightly the behavior of gitweb, so that it verifies that the box isn't inundated with before attempting to serve gitweb. If the box is overloaded, it basically returns a 503 Server Unavailable until the load falls below the defined threshold. This helps dramatically if you have a box that's I/O bound, reaches a certain load and you don't want gitweb, the I/O hog that it is, increasing the pain the server is already undergoing. This behavior is controlled by $maxload configuration variable. Default is a load of 300, which for most cases should never be hit. Unset it (set it to undefined value, i.e. undef) to turn off checking. Currently it requires that '/proc/loadavg' file exists, otherwise the load check is bypassed (load is taken to be 0). So platforms that do not implement '/proc/loadavg' currently cannot use this feature (provisions are included for additional checks to be added by others). There is simple test in t/t9501-gitweb-standalone-http-status.sh to check that it correctly returns "503 Service Unavailable" if load is too high, and also if there are any Perl warnings or errors. Signed-off-by: John 'Warthog9' Hawley <warthog9@kernel.org> Signed-off-by: Jakub Narebski <jnareb@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
John 'Warthog9' Hawley committed
Jan 30, 2010 at 23:30 UTC
b62a1a98bc65f52d972a3f403c5b29d6a84642fd
4 files changed
+72
-6
gitweb/README
+6
-1
@@ -174,7 +174,7 @@ not include variables usually directly set during build):
174
Base URL for relative URLs in pages generated by gitweb,
175
(e.g. $logo, $favicon, @stylesheets if they are relative URLs),
176
needed and used only for URLs with nonempty PATH_INFO via
177
- <base href="$base_url>. Usually gitweb sets its value correctly,
177
+ <base href="$base_url">. Usually gitweb sets its value correctly,
178
and there is no need to set this variable, e.g. to $my_uri or "/".
179
* $home_link
180
Target of the home link on top of all pages (the first part of view
@@ -228,6 +228,11 @@ not include variables usually directly set during build):
228
repositories from launching cross-site scripting (XSS) attacks. Set this
229
to true if you don't trust the content of your repositories. The default
230
is false.
231
+ * $maxload
232
+ Used to set the maximum load that we will still respond to gitweb queries.
233
+ If server load exceed this value then return "503 Service Unavaliable" error.
234
+ Server load is taken to be 0 if gitweb cannot determine its value. Set it to
235
+ undefined value to turn it off. The default is 300.
236
237
238
Projects list file format
gitweb/gitweb.perl
+43
-5
@@ -221,6 +221,12 @@ our %avatar_size = (
221
'double' => 32
222
);
223
224
+# Used to set the maximum load that we will still respond to gitweb queries.
225
+# If server load exceed this value then return "503 server busy" error.
226
+# If gitweb cannot determined server load, it is taken to be 0.
227
+# Leave it undefined (or set to 'undef') to turn off load checking.
228
+our $maxload = 300;
229
+
230
# You define site-wide feature defaults here; override them with
231
# $GITWEB_CONFIG as necessary.
232
our %feature = (
@@ -551,12 +557,38 @@ if (-e $GITWEB_CONFIG) {
557
do $GITWEB_CONFIG_SYSTEM if -e $GITWEB_CONFIG_SYSTEM;
558
}
559
560
+# Get loadavg of system, to compare against $maxload.
561
+# Currently it requires '/proc/loadavg' present to get loadavg;
562
+# if it is not present it returns 0, which means no load checking.
563
+sub get_loadavg {
564
+ if( -e '/proc/loadavg' ){
565
+ open my $fd, '<', '/proc/loadavg'
566
+ or return 0;
567
+ my @load = split(/\s+/, scalar <$fd>);
568
+ close $fd;
569
+
570
+ # The first three columns measure CPU and IO utilization of the last one,
571
+ # five, and 10 minute periods. The fourth column shows the number of
572
+ # currently running processes and the total number of processes in the m/n
573
+ # format. The last column displays the last process ID used.
574
+ return $load[0] || 0;
575
+ }
576
+ # additional checks for load average should go here for things that don't export
577
+ # /proc/loadavg
578
+
579
+ return 0;
580
+}
581
+
582
# version of the core git binary
583
our $git_version = qx("$GIT" --version) =~ m/git version (.*)$/ ? $1 : "unknown";
584
$number_of_git_cmds++;
585
586
$projects_list ||= $projectroot;
587
588
+if (defined $maxload && get_loadavg() > $maxload) {
589
+ die_error(503, "The load average on the server is too high");
590
+}
591
+
592
# ======================================================================
593
# input validation and dispatch
594
@@ -3328,7 +3360,8 @@ sub git_footer_html {
3360
}
3361
3362
print qq!<script type="text/javascript" src="$javascript"></script>\n!;
3331
- if ($action eq 'blame_incremental') {
3363
+ if (defined $action &&
3364
+ $action eq 'blame_incremental') {
3365
print qq!<script type="text/javascript">\n!.
3366
qq!startBlame("!. href(action=>"blame_data", -replay=>1) .qq!",\n!.
3367
qq! "!. href() .qq!");\n!.
@@ -3354,14 +3387,19 @@ sub git_footer_html {
3387
# 500: The server isn't configured properly, or
3388
# an internal error occurred (e.g. failed assertions caused by bugs), or
3389
# an unknown error occurred (e.g. the git binary died unexpectedly).
3390
+# 503: The server is currently unavailable (because it is overloaded,
3391
+# or down for maintenance). Generally, this is a temporary state.
3392
sub die_error {
3393
my $status = shift || 500;
3394
my $error = shift || "Internal server error";
3395
3361
- my %http_responses = (400 => '400 Bad Request',
3362
- 403 => '403 Forbidden',
3363
- 404 => '404 Not Found',
3364
- 500 => '500 Internal Server Error');
3396
+ my %http_responses = (
3397
+ 400 => '400 Bad Request',
3398
+ 403 => '403 Forbidden',
3399
+ 404 => '404 Not Found',
3400
+ 500 => '500 Internal Server Error',
3401
+ 503 => '503 Service Unavailable',
3402
+ );
3403
git_header_html($http_responses{$status});
3404
print <<EOF;
3405
<div class="page_body">
t/gitweb-lib.sh
+1
@@ -25,6 +25,7 @@ our \$favicon = 'file:///$TEST_DIRECTORY/../gitweb/git-favicon.png';
25
our \$projects_list = '';
26
our \$export_ok = '';
27
our \$strict_export = '';
28
+our \$maxload = undef;
29
30
EOF
31
t/t9501-gitweb-standalone-http-status.sh
+22
@@ -112,4 +112,26 @@ test_expect_success 'snapshots: bad object id' '
112
test_debug 'cat gitweb.output'
113
114
115
+# ----------------------------------------------------------------------
116
+# load checking
117
+
118
+# always hit the load limit
119
+cat >>gitweb_config.perl <<\EOF
120
+our $maxload = 0;
121
+EOF
122
+
123
+test_expect_success 'load checking: load too high (default action)' '
124
+ gitweb_run "p=.git" &&
125
+ grep "Status: 503 Service Unavailable" gitweb.headers &&
126
+ grep "503 - The load average on the server is too high" gitweb.body
127
+'
128
+test_debug 'cat gitweb.log' # just in case
129
+test_debug 'cat gitweb.headers'
130
+
131
+# turn off load checking
132
+cat >>gitweb_config.perl <<\EOF
133
+our $maxload = undef;
134
+EOF
135
+
136
+
137
test_done