@matheusbsbs / matheus-bagliano / commits / 8435b28989

git-remote-mediawiki: test environment of git-remote-mediawiki

In order to test git-remote-mediawiki, a set of functions is needed to manage a MediaWiki: edit a page, remove a page, fetch a page, fetch all pages on a given wiki. A few helper function are also provided to check the content of directories. In addition, this patch provides Makefiles to execute tests. See the README file for more details. Signed-off-by: Simon CATHEBRAS <Simon.Cathebras@ensimag.imag.fr> Signed-off-by: Julien KHAYAT <Julien.Khayat@ensimag.imag.fr> Signed-off-by: Simon Perrat <simon.perrat@ensimag.imag.fr> Signed-off-by: Charles ROUSSEL <Charles.Roussel@ensimag.imag.fr> Signed-off-by: Guillaume SASDY <Guillaume.Sasdy@ensimag.imag.fr> Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Guillaume Sasdy committed Jul 6, 2012 at 12:03 UTC 8435b289890739600436b6c902156fc073752ffb
6 files changed +556
contrib/mw-to-git/Makefile new
+47
@@ -0,0 +1,47 @@
1 +#
2 +# Copyright (C) 2012
3 +# Charles Roussel <charles.roussel@ensimag.imag.fr>
4 +# Simon Cathebras <simon.cathebras@ensimag.imag.fr>
5 +# Julien Khayat <julien.khayat@ensimag.imag.fr>
6 +# Guillaume Sasdy <guillaume.sasdy@ensimag.imag.fr>
7 +# Simon Perrat <simon.perrat@ensimag.imag.fr>
8 +#
9 +## Build git-remote-mediawiki
10 +
11 +-include ../../config.mak.autogen
12 +-include ../../config.mak
13 +
14 +ifndef PERL_PATH
15 + PERL_PATH = /usr/bin/perl
16 +endif
17 +ifndef gitexecdir
18 + gitexecdir = $(shell git --exec-path)
19 +endif
20 +
21 +PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
22 +gitexecdir_SQ = $(subst ','\'',$(gitexecdir))
23 +SCRIPT = git-remote-mediawiki
24 +
25 +.PHONY: install help doc test clean
26 +
27 +help:
28 + @echo 'This is the help target of the Makefile. Current configuration:'
29 + @echo ' gitexecdir = $(gitexecdir_SQ)'
30 + @echo ' PERL_PATH = $(PERL_PATH_SQ)'
31 + @echo 'Run "$(MAKE) install" to install $(SCRIPT) in gitexecdir'
32 + @echo 'Run "$(MAKE) test" to run the testsuite'
33 +
34 +install:
35 + sed -e '1s|#!.*/perl|#!$(PERL_PATH_SQ)|' $(SCRIPT) \
36 + > '$(gitexecdir_SQ)/$(SCRIPT)'
37 + chmod +x '$(gitexecdir)/$(SCRIPT)'
38 +
39 +doc:
40 + @echo 'Sorry, "make doc" is not implemented yet for $(SCRIPT)'
41 +
42 +test:
43 + $(MAKE) -C t/ test
44 +
45 +clean:
46 + $(RM) '$(gitexecdir)/$(SCRIPT)'
47 + $(MAKE) -C t/ clean
contrib/mw-to-git/t/.gitignore
+2
@@ -1,2 +1,4 @@
1 WEB/
2 wiki/
3 +trash directory.t*/
4 +test-results/
contrib/mw-to-git/t/Makefile new
+31
@@ -0,0 +1,31 @@
1 +#
2 +# Copyright (C) 2012
3 +# Charles Roussel <charles.roussel@ensimag.imag.fr>
4 +# Simon Cathebras <simon.cathebras@ensimag.imag.fr>
5 +# Julien Khayat <julien.khayat@ensimag.imag.fr>
6 +# Guillaume Sasdy <guillaume.sasdy@ensimag.imag.fr>
7 +# Simon Perrat <simon.perrat@ensimag.imag.fr>
8 +#
9 +## Test git-remote-mediawiki
10 +
11 +all: test
12 +
13 +-include ../../../config.mak.autogen
14 +-include ../../../config.mak
15 +
16 +T = $(wildcard t[0-9][0-9][0-9][0-9]-*.sh)
17 +
18 +.PHONY: help test clean all
19 +
20 +help:
21 + @echo 'Run "$(MAKE) test" to launch test scripts'
22 + @echo 'Run "$(MAKE) clean" to remove trash folders'
23 +
24 +test:
25 + @for t in $(T); do \
26 + echo "$$t"; \
27 + "./$$t" || exit 1; \
28 + done
29 +
30 +clean:
31 + $(RM) -r 'trash directory'.*
contrib/mw-to-git/t/README new
+124
@@ -0,0 +1,124 @@
1 +Tests for Mediawiki-to-Git
2 +==========================
3 +
4 +Introduction
5 +------------
6 +This manual describes how to install the git-remote-mediawiki test
7 +environment on a machine with git installed on it.
8 +
9 +Prerequisite
10 +------------
11 +
12 +In order to run this test environment correctly, you will need to
13 +install the following packages (Debian/Ubuntu names, may need to be
14 +adapted for another distribution):
15 +
16 +* lighttpd
17 +* php5
18 +* php5-cgi
19 +* php5-cli
20 +* php5-curl
21 +* php5-sqlite
22 +
23 +Principles and Technical Choices
24 +--------------------------------
25 +
26 +The test environment makes it easy to install and manipulate one or
27 +several MediaWiki instances. To allow developers to run the testsuite
28 +easily, the environment does not require root priviledge (except to
29 +install the required packages if needed). It starts a webserver
30 +instance on the user's account (using lighttpd greatly helps for
31 +that), and does not need a separate database daemon (thanks to the use
32 +of sqlite).
33 +
34 +Run the test environment
35 +------------------------
36 +
37 +Install a new wiki
38 +~~~~~~~~~~~~~~~~~~
39 +
40 +Once you have all the prerequisite, you need to install a MediaWiki
41 +instance on your machine. If you already have one, it is still
42 +strongly recommended to install one with the script provided. Here's
43 +how to work it:
44 +
45 +a. change directory to contrib/mw-to-git/t/
46 +b. if needed, edit test.config to choose your installation parameters
47 +c. run `./install-wiki.sh install`
48 +d. check on your favourite web browser if your wiki is correctly
49 + installed.
50 +
51 +Remove an existing wiki
52 +~~~~~~~~~~~~~~~~~~~~~~~
53 +
54 +Edit the file test.config to fit the wiki you want to delete, and then
55 +execute the command `./install-wiki.sh delete` from the
56 +contrib/mw-to-git/t directory.
57 +
58 +Run the existing tests
59 +~~~~~~~~~~~~~~~~~~~~~~
60 +
61 +The provided tests are currently in the `contrib/mw-to-git/t` directory.
62 +The files are all the t936[0-9]-*.sh shell scripts.
63 +
64 +a. Run all tests:
65 +To do so, run "make test" from the contrib/mw-to-git/ directory.
66 +
67 +b. Run a specific test:
68 +To run a given test <test_name>, run ./<test_name> from the
69 +contrib/mw-to-git/t directory.
70 +
71 +How to create new tests
72 +-----------------------
73 +
74 +Available functions
75 +~~~~~~~~~~~~~~~~~~~
76 +
77 +The test environment of git-remote-mediawiki provides some functions
78 +useful to test its behaviour. for more details about the functions'
79 +parameters, please refer to the `test-gitmw-lib.sh` and
80 +`test-gitmw.pl` files.
81 +
82 +** `test_check_wiki_precond`:
83 +Check if the tests must be skipped or not. Please use this function
84 +at the beggining of each new test file.
85 +
86 +** `wiki_getpage`:
87 +Fetch a given page from the wiki and puts its content in the
88 +directory in parameter.
89 +
90 +** `wiki_delete_page`:
91 +Delete a given page from the wiki.
92 +
93 +** `wiki_edit_page`:
94 +Create or modify a given page in the wiki. You can specify several
95 +parameters like a summary for the page edition, or add the page to a
96 +given category.
97 +See test-gitmw.pl for more details.
98 +
99 +** `wiki_getallpage`:
100 +Fetch all pages from the wiki into a given directory. The directory
101 +is created if it does not exists.
102 +
103 +** `test_diff_directories`:
104 +Compare the content of two directories. The content must be the same.
105 +Use this function to compare the content of a git directory and a wiki
106 +one created by wiki_getallpage.
107 +
108 +** `test_contains_N_files`:
109 +Check if the given directory contains a given number of file.
110 +
111 +** `wiki_page_exists`:
112 +Tests if a given page exists on the wiki.
113 +
114 +** `wiki_reset`:
115 +Reset the wiki, i.e. flush the database. Use this function at the
116 +begining of each new test, except if the test re-uses the same wiki
117 +(and history) as the previous test.
118 +
119 +How to write a new test
120 +~~~~~~~~~~~~~~~~~~~~~~~
121 +
122 +Please, follow the standards given by git. See git/t/README.
123 +New file should be named as t936[0-9]-*.sh.
124 +Be sure to reset your wiki regulary with the function `wiki_reset`.
contrib/mw-to-git/t/test-gitmw-lib.sh
+143
@@ -13,6 +13,7 @@
13
14 . ./test.config
15
16 +WIKI_URL=http://"$SERVER_ADDR:$PORT/$WIKI_DIR_NAME"
17 CURR_DIR=$(pwd)
18 TEST_OUTPUT_DIRECTORY=$(pwd)
19 TEST_DIRECTORY="$CURR_DIR"/../../../t
@@ -25,6 +26,148 @@ else
26 WIKI_DIR_INST="$CURR_DIR/$WEB_WWW"
27 fi
28
29 +
30 +wiki_getpage () {
31 + "$CURR_DIR"/test-gitmw.pl get_page "$@"
32 +}
33 +
34 +wiki_delete_page () {
35 + "$CURR_DIR"/test-gitmw.pl delete_page "$@"
36 +}
37 +
38 +wiki_editpage () {
39 + "$CURR_DIR"/test-gitmw.pl edit_page "$@"
40 +}
41 +
42 +die () {
43 + die_with_status 1 "$@"
44 +}
45 +
46 +die_with_status () {
47 + status=$1
48 + shift
49 + echo >&2 "$*"
50 + exit "$status"
51 +}
52 +
53 +
54 +# Check the preconditions to run git-remote-mediawiki's tests
55 +test_check_precond () {
56 + if ! test_have_prereq PERL
57 + then
58 + skip_all='skipping gateway git-mw tests, perl not available'
59 + test_done
60 + fi
61 +
62 + if [ ! -f "$GIT_BUILD_DIR"/git-remote-mediawiki ];
63 + then
64 + echo "No remote mediawiki for git found. Copying it in git"
65 + echo "cp $GIT_BUILD_DIR/contrib/mw-to-git/git-remote-mediawiki $GIT_BUILD_DIR/"
66 + ln -s "$GIT_BUILD_DIR"/contrib/mw-to-git/git-remote-mediawiki "$GIT_BUILD_DIR"
67 + fi
68 +
69 + if [ ! -d "$WIKI_DIR_INST/$WIKI_DIR_NAME" ];
70 + then
71 + skip_all='skipping gateway git-mw tests, no mediawiki found'
72 + test_done
73 + fi
74 +}
75 +
76 +# test_diff_directories <dir_git> <dir_wiki>
77 +#
78 +# Compare the contents of directories <dir_git> and <dir_wiki> with diff
79 +# and errors if they do not match. The program will
80 +# not look into .git in the process.
81 +# Warning: the first argument MUST be the directory containing the git data
82 +test_diff_directories () {
83 + rm -rf "$1_tmp"
84 + mkdir -p "$1_tmp"
85 + cp "$1"/*.mw "$1_tmp"
86 + diff -r -b "$1_tmp" "$2"
87 +}
88 +
89 +# $1=<dir>
90 +# $2=<N>
91 +#
92 +# Check that <dir> contains exactly <N> files
93 +test_contains_N_files () {
94 + if test `ls -- "$1" | wc -l` -ne "$2"; then
95 + echo "directory $1 sould contain $2 files"
96 + echo "it contains these files:"
97 + ls "$1"
98 + false
99 + fi
100 +}
101 +
102 +
103 +# wiki_check_content <file_name> <page_name>
104 +#
105 +# Compares the contents of the file <file_name> and the wiki page
106 +# <page_name> and exits with error 1 if they do not match.
107 +wiki_check_content () {
108 + mkdir -p wiki_tmp
109 + wiki_getpage "$2" wiki_tmp
110 + # replacement of forbidden character in file name
111 + page_name=$(printf "%s\n" "$2" | sed -e "s/\//%2F/g")
112 +
113 + diff -b "$1" wiki_tmp/"$page_name".mw
114 + if test $? -ne 0
115 + then
116 + rm -rf wiki_tmp
117 + error "ERROR: file $2 not found on wiki"
118 + fi
119 + rm -rf wiki_tmp
120 +}
121 +
122 +# wiki_page_exist <page_name>
123 +#
124 +# Check the existence of the page <page_name> on the wiki and exits
125 +# with error if it is absent from it.
126 +wiki_page_exist () {
127 + mkdir -p wiki_tmp
128 + wiki_getpage "$1" wiki_tmp
129 + page_name=$(printf "%s\n" "$1" | sed "s/\//%2F/g")
130 + if test -f wiki_tmp/"$page_name".mw ; then
131 + rm -rf wiki_tmp
132 + else
133 + rm -rf wiki_tmp
134 + error "test failed: file $1 not found on wiki"
135 + fi
136 +}
137 +
138 +# wiki_getallpagename
139 +#
140 +# Fetch the name of each page on the wiki.
141 +wiki_getallpagename () {
142 + "$CURR_DIR"/test-gitmw.pl getallpagename
143 +}
144 +
145 +# wiki_getallpagecategory <category>
146 +#
147 +# Fetch the name of each page belonging to <category> on the wiki.
148 +wiki_getallpagecategory () {
149 + "$CURR_DIR"/test-gitmw.pl getallpagename "$@"
150 +}
151 +
152 +# wiki_getallpage <dest_dir> [<category>]
153 +#
154 +# Fetch all the pages from the wiki and place them in the directory
155 +# <dest_dir>.
156 +# If <category> is define, then wiki_getallpage fetch the pages included
157 +# in <category>.
158 +wiki_getallpage () {
159 + if test -z "$2";
160 + then
161 + wiki_getallpagename
162 + else
163 + wiki_getallpagecategory "$2"
164 + fi
165 + mkdir -p "$1"
166 + while read -r line; do
167 + wiki_getpage "$line" $1;
168 + done < all.txt
169 +}
170 +
171 # ================= Install part =================
172
173 error () {
contrib/mw-to-git/t/test-gitmw.pl new
+209
@@ -0,0 +1,209 @@
1 +#!/usr/bin/perl -w -s
2 +# Copyright (C) 2012
3 +# Charles Roussel <charles.roussel@ensimag.imag.fr>
4 +# Simon Cathebras <simon.cathebras@ensimag.imag.fr>
5 +# Julien Khayat <julien.khayat@ensimag.imag.fr>
6 +# Guillaume Sasdy <guillaume.sasdy@ensimag.imag.fr>
7 +# Simon Perrat <simon.perrat@ensimag.imag.fr>
8 +# License: GPL v2 or later
9 +
10 +# Usage:
11 +# ./test-gitmw.pl <command> [argument]*
12 +# Execute in terminal using the name of the function to call as first
13 +# parameter, and the function's arguments as following parameters
14 +#
15 +# Example:
16 +# ./test-gitmw.pl "get_page" foo .
17 +# will call <wiki_getpage> with arguments <foo> and <.>
18 +#
19 +# Available functions are:
20 +# "get_page"
21 +# "delete_page"
22 +# "edit_page"
23 +# "getallpagename"
24 +
25 +use MediaWiki::API;
26 +use Getopt::Long;
27 +use encoding 'utf8';
28 +use DateTime::Format::ISO8601;
29 +use open ':encoding(utf8)';
30 +use constant SLASH_REPLACEMENT => "%2F";
31 +
32 +#Parsing of the config file
33 +
34 +my $configfile = "$ENV{'CURR_DIR'}/test.config";
35 +my %config;
36 +open my $CONFIG, "<", $configfile or die "can't open $configfile: $!";
37 +while (<$CONFIG>)
38 +{
39 + chomp;
40 + s/#.*//;
41 + s/^\s+//;
42 + s/\s+$//;
43 + next unless length;
44 + my ($key, $value) = split (/\s*=\s*/,$_, 2);
45 + $config{$key} = $value;
46 + last if ($key eq 'LIGHTTPD' and $value eq 'false');
47 + last if ($key eq 'PORT');
48 +}
49 +close $CONFIG or die "can't close $configfile: $!";
50 +
51 +my $wiki_address = "http://$config{'SERVER_ADDR'}".":"."$config{'PORT'}";
52 +my $wiki_url = "$wiki_address/$config{'WIKI_DIR_NAME'}/api.php";
53 +my $wiki_admin = "$config{'WIKI_ADMIN'}";
54 +my $wiki_admin_pass = "$config{'WIKI_PASSW'}";
55 +my $mw = MediaWiki::API->new;
56 +$mw->{config}->{api_url} = $wiki_url;
57 +
58 +
59 +# wiki_login <name> <password>
60 +#
61 +# Logs the user with <name> and <password> in the global variable
62 +# of the mediawiki $mw
63 +sub wiki_login {
64 + $mw->login( { lgname => "$_[0]",lgpassword => "$_[1]" } )
65 + || die "getpage: login failed";
66 +}
67 +
68 +# wiki_getpage <wiki_page> <dest_path>
69 +#
70 +# fetch a page <wiki_page> from the wiki referenced in the global variable
71 +# $mw and copies its content in directory dest_path
72 +sub wiki_getpage {
73 + my $pagename = $_[0];
74 + my $destdir = $_[1];
75 +
76 + my $page = $mw->get_page( { title => $pagename } );
77 + if (!defined($page)) {
78 + die "getpage: wiki does not exist";
79 + }
80 +
81 + my $content = $page->{'*'};
82 + if (!defined($content)) {
83 + die "getpage: page does not exist";
84 + }
85 +
86 + $pagename=$page->{'title'};
87 + # Replace spaces by underscore in the page name
88 + $pagename =~ s/ /_/g;
89 + $pagename =~ s/\//%2F/g;
90 + open(my $file, ">$destdir/$pagename.mw");
91 + print $file "$content";
92 + close ($file);
93 +
94 +}
95 +
96 +# wiki_delete_page <page_name>
97 +#
98 +# delete the page with name <page_name> from the wiki referenced
99 +# in the global variable $mw
100 +sub wiki_delete_page {
101 + my $pagename = $_[0];
102 +
103 + my $exist=$mw->get_page({title => $pagename});
104 +
105 + if (defined($exist->{'*'})){
106 + $mw->edit({ action => 'delete',
107 + title => $pagename})
108 + || die $mw->{error}->{code} . ": " . $mw->{error}->{details};
109 + } else {
110 + die "no page with such name found: $pagename\n";
111 + }
112 +}
113 +
114 +# wiki_editpage <wiki_page> <wiki_content> <wiki_append> [-c=<category>] [-s=<summary>]
115 +#
116 +# Edit a page named <wiki_page> with content <wiki_content> on the wiki
117 +# referenced with the global variable $mw
118 +# If <wiki_append> == true : append <wiki_content> at the end of the actual
119 +# content of the page <wiki_page>
120 +# If <wik_page> doesn't exist, that page is created with the <wiki_content>
121 +sub wiki_editpage {
122 + my $wiki_page = $_[0];
123 + my $wiki_content = $_[1];
124 + my $wiki_append = $_[2];
125 + my $summary = "";
126 + my ($summ, $cat) = ();
127 + GetOptions('s=s' => \$summ, 'c=s' => \$cat);
128 +
129 + my $append = 0;
130 + if (defined($wiki_append) && $wiki_append eq 'true') {
131 + $append=1;
132 + }
133 +
134 + my $previous_text ="";
135 +
136 + if ($append) {
137 + my $ref = $mw->get_page( { title => $wiki_page } );
138 + $previous_text = $ref->{'*'};
139 + }
140 +
141 + my $text = $wiki_content;
142 + if (defined($previous_text)) {
143 + $text="$previous_text$text";
144 + }
145 +
146 + # Eventually, add this page to a category.
147 + if (defined($cat)) {
148 + my $category_name="[[Category:$cat]]";
149 + $text="$text\n $category_name";
150 + }
151 + if(defined($summ)){
152 + $summary=$summ;
153 + }
154 +
155 + $mw->edit( { action => 'edit', title => $wiki_page, summary => $summary, text => "$text"} );
156 +}
157 +
158 +# wiki_getallpagename [<category>]
159 +#
160 +# Fetch all pages of the wiki referenced by the global variable $mw
161 +# and print the names of each one in the file all.txt with a new line
162 +# ("\n") between these.
163 +# If the argument <category> is defined, then this function get only the pages
164 +# belonging to <category>.
165 +sub wiki_getallpagename {
166 + # fetch the pages of the wiki
167 + if (defined($_[0])) {
168 + my $mw_pages = $mw->list ( { action => 'query',
169 + list => 'categorymembers',
170 + cmtitle => "Category:$_[0]",
171 + cmnamespace => 0,
172 + cmlimit => 500 },
173 + )
174 + || die $mw->{error}->{code}.": ".$mw->{error}->{details};
175 + open(my $file, ">all.txt");
176 + foreach my $page (@{$mw_pages}) {
177 + print $file "$page->{title}\n";
178 + }
179 + close ($file);
180 +
181 + } else {
182 + my $mw_pages = $mw->list({
183 + action => 'query',
184 + list => 'allpages',
185 + aplimit => 500,
186 + })
187 + || die $mw->{error}->{code}.": ".$mw->{error}->{details};
188 + open(my $file, ">all.txt");
189 + foreach my $page (@{$mw_pages}) {
190 + print $file "$page->{title}\n";
191 + }
192 + close ($file);
193 + }
194 +}
195 +
196 +# Main part of this script: parse the command line arguments
197 +# and select which function to execute
198 +my $fct_to_call = shift;
199 +
200 +wiki_login($wiki_admin, $wiki_admin_pass);
201 +
202 +my %functions_to_call = qw(
203 + get_page wiki_getpage
204 + delete_page wiki_delete_page
205 + edit_page wiki_editpage
206 + getallpagename wiki_getallpagename
207 +);
208 +die "$0 ERROR: wrong argument" unless exists $functions_to_call{$fct_to_call};
209 +&{$functions_to_call{$fct_to_call}}(@ARGV);