@matheusbsbs / matheus-bagliano / commits / b0d8923ec0
Fork 0

[PATCH] Improve git-rev-list memory usage further

This avoids keeping tree entries around, and free's them as it traverses the list. This avoids building up a huge memory footprint just for these small but very common allocations. Before: $ /usr/bin/time git-rev-list --objects v2.6.12..HEAD | wc -l 11.65user 0.38system 0:12.65elapsed 95%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+42934minor)pagefaults 0swaps 59124 After: $ /usr/bin/time git-rev-list --objects v2.6.12..HEAD | wc -l 12.28user 0.29system 0:12.57elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+26718minor)pagefaults 0swaps 59124 Note how the minor fault numbers - which ends up being how many pages we needed to map - go down from 42934 (167 MB) to 26718 (104 MB). That is: Before: 42934 minor pagefaults After: 26718 minor pagefaults This is all in _addition_ to the previous fixes. It used to be ~48,000 pagefaults. That's still a honking big memory footprint, but it's about half of what it was just a day or two ago (and this is the object list for a pretty big update - almost 60,000 objects. Smaller updates need less memory). Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>

Linus Torvalds committed Sep 16, 2005 at 14:32 UTC b0d8923ec01fd91b75ab079034f89ced91500157
1 file changed +10 -2
rev-list.c
+10 -2
@@ -147,11 +147,16 @@ static struct object_list **process_tree(struct tree *tree, struct object_list *
147 die("bad tree object %s", sha1_to_hex(obj->sha1));
148 obj->flags |= SEEN;
149 p = add_object(obj, p, name);
150 - for (entry = tree->entries ; entry ; entry = entry->next) {
150 + entry = tree->entries;
151 + tree->entries = NULL;
152 + while (entry) {
153 + struct tree_entry_list *next = entry->next;
154 if (entry->directory)
155 p = process_tree(entry->item.tree, p, entry->name);
156 else
157 p = process_blob(entry->item.blob, p, entry->name);
158 + free(entry);
159 + entry = next;
160 }
161 return p;
162 }
@@ -218,12 +223,15 @@ static void mark_tree_uninteresting(struct tree *tree)
223 if (parse_tree(tree) < 0)
224 die("bad tree %s", sha1_to_hex(obj->sha1));
225 entry = tree->entries;
226 + tree->entries = NULL;
227 while (entry) {
228 + struct tree_entry_list *next = entry->next;
229 if (entry->directory)
230 mark_tree_uninteresting(entry->item.tree);
231 else
232 mark_blob_uninteresting(entry->item.blob);
226 - entry = entry->next;
233 + free(entry);
234 + entry = next;
235 }
236 }
237