Merge ~ilasc/turnip:trigger-auto-gc into turnip:master

[Ioana Lasc]

Add test to trigger the Git GC

This unit test confirms the inner workings of the Git GC with regard to the threshold where it considers the repository insufficiently packed.

This investigation is work in preparation for the feature document: https://docs.google.com/document/d/1Y4fnuciBsbRQhPjZcbKTVQqHcT7kkrIhHwdByNAIJCo/edit?usp=sharing

The next step here is to trigger the Git GC and have it error out the same way it does in Prod today with: "error: The last gc run reported the following. Please correct the root
cause and remove gc.log."

I'm not proposing this as an actual "I intend to merge this once approved", it's a "keeping progress visible" MP for now.

[Ioana Lasc (ilasc)]

Rejecting this as merging this Unit test into Master was never the purpose of the MP.

Practical elements from it have already been used in https://code.launchpad.net/~ilasc/turnip/+git/turnip/+merge/394581 and "theory verification" elements will be documented in the feature document: https://docs.google.com/document/d/1Y4fnuciBsbRQhPjZcbKTVQqHcT7kkrIhHwdByNAIJCo/edit?usp=sharing

Unmerged commits

1972951... by Ioana Lasc on 2020-11-25

Add test to trigger the Git GC

diff --git a/turnip/api/tests/test_store.py b/turnip/api/tests/test_store.py
index 5108581..23f7de1 100644
--- a/turnip/api/tests/test_store.py
+++ b/turnip/api/tests/test_store.py
@@ -10,6 +10,7 @@ from __future__ import (
 import os.path
 import re
 import subprocess
+import time
 import uuid
 
 from fixtures import (
@@ -446,3 +447,62 @@ class InitTestCase(TestCase):
         self.assertEqual(before_refs_len - 1, len(orig.references.objects))
         self.assertNotIn(
             new_branch_name, [i.name for i in orig.references.objects])
+
+    def test_pre_gc_auto_hook(self):
+
+        curdir = os.getcwd()
+        gc_path = os.path.join(self.repo_store, 'test_gc/')
+        gc_factory = RepoFactory(
+            gc_path, num_branches=3, num_commits=300, num_tags=3)
+        os.mkdir(os.path.join(gc_path, 'worktree'))
+        gc_factory.build()
+        os.chdir(os.path.join(gc_path, 'worktree'))
+        repo_config = pygit2.Repository(gc_path).config
+
+        # set gc.auto to a low value to verify git's threshlod
+        # for triggering auto packing of the repository
+        repo_config['gc.auto'] = 1
+        repo_config['core.bare'] = False
+        repo_config['core.worktree'] = os.path.join(gc_path, 'worktree')
+        assert 'gc.auto' in repo_config
+        assert repo_config.get_int('gc.auto') == 1
+        assert 'core.worktree' in repo_config
+        assert repo_config['core.worktree']
+
+        files = os.listdir(os.path.join(gc_path, 'objects/17'))
+        self.assertGreater(len(files), repo_config.get_int('gc.auto'))
+        repo = pygit2.Repository(gc_path)
+        branch = repo.lookup_branch('master')
+        ref = repo.lookup_reference(branch.name)
+        repo.checkout(ref)
+        oid = gc_factory.add_commit('foo', 'foobar.txt')
+        repo.head.set_target(oid)
+        repo.index.write()
+
+        # a repository with 300 commits and 3 branches should have
+        # around 2700 loose objects under objects/17
+        # we assert that their number is above 0 at this point
+        objects = subprocess.check_output(['git', 'count-objects'])
+        self.assertGreater(int(objects[0:(objects.find(' objects'))]), 0)
+
+        # at this point we have more objects in forlder objects/17 than
+        # we define in the config file under gc.auto and
+        # with the next commit automated garbace collection will trigger
+        commit_output = subprocess.check_output(
+            ['git commit -m \"test_commit_message\"'],
+            stderr=subprocess.STDOUT, shell=True)
+
+        m1 = "Auto packing the repository in background"
+        m2 = "for optimum performance."
+        self.assertIn(m1+" "+m2, commit_output)
+
+        # give gc 2 seconds to run
+        time.sleep(2)
+
+        # after above commit automated gc ran and there will
+        # be 0 loose objects as they just got repacked by gc
+        free_objects = subprocess.check_output(['git', 'count-objects'])
+        self.assertEquals(
+            int(free_objects[0:(free_objects.find(' objects'))]),
+            0)
+        os.chdir(curdir)