Merge lp:~stub/launchpad/replication into lp:launchpad/db-devel

[Stuart Bishop]

[Stuart Bishop (stub)]

Bug #504807

Changes made late last year to the staging rebuild broke things - sequence values in the authdb replication set stopped being restored. This broke parts of the system, such as account creation, adding email addresses etc. and required manual repair by the losas.

This branch automates the repair work as part of the rebuild process, as restoring the sequence from the actual database dumps would involve much more invasive refactoring and gain us nothing.

[Stuart Bishop (stub)]

Also fix Bug #504751, where new-slave.py stopped subscribing the new slave to the authdb replication set after we move its master to a new database.

[Henning Eggers (henninge)]

Looks good, thank you!

review: Approve (code)

=== modified file 'database/replication/Makefile'
--- database/replication/Makefile	2010-01-06 15:53:35 +0000
+++ database/replication/Makefile	2010-01-13 08:43:16 +0000
@@ -110,6 +110,7 @@
 		--no-acl --no-owner --disable-triggers --data-only \
 		--table=$$table ${STAGING_DUMP}; \
 	done
+	psql -q -d lpmain_staging_new -f authdb_sequences.sql
 	# Uninstall Slony-I if it is installed - a pg_dump of a DB with
 	# Slony-I installed isn't usable without this step.
 	LPCONFIG=${NEW_STAGING_CONFIG} ./repair-restored-db.py

=== added file 'database/replication/authdb_sequences.sql'
--- database/replication/authdb_sequences.sql	1970-01-01 00:00:00 +0000
+++ database/replication/authdb_sequences.sql	2010-01-13 08:43:16 +0000
@@ -0,0 +1,22 @@
+-- Repair sequences in the authdb replication set. We need to do this because
+-- we cannot restore the sequence values from the dump when restoring the
+-- data using pg_restore --data-only.
+
+SELECT setval('account_id_seq', max(id)) AS Account
+FROM Account;
+
+SELECT setval('accountpassword_id_seq', max(id)) AS AccountPassword
+FROM AccountPassword;
+
+SELECT setval('authtoken_id_seq', max(id)) AS AuthToken
+FROM AuthToken;
+
+SELECT setval('emailaddress_id_seq', max(id)) AS EmailAddress
+FROM EmailAddress;
+
+SELECT setval('openidauthorization_id_seq', max(id)) AS OpenIDAuthorization
+FROM OpenIDAuthorization;
+
+SELECT setval('openidrpsummary_id_seq', max(id)) AS OpenIDRPSummary
+FROM OpenIDRPSummary;
+

=== modified file 'database/replication/new-slave.py'
--- database/replication/new-slave.py	2009-12-03 09:47:24 +0000
+++ database/replication/new-slave.py	2010-01-13 08:43:16 +0000
@@ -162,13 +162,22 @@
     target_con.commit()
     del target_con
 
-    # Get a list of existing set ids.
+    # Get a list of existing set ids that can be subscribed too. This
+    # is all sets where the origin is the master_node, and set 2 if
+    # the master happens to be configured as a forwarding slave. We
+    # don't allow other sets where the master is configured as a
+    # forwarding slave as we have to special case rebuilding the database
+    # schema (such as we do for the authdb replication set 2).
     source_connection.rollback()
     master_node = replication.helpers.get_master_node(source_connection)
     cur = source_connection.cursor()
-    cur.execute(
-        "SELECT set_id FROM _sl.sl_set WHERE set_origin=%d"
-        % master_node.node_id)
+    cur.execute("""
+        SELECT set_id FROM _sl.sl_set WHERE set_origin=%d
+        UNION
+        SELECT sub_set AS set_id FROM _sl.sl_subscribe
+        WHERE sub_receiver=%d AND sub_forward IS TRUE AND sub_active IS TRUE
+            AND sub_set=2
+        """ % (master_node.node_id, master_node.node_id))
     set_ids = [set_id for set_id, in cur.fetchall()]
     log.debug("Discovered set ids %s" % repr(list(set_ids)))