Created by Max Bowsher on 2011-01-03 and last modified on 2017-06-24
Get this branch:
bzr branch lp:postgresql

Branch merges

Related bugs

Related blueprints

Branch information

VCS imports

Import details

Import Status: Reviewed

This branch is an import of the HEAD branch of the Git repository at git://git.postgresql.org/git/postgresql.git.

The next import is scheduled to run in 2 hours.

Last successful import was 3 hours ago.

Import started 3 hours ago on pear and finished 3 hours ago taking 50 seconds — see the log
Import started 9 hours ago on pear and finished 9 hours ago taking 1 minute — see the log
Import started 15 hours ago on pear and finished 15 hours ago taking 50 seconds — see the log
Import started 21 hours ago on pear and finished 21 hours ago taking 50 seconds — see the log
Import started on 2017-06-24 on pear and finished on 2017-06-24 taking 1 minute — see the log
Import started on 2017-06-24 on pear and finished on 2017-06-24 taking 1 minute — see the log
Import started on 2017-06-24 on pear and finished on 2017-06-24 taking 1 minute — see the log
Import started on 2017-06-23 on russkaya and finished on 2017-06-23 taking 2 minutes — see the log
Import started on 2017-06-23 on pear and finished on 2017-06-23 taking 2 minutes — see the log
Import started on 2017-06-23 on pear and finished on 2017-06-23 taking 50 seconds — see the log

Recent revisions

42998. By Tom Lane <email address hidden> on 2017-06-24

Further hacking on ICU collation creation and usage.

pg_import_system_collations() refused to create any ICU collations if
the current database's encoding didn't support ICU. This is wrongheaded:
initdb must initialize pg_collation in an encoding-independent way
since it might be used in other databases with different encodings.
The reason for the restriction seems to be that get_icu_locale_comment()
used icu_from_uchar() to convert the UChar-format display name, and that
unsurprisingly doesn't know what to do in unsupported encodings.
But by the same token that the initial catalog contents must be
encoding-independent, we can't allow non-ASCII characters in the comment
strings. So we don't really need icu_from_uchar() here: just check for
Unicode codes outside the ASCII range, and if there are none, the format
conversion is trivial. If there are some, we can simply not install the
comment. (In my testing, this affects only Norwegian Bokmål, which has
given us trouble before.)

For paranoia's sake, also check for non-ASCII characters in ICU locale
names, and skip such locales, as we do for libc locales. I don't
currently have a reason to believe that this will ever reject anything,
but then again the libc maintainers should have known better too.

With just the import changes, ICU collations can be found in pg_collation
in databases with unsupported encodings. This resulted in more or less
clean failures at runtime, but that's not how things act for unsupported
encodings with libc collations. Make it work the same as our traditional
behavior for libc collations by having collation lookup take into account
whether is_encoding_supported_by_icu().

Adjust documentation to match. Also, expand Table 23.1 to show which
encodings are supported by ICU.

catversion bump because of likely change in pg_collation/pg_description
initial contents in ICU-enabled builds.

Discussion: https://<email address hidden>

42997. By Simon Riggs <simon@2ndQuadrant.com> on 2017-06-24

Fix typo in comment in SerializeSnapshot

Author: Masahiko Sawada

42996. By Simon Riggs <simon@2ndQuadrant.com> on 2017-06-24

Revert 1f30295eab65eddaa88528876ab66e7095f4bb65

Reported-by: Tom Lane

42995. By Tom Lane <email address hidden> on 2017-06-23

Fix incorrect buffer-length argument to uloc_getDisplayName().

The maxResultSize argument of uloc_getDisplayName is the number of
UChars in the output buffer, not the number of bytes. In principle
this could result in a stack smash, although at least in my Fedora 25
install there are no ICU locales with display names long enough to
overrun the buffer. But it's easily proven to be wrong by reducing
the length of displayname to around 20, whereupon a stack smash
does happen.

(This is a rather scary bug, because the same mistake could easily
have been made in other places; but in a quick code search looking
at uses of UChar I could not find any other instances.)

42994. By Peter Eisentraut on 2017-06-23

Fix replication with replica identity full

The comparison with the target rows on the subscriber side was done with
datumIsEqual(), which can have false negatives. For instance, it didn't
work reliably for text columns. So use the equality operator provided
by the type cache instead.

Also add more user documentation about replica identity requirements.

Reported-by: Tatsuo Ishii <email address hidden>

42993. By Tom Lane <email address hidden> on 2017-06-23

Rethink behavior of pg_import_system_collations().

Marco Atzeri reported that initdb would fail if "locale -a" reported
the same locale name more than once. All previous versions of Postgres
implicitly de-duplicated the results of "locale -a", but the rewrite
to move the collation import logic into C had lost that property.
It had also lost the property that locale names matching built-in
collation names were silently ignored.

The simplest way to fix this is to make initdb run the function in
if-not-exists mode, which means that there's no real use-case for
non if-not-exists mode; we might as well just drop the boolean argument
and simplify the function's definition to be "add any collations not
already known". This change also gets rid of some odd corner cases
caused by the fact that aliases were added in if-not-exists mode even
if the function argument said otherwise.

While at it, adjust the behavior so that pg_import_system_collations()
doesn't spew "collation foo already exists, skipping" messages during a
re-run; that's completely unhelpful, especially since there are often
hundreds of them. And make it return a count of the number of collations
it did add, which seems like it might be helpful.

Also, re-integrate the previous coding's property that it would make a
deterministic selection of which alias to use if there were conflicting
possibilities. This would only come into play if "locale -a" reports
multiple equivalent locale names, say "de_DE.utf8" and "de_DE.UTF-8",
but that hardly seems out of the question.

In passing, fix incorrect behavior in pg_import_system_collations()'s
ICU code path: it neglected CommandCounterIncrement, which would result
in failures if ICU returns duplicate names, and it would try to create
comments even if a new collation hadn't been created.

Also, reorder operations in initdb so that the 'ucs_basic' collation
is created before calling pg_import_system_collations() not after.
This prevents a failure if "locale -a" were to report a locale named
that. There's no reason to think that that ever happens in the wild,
but the old coding would have survived it, so let's be equally robust.

Discussion: https://<email address hidden>

42992. By Simon Riggs <simon@2ndQuadrant.com> on 2017-06-23

Improve replication lag interpolation after idle period

After sitting idle and fully replayed for a while and then encountering
a new burst of WAL activity, we interpolate between an ancient sample and the
not-yet-reached one for the new traffic. That produced a corner case report
of lag after receiving first new reply from standby, which might sometimes
be a large spike.

Correct this by resetting last_read time and handle that new case.

Author: Thomas Munro

42991. By Simon Riggs <simon@2ndQuadrant.com> on 2017-06-23

Minor corrections to high availability docs

Startup process is displayed in pg_stat_activity, noted by Yugo Nagata.
Transactions can be resolved at end of recovery.

Author: Yugo Nagata, with addition by me

42990. By Tom Lane <email address hidden> on 2017-06-23

Fix memory leakage in ICU encoding conversion, and other code review.

Callers of icu_to_uchar() neglected to pfree the result string when done
with it. This results in catastrophic memory leaks in varstr_cmp(),
because of our prevailing assumption that btree comparison functions don't
leak memory. For safety, make all the call sites clean up leaks, though
I suspect that we could get away without it in formatting.c. I audited
callers of icu_from_uchar() as well, but found no places that seemed to
have a comparable issue.

Add function API specifications for icu_to_uchar() and icu_from_uchar();
the lack of any thought-through specification is perhaps not unrelated
to the existence of this bug in the first place. Fix icu_to_uchar()
to guarantee a nul-terminated result; although no existing caller appears
to care, the fact that it would have been nul-terminated except in
extreme corner cases seems ideally designed to bite someone on the rear
someday. Fix ucnv_fromUChars() destCapacity argument --- in the worst
case, that could perhaps have led to a non-nul-terminated result, too.
Fix icu_from_uchar() to have a more reasonable definition of the function
result --- no callers are actually paying attention, so this isn't a live
bug, but it's certainly sloppily designed. Const-ify icu_from_uchar()'s
input string for consistency.

That is not the end of what needs to be done to these functions, but
it's as much as I have the patience for right now.

Discussion: https://<email address hidden>

42989. By Tom Lane <email address hidden> on 2017-06-23

Add testing to detect errors of omission in "pin" dependency creation.

It's essential that initdb.c's setup_depend() scan each system catalog
that could contain objects that need to have "p" (pin) entries in pg_depend
or pg_shdepend. Forgetting to add that, either when a catalog is first
invented or when it first acquires DATA() entries, is an obvious bug
hazard. We can detect such omissions at reasonable cost by probing every
OID-containing system catalog to see whether the lowest-numbered OID in it
is pinned. If so, the catalog must have been properly accounted for in
setup_depend(). If the lowest OID is above FirstNormalObjectId then the
catalog must have been empty at the end of initdb, so it doesn't matter.
There are a small number of catalogs whose first entry is made later in
initdb than setup_depend(), resulting in nonempty expected output of the
test, but these can be manually inspected to see that they are OK. Any
future mistake of this ilk will manifest as a new entry in the test's

Since pg_conversion is already in the test's output, add it to the set of
catalogs scanned by setup_depend(). That has no effect today (hence, no
catversion bump here) but it will protect us if we ever do add pin-worthy

This test is very much like the catalog sanity checks embodied in
opr_sanity.sql and type_sanity.sql, but testing pg_depend doesn't seem to
fit naturally into either of those scripts' charters. Hence, invent a new
test script misc_sanity.sql, which can be a home for this as well as tests
on any other catalogs we might want in future.

Discussion: https://<email address hidden>

Branch metadata

Branch format:
Branch format 7
Repository format:
Bazaar repository format 2a (needs bzr 1.16 or later)
This branch contains Public information 
Everyone can see this information.


No subscribers.