Ubuntu
krb5 package

krb5-user: kinit fails for OTP user when using kdc discovery via DNS

Bug #1683237 reported by Jochen Hein
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
krb5 (Debian)
Fix Released
Unknown
krb5 (Ubuntu)
Fix Released
High
Unassigned
Zesty
Fix Released
High
Andreas Hasenack

Bug Description

This is fixed in krb5 1.15-2 in artful

Upstream bug : http://krbdev.mit.edu/rt/Ticket/Display.html?id=8554
Debian bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=856307
Debian patch in 1.15-2 in artful: 0013-Fix-udp_preference_limit-with-SRV-records.patch

[Impact]

kinit does not respect udp_preference_limit and always uses TCP to talk to the KDC when using the DNS SRV records to locate the service and these records show both udp and tcp entries.

One particular scenario that fails is when OTP (one time password) is used, as reported.

The provided patch is applied upstream and debian testing.

[Test Case]

Steps to reproduce on zesty, with all services on one machine for simplicity (I suggest to use LXD):

a) install the packages from zesty (not the proposed ones yet):
$ sudo apt install krb5-kdc krb5-admin-server bind9

When prompted for the realm, choose EXAMPLE.COM
When prompted for the KDC and Admin services server address, use the IP of your test machine/container (not localhost or 127.0.0.1)
The KDC will fail to start because there is no realm yet, that's not relevant for this bug.

b) Edit /etc/krb5.conf and make the following changes:
- remove the "default_realm" line from the [libdefaults] section
- remove the EXAMPLE.COM realm block from the [realms] section
- add "dns_lookup_realm = true" to the [libdefaults] section
- add "dns_lookup_kdc = true" to the [libdefaults] section
- add "udp_preference_limit = 1" to the [libdefaults] section

c) Edit /etc/bind/named.conf.local and add this zone block (for simplicity, we are skipping the reverse zone):
zone "example.com" {
    type master;
    file "/etc/bind/db.example.com";
};

d) Create /etc/bind/db.example.com with this content:
$TTL 604800
@ IN SOA example.com. ubuntu.example.com. (
                  1 ; Serial
             604800 ; Refresh
              86400 ; Retry
            2419200 ; Expire
             604800 ) ; Negative Cache TTL
;
@ IN NS zesty-bug1683237.example.com.
zesty-bug1683237 IN A 10.0.100.249
_kerberos TXT "EXAMPLE.COM"
_kerberos._udp SRV 0 0 88 zesty-bug1683237
_kerberos._tcp SRV 0 0 88 zesty-bug1683237
_kerberos-master._udp SRV 0 0 88 zesty-bug1683237
_kerberos-master._tcp SRV 0 0 88 zesty-bug1683237
_kerberos-adm._tcp SRV 0 0 749 zesty-bug1683237
_kpasswd._udp SRV 0 0 464 zesty-bug1683237

Use the real IP of your test machine/container where I used "10.0.100.249". You can also choose another hostname if you want, just be consistent across the board. I chose "zesty-bug1683237".

e) Restart bind
$ sudo service bind9 restart

f) Do a few quick DNS tests:
$ dig +short @10.0.100.249 zesty-bug1683237.example.com
10.0.100.249
$ dig +short @10.0.100.249 -t TXT _kerberos.example.com
"EXAMPLE.COM"
$ dig +short @10.0.100.249 -t SRV _kerberos._udp.example.com
0 0 88 zesty-bug1683237.example.com.
$ dig +short @10.0.100.249 -t SRV _kerberos._tcp.example.com
0 0 88 zesty-bug1683237.example.com.

g) Edit /etc/resolv.conf, ignoring the warning since we are not going to reboot or change network interfaces:
nameserver 10.0.100.249 # USE YOUR IP HERE
search example.com

h) Create the EXAMPLE.COM kerberos realm:
$ sudo krb5_newrealm
When prompted for a password, use whatever you like. If you get an error about no default realm, then your TXT record in DNS is not working. Retrace your DNS configuration steps.

i) Start the kerberos services:
$ sudo service krb5-kdc start
sudo service krb5-admin-server start

j) Create a principal and test it:
$ sudo kadmin.local addprinc -pw ubuntu ubuntu
$ kinit ubuntu
Password for <email address hidden>:
$ klist
(...)
05/05/2017 13:10:01 05/05/2017 23:10:01 <email address hidden>
(...)

Now we are ready to test the bug.

Given that we have udp_preference_limit = 1 in /etc/krb5.conf, kinit should use TCP instead of UDP. Let's check:

$ KRB5_TRACE=/dev/stderr kinit
[7609] 1493989890.568980: Getting initial credentials for <email address hidden>
[7609] 1493989890.569904: Sending request (172 bytes) to EXAMPLE.COM
[7609] 1493989890.571991: Resolving hostname zesty-bug1683237.example.com.
[7609] 1493989890.576853: Sending initial UDP request to dgram 10.0.100.249:88
(...)

Uh oh, it's using UDP!

With the fixed packages, kinit will use TCP, thus honoring the udp_preference_limit setting:
$ KRB5_TRACE=/dev/stderr kinit
[14287] 1493990160.760430: Getting initial credentials for <email address hidden>
[14287] 1493990160.761590: Sending request (172 bytes) to EXAMPLE.COM
[14287] 1493990160.763783: Resolving hostname zesty-bug1683237.example.com.
[14287] 1493990160.767803: Resolving hostname zesty-bug1683237.example.com.
[14287] 1493990160.770588: Initiating TCP connection to stream 10.0.100.249:88
[14287] 1493990160.771724: Sending TCP request to stream 10.0.100.249:88
(...)

And if udp_preference_limit is removed from /etc/krb5.conf, the updated packages start with UDP again.

[Regression Potential]
Sites who were inadvertently relying on this bug (by having udp_preference_limit set to a low value but being ignored) will see their clients suddenly switch (correctly) from UDP to TCP when talking to the KDC and could require firewall changes to allow that exchange.

== Original description ==

Zesty is now affected, please see the debian bug https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=856307
and upstream bug http://krbdev.mit.edu/rt/Ticket/Display.html?id=8554

Would it be possible to get 1.15.1 (already released upstream) in zesty/zesty-updates?

Thanks
Jochen

See original description
Bug Watch Updater (bug-watch-updater)
Changed in krb5 (Debian):
status: Unknown → New
Joshua Powers (powersj)
Changed in krb5 (Ubuntu):
status: New → Triaged
importance: Undecided → High
tags: added: server-next
Revision history for this message
Joshua Powers (powersj) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu
better. Thank you also for reporting this to Debian and finding the
upstream record.

If you need a fix for the bug in previous versions of Ubuntu, then the
relevant procedure is documented here:

https://wiki.ubuntu.com/StableReleaseUpdates

However, uploading point releases is generally not done to existing
releases in an effort to prevent additional regressions. Because
we currently do not have a delta with Debian, ideally we would sync over
from Debian directly and get the fix. However, with the existing freeze
this may not happen right away. I have added this to the server-next
backlog to get looked at during the next release.

Also I believe this is the commit to fix:
https://github.com/krb5/krb5/commit/bc7594058011c2f9711f24af4fa15a421a8d5b62

Revision history for this message
Sam Hartman (hartmans) wrote : Re: [Bug 1683237] Re: krb5-user: kinit fails for OTP user when using kdc discovery via DNS

I can put something in debian experimental if that makes the sync
easier.
So, you'd prefer just the Debian 1.15-1 with bug fixes rather than a
1.15.1?

Revision history for this message
Joshua Powers (powersj) wrote :

If we want to fix this in zesty, then a release with only the bug fixes would be desired. That could be SRU'ed assuming it is not too big of a change such that it would limit the exposure to new issues or changes in features/functionality.

For fixing this in the 17.10 release, then picking up the new release with fixes makes sense.

Revision history for this message
Sam Hartman (hartmans) wrote :

>>>>> "Joshua" == Joshua Powers <email address hidden> writes:

    Joshua> If we want to fix this in zesty, then a release with only
    Joshua> the bug fixes would be desired. That could be SRU'ed
    Joshua> assuming it is not too big of a change such that it would
    Joshua> limit the exposure to new issues or changes in
    Joshua> features/functionality.

OK, I'll look at pulling a bugfix release into experimental now. I sure
hope stretch releases before 17.10:-)

Revision history for this message
Sam Hartman (hartmans) wrote :

>>>>> "Joshua" == Joshua Powers <email address hidden> writes:

    Joshua> If we want to fix this in zesty, then a release with only
    Joshua> the bug fixes would be desired. That could be SRU'ed
    Joshua> assuming it is not too big of a change such that it would
    Joshua> limit the exposure to new issues or changes in
    Joshua> features/functionality.

Hi.
I uploaded 1.15-2 to Debian experimental.
It includes two fixes to regressions from jessie (also regressions in
Ubuntu):
1) this bug -- OTP users don't work with DNS discovery

2) Another upstream issue where enhancements to the IPv6 support break
IPv4 only systems.
I haven't seen Debian reports of that issue, but code changes are
minimal and are easy to audit for impact.

I'm not volunteering to file the SRU paperwork on the Ubuntu side, but
I'm happy to provide technical assistance for anyone who wants to do
that.
I believve that a sync of 1.15-2 over 1.15-1 would be appropriate and in
terms of code change complexity would be within Ubuntu's policies.

Revision history for this message
Jochen Hein (jochen-jochen) wrote :

Sam Hartman <email address hidden> writes:

>>>>>> "Joshua" == Joshua Powers <email address hidden> writes:
>
> Joshua> If we want to fix this in zesty, then a release with only
> Joshua> the bug fixes would be desired. That could be SRU'ed
> Joshua> assuming it is not too big of a change such that it would
> Joshua> limit the exposure to new issues or changes in
> Joshua> features/functionality.
>
> I uploaded 1.15-2 to Debian experimental.
> It includes two fixes to regressions from jessie (also regressions in
> Ubuntu):
> 1) this bug -- OTP users don't work with DNS discovery
>
> 2) Another upstream issue where enhancements to the IPv6 support break
> IPv4 only systems.

Thanks.

Another useful fix for OTP users might be
http://krbdev.mit.edu/rt/Ticket/Display.html?id=8580
(kinit fails for OTP users when using KdcProxy with both IPv4&6 DNS)

With the patch from 1) and that one my laptop once again can
authenticate with OTP both internally as well as externally via
KdcProxy.

Jochen

--
This space is intentionally left blank.

Bug Watch Updater (bug-watch-updater)
Changed in krb5 (Debian):
status: New → Fix Released
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Thank you Sam to make this available!
I agree that you hopefully release before 17.10.

Since the auto sync won't catch it there I checked if things would work out as intended and they seem good with:
$ syncpackage --distribution=experimental --simulate --verbose krb5

What is needed is:
1. artful sync
2. backport change as a 1.15-1ubuntu0.1 version
3. Add SRU Paperwork

Since this is on core-dev upload permissions neither myself nor Josh can help here atm, but I'll ping a few people who can.

Robie Basak (racb)
Changed in krb5 (Ubuntu Zesty):
status: New → Triaged
Revision history for this message
Robie Basak (racb) wrote :

I sync this yesterday but it's stuck in proposed. Two failing dep8 tests. Both look intermittent so I've retried them.

Changed in krb5 (Ubuntu):
status: Triaged → Fix Committed
Revision history for this message
Robie Basak (racb) wrote :

This bug was fixed in the package krb5 - 1.15-2
Sponsored for ChristianEhrhardt (paelzer)

---------------
krb5 (1.15-2) experimental; urgency=medium

  * Upstream patches to fix startup if getaddrinfo() returns a wildcard v6
    address, and to fix handling of explicitly specified v4 wildcard
    address; regression over previous versions, Closes: #860767
  * Fix SRV lookups to respect udp_preference_limit, regression over
    previous versions with OTP, Closes: #856307

 -- Sam Hartman <email address hidden> Wed, 19 Apr 2017 16:50:01 -0400

Changed in krb5 (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
Robie Basak (racb) wrote :

I think this bug is fixed in Artful then, but still affects Zesty?

To fix Zesty, please follow the steps documented at https://wiki.ubuntu.com/StableReleaseUpdates#Procedure to get a fixed version released to zesty-updates.

Joshua Powers (powersj)
tags: removed: server-next
Andreas Hasenack (ahasenack)
Changed in krb5 (Ubuntu Zesty):
assignee: nobody → Andreas Hasenack (ahasenack)
status: Triaged → In Progress
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

This launchpad bug was "overloaded" and is talking about 3 issues:

a) kinit fails for OTP user when using kdc discovery via DNS
- upstream: http://krbdev.mit.edu/rt/Ticket/Display.html?id=8554
- debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=856307
- debian patch: 0013-Fix-udp_preference_limit-with-SRV-records.patch

b) KDC/kadmind explicit wildcard listener addresses do not use pktinfo
- no LP bug
- upstream: http://krbdev.mit.edu/rt/Ticket/Display.html?id=8530
- debian: conflated into https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=860767
- debian patch: 0012-Use-pktinfo-for-explicit-UDP-wildcard-listeners.patch

c) KDC/kadmind may fail to start on IPv4-only systems
- no LP bug
- upstream: http://krbdev.mit.edu/rt/Ticket/Display.html?id=8531
- debian: also conflated into https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=860767
- debian patch: 0011-Fix-KDC-kadmind-startup-on-some-IPv4-only-systems.patch

I'll file separate bugs for (b) and (c) including test cases and then the SRU can address them too. I'm now working on a test case for (a).

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

https://bugs.launchpad.net/ubuntu/+source/krb5/+bug/1688121 filed for (b)

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

https://bugs.launchpad.net/ubuntu/+source/krb5/+bug/1688310 filed for (c)

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Ok, I got a simpler test case for (a) that doesn't involve setting up FreeIPA, PKINIT or OTP. I'll update the bug description about it tomorrow and then proceed with the SRU paperwork and actual packages.

Andreas Hasenack (ahasenack)
description: updated
description: updated
Andreas Hasenack (ahasenack)
description: updated
Andreas Hasenack (ahasenack)
description: updated
Andreas Hasenack (ahasenack)
description: updated
Mathew Hodson (mhodson)
Changed in krb5 (Ubuntu Zesty):
importance: Undecided → High
Revision history for this message
Adam Conrad (adconrad) wrote : Please test proposed package

Hello Jochen, or anyone else affected,

Accepted krb5 into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/krb5/1.15-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in krb5 (Ubuntu Zesty):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Jochen Hein (jochen-jochen) wrote :

The updated package works for my OTP user. Thanks a lot!
Jochen

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package krb5 - 1.15-1ubuntu0.1

---------------
krb5 (1.15-1ubuntu0.1) zesty; urgency=medium

  * Pulled in Debian fixes from Sam Hartman for:
    - kinit fails for OTP user when using kdc discovery via DNS
      (LP: #1683237)
    - KDC/kadmind explicit wildcard listener addresses do not use pktinfo
      (LP: #1688121)
    - KDC/kadmind may fail to start on IPv4-only systems (LP: #1688310)

 -- Andreas Hasenack <email address hidden> Fri, 05 May 2017 14:05:38 +0000

Changed in krb5 (Ubuntu Zesty):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for krb5 has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.