Merge lp:~lqb/update-notifier/kdeproxy+kwallet-support into lp:update-notifier/ubuntu

[Lorenz]

- Added proxy support configured in kde
- Added kwallet support
- Added simple GUI Password dialog for proxy authentification

[Sebastien Bacher (seb128)]

thank you for your work, the current version of the merge request has conflicts (in data/package-data-downloader and debian/control), could you resolve those and resubmit it?

review: Needs Fixing

[lp:~lqb/update-notifier/kdeproxy+kwallet-support]

706. By Lorenz on 2013-03-15

  * resolved conflict debian/control
  * resolved conflict data/package-data-downloader

[Lorenz (lqb)]

> thank you for your work, the current version of the merge request has
> conflicts (in data/package-data-downloader and debian/control), could you
> resolve those and resubmit it?

My try to do that failed. I have to find out what I made wrong. Perhaps you might give me a hint.
Thanks

[Sebastien Bacher (seb128)]

Not sure what happened, in case there were need commits in the vcs since you did the change you might want to "bzr merge lp:update-notifier" in your branch, then resolve the conflicts and use "bzr resolve <files_that_conflicted>"

[lp:~lqb/update-notifier/kdeproxy+kwallet-support]

707. By Lorenz on 2013-03-15

merged data/package-data-downloader

708. By Lorenz on 2013-03-18

merge with trunk

[Lorenz (lqb)]

> Not sure what happened, in case there were need commits in the vcs since you
> did the change you might want to "bzr merge lp:update-notifier" in your
> branch, then resolve the conflicts and use "bzr resolve
> <files_that_conflicted>"

That is not realy what I wanted but it works.
Thanks

[lp:~lqb/update-notifier/kdeproxy+kwallet-support]

709. By Lorenz on 2013-04-11

merged with trunk

710. By Lorenz on 2013-04-11

bufix for password dialog problem

Unmerged revisions

710. By Lorenz on 2013-04-11

bufix for password dialog problem

709. By Lorenz on 2013-04-11

merged with trunk

708. By Lorenz on 2013-03-18

merge with trunk

707. By Lorenz on 2013-03-15

merged data/package-data-downloader

706. By Lorenz on 2013-03-15

  * resolved conflict debian/control
  * resolved conflict data/package-data-downloader

705. By Lorenz on 2013-03-14

  * added policykit rule to allow pkexec to use the GUI
  * Enhanced proxy support for kde with kwallet support
  * added partial autoconfiguration support (if configured in kde)
  * added gui prompt for proxy password
  * added passwordcache

704. By Lorenz on 2013-03-12

Merged from lp:~lqb/update-notifier/bug1152203

=== modified file 'TODO'
--- TODO	2007-07-27 21:00:33 +0000
+++ TODO	2013-04-11 13:43:22 +0000
@@ -6,4 +6,10 @@
 * use a g_hash_table for hook_files
 * remove hook-files that are no longer in the HOOK_DIR
   from the hook_seen file (by discarding them when building 
-  the hook_files list)
\ No newline at end of file
+  the hook_files list)
+
+get_kde_proxy_servers:
+	get wpad autoconfiguration from DHCP
+
+get_gconf_proxy_servers:
+	implement this function

=== added file 'data/kwalletdbuscli.py'
--- data/kwalletdbuscli.py	1970-01-01 00:00:00 +0000
+++ data/kwalletdbuscli.py	2013-04-11 13:43:22 +0000
@@ -0,0 +1,94 @@
+#!/usr/bin/python
+
+import sys
+import os
+import subprocess
+import shlex
+import argparse
+import dbus
+from pprint import pprint
+
+appid='Python KWallet D-Bus CLI'
+wallet='kdewallet'
+
+sOK=0
+sError=1
+sCancel=10
+
+def askpass(title="KWallet DBUS CLI",text="Enter username and password"):
+	appid="test"
+	description="desc"
+	realm="re"
+	cmd='zenity --title="'+title+'" --forms --text="'+text+'" --add-entry=username --add-password=password'
+	p=subprocess.Popen(args=shlex.split(cmd),stdout=subprocess.PIPE,stdin=subprocess.PIPE,stderr=subprocess.PIPE)
+	ret=p.wait()
+	out,err=p.communicate()
+	if ret != 0:
+		return None
+	user,password=out.split('|')
+	return user+':'+password
+	
+
+def main():
+	global appid
+	STATUS=sOK
+	parser=argparse.ArgumentParser()
+	parser.add_argument('-a','--app',default=appid,dest='appid')
+
+	subparsers=parser.add_subparsers(dest='parser_name')
+
+	pclose=subparsers.add_parser('close')
+
+	pget=subparsers.add_parser('get')
+	pget.add_argument('-f','--folder', required=True)
+	pget.add_argument('-e','--entry', required=True)
+
+	pset=subparsers.add_parser('set')
+	pset.add_argument('-f','--folder', required=True)
+	pset.add_argument('-e','--entry', required=True)
+	pset.add_argument('-p','--password', required=True)
+
+	pset=subparsers.add_parser('ask')
+	pset.add_argument('-f','--folder', required=True)
+	pset.add_argument('-e','--entry', required=True)
+	pset.add_argument('-t','--text',default='Enter username and password', required=False)
+
+	args=parser.parse_args()
+
+	appid=args.appid
+	bus=dbus.SessionBus()
+	remote_object = bus.get_object('org.kde.kwalletd', '/modules/kwalletd' )
+	iface = dbus.Interface(remote_object, 'org.kde.KWallet')
+
+	if args.parser_name == 'close':
+		iface.disconnectApplication(wallet, appid)
+	if args.parser_name == 'get':
+		wid=iface.open(wallet, 0, appid)
+		password=iface.readPasswordList(wid, args.folder, args.entry, appid)
+		if len(password.keys()) != 0:
+			password=password[args.entry]
+			print password
+		else:
+			STATUS=sError
+	if args.parser_name == 'ask':
+		password=askpass(appid,args.text)
+		if password is None:
+			print >> sys.stderr, "askpass canceled"
+			STATUS=sCancel
+		else:
+			wid=iface.open(wallet, 0, appid)
+			iface.writePassword(wid, args.folder, args.entry, password, appid)
+			print password	
+	if args.parser_name == 'set':
+		wid=iface.open(wallet, 0, appid)
+		password=args.password
+		iface.writePassword(wid, args.folder, args.entry, password, appid)
+		print password
+	sys.exit(STATUS)
+
+
+if __name__ == "__main__":
+	main()
+
+
+

=== added file 'data/org.freedesktop.policykit.pkexec.package-data-downloader.policy'
--- data/org.freedesktop.policykit.pkexec.package-data-downloader.policy	1970-01-01 00:00:00 +0000
+++ data/org.freedesktop.policykit.pkexec.package-data-downloader.policy	2013-04-11 13:43:22 +0000
@@ -0,0 +1,19 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE policyconfig PUBLIC
+ "-//freedesktop//DTD PolicyKit Policy Configuration 1.0//EN"
+ "http://www.freedesktop.org/standards/PolicyKit/1/policyconfig.dtd">
+<policyconfig>
+
+  <action id="org.freedesktop.policykit.pkexec.run-package-data-downloader">
+    <description>Run package-data-downloader</description>
+    <message>Authentication is required to run package-data-downloader</message>
+    <defaults>
+      <allow_any>no</allow_any>
+      <allow_inactive>no</allow_inactive>
+      <allow_active>auth_admin_keep</allow_active>
+    </defaults>
+    <annotate key="org.freedesktop.policykit.exec.path">/usr/lib/update-notifier/package-data-downloader</annotate>
+    <annotate key="org.freedesktop.policykit.exec.allow_gui">TRUE</annotate>
+  </action>
+
+</policyconfig>

=== modified file 'data/package-data-downloader'
--- data/package-data-downloader	2012-08-25 14:18:38 +0000
+++ data/package-data-downloader	2013-04-11 13:43:22 +0000
@@ -27,7 +27,18 @@
 import string
 import apt
 import debconf
+import pacparser
+import pwd
+import shlex
+import getpass
+import tempfile
+import socket
 from datetime import datetime
+try:
+        import configparser as config_parser
+except:
+        import ConfigParser as config_parser
+
 
 DATADIR = "/usr/share/package-data-downloads/"
 STAMPDIR = "/var/lib/update-notifier/package-data-downloads/"
@@ -39,6 +50,274 @@
 failures = []
 permanent_failures = []
 
+PROGNAME="update-notifier-downloader"
+
+class PasswordCache():
+    __data={}
+    def set(self, key, user, password):
+        self.__data[key]={'user':user,'password':password}
+    def get(self, key=None):
+        if key is None:
+            return self.__data
+        else:
+            if key in self.__data.keys():
+                return self.__data[key]
+            else:
+                return None
+
+
+
+class kwalletFancyURLopener(urllib.FancyURLopener):
+    appid=PROGNAME
+    """Inheriited from urllib.FancyURLopener to add kwallet communication.
+    Communication with kwallet over dbus seems to work only if DISPLAY is set.
+    If TERM is set, we can communicate with the user.
+    If neither DISPLAY or TERM is set communication might not be possible.
+    """
+    count=0
+
+    def prompt_user_passwd(self, realm, description):
+	self.count+=1
+	if self.count > 20:
+		sys.exit(1)
+        appid=kwalletFancyURLopener.appid
+        kwallet_bin='/usr/lib/update-notifier/kwalletdbuscli.py'
+        key=realm + "-" + description
+        TERM=os.getenv('TERM',None)
+        DISPLAY=os.getenv('DISPLAY',None)    
+        user=None
+        password=None
+        pwcache=PasswordCache()
+
+        uid=os.getenv('PKEXEC_UID',None)
+        if uid==None:
+            uid=os.getenv('SUDO_UID',None)
+        if uid==None:
+            if 'USER' in os.environ:
+                uid=int(pwd.getpwnam(os.environ['USER']).pw_uid)
+        if uid==None:
+            uid=os.getuid()
+
+        credentials=pwcache.get(key)
+        #if credentials is not None:
+        if credentials is not None and self.count < 2:
+            return (credentials['user'], credentials['password'])
+        else:
+            if (TERM == None or DISPLAY == None):
+                return ('','')
+            else:
+                out=''
+                if DISPLAY != None:
+                    print "Accessing kwallet"
+                    cmd='sudo -u "#'+str(uid)+'" '+kwallet_bin+' -a "'+appid+'" get -f "'+appid+'" -e "'+key+'"'
+                    p=subprocess.Popen(args=shlex.split(cmd),stdout=subprocess.PIPE,stdin=subprocess.PIPE,stderr=subprocess.PIPE)
+                    ret=p.wait()
+                    out,err=p.communicate()
+                    out=out.strip()    
+                    if ret != 0 or self.count > 2 or (out=="" or out==":"):	# No entry in wallet or wrong credentials or empty credentials
+                        cmd='sudo -u "#'+str(uid)+'" '+kwallet_bin+' -a "'+appid+'" ask -f "'+appid+'" -e "'+key+'" -t "Enter credentials for '+description.replace('&','&amp;')+' at '+realm+'"'
+                        p=subprocess.Popen(args=shlex.split(cmd),stdout=subprocess.PIPE,stdin=subprocess.PIPE,stderr=subprocess.PIPE)
+                        ret=p.wait()
+                        out,err=p.communicate()
+                        out=out.strip()
+                        if ret==10:
+                            return (None,None) # Don't ask again
+                        elif ret != 0:
+                            return ("\n","\n")	# Force to ask again
+                        elif out=="" or out==":":
+                            return ("\n","\n")  # Force to ask again. Treats empty credentials as error
+                    user,password=out.split(':')
+        pwcache.set(key, user, password)
+        credentials=pwcache.get(key)
+        return (user, password)
+
+
+def get_gconf_proxy_servers(url,uid=None):
+    """Get proxy server from (system) gconf settings."""
+    print "Not yet Implemented!"
+    return {}
+
+
+def get_kde_proxy_servers(url,uid=None):
+    """Get proxy server from kde settings"""
+    proxies={}
+    kioslave=config_parser.ConfigParser(allow_no_value=True)
+    if uid is not None:
+        home=pwd.getpwuid(int(uid)).pw_dir
+        #print "HOME="+home
+        config_files=['/etc/kde4/kioslaverc',home+'/.kde/share/config/kioslaverc']
+    else:
+        config_files=['/etc/kde4/kioslaverc']
+    
+    for config_file in config_files:
+        try:
+            fh=open(config_file)
+            line=fh.readline()      # discard first lines until line contains "["
+            while not '[' in line:
+                line=fh.readline()
+                if line=='':
+                    break
+            fh.seek(-1 * len(line), 1) # rewind file pointer
+            kioslave.readfp(fh)
+        except:            
+            pass
+
+    if not kioslave.has_section('Proxy Settings'):
+        return proxies
+
+    if (kioslave.get('Proxy Settings','ProxyType') == '0'): # No Proxy
+        proxies={}
+    
+    if (kioslave.get('Proxy Settings','ProxyType') == '1'): # Use manually specified proxy configuration
+        proxies['http']=kioslave.get('Proxy Settings','httpProxy').replace(' ',':')
+        proxies['ftp']=kioslave.get('Proxy Settings','ftpProxy').replace(' ',':')
+        proxies['socks']=kioslave.get('Proxy Settings','socksProxy').replace(' ',':')
+        proxies['https']=kioslave.get('Proxy Settings','httpsProxy').replace(' ',':')
+        proxies['no']=kioslave.get('Proxy Settings','NoProxyFor') # does this work?
+    
+    if (kioslave.get('Proxy Settings','ProxyType') == '2'): # Use proxy auto configuration URL
+        pacurl=kioslave.get('Proxy Settings','Proxy Config Script')
+        if pacurl is None:
+            return {}
+        tmpfn=tempfile.mkstemp(prefix="proxy_",suffix=".pac")[1]
+        urlh=urllib.urlopen(pacurl,proxies={}) # Force direct download
+        tmph=open(tmpfn,"w")
+        for line in urlh:
+            tmph.write(line)
+        tmph.close()
+        proxy_string=pacparser.just_find_proxy(tmpfn, url)
+        os.remove(tmpfn)
+        proxylist = proxy_string.split(";")
+        proxies = None
+        while proxylist:
+            proxy = proxylist.pop(0).strip()
+            if 'DIRECT' in proxy:
+                proxies = {}
+                break
+            if proxy[0:5].upper() == 'PROXY':
+                proxy = proxy[6:].strip()
+                proxies = {'http': 'http://%s' % proxy}    
+
+    if (kioslave.get('Proxy Settings','ProxyType') == '3'): # Detect proxy configuration automatically
+        # usually a DHCPINFORM shoud be used first to determine the proxy pac url but I currently can't test, so I don't implement that yet
+        pacurl=None
+        tmpfn=None
+        fqdn=socket.getfqdn()
+        domparts=fqdn.split('.')
+        proxylist=None
+        while len(domparts) > 2:
+            domparts.remove(domparts[0])
+            pacurl="http://wpad."+string.join(domparts,".")+"/wpad.dat"    
+            tmpfn=tempfile.mkstemp(prefix="proxy_",suffix=".pac")[1]
+            try:
+                urlh=urllib.urlopen(pacurl,proxies={}) # Force direct download
+            except:
+                continue
+            tmph=open(tmpfn,"w")
+            if urlh.info().gettype() == 'application/x-ns-proxy-autoconfig':
+                for line in urlh:
+                    tmph.write(line)
+                tmph.close()
+                proxy_string=pacparser.just_find_proxy(tmpfn, url)
+                proxylist = proxy_string.split(";")
+            os.remove(tmpfn)
+        if proxylist==None:
+            print "No WPAD configuration found. Using direct access"
+        proxies = {}
+        while proxylist:
+            proxy = proxylist.pop(0).strip()
+            if 'DIRECT' in proxy:
+                proxies = {}
+                break
+            if proxy[0:5].upper() == 'PROXY':
+                proxy = proxy[6:].strip()
+                proxies = {'http': 'http://%s' % proxy}    
+
+
+    if (kioslave.get('Proxy Settings','ProxyType') == '4'): # Use system proxy configuration
+        for varname in kioslave.get('Proxy Settings','httpProxy').split(','):
+            try:
+                proxies['http']=os.environ[varname]
+            except:
+                pass
+            if 'http' in proxies.keys():
+                break
+        for varname in kioslave.get('Proxy Settings','ftpProxy').split(','):
+            try:
+                proxies['ftp']=os.environ[varname]
+            except:
+                pass
+            if 'ftp' in proxies.keys():
+                break
+        for varname in kioslave.get('Proxy Settings','httpsProxy').split(','):
+            try:
+                proxies['https']=os.environ[varname]
+            except:
+                pass
+            if 'https' in proxies.keys():
+                break
+        for varname in kioslave.get('Proxy Settings','socksProxy').split(','):
+            try:
+                proxies['socks']=os.environ[varname]
+            except:
+                pass
+            if 'socks' in proxies.keys():
+                break
+        for varname in kioslave.get('Proxy Settings','NoProxyFor').split(','): # does this work?
+            try:
+                proxies['no']=os.environ[varname]
+            except:
+                pass
+            if 'no' in proxies.keys():
+                break
+    return proxies
+
+        
+def get_env_proxy_servers():
+    """Get proxy server from environment settings"""
+    # Get our proxy settings from the environment
+    proxies = {}
+    try:
+        for proto in ('http','https','ftp','no'):
+            try:
+                proxies[proto] = os.environ[proto+"_proxy"]
+            except KeyError:
+                pass
+    except Exception:
+        pass
+    return proxies
+
+def get_proxy_servers(url):
+    """Get proxy settings dependent on environment and call
+
+    This script could be called through different ways. 
+    It is not possible to fetch proxy setting e.g. from kde 
+    settings if this script was called from a cronjob
+    """
+
+    UID=os.getuid()
+    PKEXEC_UID=os.getenv('PKEXEC_UID',None)
+    SUDO_UID=os.getenv('SUDO_UID',None)
+    proxies={}
+
+    if (PKEXEC_UID != None) and (PKEXEC_UID != 0):
+        proxies=get_kde_proxy_servers(url,PKEXEC_UID)    
+    elif (SUDO_UID != None) and (SUDO_UID != 0):
+        proxies=get_kde_proxy_servers(url,SUDO_UID)
+    elif (UID != None) and (UID != 0):
+        proxies=get_kde_proxy_servers(url,UID)
+    else:
+        proxies=get_kde_proxy_servers(url)
+
+    if not proxies:
+        proxies=get_gconf_proxy_servers(url)
+
+    if not proxies:
+        proxies=get_env_proxy_servers()
+
+    return proxies
+
+
 def create_or_update_stampfile(file):
     """Create or update the indicated stampfile, and remove failure flags"""
 
@@ -145,8 +424,9 @@
         sys.exit(1)
 
     packages = [os.path.basename(failure) for failure in failures]
+
     output_file.write(
-        string.Template(input).substitute(
+        string.Template(input).safe_substitute(
             {'packages' : ", ".join(packages)}))
     output_file.close()
 
@@ -160,6 +440,17 @@
         res.append(relfile)
     return res
 
+def mktmp4url(url):
+    import tempfile
+    garbage, path = urllib.splittype(url)
+    garbage, path = urllib.splithost(path or "")
+    path, garbage = urllib.splitquery(path or "")
+    path, garbage = urllib.splitattr(path or "")
+    suffix = os.path.splitext(path)[1]
+    (fd, filename) = tempfile.mkstemp(suffix)
+    return (fd, filename)
+
+
 def process_download_requests():
     """Process requests to download package data files
 
@@ -182,7 +473,6 @@
                 proxies[proto] = os.environ[proto+"_proxy"]
             except KeyError:
                 pass
-
         if proxies:
             urllib._urlopener = urllib.FancyURLopener(proxies)
     except Exception:
@@ -214,6 +504,7 @@
                     record_failure(relfile)
                     break
                 command = [para['script']]
+                tmpfiles=[]
 
                 if 'should-download' in para:
                     db = debconf.DebconfCommunicator('update-notifier')
@@ -231,7 +522,12 @@
                 try:
                     for i in range(len(files)):
                         print "%s: downloading %s" % (relfile, files[i])
-                        dest_file = urllib.urlretrieve(files[i])[0]
+                        proxies=get_proxy_servers(files[i])    # Get our proxy settings for each url
+                        if proxies:
+                            urllib._urlopener = kwalletFancyURLopener(proxies)
+                            (fd,filename)=mktmp4url(files[i])
+                            tmpfiles.append(filename)
+                            dest_file = urllib.urlretrieve(files[i], filename)[0]
                         output = subprocess.check_output(["sha256sum", dest_file])
                         output = output.split(' ')[0]
                         if output == sums[i]:
@@ -241,7 +537,6 @@
                             break
                     if relfile in failures + permanent_failures:
                         break
-
                     sys.stdout.flush()
                     result = subprocess.call(command)
                     if result:
@@ -252,7 +547,6 @@
                     break
                 except Exception:
                     traceback.print_exc(file=sys.stderr)
-
                 record_failure(relfile)
                 # The 'script' is always the last stanza
                 break
@@ -266,7 +560,11 @@
                     traceback.print_exc(file=sys.stderr)
                 record_failure(relfile)
                 break
-
+        for f in tmpfiles:
+            try:
+                os.remove(f)
+            except:
+                pass
     previous_failures = existing_permanent_failures()
 
     # We only report about "permanent" failures when there are new ones,
@@ -294,3 +592,17 @@
 
 if __name__ == "__main__":
     process_download_requests()
+
+
+    # close wallet
+    uid=os.getenv('PKEXEC_UID',None)
+    if uid==None:
+        uid=os.getenv('SUDO_UID',None)
+    if uid==None:
+        uid=os.getuid()
+    if uid!=0:
+        cmd='sudo -u "#'+str(uid)+'" /usr/lib/update-notifier/kwalletdbuscli.py -a "'+PROGNAME+'" close'
+        p=subprocess.Popen(args=shlex.split(cmd),stdout=subprocess.PIPE,stdin=subprocess.PIPE,stderr=subprocess.PIPE)
+        p.wait()
+
+    print PROGNAME+" finished"

=== modified file 'debian/changelog'
--- debian/changelog	2013-04-09 10:31:08 +0000
+++ debian/changelog	2013-04-11 13:43:22 +0000
@@ -218,10 +218,20 @@
 
 update-notifier (0.119ubuntu10) quantal; urgency=low
 
+  [ Steve Langasek ]
   * Merge in changes from 0.119ubuntu8.1 which had been uploaded to
     precise-proposed.
 
- -- Steve Langasek <[e-mail]>  Tue, 15 May 2012 16:20:43 -0700
+  [ Lorenz ]
+  * Fix Translation error and use safe_substitute instead of substitute on
+    template substitution. LP: #1152203
+  * added policykit rule to allow pkexec to use the GUI
+  * Enhanced proxy support for kde with kwallet support
+  * added partial autoconfiguration support (if configured in kde)
+  * added gui prompt for proxy password
+  * added passwordcache
+
+ -- Lorenz <[e-mail]>  Tue, 12 Mar 2013 18:44:45 +0100
 
 update-notifier (0.119ubuntu9) quantal; urgency=low
 

=== modified file 'debian/control'
--- debian/control	2013-04-09 10:07:10 +0000
+++ debian/control	2013-04-11 13:43:22 +0000
@@ -47,7 +47,11 @@
 Depends: ${shlibs:Depends},
          ${misc:Depends},
          python,
-         python-apt (>= 0.6.12), python-debian, debconf, patch
+         python-apt (>= 0.6.12), 
+         python-debian,
+         debconf, 
+         patch,
+         python-pacparser
 Recommends: libpam-modules (>= 1.0.1-9ubuntu3)
 Suggests: policykit-1
 Description: Files shared between update-notifier and other packages

=== modified file 'debian/update-notifier-common.install'
--- debian/update-notifier-common.install	2013-02-01 15:13:06 +0000
+++ debian/update-notifier-common.install	2013-04-11 13:43:22 +0000
@@ -12,3 +12,6 @@
 usr/lib/update-notifier/
 usr/share/locale
 debian/update-manager-downloader-fix2.diff usr/share/update-notifier/upgrader-patches
+data/org.freedesktop.policykit.pkexec.package-data-downloader.policy   usr/share/polkit-1/actions/
+data/kwalletdbuscli.py usr/lib/update-notifier
+