Merge lp:~mgorven/ibid/port-numbers into lp:~ibid-core/ibid/old-trunk-1.6

[Michael Gorven]

[Michael Gorven (mgorven)]

Simple port/protocol lookup using /etc/services.

[Keegan Carruthers-Smith (keegan-csmith)]

Looks good

review: Approve

[Max Rabkin (max-rabkin)]

Are comments in /etc/services guaranteed to be at the start of the
line or be preceded by whitespace? On Debian, /etc/services points to
/usr/share/nmap/nmap-services for a more complete list. Perhaps we
should use that if it's available? Unfortunately, it's in a slightly
different format from /etc/services (it has an open probability where
/etc/services has aliases).

If the answer to my questions are yes and no, then it looks good.

Review: Approve

[Keegan Carruthers-Smith (keegan-csmith)]

> Are comments in /etc/services guaranteed to be at the start of the
> line or be preceded by whitespace?
No, they also occur after an entry but the code correctly deals with that.

[Max Rabkin (max-rabkin)]

Also, specifying a transport should be optional (bot should reply to
"port 25" with "smtp on TCP", or something like that.

[Michael Gorven (mgorven)]

* Use Nmap's services file if it exists
* Make TCP/UDP optional when asking about port numbers
* Handle SCTP as well

[Max Rabkin (max-rabkin)]

review: Approve

[Michael Gorven (mgorven)]

Moved to network plugin and tweaked portfor regex.

[Stefano Rivera (stefanor)]

> from collections import defaultdict

Get that from ibid.compat

More review coming when I look at it properly

[Stefano Rivera (stefanor)]

I'm not convinced that this needs to be cached from load time. Generating the dict on use would save memory. But that's a more general issue, not specific to this plugin.

ICMP ports?

Query: usage ports
Response: I don't know about that protocol
Query: help ports
Response: I don't know about that protocol

review: Needs Fixing

[Michael Gorven (mgorven)]

> ICMP ports?

The services files doesn't have that information, and it would be used
extremely rarely.

[Michael Gorven (mgorven)]

Fix masking of help plugin, and only load services file during first request.

[marcog (marco-gallotta)]

Query: port for http
Response: 80/tcp and 80/udp
Query: port 80
Response: http (TCP) and http (UDP)

Perhaps group them if the protocol is the same?

Query: port for http
Response: 80 (tcp and udp)
Query: port 80
Response: http (TCP and UDP)

The one response uses / and the other uses (), perhaps they should match. The case should also match.

Query: X11:1 port
Response: I don't know about that protocol
Query: port 6001
Response: X11:1 (TCP)

?

review: Needs Fixing

[lp:~mgorven/ibid/port-numbers]

893. By Michael Gorven on 2010-02-13

Fix case sensitivity in protocol name lookup.

894. By Michael Gorven on 2010-02-13

Display transport consistently.

[Michael Gorven (mgorven)]

> The one response uses / and the other uses (), perhaps they should match.
> The case should also match.

r894

> Query: X11:1 port
> Response: I don't know about that protocol

r893

[Michael Gorven (mgorven)]

> Perhaps group them if the protocol is the same?

Not really worth the effort IMHO.

[marcog (marco-gallotta)]

review: Approve

[Keegan Carruthers-Smith (keegan-csmith)]

review: Approve

[Stefano Rivera (stefanor)]

review: Approve

=== modified file 'ibid/plugins/network.py'
--- ibid/plugins/network.py	2010-01-29 23:33:18 +0000
+++ ibid/plugins/network.py	2010-02-13 09:40:30 +0000
@@ -3,11 +3,13 @@
 
 import re
 import socket
+from ibid.compat import defaultdict
 from httplib import HTTPConnection, HTTPSConnection
+from os.path import exists
 from subprocess import Popen, PIPE
+from sys import version_info
 from urllib import getproxies_environment
 from urlparse import urlparse
-from sys import version_info
 
 try:
     from dns.resolver import Resolver, NoAnswer, NXDOMAIN
@@ -431,4 +433,72 @@
 
         event.addresponse(u"ISO doesn't know about any TLD for %s", location)
 
+help['ports'] = u'Looks up port numbers for protocols'
+class Ports(Processor):
+    u"""port for <protocol>
+    (tcp|udp) port <number>"""
+    feature = 'ports'
+    priority = 10
+
+    services = Option('services', 'Path to services file', '/etc/services')
+    nmapservices = Option('nmap-services', "Path to Nmap's services file", '/usr/share/nmap/nmap-services')
+    protocols = {}
+    ports = {}
+
+    def setup(self):
+        self.filename = None
+        if exists(self.nmapservices):
+            self.filename = self.nmapservices
+            self.nmap = True
+        elif exists(self.services):
+            self.filename = self.services
+            self.nmap = False
+
+        if not self.filename:
+            raise Exception(u"Services file doesn't exist")
+
+    def _load_services(self):
+        if self.protocols:
+            return
+
+        self.protocols = defaultdict(list)
+        self.ports = defaultdict(list)
+        f = open(self.filename)
+        for line in f.readlines():
+            parts = line.split()
+            if parts and not parts[0].startswith('#') and parts[0] != 'unknown':
+                number, transport = parts[1].split('/')
+                port = '%s (%s)' % (number, transport.upper())
+                self.protocols[parts[0].lower()].append(port)
+                self.ports[parts[1]].append(parts[0])
+                if not self.nmap:
+                    for proto in parts[2:]:
+                        if proto.startswith('#'):
+                            break
+                        self.protocols[proto.lower()].append(port)
+
+    @match(r'^(?:(.+)\s+)?ports?(?:\s+numbers?)?(?(1)|\s+for\s+(.+))$')
+    def portfor(self, event, proto1, proto2):
+        self._load_services()
+        protocol = (proto1 or proto2).lower()
+        if protocol in self.protocols:
+            event.addresponse(human_join(self.protocols[protocol]))
+        else:
+            event.addresponse(u"I don't know about that protocol")
+
+    @match(r'^(?:(udp|tcp|sctp)\s+)?port\s+(\d+)$')
+    def port(self, event, transport, number):
+        self._load_services()
+        results = []
+        if transport:
+            results.extend(self.ports.get('%s/%s' % (number, transport.lower()), []))
+        else:
+            for transport in ('tcp', 'udp', 'sctp'):
+                results.extend('%s (%s)' % (protocol, transport.upper()) for protocol in self.ports.get('%s/%s' % (number, transport.lower()), []))
+
+        if results:
+            event.addresponse(human_join(results))
+        else:
+            event.addresponse(u"I don't know about any protocols using that port")
+
 # vi: set et sta sw=4 ts=4:

=== modified file 'ibid/plugins/sysadmin.py'
--- ibid/plugins/sysadmin.py	2010-01-18 23:20:33 +0000
+++ ibid/plugins/sysadmin.py	2010-02-13 09:40:30 +0000
@@ -1,8 +1,8 @@
 # Copyright (c) 2009-2010, Michael Gorven, Stefano Rivera
 # Released under terms of the MIT/X/Expat Licence. See COPYING for details.
 
+import os
 from subprocess import Popen, PIPE
-import os
 
 from ibid.plugins import Processor, match
 from ibid.config import Option