broken version of netcat installed by default

Please try to keep a constructive tone. Your bug report does not, as far as I can see, describe the actual symptoms you're seeing at all. What is it, specifically, that you tried and how did it fail?

I believe I was quite clear:

(a) "nc -l -p 1234" spits out a usage message. The traditional version actually listens on port 1234. Having finally found nc's manpage, I see that the OpenBSD version uses completely incompatible arguments, hence breaking scripts that expect the decades-old version and giving no real clue what happened, because:
(b) nc's usage message does not say it's from OpenBSD and has no version number, hence making debugging the change in behavior needlessly frustrating. (Not to mention that it has -only- a short-form usage message and fails to take --version or --help or any of the other canonical ways of getting more info, so it fakes people out into believing that it's basically the same set of options when in fact they're totally different.)

This is true in both 32- and 64-bit Lucid on any machine I've tried.

I note, btw, that using update-alternatives did -not- replace the OpenBSD manpage with the traditional one, even though it replaced the links in /etc/alternatives, so I now have the manpage for the OpenBSD version installed even though nc & netcat on my machines are actually the traditional version. This also seems buggy to me, since it's a really easy way to totally confuse people by showing them the manpage for the wrong program. I suppose I should file a bug report about that, too.

On Mon, Jun 07, 2010 at 08:52:50PM -0000, Grondr wrote:
> I believe I was quite clear:
>
> (a) "nc -l -p 1234" spits out a usage message.

I think this is the first time you shared any of the commands you
attempted to run.

You are asking people (volunteers, mostly) to help you with your
problems. Please don't make their lives more difficult by making them
guess what you've done when you could just tell them.

> The traditional version actually listens on port 1234. Having finally
> found nc's manpage, I see that the OpenBSD version uses completely
> incompatible arguments,

Ok, I'll take your flame bait.

-c and -e are missing. Noone should be using them anyway, but yes, they
are missing.

-b is missing. The openbsd version seems to allow broadcast by default,
so we could just make it accept the option and ignore it.

-g and -G are missing, so you don't get to specify source routing.

Both have -h. It does the same.

Both have -i. It does the same.

Both have -k. They seem to do the same.

Both have -l. They do the same.

Both have -n. They do the same.

-o is missing in the openbsd version. I think it's a silly option, but
yes, it's missing.

-p is different between the two. The OpenBSD version uses it to set a
source port for remote connections, while the traditional nc uses it to
specify the local port on which it will listen when in listen mode.

Both have -r. They do the same.

Both have -q. They do the same, but have different defaults (see
#544935).

Both have -s. They do the same.

Both have -T. They do the same.

Both have -t. They do the same.

Both have -u. They do the same.

Both have -v. They do the same.

Both have -w. They do the same.

Both have -z. They do the same.

In addition, the openbsd version has all these cool new things:

-4 and -6 to specify ipv4 or ipv6, respectively.

-D to enable a debug socket (don't really know what this is).

-d detaches from stdin.

-P to specify username for proxy auth.

-S to enable TCP MD5 signatures.

-C to do LF/CRLF translation.

-U to talk to UNIX domain sockets.

-X to specify proxy protocol.

-x to specify proxy address.

So no, they're not identical, but they're also hardly "completely
incompatible". This is the sort of stuff I'm talking about when I say
"constructive tone". Let's focus on the specific issues and try to work
on them rather than tossing around (incorrect) absolutes, shall we?

> hence breaking scripts that expect the decades-old version and giving
> no real clue what happened, because: (b) nc's usage message does not
> say it's from OpenBSD and has no version number, hence making
> debugging the change in behavior needlessly frustrating.

True. The "nc -h" output says it loud and clear, though:
OpenBSD netcat (Debian patchlevel 1.89-3ubuntu2)

Would you like to file a feature request about making it output this as
part of its usage message? If so, could you file a similar bug against
the traditional netcat as it has the same behaviour?

> (Not to mention that it has -only- a short-form usage message and
> fails to take --version or --help or any of the other canonical ways
> of getting more info, so it fakes people out into believing that it's
> basically the same ...

I'll try one more time to make the point I'm trying to make, and then I'll give up until someone who seems to understand it chimes in. (And by the way, calling a usability complaint "flamebait" isn't exactly setting the right tone yourself.)

100% of my usage of nc is in building tunnels. 50% of nc's invocations are thus listeners, and 50% are senders. 100% of listeners require (in the old nc) specifying both -l and -p. Since OpenBSD apparently decided to take what seems to me the case that is -required- in -every- usage of nc for this purpose and incompatibly break it (not -just- by dropping -p's original meaning---which at least would mean it wouldn't show up in a usage messsage---but by making it mean something -completely different- while -simultaneously- making it fail when used with -l!), this means that anyone expecting the traditional nc (which is not only something like 20 years old, but has also been shipping by default in Ubuntu as long as Ubuntu has been around---until now) will be screwed by the change in behavior. So it really doesn't matter how many -other- options -haven't- been changed---they picked the -one- option I use -every single time- and made the same option mean something completely different, in a way guaranteed to cause the program to (apparently inexplicably) fall over and emit a usage message instead.

You complain that I called it "broken", but it wasn't until after I posted my first entry in this that I even -realized- that the currently-default nc -did not allow- both -l and -p in the same command line! It just never even crossed my mind that the old nc would have been replaced with one that didn't take the same command line args in an upward-compatible fashion, but instead decided to just break if called with the old ones. So yes, when I first posted, I truly believed that nc was absolutely, completely broken, because I could not come up with a single command line that failed to do anything but emit a usage message. When I've been using the same program for decades and it suddenly won't take the same commands I've been giving it all that time, my first inclination is -not- to turn to a manpage to see if someone has scrambled the allowed options---especially when I see in the usage it spits out that the only options I was trying (-l and -p) -were- still there. Suuuure, if I'd really read it carefully, I'd have seen "source" at -p and said, "Eh?", but even that might not have necessarily been enough to raise a big red flag.

So yeah, when I wrote that report, I honestly believed that Ubuntu had shipped a completely broken netcat---one that, despite my best efforts, could not be made into a listener. I just couldn't believe it. It seemed incomprehensible that their quality control had fallen so far that they could (a) modify and (b) -break- such a fundamental part of netcat.

As I see it, there's plenty of blame to go around:
(a) OpenBSD, for breaking the established argline for no particularly good reason.
(b) Ubuntu, for substituting in their version without making some changes (like making it -really clear- from the usage that this isn't the nc you were expecting, e.g., by altering the usage messag...

On Thu, Jun 10, 2010 at 04:52:32AM -0000, Grondr wrote:
> I'll try one more time to make the point I'm trying to make, and then
> I'll give up until someone who seems to understand it chimes in. (And
> by the way, calling a usability complaint "flamebait" isn't exactly
> setting the right tone yourself.)

I'm not calling the usability complaint itself flamebait. I'm calling
the language you used to convey said usability complaint flamebait. And
counter-productive.

> You complain that I called it "broken", but it wasn't until after I
> posted my first entry in this that I even -realized- that the
> currently- default nc -did not allow- both -l and -p in the same
> command line!

I don't see where you're going with this /at all/.

> So yeah, when I wrote that report, I honestly believed that Ubuntu had
> shipped a completely broken netcat---one that, despite my best
> efforts, could not be made into a listener. I just couldn't believe
> it. It seemed incomprehensible that their quality control had fallen
> so far that they could (a) modify and (b) -break- such a fundamental
> part of netcat.
>
> As I see it, there's plenty of blame to go around:

Yes, everyone except you suck. Now that we've established this, can we
/please/ get on with fixing the 1 (one!) bug that you actually seem to
have encountered?

> Unless Ubuntu can successfully apply backpressure upstream (I'd have
> said, "We won't make OpenBSD's version of nc the default until the
> following fake-out usability issues are addressed"),

The OpenBSD folks never asked for us to include their software, much
less make it our default netcat. This is not politics. There's no need
to go and make demands to all sorts of people. It's simple engineering.
In the time I've spent writing this response I could have fixed this bug
several times.

> Ubuntu certainly isn't shy about making their own downstream patches
> in other things---this seems like an issue that begs for it.

You'd be surprised how much the OpenBSD netcat in Ubuntu has been
patched compared to what's in OpenBSD, /precisely/ to make this
transition smooth. Yes, I missed this "-p" option when I did this. Shit
happens.

> As for filing a feature request with OpenBSD for better and clearer
> version/origin info, I can try it, but I'm fairly sure the OpenBSD
> crowd will just say, "We've been using this for years and it's -our-
> program, so why should -we- be the ones to include something that
> points out it's from OpenBSD?"

Yeah, openbsd-devel seems like quite an argumentative place. I've met my
quota for argumentativity for this week, though.

> They might object that it's Ubuntu's job (or maybe Debian's---I don't
> know who made this change)

I did.

> to make it clear, since U/D are the ones who decided to -switch-,
> after years of the traditional nc, to using OpenBSD's version in the
> first place.

Yeah. And as is always the case when humans are involved, mistakes
happen, which brings me back to the 1 (one!), small bug you seem to be
experiencing:

  netcat-openbsd should accept "-p" along with "-l".

I would update the bug title and description (as appropriate), but I
can't be bothered to respond to another burst of o...

Wait, what? Are you saying that nc -should- accept -p with -l? In other words, that "nc -l -p 1234" is the same thing as "nc -l 1234" in OpenBSD nc? If so, I very much agree with that sentiment!

But that wasn't what the manpage seemed to indicate -at all-. It goes out of its way, in several places, to claim that this isn't possible, which was why I assumed that they had incompatibly changed the meaning of -p such that it no longer made sense to use it with -l, and then Ubuntu had (knowingly) shipped a version of nc that no longer accepted args that previous versions had accepted. If that's not the case, and the two cases really -are- supposed to be equivalent, then yes, changing this will restore a huge amount of compatibility with the traditional nc, and I don't understand why OpenBSD went out of their way (in both the code and the manpage) to make this not work in the first place.

Since you say this is a bug, I'll change the status.

(And yes, Hobbit really does have a real name, but he almost never uses it.)

P.S. You say you don't know where I was going w/my comment about "broken" vs -l -p. I was simply trying to say that the current (bug? deliberate change?) to the behavior of -l -p made it look like nc simply couldn't be used as a listener at all, and that, when I wrote the report, I hadn't yet discovered that the manpage for the OpenBSD's version claimed that -l -p was simply incorrect usage (based on what looked like a non-upward-compatible change) that that's why I was so intemperate in my tone---until I found what looked like a deliberate change in semantics (which you now say is a bug---I agree!), it looked like the intended way to use nc as a listener had just somehow been entirely broken instead.

On Thu, Jun 10, 2010 at 04:38:21PM -0000, Grondr wrote:
> Wait, what? Are you saying that nc -should- accept -p with -l?

Of course. Otherwise it'd be pointlessly incompatible with the old
netcat.

> In other words, that "nc -l -p 1234" is the same thing as "nc -l 1234"
> in OpenBSD nc?

No, it /is/ not. It should be.

> But that wasn't what the manpage seemed to indicate -at all-.

No, because it's not the case now.

> It goes out of its way, in several places, to claim that this isn't
> possible,

It's not. From netcat.c:

        if (lflag && pflag)
                errx(1, "cannot use -p and -l");

We should just change that.

> yes, changing this will restore a huge amount of compatibility with
> the traditional nc, and I don't understand why OpenBSD went out of
> their way (in both the code and the manpage) to make this not work in
> the first place.

Well, having to pass -p /is/ kind of silly. If you want to listen, you
want to specify a port on which to do so, so making you pass the -p
*option* only makes sense because of tradition.

> P.S. You say you don't know where I was going w/my comment about
> "broken" vs -l -p. I was simply trying to say that the current (bug?
> deliberate change?) to the behavior of -l -p made it look like nc
> simply couldn't be used as a listener at all, and that, when I wrote
> the report, I hadn't yet discovered that the manpage for the OpenBSD's
> version claimed that -l -p was simply incorrect usage (based on what
> looked like a non-upward-compatible change) that that's why I was so
> intemperate in my tone---until I found what looked like a deliberate
> change in semantics (which you now say is a bug---I agree!), it looked
> like the intended way to use nc as a listener had just somehow been
> entirely broken instead.

The OpenBSD folks may not see it as a bug at all. For us, it's a bug,
because it pointlessly breaks compatibility. No doubt about that.

--
Soren Hansen
Ubuntu Developer
http://www.ubuntu.com/

Aha! In that case, I agree with everything you've said. :)

(Yes, I agree that it's pointlessly incompatible, but I thought someone must have had a good reason for it---e.g., I'd simply assumed there was -some- point that I was missing, and that Ubuntu had gone along. I also agree that -having- to pass -p is silly if it can take it as a parameter without -p [it certainly makes both the client & server syntax much more similar to not require -p), but yes, for backward compatibility, -p should be allowed.)

Thanks!

P.S. Huh. I haven't gone and looked at nc's code carefully, but that clause you quote that includes the string "cannot use -p and -l" doesn't seem to make it through to any user-visible output. If I try just "nc -l -p 1234",
I get the "usage: nc [blah blah blah]" that lists all the options but does not include that string, which was one of the problems---the message I saw never said "you're using incompatible options", and since I was using the same options as I always had before, it seemed like option parsing had simply gotten broken somehow instead. Anyway, thanks again.

On Fri, Jun 11, 2010 at 06:49:26PM -0000, Grondr wrote:
> P.S. Huh. I haven't gone and looked at nc's code carefully, but that
> clause you quote that includes the string "cannot use -p and -l"
> doesn't seem to make it through to any user-visible output.

Umm. Yeah, that's really stupid. Someone put those things *after* the
other cmdline syntax checks, so they'd (hardly?) ever be shown. I'll fix
that while I'm at it.

--
Soren Hansen
Ubuntu Developer
http://www.ubuntu.com/

This bug has also affected me though the work around below

sudo apt-get install netcat
sudo update-alternatives --set nc /bin/nc.traditional

The bug I had with the netcat-openbsd missing -e and -u (udp mode) does not work.

The bsd version does not support requests via piping too.
This works with nc.traditional, but does not with nc.openbsd:
printf 'GET / HTTP/1.0\n\n' | nc example.com 80

The same with the following:
nc example.com 80 <<request
GET / HTTP/1.0

request

Yea. I just spent all morning on this not having the piping thing. I had to use
-q-1

printf 'GET / HTTP/1.0\n\n' | nc -q-1 example.com 80

The same with the following:
nc example.com 80 <<request
GET / HTTP/1.0

request

I just went through the ringer tracing down a problem with nc on lucid and discovered that the problem is in a major difference between nc.openbsd and nc.traditional. Above, Soren states:

> Both have -w. They do the same.

Well, they don't. Not by a longshot. If you surf the 'net for ssh ProxyCommand usage you will see a lot of suggestions to do this in your ~/.ssh/config file:

   Host foobar
       ProxyCommand ssh mygateway nc -w 1 foobar 22

The above allows one to ssh into host foobar which sits behind a ssh gateway, mygateway. The -w option in nc.traditional is a timeout "on connections and EOFs". Add to that for nc.openbsd, "stdin". In other words, my users are discovering that if they don't send input every second they will be bumped off.

Man page for nc_traditional:

     -w secs timeout for connects and final net reads

Man page for nc (openbsd):

    -w timeout
             If a connection and stdin are idle for more than timeout seconds, then the connection is silently closed.

I installed both netcat-openbsd and netcat-traditional on mygateway. I put this in my ~/.ssh/config file on a remote computer:

    Host foobar
       ProxyCommand ssh mygateway nc.openbsd -w 1 foobar 22

    Host foobar2
       ProxyCommand ssh mygateway nc.traditional -w 1 foobar 22

If I ssh to foobar, I get disconnected after one second of idle time. Foobar2, I stay logged in.

Cheers,

Dan

This also breaks the web, resulting in major confusion.

Searching the web for netcat linux howto or tutorial returns almost 100% broken examples.

Netcat fails without any hint of why, while saying the flags you used where valid.
using man nc does does not solve this confusion.
From 'man nc'
> -l Used to specify that nc should listen for an incoming connection rather than initiate a connection to a
> remote host. It is an error to use this option in conjunction with the -p, -s, or -z options. Additionally,
> any timeouts specified with the -w option are ignored.

This does not say you specify the port with -l port
Instead it says how not to use it. Not much help.

I propose the following changes

1. Change short usage message to include ref to the man page and the date of switch of behaviour
 - nc usage flags where changed %date%. Use man nc for more information.

2. Remove the "It is a n error" (way too strong, suggests that this will result in some kind of damage) and at the end (of each parameter) say "can not be used with -p" etc.

3. Add information on HOW -l is used. -l port. See "-i interval" and compare the two. -i has a parameter. -l does not.

Sounds good?

Regards,

Torbjorn

This bug was fixed in the package netcat-openbsd - 1.89-3ubuntu4

---------------
netcat-openbsd (1.89-3ubuntu4) natty; urgency=low

  * debian/patches/quit-timer.patch: Drop quit-time.patch as per disccussion on
    ubuntu-server. (LP: #590925)
 -- Chuck Short <email address hidden> Mon, 03 Jan 2011 10:08:24 -0500

Manpage should be corrected. The -q option is not working now. And you should report this change to others packages maintainers, for example maintainers of libvirt and virt-manager, which expects this -q option and wrote ugly hacks for compability with other non-debian distros.

This bug was fixed in the package netcat-openbsd - 1.89-3ubuntu5

---------------
netcat-openbsd (1.89-3ubuntu5) natty; urgency=low

  * debian/patches/quit-timer.patch: Re-enabled, but set default to 0 to match
    current behavior.
  * debian/patches/netcat-info.patch: Add info about netcat-traditional
    if you are looking for an option when it is not available.
    (LP: #590925)
 -- Chuck Short <email address hidden> Tue, 04 Jan 2011 14:31:12 -0500

Hi all,

How does this fix the bug in lucid.

I wish to perform this command in lucid

nc -l -p xxxx. which would have been a normal nc command.

what do I have to do in lucid to have this work.

Thanks.

@sefs: the bug has been fixed in Natty. It won't be fixed in previous versions of Ubuntu because the package doesn't fit the requirements for backporting. See https://help.ubuntu.com/community/UbuntuBackports for more information.

Hello All,

>-b is missing. The openbsd version seems to allow broadcast by default,
>so we could just make it accept the option and ignore it.

I found that no broadcast is being made when trying to send a udp broadcast. The openbsd version will return with error code 1

Regards,

Andras

A few years later, but I just have to join in: I don't think nc.openbsd should ever have been added as an alternative to nc.traditional, as they are not compatible. They should be separate packages - nc.traditional should be netcat, and nc.openbsd should be netcat-openbsd. They should have separate binaries that are not part of an alternatives group, and if someone wants to use nc.openbsd then they can call it as such - thereby specifically requesting the openbsd interface instead of the traditional one.

Probably too late to fix that now. But just in general, barring the obvious case of one implementation having extra functionality and options that are not present in the other, alternatives should ALWAYS share the same interface (think of the vast number of options to GNU binaries that are ignored or mirror other options, PURELY to provide the same interface that has been used by other implementations for decades). Kind of a 'duh' moment for any programmer reading this thread.