files missing in directory listing from smbclient 'dir' command with windows xp server

Automatically imported from Debian bug report #297771 http://bugs.debian.org/297771

Message-Id: <email address hidden>
Date: Wed, 02 Mar 2005 11:51:36 -0800
From: Kevin Dalley <email address hidden>
To: Debian Bug Tracking System <email address hidden>
Subject: files missing in directory listing from smbclient 'dir' command with windows
 xp server

Package: smbclient
Version: 3.0.10-1
Severity: grave
Justification: causes non-serious data loss

smbclient often loses files when getting a directory listing. Using
amanda for backup with samba results in loss of data, causing
incomplete backups. The problem is most likely to be occur when
dealing with larger directories, perhaps directories with over 300 or
1000 files.

I have seen the problem on my Debian system.

Here's my test, where /Kathy/DATA/2120 WNmod/ is a large directory.

for num in 1 2 3 4 5 6 7 8 9; do smbclient '\\puffin\c$' -U 'amanda%password' -E -d1 -Tcan /dev/null '/Kathy/DATA/2120 WNmod/*' > /tmp/foo/foolog$num.txt 2>&1; done
for num in 2 3 4 5 6 7 8 9; do diff -u /tmp/foo/foolog1.txt
/tmp/foo/foolog${num}.txt; done

The diff output includes the following, which shows 4 files which
disappeared between the 2 runs:

--- /tmp/foo/foolog1.txt 2005-03-02 11:46:45.000000000 -0800
+++ /tmp/foo/foolog9.txt 2005-03-02 11:46:46.000000000 -0800
@@ -1,4 +1,4 @@
 dry_run set
 Domain=[KECK] OS=[Windows 5.0] Server=[Windows 2000 LAN Manager]
-tar: dumped 1193 files and directories
-Total bytes written: 807012352
+tar: dumped 1189 files and directories
+Total bytes written: 805666304

The bug has been reported to the samba bugzilla list, but the bug is
important enough to be listed under Debian as well. The bug is
considered critical on the bugzilla list. I'm marking it as grave for
Debian, though I could probably justify critical.

https://bugzilla.samba.org/show_bug.cgi?id=2271

-- System Information:
Debian Release: 3.1
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: i386 (i686)
Kernel: Linux 2.6.8
Locale: LANG=en_US, LC_CTYPE=en_US (charmap=ISO-8859-1)

Versions of packages smbclient depends on:
ii libc6 2.3.2.ds1-20 GNU C Library: Shared libraries an
ii libcomerr2 1.35-6 The Common Error Description libra
ii libkrb53 1.3.6-1 MIT Kerberos runtime libraries
ii libldap2 2.1.30-3 OpenLDAP libraries
ii libncurses5 5.4-4 Shared libraries for terminal hand
ii libpopt0 1.7-5 lib for parsing cmdline parameters
ii libreadline4 4.3-11 GNU readline and history libraries
ii samba-common 3.0.10-1 Samba common files used by both th

-- no debconf information

Message-ID: <email address hidden>
Date: Sun, 13 Mar 2005 08:23:46 +0100 (CET)
From: Alban BROWAEYS <email address hidden>
To: <email address hidden>
Subject: Re: files missing in directory listing from smbclient 'dir' command with windows xp server

Just wanted to point out that the problem is fixed
upstream.

It was not a winxp problem, it affected all windows
servers.

https://bugzilla.samba.org/show_bug.cgi?id=2271
the patches are:
- Patch I've committed
- Additional patch

It was tested by the reported and fixed the issue (the
second patch fixing afteraffects of the first :
slowness)

Cheers
Alban

D�uvrez le nouveau Yahoo! Mail : 250 Mo d'espace de stockage pour vos mails !
Cr� votre Yahoo! Mail sur http://fr.mail.yahoo.com/

Message-Id: <E1DAS6y-0006vj-00@andromeda>
Date: Sun, 13 Mar 2005 07:22:23 -0500
From: Justin Pryzby <email address hidden>
To: <email address hidden>
Subject: tagging 297771

# Automatically generated email from bts, devscripts version 2.8.10
tags 297771 patch

Message-Id: <E1DASCp-0006w7-00@andromeda>
Date: Sun, 13 Mar 2005 07:28:27 -0500
From: Justin Pryzby <email address hidden>
To: <email address hidden>
Subject: tagging 297771

# Automatically generated email from bts, devscripts version 2.8.10
tags 297771 fixed-upstream

Message-Id: <E1DASD0-0006wI-00@andromeda>
Date: Sun, 13 Mar 2005 07:28:38 -0500
From: Justin Pryzby <email address hidden>
To: <email address hidden>
Subject: bug 297771 is forwarded to https://bugzilla.samba.org/show_bug.cgi?id=2271

# Automatically generated email from bts, devscripts version 2.8.10
forwarded 297771 https://bugzilla.samba.org/show_bug.cgi?id=2271

Message-ID: <20050313143301.GA27986@andromeda>
Date: Sun, 13 Mar 2005 09:33:01 -0500
From: Justin Pryzby <email address hidden>
To: <email address hidden>
Subject: patch

--dDRMvlgZJXvWKvBx
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Attaching a patch which is the result of the two upstream svn commits,
applied cleanly to Debian's clilist.c, and reran diff.
Justin

--dDRMvlgZJXvWKvBx
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename=smbpatch

--- clilist.c 2005-03-13 09:29:52.541928856 -0500
+++ old/clilist.c 2005-03-13 09:30:02.151467984 -0500
@@ -185,7 +185,7 @@
    setup = TRANSACT2_FINDFIRST;
    SSVAL(param,0,attribute); /* attribute */
    SSVAL(param,2,max_matches); /* max count */
- SSVAL(param,4,(FLAG_TRANS2_FIND_REQUIRE_RESUME|FLAG_TRANS2_FIND_CLOSE_IF_END)); /* resume required + close on end */
+ SSVAL(param,4,4+2); /* resume required + close on end */
    SSVAL(param,6,info_level);
    SIVAL(param,8,0);
    p = param+12;
@@ -197,9 +197,7 @@
    SSVAL(param,2,max_matches); /* max count */
    SSVAL(param,4,info_level);
    SIVAL(param,6,0); /* ff_resume_key */
- /* NB. *DON'T* use continue here. If you do it seems that W2K and bretheren
- can miss filenames. Use last filename continue instead. JRA */
- SSVAL(param,4,(FLAG_TRANS2_FIND_REQUIRE_RESUME|FLAG_TRANS2_FIND_CLOSE_IF_END)); /* resume required + close on end */
+ SSVAL(param,10,8+4+2); /* continue + resume required + close on end */
    p = param+12;
    p += clistr_push(cli, param+12, mask, sizeof(param)-12,
       STR_TERMINATE);
@@ -264,6 +262,24 @@
   p = rdata;

   /* we might need the lastname for continuations */
+ if (ff_lastname > 0) {
+ switch(info_level) {
+ case 260:
+ clistr_pull(cli, mask, p+ff_lastname,
+ sizeof(mask),
+ data_len-ff_lastname,
+ STR_TERMINATE);
+ break;
+ case 1:
+ clistr_pull(cli, mask, p+ff_lastname+1,
+ sizeof(mask),
+ -1,
+ STR_TERMINATE);
+ break;
+ }
+ } else {
+ pstrcpy(mask,"");
+ }

   /* and add them to the dirlist pool */
   tdl = SMB_REALLOC(dirlist,dirlist_len + data_len);
@@ -277,18 +293,10 @@

   /* put in a length for the last entry, to ensure we can chain entries
      into the next packet */
- for (p2=p,i=0;i<(ff_searchcount-1);i++) {
- p2 += interpret_long_filename(cli,info_level,p2,&finfo);
- }
+ for (p2=p,i=0;i<(ff_searchcount-1);i++)
+ p2 += interpret_long_filename(cli,info_level,p2,NULL);
   SSVAL(p2,0,data_len - PTR_DIFF(p2,p));

- /* we might need the lastname for continuations */
- if (ff_lastname > 0) {
- pstrcpy(mask, finfo.name);
- } else {
- pstrcpy(mask,"");
- }
-
   /* grab the data for later use */
   memcpy(dirlist+dirlist_len,p,data_len);
   dirlist_len += data_len;

--dDRMvlgZJXvWKvBx--

Message-Id: <E1DAaQ1-0000AD-00@andromeda>
Date: Sun, 13 Mar 2005 16:14:36 -0500
From: Justin Pryzby <email address hidden>
To: <email address hidden>
Subject: bug 297771 is not forwarded

# Automatically generated email from bts, devscripts version 2.8.10
notforwarded 297771

Not pendingupload anymore. The patches here make a problem that I can't
reproduce into one that I can. These patches have definetly been applied
upstream, but they're related to some other patch.

Also given that this only affects unusual cases, and has certainly been around
for a while, I'm not sure whether it justifies the 'Maj' setting.

Message-ID: <email address hidden>
Date: Wed, 23 Mar 2005 03:11:41 -0800
From: Steve Langasek <email address hidden>
To: <email address hidden>
Subject: 3.0.11-1 in preparation

tags 292957 pending
tags 274969 pending
tags 286174 pending
tags 284125 pending
tags 266693 pending
tags 297771 pending
thanks

Message-Id: <email address hidden>
Date: Thu, 24 Mar 2005 00:17:14 -0500
From: Steve Langasek <email address hidden>
To: <email address hidden>
Subject: Bug#297771: fixed in samba 3.0.11-1

Source: samba
Source-Version: 3.0.11-1

We believe that the bug you reported is fixed in the latest version of
samba, which is due to be installed in the Debian FTP archive:

libpam-smbpass_3.0.11-1_i386.deb
  to pool/main/s/samba/libpam-smbpass_3.0.11-1_i386.deb
libsmbclient-dev_3.0.11-1_i386.deb
  to pool/main/s/samba/libsmbclient-dev_3.0.11-1_i386.deb
libsmbclient_3.0.11-1_i386.deb
  to pool/main/s/samba/libsmbclient_3.0.11-1_i386.deb
python2.3-samba_3.0.11-1_i386.deb
  to pool/main/s/samba/python2.3-samba_3.0.11-1_i386.deb
samba-common_3.0.11-1_i386.deb
  to pool/main/s/samba/samba-common_3.0.11-1_i386.deb
samba-dbg_3.0.11-1_i386.deb
  to pool/main/s/samba/samba-dbg_3.0.11-1_i386.deb
samba-doc_3.0.11-1_all.deb
  to pool/main/s/samba/samba-doc_3.0.11-1_all.deb
samba_3.0.11-1.diff.gz
  to pool/main/s/samba/samba_3.0.11-1.diff.gz
samba_3.0.11-1.dsc
  to pool/main/s/samba/samba_3.0.11-1.dsc
samba_3.0.11-1_i386.deb
  to pool/main/s/samba/samba_3.0.11-1_i386.deb
samba_3.0.11.orig.tar.gz
  to pool/main/s/samba/samba_3.0.11.orig.tar.gz
smbclient_3.0.11-1_i386.deb
  to pool/main/s/samba/smbclient_3.0.11-1_i386.deb
smbfs_3.0.11-1_i386.deb
  to pool/main/s/samba/smbfs_3.0.11-1_i386.deb
swat_3.0.11-1_i386.deb
  to pool/main/s/samba/swat_3.0.11-1_i386.deb
winbind_3.0.11-1_i386.deb
  to pool/main/s/samba/winbind_3.0.11-1_i386.deb

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to <email address hidden>,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Steve Langasek <email address hidden> (supplier of updated samba package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing <email address hidden>)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Wed, 23 Mar 2005 00:13:16 -0800
Source: samba
Binary: samba-doc libsmbclient libpam-smbpass swat winbind smbclient samba samba-common libsmbclient-dev samba-dbg python2.3-samba smbfs
Architecture: source i386 all
Version: 3.0.11-1
Distribution: unstable
Urgency: high
Maintainer: Eloy A. Paris <email address hidden>
Changed-By: Steve Langasek <email address hidden>
Desription:
 libpam-smbpass - pluggable authentication module for SMB password database
 libsmbclient - shared library that allows applications to talk to SMB servers
 libsmbclient-dev - libsmbclient static libraries and headers
 python2.3-samba - Python bindings that allow access to various aspects of Samba
 samba - a LanManager-like file and printer server for Unix
 samba-common - Samba common files used by both the server and the client
 samba-dbg - Samba debugging symbols
 samba-doc - Samba documentation
 smbclient - a LanManager-like simple client for Unix
 smbfs - mount and umount commands for the smbfs (for kern...

Confimed fixed with current smbclient in Breezy.