Pushing to shared repo using FTP doesn't work: "Append/Restart not permitted, try again"

On Thu, 2008-11-06 at 14:35 +0000, Jonny Dee wrote:
> FTP temporary error: 451 /bzr-repo/.bzr/repository/upload/k2a94q9mf00jznz50sct.fetch: Append/Restart not permitted, try again. Retrying.
> [...]
> bzr: ERROR: Transport error: FTP temporary error during APPEND /bzr-repo/.bzr/repository/upload/k2a94q9mf00jznz50sct.fetch.Aborting. 451 /bzr-repo/.bzr/repository/upload/k2a94q9mf00jznz50sct.fetch: Append/Restart not permitted, try again
It seems like we should be able to work around this by doing 'get remote
file', 'perform append locally, 'push new file'.

On Thu, 2008-11-06 at 14:49 +0000, Daniel Watkins wrote:
>
> > bzr: ERROR: Transport error: FTP temporary error during
> APPEND /bzr-repo/.bzr/repository/upload/k2a94q9mf00jznz50sct.fetch.Aborting. 451 /bzr-repo/.bzr/repository/upload/k2a94q9mf00jznz50sct.fetch: Append/Restart not permitted, try again
> It seems like we should be able to work around this by doing 'get
> remote
> file', 'perform append locally, 'push new file'.

We don't actually append to any files. The .fetch file is streamed out
by the push command - it starts empty.

This is the FTP implementation of a stream expecting APPE to work (which
is a reasonable expectation :() and being denied.

So - its a FTP transport bug, and probably what the FTP transport needs
to do is to write the first block of data its given in two parts
(keeping both in memory) so it can be sure that APPE works; if it
doesn't, make a temp file and buffer, if it does, cool.

Note that using a temp file will require redirecting read/readv requests
to the streams name into the local temp file.

 status confirmed
 importance medium

-Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.

I had a similar problem and solved it by using proftpd and adding/setting the parameter "AllowStoreRestart on".

Hi everybudy, first : THANK YOU for your work ;).

I tried to push a repository online and I have the same problem.
I can't change things like FTP server and so on, I can only use FTP, I need this to work with my friend on something, is there a way? a workaround?

2009/4/29 Aldo "xoen" Giambelluca <email address hidden>:
> Hi everybudy, first : THANK YOU for your work ;).
>
> I tried to push a repository online and I have the same problem.
> I can't change things like FTP server and so on, I can only use FTP, I need this to work with my friend on something, is there a way? a workaround?

Hi Aldo,

Unfortunately if you can't change the server I don't think there is a
workaround until this is fixed in Bazaar. If you're interested in
making a patch as described by Robert it should not be very hard and
we will review it for you.

--
Martin <http://launchpad.net/~mbp/>

Hi,

the attached patch adds a fallback to GET + PUT if APPE fails.
Tested against Twisted 8.2.0 FTP Server.

Best,

Nils Durner

2009/7/14 Nils Durner <email address hidden>:
> Hi,
>
> the attached patch adds a fallback to GET + PUT if APPE fails.
> Tested against Twisted 8.2.0 FTP Server.

Hi Nils,

Thanks for the patch. Could you please put it in a branch on
Launchpad and create a merge proposal so that we can review and track
it?

The patch looks reasonable to me. In some cases this may be a lot
slower so we should at least write a warning to the user the first
time this happens.

It would be nice to add a test that it works correctly, using our test
suite's medusa ftp server. I wouldn't strictly insist on having one
but at least adding a TODO to the test file would be good.

--
Martin <http://launchpad.net/~mbp/>

Nils: How does this issue affect TahoeLAFS?

@Martin: ok!

@Zooko: Bazaar uses extended FTP commands like APPE and SITE which are not supported by some FTP servers (like the one that is integrated in Tahoe). With my patch applied, users can store their Bazaar repositories in a Tahoe storage grid (aka "SCM in the cloud").

Nils: Awesome! Thank you very much for this patch. If it gets accepted into bzr in time for the TahoeLAFS v1.5 release (due any day now), then I'll include the fact of its existence in the TahoeLAFS release announcement! :-)

I'll just mention that higher level code operations assume things about being able to append. Which means that while your code will probably work, I believe it will become quadratic if you try to push up a large branch.

1) Push up rev 1
2) Download rev 1 + push up rev 1 & 2
3) Download rev 1&2, push up rev 1-3
4) Download rev 1-3, push up rev 1-4
...
100) Download revs 1-99, push up revs 1-100

Now we do a little bit with buffering, so we should only append XX data per attempt, but I'm curious if supporting something with abysmal performance is worthwhile...

I agree that large pushes are problematic. But depending on your workflow, small pushes may be much more common than really big (and thus expensive) ones. This is especially true for solo repositories that are usually always reachable: one push (pack) would virtually always consist of just one commit.

Patch sent to pqm

See bug 409615 - we can probably eliminate at least some cases where we're unnecessarily asking to append when creating a new file, which may reduce/eliminate the need to use this fallback. Want to try, Nils?

In duplicate Bug #409507, Martin Pool asked: "Do you know what (FTP) server software it is?"

I checked with my hosting provider and they said

"Our servers use proftpd. http://www.proftpd.org/"

I'm really impressed with how fast you've replied to my bug submission. I guess I'll keep an
eye out for v1.18.

I'm a little surprised that you haven't hit this sever problem long ago, but then again I
know next to nothing about how FTP works and how many server implementations there are.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Peter Liepa wrote:
> In duplicate Bug #409507, Martin Pool asked: "Do you know what (FTP)
> server software it is?"
>
> I checked with my hosting provider and they said
>
> "Our servers use proftpd. http://www.proftpd.org/"
>
> I'm really impressed with how fast you've replied to my bug submission. I guess I'll keep an
> eye out for v1.18.
>
> I'm a little surprised that you haven't hit this sever problem long ago, but then again I
> know next to nothing about how FTP works and how many server implementations there are.
>

We've certainly hit servers that didn't support APPE (append) before.
Just that nobody cared enough to implement support for it. As usually
switching to sftp is a *much* better decision for a large variety of
reasons. (FTP also doesn't support partial *reads* which causes some
significant performance implications when fetching data from a repository.)

FTP as a protocol is not a rich protocol, and not particularly secure
(passwords are transmitted in plain-text across the wire).

It is supported, because it was possible to do so, but it certainly
isn't something we would specifically *recommend*.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkp7I5kACgkQJdeBCYSNAAOA4gCgtgldGdrL/6IqvTfxa24Z03J3
alMAoNmMeQDanHWaXxF/iRvKa52pPV4m
=0Du8
-----END PGP SIGNATURE-----

>>>>> Peter Liepa writes:

    > I checked with my hosting provider and they said

    > "Our servers use proftpd. http://www.proftpd.org/"

proftpd certainly support APPE. It may be configured to *not*
support it though and that may be the case for the server you're
using, you may want to ask "them" why "they" did so though...

John A Meinel wrote:
>We've certainly hit servers that didn't support APPE (append) before.
>Just that nobody cared enough to implement support for it. As usually
>switching to sftp is a *much* better decision for a large variety of
>reasons [...]

Ok, thanks, the docs could be clearer about this, and maybe bazaar could
avoid crashing. I'm checking with my hosting
provider whether SFTP is available (probably is, I've just never used it before,
and am having trouble getting a successful "hello world" connection). But
that sounds like the way to go. From an above comment, maybe the
proftpd server has AllowStoreRestart set the wrong way, but I doubt I
could get them to change it.

2009/8/8 Peter Liepa <email address hidden>:
> John A Meinel  wrote:
>>We've certainly hit servers that didn't support APPE (append) before.
>>Just that nobody cared enough to implement support for it. As usually
>>switching to sftp is a *much* better decision for a large variety of
>>reasons [...]
>
> Ok, thanks, the docs could be clearer about this, and maybe bazaar could
> avoid crashing.

Right, from the next release of bzr we will give a message and continue.

Where do you think this should be described in the Bazaar documentation?

--
Martin <http://launchpad.net/~mbp/>

There is a user guide at http://doc.bazaar-vcs.org/bzr.dev/en/user-guide/index.html. This is where I read about FTP being supported. It's clear in retrospect that SFTP is preferred, because this is what the user guide refers to the most. FTP is referred to, in passing, just a couple of times. I think that you may mention that not all FTP configurations are supported (the details of APPE and AllowStoreRestart may be too arcane for users at my level), and that you might implement a specific test that validates some pre-conditions for FTP working (or just fix the crash and try to print out a clear message). I'd suggest adding a new section somewhere in the guide, and referring to it from anywhere that FTP is mentioned.

My hosting service has disabled proftpd APPE, rightly or wrongly, because of the possibility of malicious use. Also, my cheapo plan does not support SSH or SFTP, and I'm unwilling to pay the extra $ to upgrade. Also, our volunteer-run website will soon be migrating to a server that does not support SSH at all, or so I'm told. I can understand that the bazaar project may be concentrating on serving a more savvy and professional grade of user than myself. I'm going to try springloops.com, because they have a different model of FTP deployment - but I'd be more than happy to revisit bazaar when the limitations of FTP have been eased.

In closing, I'd like to mention that FTP website deployment via VCS seems to be almost a "killer app" that is under-addressed for the masses like me. I'm a relative web newbie who doesn't have the resources or inclination to use SSH, configure Apache, install server-side VCS, write Python commit hooks, etc, etc, yet would love to deploy websites via FTP from a VCS in a platform independent (mac and pc) way. So far I haven't found a solution that addresses my needs, although I'll be looking at Bespin and Springloops. I'll also be keeping an eye on how you guys progress on this. Meanwhile, thanks for your rapid responses and helpful attention.

FTP service is definitely a feature we want to support. It's true
that there seem to be more quirks in FTP servers than in SFTP and the
protocol is not quite as capable - understandable considering their
relative ages.

If you would like to try 1.18rc1 when it comes out and report on the
other bug (or a new bug) whether it works ok that would be
appreciated.

--
Martin <http://launchpad.net/~mbp/>

>>>>> Peter Liepa <email address hidden> writes:

<snip/>

    > My hosting service has disabled proftpd APPE, rightly or
    > wrongly, because of the possibility of malicious use.

Shudder.

Have you tried contacting them to see if that could be activated
on a per-user or per-directory basis ?

    > Also, my cheapo plan does not support SSH or SFTP, and I'm
    > unwilling to pay the extra $ to upgrade.

Hmm, I think that answer my question above :-/

    > I can understand that the bazaar project may be
    > concentrating on serving a more savvy and professional
    > grade of user than myself.

That's not how I feel about ftp at all. There are definitely
performance and security issues with ftp and *these* are the main
reasons why the ftp transport got less love.

But the truth is also that we have few bug reports about ftp
*users* and gives the (maybe wrong) impression that the existing
ftp servers are ok.

If propftpd doesn't allow APPE by default (but how many admins
knows about to override that ?), now, that kind of change the
game.

Indeed, with my local_test_server plugin I had to configure
proftpd like:

<Anonymous /tmp>
   User %(user)s
   UserAlias anonymous %(user)s
   AllowOverwrite on
   AllowStoreRestart on
   <Limit ALL>
          AllowAll
   </Limit>
</Anonymous>

    > In closing, I'd like to mention that FTP website deployment
    > via VCS seems to be almost a "killer app" that is
    > under-addressed for the masses like me.

bzr is committed to addres what we call the 'dumb servers' (as
opposed to the smart servers which use bzr itself) which only
provides a remote file system.

<snip/>

    > So far I haven't found a solution that addresses my needs,
    > although I'll be looking at Bespin and Springloops. I'll
    > also be keeping an eye on how you guys progress on this.
    > Meanwhile, thanks for your rapid responses and helpful
    > attention.

Launchpad itself offers code hosting, did you check it ?

I'm beginning to think that satisfying users like me may be counterproductive for your project.

To recap, I'm helping a small group of volunteers to maintain a ski club website. I'd like to set up version control on server A, and deploy the website content to server B. I don't mind whether A and B are the same or different. My main restriction is that my only access to server B is via FTP. And of course I want the VCS to do the deployment -- it should only update the files that have changed or been added, while deleting those files that have been deleted.

In researching this, I found descriptions of people setting up a VCS on server B, and using the working tree as the actual website content. I.e A and B are the same. That meant that deployment was pretty well automatic .. whenever you committed/pushed a change to the VCS, the working tree was updated, and so the website was automatically updated. But these people were able to use SSH and set up a VCS on their server. So I couldn't do what they were doing.

I was hoping that I could pull off this trick with Bazaar, because it handled FTP updates of the VCS. Yesterday, I tried making this happen on a host server that didn't have the FTP APPE disabled. This time, the push to server B was successful. Unfortunately, the working tree was not populated. There was a long message explaining this and why it was so, but nevertheless it was a showstopper for me.

Even if this worked, there would potentially be the problem that the .bzr directory is actually part of my website. Probably not a showstopper -- I'd have to configure the server to make that directory's contents inaccessible to HTTP.

Meanwhile, springloops.com works for me. The configuration in this case is that my web host is server B, and springloops is server A. I.e. A and B are different.

Springloops seems to be optimized for website development and deployment. Bazaar seems to be optimized for code base maintenance and release deployment. Although the two are philosophically similar, it may be that you have to optimize for one or the other.

So. getting back to my opening point, it may be counterproductive for you to try to address my problems. A few developers have commented that FTP is somewhat unworthy of their love, and I construe that to mean that making it "just work" may be a source of unending aggravation. And I don't quite understand what it is that does not allow you to populate a remote working directory, but without that I still wouldn't be able to use Bazaar for my purposes.

I guess you could turn my comments around and, rather than concentrate on building a VCS for code development and deployment, you could look at supporting workflows for website development and deployment as well, and that would be great. Mercurial has made a step in that direction - there is a plugin called FTPExtension, but rightly or wrongly I got the feeling it was too bleeding edge for my purposes.

Hope this helps.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

...
> Meanwhile, springloops.com works for me. The configuration in this case
> is that my web host is server B, and springloops is server A. I.e. A
> and B are different.
>
> Springloops seems to be optimized for website development and
> deployment. Bazaar seems to be optimized for code base maintenance and
> release deployment. Although the two are philosophically similar, it
> may be that you have to optimize for one or the other.

You might want to look at "bzr-upload" which is a plugin specifically
for uploading websites to other locations via the bzr transport
mechanism. I think it fits pretty much exactly what you are looking for.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkqBkW4ACgkQJdeBCYSNAAMulQCdE5jXyFR6vDRy91UG7njr9+kK
xCsAn3pSayUff8e0CJZJifUHGeVBWgRy
=xCin
-----END PGP SIGNATURE-----

'Invalid' only because it didn't require any change to Tahoe-LAFS.

This bug also affects me

bzr --version
Bazaar (bzr) 2.3b1
  Python interpreter: /usr/bin/python 2.6.5
  Python standard library: /usr/lib/python2.6
  Platform: Linux-2.6.32-25-generic-x86_64-with-Ubuntu-10.04-lucid
  bzrlib: /usr/lib/python2.6/dist-packages/bzrlib

The error that return the ftp server is 451
and the bzrlib ftp layer cannot detect it

the file attached is my .bzr-log (i removed ftp address and password and my home address for security....)

the relevant lines are

9.816 FTP stat: /htdocs/.bzr/repository
10.045 FTP get: /htdocs/.bzr/repository/pack-names
10.333 FTP rm: /htdocs/.bzr/repository/no-working-trees
10.393 FTP get: /htdocs/.bzr/repository/pack-names
10.693 Using fetch logic to copy between CHKInventoryRepository('file://~/www-data/.bzr/repository/')(RepositoryFormat2a()) and CHKInventoryRepository('ftp://ftp.ciccio.com/htdocs/.bzr/repository/')(RepositoryFormat2a())
10.695 FTP put: /htdocs/.bzr/repository/upload/z1lkk39vtt2ohonyndb1.pack
11.164 FTP has check: z1lkk39vtt2ohonyndb1.pack => /htdocs/.bzr/repository/upload/z1lkk39vtt2ohonyndb1.pack
11.221 FTP has: /htdocs/.bzr/repository/upload/z1lkk39vtt2ohonyndb1.pack
11.278 FTP appe to /htdocs/.bzr/repository/upload/z1lkk39vtt2ohonyndb1.pack
11.279 FTP appe (try 0) to /htdocs/.bzr/repository/upload/z1lkk39vtt2ohonyndb1.pack
[ 4808] 2010-10-12 06:33:27.985 WARNING: FTP temporary error: 451 /htdocs/.bzr/repository/upload/z1lkk39vtt2ohonyndb1.pack: Append/Restart not permitted, try again. Retrying.
11.452 Constructing FTP instance against ('ftp.ciccio.com', None, '++++++++', '********', False)
11.863 FTP appe (try 1) to /htdocs/.bzr/repository/upload/z1lkk39vtt2ohonyndb1.pack
[ 4808] 2010-10-12 06:33:28.569 WARNING: FTP temporary error: 451 /htdocs/.bzr/repository/upload/z1lkk39vtt2ohonyndb1.pack: Append/Restart not permitted, try again. Retrying.
12.036 Constructing FTP instance against ('ftp.ciccio.com', None, '++++++++', '********', False)
12.449 FTP appe (try 2) to /htdocs/.bzr/repository/upload/z1lkk39vtt2ohonyndb1.pack
[ 4808] 2010-10-12 06:33:29.156 WARNING: FTP temporary error: 451 /htdocs/.bzr/repository/upload/z1lkk39vtt2ohonyndb1.pack: Append/Restart not permitted, try again. Retrying.
12.623 Constructing FTP instance against ('ftp.ciccio.com', None, '++++++++', '********', False)
13.048 FTP appe (try 3) to /htdocs/.bzr/repository/upload/z1lkk39vtt2ohonyndb1.pack
13.339 Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 912, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 1112, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 690, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 705, in run
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 135, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 165, in _do_with_cleanups
    result = func(*args, **kwargs)
...