Merge lp:~lenharo-h/duplicity/duplicity into lp:duplicity/0.6

[Marcos Lenharo]

Hi,

I thought that we could also generate encrypted backups without revealing the user's key id.
So I created a new switch (--hidden-encrypt-key) that will act like --encrypt-key but will use gpg's --hidden-recipient switch instead of --recipient.

Please, let me know if you need more info.

Thanks,
Marcos

[edso (ed.so)]

hi,

1. there is a typo in the manpage change.
2. how does it work on restoring? wouldn't gpg have to ask for a keyid to use?

..ede

review: Needs Fixing

[Marcos Lenharo (lenharo-h)]

Hi,

the typo was fixed and already pushed.

Regarding restoring, if you only have hidden-recipients on you encrypted
file, gpg will try every secret key you have on your keyring and use the
right one if it finds it.
Otherwise it will say that didn't find a secret key. Like the following:

[tmp]$ gpg duplicity-full.20121121T001521Z.vol1.difftar.gpg
gpg: anonymous recipient; trying secret key 12345678 ...
gpg: anonymous recipient; trying secret key 9ABCDEF0...
*gpg: okay, we are the anonymous recipient.*
gpg: encrypted with RSA key, ID 00000000
...

or

[tmp]$ gpg duplicity-full.20121121T001521Z.vol1.difftar.gpg
gpg: anonymous recipient; trying secret key 12345678 ...
gpg: anonymous recipient; trying secret key 87654321 ...
gpg: encrypted with RSA key, ID 00000000
*gpg: decryption failed: No secret key*

Please let me know if you have questions on other scenarios.
Best Regards,

Marcos A. Lenharo

On 21-11-2012 08:11, edso wrote:
> Review: Needs Fixing
>
> hi,
>
> 1. there is a typo in the manpage change.
> 2. how does it work on restoring? wouldn't gpg have to ask for a keyid to use?
>
> ..ede

[edso (ed.so)]

On 21.11.2012 14:03, Marcos Aurelio Lenharo wrote:
> Hi,
>
> the typo was fixed and already pushed.
>
> Regarding restoring, if you only have hidden-recipients on you encrypted file, gpg will try every secret key you have on your keyring and use the right one if it finds it.

[snip]
>
>
> Please let me know if you have questions on other scenarios.
> Best Regards,
>

otherwise it looked good to me.

maybe it'd make sense to mention 'uses gpg --hidden-recipient' and 'try's every key' in the manpage as well.

..ede/duply.net

[Marcos Lenharo (lenharo-h)]

Hi,

It seems to be better now. Please, take a look on revision 900.

Best Regards,

Marcos A. Lenharo

On 21-11-2012 11:46, edso wrote:
> On 21.11.2012 14:03, Marcos Aurelio Lenharo wrote:
>> Hi,
>>
>> the typo was fixed and already pushed.
>>
>> Regarding restoring, if you only have hidden-recipients on you encrypted file, gpg will try every secret key you have on your keyring and use the right one if it finds it.
> [snip]
>>
>> Please let me know if you have questions on other scenarios.
>> Best Regards,
>>
> otherwise it looked good to me.
>
> maybe it'd make sense to mention 'uses gpg --hidden-recipient' and 'try's every key' in the manpage as well.
>
> ..ede/duply.net
>

[Kenneth Loafman (kenneth-loafman)]

Has anyone tried restore with the _ssh_pexpect backend?

[Marcos Lenharo (lenharo-h)]

No, but I can try it later.
Why?

Regards
Em 22/11/2012 10:52, "Kenneth Loafman" <email address hidden> escreveu:

> Has anyone tried restore with the _ssh_pexpect backend?
> --
> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
> You are the owner of lp:~lenharo-h/duplicity/duplicity.
>

[Kenneth Loafman (kenneth-loafman)]

It will need to be modded, probably, since the responses from gpg are
different.

On Thu, Nov 22, 2012 at 7:01 AM, Marcos Lenharo <email address hidden> wrote:

> No, but I can try it later.
> Why?
>
> Regards
> Em 22/11/2012 10:52, "Kenneth Loafman" <email address hidden> escreveu:
>
> > Has anyone tried restore with the _ssh_pexpect backend?
> > --
> > https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
> > You are the owner of lp:~lenharo-h/duplicity/duplicity.
> >
>
> --
> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
> You are subscribed to branch lp:duplicity.
>

[edso (ed.so)]

how does a specific backend directly deal with gpg? isn't that the job of gpg_interface?

..ede

On 22.11.2012 15:38, Kenneth Loafman wrote:
> It will need to be modded, probably, since the responses from gpg are
> different.
>
> On Thu, Nov 22, 2012 at 7:01 AM, Marcos Lenharo <email address hidden> wrote:
>
>> No, but I can try it later.
>> Why?
>>
>> Regards
>> Em 22/11/2012 10:52, "Kenneth Loafman" <email address hidden> escreveu:
>>
>>> Has anyone tried restore with the _ssh_pexpect backend?
>>> --
>>> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
>>> You are the owner of lp:~lenharo-h/duplicity/duplicity.
>>>
>>
>> --
>> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
>> You are subscribed to branch lp:duplicity.
>>
>

[Kenneth Loafman (kenneth-loafman)]

It uses pexpect to check the return strings. Sometimes that's from gpg.

...Ken

On Thu, Nov 22, 2012 at 8:46 AM, edso <email address hidden> wrote:

> how does a specific backend directly deal with gpg? isn't that the job of
> gpg_interface?
>
> ..ede
>
> On 22.11.2012 15:38, Kenneth Loafman wrote:
> > It will need to be modded, probably, since the responses from gpg are
> > different.
> >
> > On Thu, Nov 22, 2012 at 7:01 AM, Marcos Lenharo <email address hidden>
> wrote:
> >
> >> No, but I can try it later.
> >> Why?
> >>
> >> Regards
> >> Em 22/11/2012 10:52, "Kenneth Loafman" <email address hidden> escreveu:
> >>
> >>> Has anyone tried restore with the _ssh_pexpect backend?
> >>> --
> >>>
> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
> >>> You are the owner of lp:~lenharo-h/duplicity/duplicity.
> >>>
> >>
> >> --
> >> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
> >> You are subscribed to branch lp:duplicity.
> >>
> >
>
>
> --
> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
> You are subscribed to branch lp:duplicity.
>

[edso (ed.so)]

just scanned
http://bazaar.launchpad.net/~duplicity-team/duplicity/0.6-series/view/head:/duplicity/backends/_ssh_pexpect.py
and can only see sftp and scp being used.

what am i missing?

..ede

On 22.11.2012 15:55, Kenneth Loafman wrote:
> It uses pexpect to check the return strings. Sometimes that's from gpg.
>
> ...Ken
>
> On Thu, Nov 22, 2012 at 8:46 AM, edso <email address hidden> wrote:
>
>> how does a specific backend directly deal with gpg? isn't that the job of
>> gpg_interface?
>>
>> ..ede
>>
>> On 22.11.2012 15:38, Kenneth Loafman wrote:
>>> It will need to be modded, probably, since the responses from gpg are
>>> different.
>>>
>>> On Thu, Nov 22, 2012 at 7:01 AM, Marcos Lenharo <email address hidden>
>> wrote:
>>>
>>>> No, but I can try it later.
>>>> Why?
>>>>
>>>> Regards
>>>> Em 22/11/2012 10:52, "Kenneth Loafman" <email address hidden> escreveu:
>>>>
>>>>> Has anyone tried restore with the _ssh_pexpect backend?
>>>>> --
>>>>>
>> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
>>>>> You are the owner of lp:~lenharo-h/duplicity/duplicity.
>>>>>
>>>>
>>>> --
>>>> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
>>>> You are subscribed to branch lp:duplicity.
>>>>
>>>
>>
>>
>> --
>> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
>> You are subscribed to branch lp:duplicity.
>>
>

[Kenneth Loafman (kenneth-loafman)]

Look at the responses from sftp and scp. If the gpg response is expected,
check that the new option supplies that, otherwise we have to match on
additional strings.

...Ken

On Thu, Nov 22, 2012 at 9:04 AM, edso <email address hidden> wrote:

> just scanned
>
> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.6-series/view/head:/duplicity/backends/_ssh_pexpect.py
> and can only see sftp and scp being used.
>
> what am i missing?
>
> ..ede
>
> On 22.11.2012 15:55, Kenneth Loafman wrote:
> > It uses pexpect to check the return strings. Sometimes that's from gpg.
> >
> > ...Ken
> >
> > On Thu, Nov 22, 2012 at 8:46 AM, edso <email address hidden> wrote:
> >
> >> how does a specific backend directly deal with gpg? isn't that the job
> of
> >> gpg_interface?
> >>
> >> ..ede
> >>
> >> On 22.11.2012 15:38, Kenneth Loafman wrote:
> >>> It will need to be modded, probably, since the responses from gpg are
> >>> different.
> >>>
> >>> On Thu, Nov 22, 2012 at 7:01 AM, Marcos Lenharo <email address hidden>
> >> wrote:
> >>>
> >>>> No, but I can try it later.
> >>>> Why?
> >>>>
> >>>> Regards
> >>>> Em 22/11/2012 10:52, "Kenneth Loafman" <email address hidden>
> escreveu:
> >>>>
> >>>>> Has anyone tried restore with the _ssh_pexpect backend?
> >>>>> --
> >>>>>
> >> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
> >>>>> You are the owner of lp:~lenharo-h/duplicity/duplicity.
> >>>>>
> >>>>
> >>>> --
> >>>>
> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
> >>>> You are subscribed to branch lp:duplicity.
> >>>>
> >>>
> >>
> >>
> >> --
> >> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
> >> You are subscribed to branch lp:duplicity.
> >>
> >
>
> --
> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
> You are subscribed to branch lp:duplicity.
>

[edso (ed.so)]

you totally loose me here :( sorry...

the patch provided by Lenharo merely changes the way how gpg_interface.py encrypts the datastream locally. did you have a look at what is actually patched?
http://bazaar.launchpad.net/~lenharo-h/duplicity/duplicity/revision/898

it does in no way affect up/downloading in any backend as far as i can see. still you come across convincing, so i might miss something ;)

..ede

On 22.11.2012 17:31, Kenneth Loafman wrote:
> Look at the responses from sftp and scp. If the gpg response is expected,
> check that the new option supplies that, otherwise we have to match on
> additional strings.
>
> ...Ken
>
> On Thu, Nov 22, 2012 at 9:04 AM, edso <email address hidden> wrote:
>
>> just scanned
>>
>> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.6-series/view/head:/duplicity/backends/_ssh_pexpect.py
>> and can only see sftp and scp being used.
>>
>> what am i missing?
>>
>> ..ede
>>
>> On 22.11.2012 15:55, Kenneth Loafman wrote:
>>> It uses pexpect to check the return strings. Sometimes that's from gpg.
>>>
>>> ...Ken
>>>
>>> On Thu, Nov 22, 2012 at 8:46 AM, edso <email address hidden> wrote:
>>>
>>>> how does a specific backend directly deal with gpg? isn't that the job
>> of
>>>> gpg_interface?
>>>>
>>>> ..ede
>>>>
>>>> On 22.11.2012 15:38, Kenneth Loafman wrote:
>>>>> It will need to be modded, probably, since the responses from gpg are
>>>>> different.
>>>>>
>>>>> On Thu, Nov 22, 2012 at 7:01 AM, Marcos Lenharo <email address hidden>
>>>> wrote:
>>>>>
>>>>>> No, but I can try it later.
>>>>>> Why?
>>>>>>
>>>>>> Regards
>>>>>> Em 22/11/2012 10:52, "Kenneth Loafman" <email address hidden>
>> escreveu:
>>>>>>
>>>>>>> Has anyone tried restore with the _ssh_pexpect backend?
>>>>>>> --
>>>>>>>
>>>> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
>>>>>>> You are the owner of lp:~lenharo-h/duplicity/duplicity.
>>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
>>>>>> You are subscribed to branch lp:duplicity.
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
>>>> You are subscribed to branch lp:duplicity.
>>>>
>>>
>>
>> --
>> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
>> You are subscribed to branch lp:duplicity.
>>
>

[Marcos Lenharo (lenharo-h)]

Hi,

without looking into the code ede seems right.

Anyway, I made the following test:

duplicity --hidden-encrypt-key=12345678 --use-scp --ssh-backend pexpect
source scp://target//tmp/test
duplicity --use-scp --ssh-backend pexpect scp://target//tmp/test /tmp/scr

And it behaved like expected: asked for GPG passphrase and restored the
backup.
Did I do it right?

Best Regards,

Marcos A. Lenharo

On 22-11-2012 14:40, edso wrote:
> you totally loose me here :( sorry...
>
> the patch provided by Lenharo merely changes the way how gpg_interface.py encrypts the datastream locally. did you have a look at what is actually patched?
> http://bazaar.launchpad.net/~lenharo-h/duplicity/duplicity/revision/898
>
> it does in no way affect up/downloading in any backend as far as i can see. still you come across convincing, so i might miss something ;)
>
> ..ede
>
>
> On 22.11.2012 17:31, Kenneth Loafman wrote:
>> Look at the responses from sftp and scp. If the gpg response is expected,
>> check that the new option supplies that, otherwise we have to match on
>> additional strings.
>>
>> ...Ken
>>
>> On Thu, Nov 22, 2012 at 9:04 AM, edso <email address hidden> wrote:
>>
>>> just scanned
>>>
>>> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.6-series/view/head:/duplicity/backends/_ssh_pexpect.py
>>> and can only see sftp and scp being used.
>>>
>>> what am i missing?
>>>
>>> ..ede
>>>
>>> On 22.11.2012 15:55, Kenneth Loafman wrote:
>>>> It uses pexpect to check the return strings. Sometimes that's from gpg.
>>>>
>>>> ...Ken
>>>>
>>>> On Thu, Nov 22, 2012 at 8:46 AM, edso <email address hidden> wrote:
>>>>
>>>>> how does a specific backend directly deal with gpg? isn't that the job
>>> of
>>>>> gpg_interface?
>>>>>
>>>>> ..ede
>>>>>
>>>>> On 22.11.2012 15:38, Kenneth Loafman wrote:
>>>>>> It will need to be modded, probably, since the responses from gpg are
>>>>>> different.
>>>>>>
>>>>>> On Thu, Nov 22, 2012 at 7:01 AM, Marcos Lenharo <email address hidden>
>>>>> wrote:
>>>>>>> No, but I can try it later.
>>>>>>> Why?
>>>>>>>
>>>>>>> Regards
>>>>>>> Em 22/11/2012 10:52, "Kenneth Loafman" <email address hidden>
>>> escreveu:
>>>>>>>> Has anyone tried restore with the _ssh_pexpect backend?
>>>>>>>> --
>>>>>>>>
>>>>> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
>>>>>>>> You are the owner of lp:~lenharo-h/duplicity/duplicity.
>>>>>>>>
>>>>>>> --
>>>>>>>
>>> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
>>>>>>> You are subscribed to branch lp:duplicity.
>>>>>>>
>>>>>
>>>>> --
>>>>> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
>>>>> You are subscribed to branch lp:duplicity.
>>>>>
>>> --
>>> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
>>> You are subscribed to branch lp:duplicity.
>>>

[Kenneth Loafman (kenneth-loafman)]

You did it right and it worked, so that's all I was asking.

...Ken

On Thu, Nov 22, 2012 at 5:55 PM, Marcos Aurelio Lenharo
<email address hidden>wrote:

> Hi,
>
> without looking into the code ede seems right.
>
> Anyway, I made the following test:
>
> duplicity --hidden-encrypt-key=12345678 --use-scp --ssh-backend pexpect
> source scp://target//tmp/test
> duplicity --use-scp --ssh-backend pexpect scp://target//tmp/test /tmp/scr
>
> And it behaved like expected: asked for GPG passphrase and restored the
> backup.
> Did I do it right?
>
> Best Regards,
>
> Marcos A. Lenharo
>
> On 22-11-2012 14:40, edso wrote:
> > you totally loose me here :( sorry...
> >
> > the patch provided by Lenharo merely changes the way how
> gpg_interface.py encrypts the datastream locally. did you have a look at
> what is actually patched?
> > http://bazaar.launchpad.net/~lenharo-h/duplicity/duplicity/revision/898
> >
> > it does in no way affect up/downloading in any backend as far as i can
> see. still you come across convincing, so i might miss something ;)
> >
> > ..ede
> >
> >
> > On 22.11.2012 17:31, Kenneth Loafman wrote:
> >> Look at the responses from sftp and scp. If the gpg response is
> expected,
> >> check that the new option supplies that, otherwise we have to match on
> >> additional strings.
> >>
> >> ...Ken
> >>
> >> On Thu, Nov 22, 2012 at 9:04 AM, edso <email address hidden> wrote:
> >>
> >>> just scanned
> >>>
> >>>
> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.6-series/view/head:/duplicity/backends/_ssh_pexpect.py
> >>> and can only see sftp and scp being used.
> >>>
> >>> what am i missing?
> >>>
> >>> ..ede
> >>>
> >>> On 22.11.2012 15:55, Kenneth Loafman wrote:
> >>>> It uses pexpect to check the return strings. Sometimes that's from
> gpg.
> >>>>
> >>>> ...Ken
> >>>>
> >>>> On Thu, Nov 22, 2012 at 8:46 AM, edso <email address hidden> wrote:
> >>>>
> >>>>> how does a specific backend directly deal with gpg? isn't that the
> job
> >>> of
> >>>>> gpg_interface?
> >>>>>
> >>>>> ..ede
> >>>>>
> >>>>> On 22.11.2012 15:38, Kenneth Loafman wrote:
> >>>>>> It will need to be modded, probably, since the responses from gpg
> are
> >>>>>> different.
> >>>>>>
> >>>>>> On Thu, Nov 22, 2012 at 7:01 AM, Marcos Lenharo <email address hidden>
> >>>>> wrote:
> >>>>>>> No, but I can try it later.
> >>>>>>> Why?
> >>>>>>>
> >>>>>>> Regards
> >>>>>>> Em 22/11/2012 10:52, "Kenneth Loafman" <email address hidden>
> >>> escreveu:
> >>>>>>>> Has anyone tried restore with the _ssh_pexpect backend?
> >>>>>>>> --
> >>>>>>>>
> >>>>>
> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
> >>>>>>>> You are the owner of lp:~lenharo-h/duplicity/duplicity.
> >>>>>>>>
> >>>>>>> --
> >>>>>>>
> >>>
> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
> >>>>>>> You are subscribed to branch lp:duplicity.
> >>>>>>>
> >>>>>
> >>>>> --
> >>>>>
> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
> >>>>> You are subscribed to branch lp:duplicity.
> >>>>>
> >>> --
> >>>
> https://code.launchpad.net/~lenharo-h/duplicity/duplicity/+merge/135300
> >>> You are subscribed to branch lp:duplicity.
> >>>
>
>

[edso (ed.so)]

On 23.11.2012 16:46, Kenneth Loafman wrote:
> We should check against the other backends,

that does not hurt. but why is it necessary? gpg encryption is in no way connected to our backends. volumes are built. when finished uploaded. the same in reverse for restoring. a two step process, which is totally separated. gpg is already done when volumes are loaded up or the other way around.

>plus check what happens if this option is used on an older gpg.

just checked the gpg-1.4 changelog (attached). the options seems to around at least since 2002, so it is actually no new option.

also gpg errors are shown to the user, so if gpg would not support the option it'll tell the user so.

> Do we catch the errors properly or simply ignore them? We probably need to add some test cases for this one as well.

that's analogue to our current scenario if the private key on restore is missing or the entered passphrase is plain wrong. gpg fails and a stack is shown.

while i really don't insist on this functionality (don't need it), i don't see why you treat as a major change, as if it had the possibility to break the application completely.

on the other hand, it's only fair to have contributors with new functionality add a test case.
@Lenharo: would you do so?

..ede

> ...Ken
>
> On Fri, Nov 23, 2012 at 9:24 AM, <<email address hidden> <mailto:<email address hidden>>> wrote:
>
> still i wonder why it was necessary, as opposed testing it against any other backend we got.
>
> ..ede
>
> On 23.11.2012 13:42, Kenneth Loafman wrote:
> > You did it right and it worked, so that's all I was asking.
> >
> > ...Ken
> >
> > On Thu, Nov 22, 2012 at 5:55 PM, Marcos Aurelio Lenharo <<email address hidden> <mailto:<email address hidden>> <mailto:<email address hidden> <mailto:<email address hidden>>>> wrote:
> >
> > Hi,
> >
> > without looking into the code ede seems right.
> >
> > Anyway, I made the following test:
> >
> > duplicity --hidden-encrypt-key=12345678 --use-scp --ssh-backend pexpect
> > source scp://target//tmp/test
> > duplicity --use-scp --ssh-backend pexpect scp://target//tmp/test /tmp/scr
> >
> > And it behaved like expected: asked for GPG passphrase and restored the
> > backup.
> > Did I do it right?
> >
> > Best Regards,
> >
> > Marcos A. Lenharo
> >
> > On 22-11-2012 14:40, edso wrote:
> > > you totally loose me here :( sorry...
> > >
> > > the patch provided by Lenharo merely changes the way how gpg_interface.py encrypts the datastream locally. did you have a look at what is actually patched?
> > > http://bazaar.launchpad.net/~lenharo-h/duplicity/duplicity/revision/898 <http://bazaar.launchpad.net/%7Elenharo-h/duplicity/duplicity/revision/898> <http://bazaar.launchpad.net/%7Elenharo-h/duplicity/duplicity/revision/898>
> > >
> > > it does in no way affect up/downloading in any backend as far as i can see. still you come across convincing, so i might miss something ;)
> > >
> > > ..ede
> > >
> > >
> > > On 22.11.2012 17:31, K...

[Marcos Lenharo (lenharo-h)]

Hi guys,

I updated test cases to also test the new switch. They are under:
http://bazaar.launchpad.net/~lenharo-h/duplicity/duplicity/revision/901

By the way, I am using Arch Linux, which has python 2.7 and python 3
installed and the line #!/usr/bin/env *python* inside duplicity and
rdiffdir is causing python 3 to be used and the tests are failing
(mainly due to new print function).
In my repository I changed them to #!/usr/bin/env *python2* and all test
passed.

I am not a python developer so I didn't get why duplicity is working at
command line but not when called by the tests.

Best Regards,

Marcos A. Lenharo

On 23-11-2012 14:17, edso wrote:
> On 23.11.2012 16:46, Kenneth Loafman wrote:
>> We should check against the other backends,
> that does not hurt. but why is it necessary? gpg encryption is in no way connected to our backends. volumes are built. when finished uploaded. the same in reverse for restoring. a two step process, which is totally separated. gpg is already done when volumes are loaded up or the other way around.
>
>> plus check what happens if this option is used on an older gpg.
> just checked the gpg-1.4 changelog (attached). the options seems to around at least since 2002, so it is actually no new option.
>
> also gpg errors are shown to the user, so if gpg would not support the option it'll tell the user so.
>
>> Do we catch the errors properly or simply ignore them? We probably need to add some test cases for this one as well.
> that's analogue to our current scenario if the private key on restore is missing or the entered passphrase is plain wrong. gpg fails and a stack is shown.
>
> while i really don't insist on this functionality (don't need it), i don't see why you treat as a major change, as if it had the possibility to break the application completely.
>
> on the other hand, it's only fair to have contributors with new functionality add a test case.
> @Lenharo: would you do so?
>
> ..ede
>
>
>> ...Ken
>>
>> On Fri, Nov 23, 2012 at 9:24 AM, <<email address hidden> <mailto:<email address hidden>>> wrote:
>>
>> still i wonder why it was necessary, as opposed testing it against any other backend we got.
>>
>> ..ede
>>
>> On 23.11.2012 13:42, Kenneth Loafman wrote:
>> > You did it right and it worked, so that's all I was asking.
>> >
>> > ...Ken
>> >
>> > On Thu, Nov 22, 2012 at 5:55 PM, Marcos Aurelio Lenharo <<email address hidden> <mailto:<email address hidden>> <mailto:<email address hidden> <mailto:<email address hidden>>>> wrote:
>> >
>> > Hi,
>> >
>> > without looking into the code ede seems right.
>> >
>> > Anyway, I made the following test:
>> >
>> > duplicity --hidden-encrypt-key=12345678 --use-scp --ssh-backend pexpect
>> > source scp://target//tmp/test
>> > duplicity --use-scp --ssh-backend pexpect scp://target//tmp/test /tmp/scr
>> >
>> > And it behaved like expected: asked for GPG passphrase and restored the
>> > backup.
>> > Did I do it right?
>> >
>> > Best Regards,
>> >
>> > Marcos A. Lenharo
>> >
>> > On 22-11-2012 14:40, edso wrote:
>> > > you totall...

[edso (ed.so)]

fine with me

review: Approve

=== modified file 'bin/duplicity'
--- bin/duplicity	2012-11-13 21:53:24 +0000
+++ bin/duplicity	2012-12-02 16:39:20 +0000
@@ -93,13 +93,15 @@
     # check if we can reuse an already set (signing_)passphrase
     ## if signing key is also an encryption key assume that the passphrase is identical
     if ( for_signing
-         and globals.gpg_profile.sign_key in globals.gpg_profile.recipients
+         and (globals.gpg_profile.sign_key in globals.gpg_profile.recipients
+         or globals.gpg_profile.sign_key in globals.gpg_profile.hidden_recipients)
          and 'PASSPHRASE' in os.environ ):
         log.Notice(_("Reuse configured PASSPHRASE as SIGN_PASSPHRASE"))
         return os.environ['PASSPHRASE']
     ## if one encryption key is also the signing key assume that the passphrase is identical
     if ( not for_signing
-         and globals.gpg_profile.sign_key in globals.gpg_profile.recipients
+         and (globals.gpg_profile.sign_key in globals.gpg_profile.recipients
+         or globals.gpg_profile.sign_key in globals.gpg_profile.hidden_recipients)
          and 'SIGN_PASSPHRASE' in os.environ ):
         log.Notice(_("Reuse configured SIGN_PASSPHRASE as PASSPHRASE"))
         return os.environ['SIGN_PASSPHRASE']
@@ -127,14 +129,14 @@
     # for a full backup, we don't need a password if
     # there is no sign_key and there are recipients
     elif (action == "full"
-          and globals.gpg_profile.recipients
+          and (globals.gpg_profile.recipients or globals.gpg_profile.hidden_recipients)
           and not globals.gpg_profile.sign_key):
         return ""
 
     # for an inc backup, we don't need a password if
     # there is no sign_key and there are recipients
     elif (action == "inc"
-          and globals.gpg_profile.recipients
+          and (globals.gpg_profile.recipients or globals.gpg_profile.hidden_recipients)
           and not globals.gpg_profile.sign_key):
         return ""
 
@@ -174,7 +176,7 @@
                 use_cache = False
                 continue
 
-            if not pass1 and not globals.gpg_profile.recipients and not for_signing:
+            if not pass1 and not (globals.gpg_profile.recipients or globals.gpg_profile.hidden_recipients) and not for_signing:
                 print _("Cannot use empty passphrase with symmetric encryption!  Please try again.")
                 use_cache = False
                 continue
@@ -1350,7 +1352,7 @@
 
         # if there are no recipients (no --encrypt-key), it must be a
         # symmetric key. Therefore, confirm the passphrase
-        if not globals.gpg_profile.recipients:
+        if not (globals.gpg_profile.recipients or globals.gpg_profile.hidden_recipients):
             globals.gpg_profile.passphrase = get_passphrase(2, action)
             # a limitation in the GPG implementation does not allow for
             # inputting different passphrases, this affects symmetric+sign.

=== modified file 'bin/duplicity.1'
--- bin/duplicity.1	2012-11-03 14:10:38 +0000
+++ bin/duplicity.1	2012-12-02 16:39:20 +0000
@@ -478,6 +478,17 @@
 Use the GIO backend and interpret any URLs as GIO would.
 
 .TP
+.BI "--hidden-encrypt-key " key-id
+Same as
+.BR --encrypt-key ,
+but it hides user's key id from encrypted file. It uses the gpg's
+.B --hidden-recipient
+command to obfuscate the owner of the backup. On restore, gpg will 
+automatically try all available secret keys in order to decrypt the 
+backup. See gpg(1) for more details.
+
+
+.TP
 .B --ignore-errors
 Try to ignore certain errors if they happen. This option is only
 intended to allow the restoration of a backup in the face of certain

=== modified file 'duplicity/commandline.py'
--- duplicity/commandline.py	2012-10-29 10:01:25 +0000
+++ duplicity/commandline.py	2012-12-02 16:39:20 +0000
@@ -350,6 +350,12 @@
 
     parser.add_option("--gpg-options", action="extend", metavar=_("options"))
 
+    # TRANSL: Used in usage help to represent an ID for a hidden GnuPG key. Example:
+    # --hidden-encrypt-key <gpg_key_id>
+    parser.add_option("--hidden-encrypt-key", type="string", metavar=_("gpg-key-id"),
+                      dest="", action="callback",
+                      callback=lambda o, s, v, p: globals.gpg_profile.hidden_recipients.append(v)) #@UndefinedVariable
+
     # ignore (some) errors during operations; supposed to make it more
     # likely that you are able to restore data under problematic
     # circumstances. the default should absolutely always be False unless

=== modified file 'duplicity/gpg.py'
--- duplicity/gpg.py	2012-10-15 17:13:08 +0000
+++ duplicity/gpg.py	2012-12-02 16:39:20 +0000
@@ -54,7 +54,7 @@
     Just hold some GPG settings, avoid passing tons of arguments
     """
     def __init__(self, passphrase = None, sign_key = None,
-                 recipients = None):
+                 recipients = None, hidden_recipients = None):
         """
         Set all data with initializer
 
@@ -76,6 +76,12 @@
         else:
             self.recipients = []
 
+        if hidden_recipients is not None:
+            assert type(hidden_recipients) is types.ListType # must be list, not tuple
+            self.hidden_recipients = hidden_recipients
+        else:
+            self.hidden_recipients = []
+
 
 class GPGFile:
     """
@@ -131,7 +137,10 @@
             if profile.recipients:
                 gnupg.options.recipients = profile.recipients
                 cmdlist.append('--encrypt')
-            else:
+            if profile.hidden_recipients:
+                gnupg.options.hidden_recipients = profile.hidden_recipients
+                cmdlist.append('--encrypt')
+            if not (profile.recipients or profile.hidden_recipients):
                 cmdlist.append('--symmetric')
                 # use integrity protection
                 gnupg.options.extra_args.append('--force-mdc')
@@ -149,7 +158,7 @@
                 p1.handles['passphrase'].close()
             self.gpg_input = p1.handles['stdin']
         else:
-            if profile.recipients and profile.encrypt_secring:
+            if (profile.recipients or profile.hidden_recipients) and profile.encrypt_secring:
                 cmdlist.append('--secret-keyring')
                 cmdlist.append(profile.encrypt_secring)
             self.status_fp = tempfile.TemporaryFile( dir=tempdir.default().dir() )

=== modified file 'duplicity/gpginterface.py'
--- duplicity/gpginterface.py	2012-10-15 17:13:08 +0000
+++ duplicity/gpginterface.py	2012-12-02 16:39:20 +0000
@@ -570,6 +570,7 @@
         # lists
         self.encrypt_to = []
         self.recipients = []
+        self.hidden_recipients = []
 
         # miscellaneous arguments
         self.extra_args = []
@@ -602,6 +603,7 @@
         if self.openpgp: args.append( '--openpgp' )
 
         for r in self.recipients: args.extend( [ '--recipient',  r ] )
+        for r in self.hidden_recipients: args.extend( [ '--hidden-recipient',  r ] )
         for r in self.encrypt_to: args.extend( [ '--encrypt-to', r ] )
 
         return args

=== modified file 'testing/tests/finaltest.py'
--- testing/tests/finaltest.py	2011-11-07 15:08:24 +0000
+++ testing/tests/finaltest.py	2012-12-02 16:39:20 +0000
@@ -168,6 +168,16 @@
         self.test_basic_cycle(backup_options = backup_options,
                               restore_options = restore_options)
 
+    def test_asym_with_hidden_recipient_cycle(self):
+        """Like test_basic_cycle but use asymmetric encryption (hidding key id) and signing"""
+        backup_options = ["--hidden-encrypt-key " + helper.encrypt_key1,
+                          "--sign-key " + helper.sign_key]
+        restore_options = ["--hidden-encrypt-key " + helper.encrypt_key1,
+                           "--sign-key " + helper.sign_key]
+        helper.set_environ("SIGN_PASSPHRASE", helper.sign_passphrase)
+        self.test_basic_cycle(backup_options = backup_options,
+                              restore_options = restore_options)
+
     def test_single_regfile(self):
         """Test backing and restoring up a single regular file"""
         self.runtest(["testfiles/various_file_types/regular_file"])

=== modified file 'testing/tests/gpgtest.py'
--- testing/tests/gpgtest.py	2011-11-04 12:48:04 +0000
+++ testing/tests/gpgtest.py	2012-12-02 16:39:20 +0000
@@ -87,6 +87,17 @@
                                   recipients = [helper.encrypt_key1])
         self.gpg_cycle("aoeu" * 10000, profile2)
 
+    def test_gpg_hidden_asym(self):
+        """Test GPG asymmetric encryption with hidden key id"""
+        profile = gpg.GPGProfile(passphrase = helper.sign_passphrase,
+                                 hidden_recipients = [helper.encrypt_key1,
+                                               helper.encrypt_key2])
+        self.gpg_cycle("aoensutha aonetuh saoe", profile)
+
+        profile2 = gpg.GPGProfile(passphrase = helper.sign_passphrase,
+                                  hidden_recipients = [helper.encrypt_key1])
+        self.gpg_cycle("aoeu" * 10000, profile2)
+
     def test_gpg_signing(self):
         """Test to make sure GPG reports the proper signature key"""
         self.deltmp()
@@ -107,6 +118,26 @@
         sig = decrypted_file.get_signature()
         assert sig == helper.sign_key, sig
 
+    def test_gpg_signing_and_hidden_encryption(self):
+        """Test to make sure GPG reports the proper signature key even with hidden encryption key id"""
+        self.deltmp()
+        plaintext = "hello" * 50000
+
+        signing_profile = gpg.GPGProfile(passphrase = helper.sign_passphrase,
+                                         sign_key = helper.sign_key,
+                                         hidden_recipients = [helper.encrypt_key1])
+
+        epath = path.Path("testfiles/output/encrypted_file")
+        encrypted_signed_file = gpg.GPGFile(1, epath, signing_profile)
+        encrypted_signed_file.write(plaintext)
+        encrypted_signed_file.close()
+
+        decrypted_file = gpg.GPGFile(0, epath, signing_profile)
+        assert decrypted_file.read() == plaintext
+        decrypted_file.close()
+        sig = decrypted_file.get_signature()
+        assert sig == helper.sign_key, sig
+
     def test_GPGWriteFile(self):
         """Test GPGWriteFile"""
         self.deltmp()

=== modified file 'testing/tests/restarttest.py'
--- testing/tests/restarttest.py	2012-11-19 14:13:59 +0000
+++ testing/tests/restarttest.py	2012-12-02 16:39:20 +0000
@@ -253,6 +253,23 @@
         self.backup("full", "testfiles/largefiles", options = enc_opts)
         self.verify("testfiles/largefiles")
 
+    def test_restart_sign_and_hidden_encrypt(self):
+        """
+        Test restarting a backup using same key for sign and encrypt (hidden key id)
+        https://bugs.launchpad.net/duplicity/+bug/946988
+        """
+        self.make_largefiles()
+        enc_opts = ["--sign-key " + helper.sign_key, "--hidden-encrypt-key " + helper.sign_key]
+        # Force a failure partway through
+        try:
+            self.backup("full", "testfiles/largefiles", options = ["--vols 1", "--fail 2"] + enc_opts)
+            self.fail()
+        except CmdError, e:
+            self.assertEqual(30, e.exit_status)
+        # Now finish that backup
+        self.backup("full", "testfiles/largefiles", options = enc_opts)
+        self.verify("testfiles/largefiles")
+
     def test_last_file_missing_in_middle(self):
         """
         Test restart when the last file being backed up is missing on restart.