Strip more sensitive data from the files gathered by Apport

Hi Severin,

Are you working on fixing this 'bug'? Otherwise I'd like to take this one on me. If you want me to. ;-)

Hi Marcel,

what a coincidence: Yesterday I thought for a moment that I could start making use of the "Offer mentorship" feature Launchpad provides. :-)

This bug seems pretty well-suited for that. It doesn't require you to dive into the internals of LottaNZB, because you basically only need to edit the desktop/apport/lottanzb.py file. The problem right now is that I couldn't really let other people work on the 0.6 branch as many fundamental parts of the application are being changed heavily at the moment. And as the GUI hasn't been migrated to the new SABnzbd backend yet, it's not even possible to give things a try in an easy way. I will still require a few weeks until I can offer mentorship for 0.6 tasks with a good conscience.

Regarding this bug: I guess fixing this bug won't be possible without getting used to regular expressions (the Python 're' module). Document your code. Having a look at PEP 8 (http://www.python.org/dev/peps/pep-0008/) isn't a bad idea either (I tend to use more newlines than they suggest ;-)). There's a typo in line 72 ("passoword"). "import getpass; getpass.getuser()" returns the name of the current user.

The fix will eventually be merged into the 0.6 branch, but it will require some new SABnzbd-specific code.

Happy hacking!

Working on this bug is a little different, because you would need to generate and install faulty Ubuntu package (with a 'raise EnvironmentError' somewhere) each time you make a change so that you can trigger Apport.

It might be easier to install the normal LottaNZB deb package (without any changes), temporarily rename the local, lottanzb.py file in desktop/apport to something else like hook.py, change into that directory and open a Python shell (python). You could then do something like:

from hook import add_info
report = {}
add_info(report)
open("out", "w").write(report["lottanzb.conf"])

You can inspect the "out" file after that.

Cool! :-)

So, what does this mentoring offer from you mean to me (or any other contributer)? Should I look at it like as a mentor at schools?

Probably yeah, more like a guide to our application. ;-)

On Wed, Sep 9, 2009 at 12:49, Marcel de Vries<email address hidden> wrote:
> Cool! :-)
>
> So, what does this mentoring offer from you mean to me (or any other
> contributer)? Should I look at it like as a mentor at schools?
>
> --
> Strip more sensitive data from the files gathered by Apport
> https://bugs.launchpad.net/bugs/425630
> You received this bug notification because you are subscribed to
> LottaNZB.
>
> Status in LottaNZB • Automated Usenet Client: Triaged
>
> Bug description:
> May be we should think about replacing usernames and machine identifiers by %APPORTREPLACE%. Also, the logic that anonymizes the defineServer line doesn't work properly and removes too much information.
>

Refer to https://help.launchpad.net/Teams/Mentoring for a verbose explanation.

This should be changed right? If not, well you know. ;-)

Question:

What does " % key" in ""%s = .+" % key"? Can't seem to figure it out. Couldn't find it in the python documentation.

http://www.python.org/doc/2.5.2/lib/typesseq-strings.html

The %s is a placeholder for the % key variable. So, at the place of
the %s, the variable key is put in. At least, that's what I hope it
does ;-)

On Wed, Sep 9, 2009 at 13:56, Marcel de Vries<email address hidden> wrote:
> Question:
>
> What does " % key" in ""%s = .+" % key"? Can't seem to figure it out.
> Couldn't find it in the python documentation.
>
> --
> Strip more sensitive data from the files gathered by Apport
> https://bugs.launchpad.net/bugs/425630
> You received this bug notification because you are subscribed to
> LottaNZB.
>
> Status in LottaNZB • Automated Usenet Client: In Progress
>
> Bug description:
> May be we should think about replacing usernames and machine identifiers by %APPORTREPLACE%. Also, the logic that anonymizes the defineServer line doesn't work properly and removes too much information.
>

D'oh! I can imagine you were laughing like hell at your desk! I was thinking waaaayyyy too difficult. Thank you anyway!

Heh, no, I was hoping it didn't have anything to do with RE. ;-)

On Wed, Sep 9, 2009 at 14:20, Marcel de Vries<email address hidden> wrote:
> D'oh! I can imagine you were laughing like hell at your desk! I was
> thinking waaaayyyy too difficult. Thank you anyway!
>
> --
> Strip more sensitive data from the files gathered by Apport
> https://bugs.launchpad.net/bugs/425630
> You received this bug notification because you are subscribed to
> LottaNZB.
>
> Status in LottaNZB • Automated Usenet Client: In Progress
>
> Bug description:
> May be we should think about replacing usernames and machine identifiers by %APPORTREPLACE%. Also, the logic that anonymizes the defineServer line doesn't work properly and removes too much information.
>

I see that the %s's might indeed be a little confusing in some lines of desktop/apport/lottanzb.py. In fact, it might even be a good idea to change

"%s = %s" % (key, repl)

to

key + " = " + repl

as it seems to look cleaner. Feel free to change whatever you think helps improve readability of the code.

How would you like me to publish my fix, as a patch or branch?

A branch would be fine. :-) I look forward to it.

Pushing right now. Why is it so slow, I have a 500KB/s upload but it's uploading at about 40-80 KB/s.

I'm not fully done with it though, but I think that everything that needs censoring is censored now. Just need to clean up and maybe comment where needed (?). Perhaps even improve.

Do you have an idea when you want to release 0.5.1? It should be done by then, if you want to include this in 0.5.1 of course.

I'm not sure why uploading the branch is so slow. It's probably a good idea to always use the latest version of Bazaar. You can use the Bazaar PPA for this: https://launchpad.net/~bzr/+archive/ppa Additionally, you might try to use stacked branches, which should significantly decrease the amount of data that needs to be transfered. "bzr branch" and "bzr push" both have a "--stacked" argument.

Depending on how many bugs will be reported during the next days, LottaNZB 0.5.1 could be released in a week or so. It's great to see that two translations were completed since the release of LottaNZB 0.5. LottaNZB 0.5.1 will definitely include both fixes you created.

Your branch seems to be ready to merge without any further changes! When I initially wrote the Apport hook, I didn't know anymore that the non-greedy '.+?' existed, which led to this bug.

You're now a member of the LottaNZB Development Team, which means that you're allowed branch the main branches lp:lottanzb/0.5 and lp:lottanzb and directly push changes to them. Especially for bigger changes (e. g. > 100 changed lines), I still recommend you to create a separate branch first and propose a merge once you think it's ready. This is exactly how we do it.

Congratulations on your second bug fix!

re.sub(luser, repl, config)

could be replaced by

config.replace(luser, repl)

as luser is not a regular expression, but both versions will do the job just fine.

You can merge the branch on your own if you want to:

bzr branch lp:lottanzb/0.5 0.5 --stacked (creates a directory 0.5)
cd 0.5
bzr merge ../the_branch_containing_your_bug_fix
bzr commit -m "A nice message."
bzr push

To begin with, thank you for having me on the team! I hope I will prove myself helpful for this wonderful application! Yey!
---

I have merge my branch with 0.5 and committed. But I couldn't use the "--stacked" option as you mentioned. Bzr is giving errors/warnings like these;
--- [ bzr branch lp:lottanzb/0.5 --stacked ] ---
Format <RepositoryFormatKnit1> for lp-45207760:///~lottanzb/lottanzb/0.5/.bzr is deprecated - please use 'bzr upgrade' to get better performance
Source repository format does not support stacking, using format:
  Packs 5 (adds stacking support, requires bzr 1.6)
Source branch format does not support stacking, using format:
  Branch format 7
Format <RepositoryFormatKnit1> for lp-45207760:///~lottanzb/lottanzb/0.5/.bzr is deprecated - please use 'bzr upgrade' to get better performance
Format <RepositoryFormatKnit1> for bzr+ssh://bazaar.launchpad.net/~lottanzb/lottanzb/0.5/.bzr/ is deprecated - please use 'bzr upgrade' to get better performance
Format <RepositoryFormatKnit1> for lp-45207760:///~lottanzb/lottanzb/0.5/.bzr is deprecated - please use 'bzr upgrade' to get better performance
Format <RepositoryFormatKnit1> for lp-45207760:///~lottanzb/lottanzb/0.5/.bzr is deprecated - please use 'bzr upgrade' to get better performance
Format <RepositoryFormatKnit1> for lp-45207760:///~lottanzb/lottanzb/0.5/.bzr is deprecated - please use 'bzr upgrade' to get better performance
Created new stacked branch referring to bzr+ssh://bazaar.launchpad.net/~lottanzb/lottanzb/0.5/.
--- [ end ] ---

--- [ bzr upgrade ] ---
Format <RepositoryFormatKnit1> for lp-45207760:///~lottanzb/lottanzb/0.5/.bzr is deprecated - please use 'bzr upgrade' to get better performance
Format <RepositoryFormatKnit1> for bzr+ssh://bazaar.launchpad.net/~lottanzb/lottanzb/0.5/.bzr/ is deprecated - please use 'bzr upgrade' to get better performance
starting upgrade of file:///home/carresmd/Projects/lottanzb/test/
making backup of file:///home/carresmd/Projects/lottanzb/test/.bzr
  to file:///home/carresmd/Projects/lottanzb/test/backup.bzr
starting repository conversion
repository converted
Format <RepositoryFormatKnit1> for lp-45207760:///~lottanzb/lottanzb/0.5/.bzr is deprecated - please use 'bzr upgrade' to get better performance
bzr: ERROR: The repository 'file:///home/carresmd/Projects/lottanzb/test/.bzr/repository/'(<RepositoryFormatKnitPack1>) is not a stackable format. You will need to upgrade the repository to permit branch stacking.
--- [ end ] ---

Any way, thank you!

Oh yes, running the latest stable bzr release from their ppa; "Bazaar (bzr) 1.18"

To be honest, I've never used the --stacked argument by myself, but only read about it. And I also get this "bzr upgrade" warning message on some operations. Since "bzr upgrade" doesn't work, telling me that everything is up-to-date, there's probably something wrong. But since it didn't prevent us from developing and releasing LottaNZB, it's probably not that severe. One could get in touch with some Launchpad devs though.

Welcome to the team, Marcel! A nice side effect of it is that it puts some pressure on me in terms of bringing the 0.6 branch to a state where contributions by you and Sander are actually possible and desirable. Right now, I'm still fiddling about some basic infrastructure that holds the application components together and the GUI code. After that, there will be quite some UI work to be done to cover all cool features SABnzbd provides! Fortunately, the backend is mostly ready.

This bug report is probably not meant to be used as a discussion channel. If feel like it, you could provide me with an IM address (no matter what protocol) by email. There also exists an IRC channel #lottanzb at freenode.net, but most of the time, it's deserted.