Ubuntu
firefox package

default search engines are removed and readded (keywords wiped) with upgrade

Bug #428306 reported by Hew
192
This bug affects 35 people
Affects Status Importance Assigned to Milestone
Mozilla Firefox
Invalid
Low
firefox (Ubuntu)
Fix Released
High
Alexander Sack

Bug Description

Binary package hint: firefox-3.5

I updated Karmic, which included upgrading Firefox from 3.5.2 to 3.5.3. I previously had the keyword 'w' set for the Wikipedia search. It appears that all the default search engines have been removed and added again to the bottom of the list of search engines, without keywords. This means I've had to add back keywords for default search engines I use, and remove all the default search engines I had previously removed (eg. amazon, ebay, etc).

The same situation occurs when upgrading from firefox-3.0 to firefox-3.5.

Steps to reproduce:
0) Install both firefox-3.0 and firefox-3.5. Move away ~/.mozilla so that you start with a fresh profile.
1) Run firefox-3.0. In 'Manage Search Engines', add keywords to some, and remove others. Close firefox-3.0.
2) Run firefox-3.5. Check 'Manage Search Engines'.

What should happen:
Search engines are the same as they were with firefox-3.0

What actually happens:
Default search engines have keywords wiped, and previously removed search engines have been added again.

See original description
Revision history for this message
Rick Spencer (rick-rickspencer3) wrote :

Please comment as to the possibility of handling this situation. Will it occur on a Stable Release to Stable Release upgrade? Please un-assign and set status as appropriate.

Changed in firefox-3.5 (Ubuntu):
assignee: nobody → Alexander Sack (asac)
Revision history for this message
Alexander Sack (asac) wrote :

have you reproduced this more than once?

Changed in firefox-3.5 (Ubuntu):
status: New → Incomplete
Revision history for this message
Hew (hew) wrote :

I just reproduced the issue.

0) I am starting with up-to-date Karmic including Firefox 3.5.3
1) Force downgrade of firefox-3.5, firefox-3.5-branding and firefox-3.5-gnome-support to 3.5.2+nobinonly-0ubuntu2 (fortunately I still had the amd64 debs in /var/cache/apt/archives)
2) Run 'firefox-3.5 -profilemanager' and create a new profile
3) Go to 'Manage Search Engines'. Add keywords to some of the default search engines, and remove some others
4) Close firefox
5) Upgrade to up-to-date Karmic (currently firefox-3.5 3.5.3+build1+nobinonly-0ubuntu3)
6) Run 'firefox-3.5 -profilemanager' and choose the previously created profile
7) Go to 'Manage Search Engines'

The search engines have no keywords, and deleted ones have been restored.

Changed in firefox-3.5 (Ubuntu):
status: Incomplete → New
Revision history for this message
Alexander Sack (asac) wrote : Re: [Bug 428306] Re: default search engines have keywords wiped with upgrade to 3.5.3

On Wed, Oct 07, 2009 at 09:25:31AM -0000, Hew McLachlan wrote:
> I just reproduced the issue.
>
> 0) I am starting with up-to-date Karmic including Firefox 3.5.3
> 1) Force downgrade of firefox-3.5, firefox-3.5-branding and firefox-3.5-gnome-support to 3.5.2+nobinonly-0ubuntu2 (fortunately I still had the amd64 debs in /var/cache/apt/archives)
> 2) Run 'firefox-3.5 -profilemanager' and create a new profile
> 3) Go to 'Manage Search Engines'. Add keywords to some of the default search engines, and remove some others
> 4) Close firefox
> 5) Upgrade to up-to-date Karmic (currently firefox-3.5 3.5.3+build1+nobinonly-0ubuntu3)
> 6) Run 'firefox-3.5 -profilemanager' and choose the previously created profile
> 7) Go to 'Manage Search Engines'
>
> The search engines have no keywords, and deleted ones have been
> restored.

Hmm. ok. can you post the versions you are downgrading to?

Another thing i would appreciate you could try would be to get the current
firefox-3.0 packages ... start with a fresh profile (move away $HOME/.mozilla dir
first) and see if directly going to the current 3.5 ones from there would expose
the same issue.

thanks!

 - Alexander

Revision history for this message
Hew (hew) wrote : Re: default search engines have keywords wiped with upgrade to 3.5.3

firefox-3.0 (3.0.14) to firefox-3.5 (3.5.3) exhibits the same issue.

Revision history for this message
Alexander Sack (asac) wrote :

Hew, so you are saying that every upgrade causes this?

Revision history for this message
Alexander Sack (asac) wrote :

Do you see any erros in tools -> error console?

Revision history for this message
Hew (hew) wrote :

Alexander, it appears that way. Has anyone been able to upgrade without this occurring? I have clarified the steps to reproduce in the bug description, so any Karmic tester should be able to test this issue.

I only have a few errors that don't look relevant to my untrained eye:

Failed to load XPCOM component: /usr/lib/xulrunner-1.9.1.3/components/pyabout.py

Failed to load XPCOM component: /usr/lib/xulrunner-1.9.1.3/components/py_test_component.py

Warning: Error in parsing value for 'text-align'. Declaration dropped.
Source File: http://start.ubuntu.com/9.10/
Line: 25

description: updated
Revision history for this message
Steve Langasek (vorlon) wrote :

I can confirm seeing this behavior here.

Changed in firefox-3.5 (Ubuntu):
importance: Undecided → Low
status: New → Confirmed
Revision history for this message
Hew (hew) wrote :

This has just happened to me again with the latest update from 3.5.3 to 3.5.4

Revision history for this message
Alexander Sack (asac) wrote : Re: [Bug 428306] Re: default search engines have keywords wiped with upgrade to 3.5.3

On Mon, Nov 02, 2009 at 11:52:47AM -0000, Hew McLachlan wrote:
> This has just happened to me again with the latest update from 3.5.3 to
> 3.5.4
>

does this happen for all search engines? which search engines do you add a
keyword for? are those search engines the ones shipped by ubuntu, or custom
ones you added to your profile later?

 - Alexander

Revision history for this message
Hew (hew) wrote : Re: default search engines have keywords wiped with upgrade to 3.5.3

I have already described the situation many times, it should be easily reproducible by anyone.

It happens for all default search engines. I have added a keyword for almost all search engines, default and otherwise. Those non-default search engines I have added myself remain with their keywords and are not affected. Those default search engines I have previously removed return with the upgrade. Those default search engines I was using and had added a keyword for have had the keyword erased.

It is like all default search engines are removed, then readded to the bottom of the search engines list. If I was a programmer, I would be looking for code that adds the default search engines to existing profiles. The only time any change to search engines for an existing profile should be allowed is when it's known that a user has never modified their search engines before (which in reality is probably most users). This could be tracked by a boolean variable that is set to true whenever a search engine is added, keyword is added, etc.

tags: added: regression-release
removed: regression-potential
summary: - default search engines have keywords wiped with upgrade to 3.5.3
+ default search engines are removed and readded (keywords wiped) with
+ upgrade
Revision history for this message
Vish (vish) wrote :

I can confirm that the default search engines keep adding themselves on every update. [yahoo , ebay , creative commons.... etc]
I dont set any keywords , and never have set them.
So, i'm not sure the keywords have anything to do with this.

Revision history for this message
Tobias Hansen (tobias-han) wrote :

I found this annoying, too, so I investigated:
Removing the checkmark at "Settings -> Enhanced -> Update -> Automatically check for updates on -> Search Engines" should fix this. I just found this out, so it isn't tested jet. Would be nice if this would be disabled by default on Ubuntu.

Regards, BigMc

Revision history for this message
Hew (hew) wrote :

This has happened again with the update to 3.5.5. I already have search engines updates unchecked, so it doesn't look like that's the problem.

Keywords don't have anything to do with this, other than they are wiped when the search engines are readded. The problem is that search engines are changed for existing profiles.

Revision history for this message
Tim Magee (j-launchpad-ir-nny-demon-co-uk) wrote :

Happens to me too, at every little point upgrade of FF.
I have search engine updates unchecked (and have had since the option came available).
It is very annoying to spend time cutting out the chaff from the search engine list only to have it reappear every time upstream plugs another security hole.

Revision history for this message
In , Sami00Sami (samiaira-gmail) wrote :

User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.7pre) Gecko/20091212 Ubuntu/8.10 (intrepid) Shiretoko/3.5.7pre
Build Identifier: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.7pre) Gecko/20091212 Ubuntu/8.10 (intrepid) Shiretoko/3.5.7pre

Every time update is applied, all the keywords for default search engines are erased. Not critical or even important, but highly annoying.

Reproducible: Always

Steps to Reproduce:
1. apply update
2. restart

Actual Results:
erases keywords

Expected Results:
shouldn't

Revision history for this message
In , Martnik (martnik) wrote :

Definitely confirmed!
This should be under Component "Search", <email address hidden>

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.6) Gecko/20091215 Ubuntu/9.10 (karmic) Firefox/3.5.6

This happened to me twice, first updating from Firefox 3.0 to 3.5.5 then from 3.5.5 to 3.5.6, it also places the default search engines in the bottom of the list of search engines so I have to go into Manage Search Engines and move them up again (and add their keywords again).

Reproducible: Always

Steps to Reproduce:
1. update firefox
2. restart

Actual Results:
Default search engines are placed at the bottom of the list in the drop-down menu of the search bar, and their keywords (if any) are removed.

Revision history for this message
LosD (d-hp23c) wrote :

Happens to me too. Most annoying bug ever.

Revision history for this message
Micah Gersten (micahg) wrote :

Marking Triaged as we have an upstream bug. I was not able to confirm this on an upstream build though, so I did not mark the upstream bug confirmed yet.

Changed in firefox-3.5 (Ubuntu):
status: Confirmed → Triaged
Bug Watch Updater (bug-watch-updater)
Changed in firefox:
status: Unknown → New
Revision history for this message
Micah Gersten (micahg) wrote :

I was able to confirm the keyword being wiped on the Ubuntu upgrade though.

Revision history for this message
In , Beltzner (beltzner) wrote :

--> Toolkit::Application Update

Not a blocker, adding to the potential point release list, though. Gavin, Rob: can you take a peek and see if this is something that we can fix?

Revision history for this message
In , Mozilla-bugs-micahscomputing (mozilla-bugs-micahscomputing) wrote :

In Launchpad I commented that I couldn't verify this on an upstream build but I could on an Ubuntu build.

Revision history for this message
In , Robert-bugzilla (robert-bugzilla) wrote :

I suspect the cause is that awhile back the mars were made to always remove the search engines in the install dir and install the ones included in the update. I'm not sure what the requirement is that drove that decision but I personally don't think we should be removing / adding the search engines everytime.

gavin, bsmedberg: can you shed some light on why this was done and if this behavior can be removed? If not, then this will likely need to be fixed in the search service.

Revision history for this message
In , Nrthomas (nrthomas) wrote :

Ubuntu builds don't use mar files AFAIK, just replace the package.

Revision history for this message
In , Robert-bugzilla (robert-bugzilla) wrote :

Right you are... over to search

Revision history for this message
In , Gavin Sharp (gavin-sharp) wrote :

Sounds like the same underlying issue as bug 427028. AFAICT, Ubuntu updates change the on-disk location of search engine description files, and the search service doesn't deal with having its files moved out from under it. In this case, search keywords and other metadata are associated with the path to the file on disk when that file isn't in the profile or appdir... I'm not sure why that's the case for Ubuntu's shipped builds.

Would be nice to figure out what the exact issue is, but I don't know anything about how search plugins are shipped in Ubuntu builds.

Revision history for this message
In , Gavin Sharp (gavin-sharp) wrote :

I guess this is also tracked at https://bugs.edge.launchpad.net/ubuntu/+source/firefox-3.0/+bug/338785 , but I don't really see any information there about what the actual problems are.

Revision history for this message
In , Mozilla-bugs-micahscomputing (mozilla-bugs-micahscomputing) wrote :

(In reply to comment #8)
> I guess this is also tracked at
> https://bugs.edge.launchpad.net/ubuntu/+source/firefox-3.0/+bug/338785 , but I
> don't really see any information there about what the actual problems are.

That bug is for issues of having to restart Firefox before being insured stable usage. This is about a setting being reverted after restarting.

Revision history for this message
In , Mozilla-bugs-micahscomputing (mozilla-bugs-micahscomputing) wrote :

(In reply to comment #7)
> Sounds like the same underlying issue as bug 427028. AFAICT, Ubuntu updates
> change the on-disk location of search engine description files, and the search
> service doesn't deal with having its files moved out from under it. In this
> case, search keywords and other metadata are associated with the path to the
> file on disk when that file isn't in the profile or appdir... I'm not sure why
> that's the case for Ubuntu's shipped builds.
>
> Would be nice to figure out what the exact issue is, but I don't know anything
> about how search plugins are shipped in Ubuntu builds.

Search engine plugins are symlink'd from APP_DIR/distribution/searchplugins -> /usr/lib/firefox-addons/searchplugins
So in reality, the search plugin location doesn't change, but the symlink location does (as the APP_DIR changes).
For some reason, the whole path including the app dir is being stored.
For a Mozilla build, it's stored in firefox/searchplugins/foo.xml and is stored in the DB as [app]/foo.xml
For an Ubuntu build, it's stored in /usr/lib/firefox-3.5.6/distribution/searchplugins/en-US/foo.xml and stored in the DB as /usr/lib/firefox-3.5.6/distribution/searchplugins/en-US/foo.xml

Revision history for this message
In , Gavin Sharp (gavin-sharp) wrote :

Thanks for the info! I think the problem is that the search service treats distribution/searchplugins as an "unknown" location (same thing it does for extensions) since it isn't equal to the appdir searchplugins folder exactly, so it falls back to full paths.

Revision history for this message
In , Mozilla-bugs-micahscomputing (mozilla-bugs-micahscomputing) wrote :

(In reply to comment #11)
> Thanks for the info! I think the problem is that the search service treats
> distribution/searchplugins as an "unknown" location (same thing it does for
> extensions) since it isn't equal to the appdir searchplugins folder exactly, so
> it falls back to full paths.

Does anyone besides Ubuntu use distribution/searchplugins?

Bug Watch Updater (bug-watch-updater)
Changed in firefox:
status: New → Confirmed
Revision history for this message
In , Gavin Sharp (gavin-sharp) wrote :

Our "partner repack" builds do (including http://byob.mozilla.com/ generated builds, I think).

Per discussion with asac on IRC, I think we want to just work around this by calling normalize() on the file objects we get in _loadEnginesFromDir().

Revision history for this message
crazy ivan (x-floc) wrote :

Can confirm for update 3.5.6 to 3.5.7 and all the ones before.. Really annoying. I speculated it could have to do with sponsorship issues, but it seems to be an ubuntu specific problem.
My workaround is to use the "quicksearch" approach and assign shortcuts to my engines, which is faster anyways..

Bookmark:
Keyword:g
Location:http://www.google.de/search?q=%s

Just enter "g searchString" into adress bar

Revision history for this message
Chuck Ritola (cobra176) wrote :

I confirm this with update to:
Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.7) Gecko/20100106 Ubuntu/9.10 (karmic) Firefox/3.5.7

Some previous updates did the same thing, roughly 3 times. Search engine list and ordering gets reset to default.

Revision history for this message
ViktigLemma (jorgsk) wrote :

I noticed this bug when I started renaming my search engine shortcuts about 6 months ago. The shortcuts have been reset with every update since.

Using Ubuntu 9.10 and Firefox 3.5 and installing all the updates.

Revision history for this message
Mike Coleman (tutufan) wrote :

This has been happening to me for years, as best as I can recall. It's a very annoying bug (higher than "Low" priority, IMO).

Revision history for this message
Micah Gersten (micahg) wrote :

@Mike Coleman

Agreed, I raised the priority since this appears to affect a decent amount of people and data is wiped.

Changed in firefox-3.5 (Ubuntu):
importance: Low → High
Revision history for this message
Alexander Sack (asac) wrote :

bzr commit -m '* fix LP: #428306 - default search engines are removed and readded (keywords
  wiped) with upgrade
  - add debian/patches/normalize_distribution_searchplugins.patch
  - update debian/patches/series' --fixes 'lp:428306' 'debian/patches/normalize_distribution_searchplugins.patch' 'debian/patches/series' 'debian/changelog'
Committing to: bzr+ssh://bazaar.launchpad.net/~mozillateam/firefox/firefox-3.6.head/
added debian/patches/normalize_distribution_searchplugins.patch
modified debian/changelog
modified debian/patches/series
Committed revision 524.

affects: firefox-3.5 (Ubuntu) → firefox (Ubuntu)
Changed in firefox (Ubuntu):
status: Triaged → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package firefox - 3.6+nobinonly-0ubuntu3

---------------
firefox (3.6+nobinonly-0ubuntu3) lucid; urgency=low

  [ Alexander Sack <email address hidden> ]
  * fix LP: #520963 - sysprefs not honoured since firefox is built without
    system xulrunner; resurrect the patch we ship in xulrunner
    - add debian/patches/add_syspref_dir.patch
    - update debian/patches/series
  * fix LP: #520682 - Only search provider is Ask.com; set en-US as
    distribution.searchplugins.defaultLocale in syspref firefox.js
    - update debian/firefox.js
  * fix LP: #428306 - default search engines are removed and readded (keywords
    wiped) with upgrade
    - add debian/patches/bz534663_attXXX_normalize_distribution_searchplugins.patch
    - update debian/patches/series
  * add ubuntu fr code for yahoo (en-US) searchplugin
    - add debian/patches/ubuntu_codes_yahoo.patch
    - update debian/patches/series

  [ Micah Gersten <email address hidden> ]
  * Rename apport hook to firefox.py (unversioned)
    - rename debian/apport/firefox-3.6.py => debian/apport/firefox.py
  * Update apport hook to pull from unversioned profile directory
  * Update apport hook to report on non-distro package and tag PPA
  * Collect version info for firefox/abrowser packages
    - update debian/apport/firefox.py
  * Install apport hook again
    - update debian/firefox.install

  [ Jamie Strandboge <email address hidden> ]
  * debian/firefox.postinst.in: move aside the old firefox-3.5 AppArmor
    profile
 -- Alexander Sack <email address hidden> Wed, 17 Feb 2010 21:48:12 +0100

Changed in firefox (Ubuntu):
status: Fix Committed → Fix Released
Bug Watch Updater (bug-watch-updater)
Changed in firefox:
importance: Unknown → Low
Revision history for this message
Victor Engmark (victor-engmark) wrote :

Still happens in Firefox 3.6.12 on Ubuntu 10.10. Any idea when this will be part of the vanilla distro?

Revision history for this message
Micah Gersten (micahg) wrote : Re: [Bug 428306] Re: default search engines are removed and readded (keywords wiped) with upgrade

This was fixed for Lucid. Please file a new bug and reference this one
if there is a regression.

On 11/17/2010 04:51 PM, l0b0 wrote:
> Still happens in Firefox 3.6.12 on Ubuntu 10.10. Any idea when this will
> be part of the vanilla distro?
>

Revision history for this message
LAZA (laza74) wrote :

Duplicate of: #368455

And it is NOT fixed!

Every update i got Ask.com again in my list of search engines!

Revision history for this message
LAZA (laza74) wrote :

Opened up a new report, please add there:

https://bugs.launchpad.net/ubuntu/+source/firefox/+bug/936920

Revision history for this message
In , Chris Coulson (chrisccoulson) wrote :

Created attachment 602368
Treat distribution/searchplugins as a known location and don't use the full path for the search engine ID's

We've been carrying a distro patch for a long time to do the normalize() in _loadEnginesFromDir(), and I'd like to be able to drop this at some point. Whilst normalize() works for us, it probably wouldn't work in all cases (ie, if the application directory can move around and nobody symlinked the distribution folder to a stable path).

Would you accept something like the attached patch (which basically treats distribution/searchplugins as a known location)?

Revision history for this message
In , Gavin Sharp (gavin-sharp) wrote :

(In reply to Chris Coulson from comment #14)
> Whilst normalize() works for us, it probably wouldn't work in all cases (ie,
> if the application directory can move around and nobody symlinked the
> distribution folder to a stable path).

As a simple interim solution for what is kind of an edge case, this doesn't seem so bad.

> Would you accept something like the attached patch (which basically treats
> distribution/searchplugins as a known location)?

Maybe, though I would prefer just taking the normalize() patch, I think.

Revision history for this message
In , Mh+mozilla (mh+mozilla) wrote :

(In reply to Chris Coulson from comment #14)
> Created attachment 602368
> Treat distribution/searchplugins as a known location and don't use the full
> path for the search engine ID's
>
> We've been carrying a distro patch for a long time to do the normalize() in
> _loadEnginesFromDir(), and I'd like to be able to drop this at some point.
> Whilst normalize() works for us, it probably wouldn't work in all cases (ie,
> if the application directory can move around and nobody symlinked the
> distribution folder to a stable path).
>
> Would you accept something like the attached patch (which basically treats
> distribution/searchplugins as a known location)?

Interestingly, I have a somehow similar patch in debian, but i treat distribution/searchplugins as [app]/ instead of [distribution]/, because i moved all the default searchplugins there. And I have some additional code to make the transition smoother, although there are apparently some edge cases where that doesn't work quite well.

Revision history for this message
LAZA (laza74) wrote :

Still active in Fx 21 (release for Ubuntu today).

Also the list is mixed up, i have to sort all wanted search engines and extensions again!

Revision history for this message
In , Standard8 (standard8) wrote :

It is unclear if this is still an issue or not, but seeing as there haven't been complaints in recent years, I will assume this has either been worked around or fixed. So for now I'll resolve it.

If this is still an issue, I'm happy to talk about options to resolve it.

Bug Watch Updater (bug-watch-updater)
Changed in firefox:
status: Confirmed → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.