not all files show up in files-place

Bug #646724 reported by KarlRelton
364
This bug affects 66 people
Affects Status Importance Assigned to Milestone
Unity
Fix Released
Medium
Mikkel Kamstrup Erlandsen
Zeitgeist Data-Sources
Invalid
Low
Unassigned
Zeitgeist Framework
Invalid
Low
Unassigned
unity-2d
Fix Released
Medium
Unassigned
unity-lens-files
Fix Released
Medium
Mikkel Kamstrup Erlandsen
unity (Ubuntu)
Fix Released
Undecided
Unassigned
unity-lens-files (Ubuntu)
Fix Released
Medium
Unassigned
zeitgeist (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

The Unity interface, with its files-place feature for user access to files/documents etc. is really great.

However there is a problem since the search feature relies wholly on zeitgeist (AFAIK), in the zeitgeist is not tracking everything.

E.g. in openoffice, I create a new file (or open an existing one) from within openoffice itself. This file does not show up, presumably because openoffice does not (yet) push its activity to zeitgeist.

I guess any number of other programs could suffer the same problem.

For the average user, this is surely going to be confusing. They will create documents via legitimate means ('New ...' buttons in their applications), and then expect to be able to search for them in the Unity files-place interface.

To avoid this confusion I would guess there really needs to be a filesystem watcher on the home dir (or key folders within it) so that zeitgeist is aware of activity caused by non-zeitgeist-aware applications.

(See also bug 897393, about warning users that the search doesn't search all files.)

Revision history for this message
Mikkel Kamstrup Erlandsen (kamstrup) wrote :

Zeitgeist and unity-place-files are both designed with the specific goal of *not* needing to monitor the home directory recursively. This is a can of worms of titanic proportions.

The best solution would be to have OO push it's changes to ~/.recently-used.xbel in which case Zeitgeist, hence, u-p-f, would pick it up automatically.

But I agree, the current situation is less than ideal.

Changed in unity:
assignee: nobody → Mikkel Kamstrup Erlandsen (kamstrup)
importance: Undecided → High
status: New → Triaged
Changed in unity-place-files:
assignee: nobody → Mikkel Kamstrup Erlandsen (kamstrup)
importance: Undecided → High
status: New → Triaged
Revision history for this message
KarlRelton (karllinuxtest-relton) wrote :

I'll go with your judgement about not wanting to monitor, though Tracker and similar have done this for a while (perhaps with penalties that are unknown to me?).

Is there any appropriateness in a static indexing service (like 'locate') launched at regular intervals?

Not easy to resolve I know, but its the usability for someone like my mum who need know nothing of zeitgeist & friends that I have in mind.

Revision history for this message
Mark Shuttleworth (sabdfl) wrote : Re: [Bug 646724] [NEW] not all files show up in files-place

Reporter, we agree that seeing the world though a Zeitgeist filter is
risky, and causes problems when apps are not integrated, directly or
indirectly, with ZG.

Mikkel, would you agree it's better to close this by aiming for more
complete Zeitgeist coverage (or .xbel coverage) than by re-architecting
for some sort of periodic Tracker like sweep of the home dir?

If so, I recommend this bug be turned into a tracking bug for ZG
integration, with tasks on each of the apps that, in time, need to be
integrated.

However, I wonder if we aren't going to *have* to do some sort of
forcing function eventually, so that we can be sure that any content
*does* eventually get picked up, one way or the other. Because it seems
weird that there might always be a hole in our search.

Mark

Revision history for this message
Seif Lotfy (seif) wrote :

Hey I would like you to notice
https://code.edge.launchpad.net/zeitgeist-dataproviders
<https://code.edge.launchpad.net/zeitgeist-dataproviders>We cover a lot of
dataproviders in the trunk of this subproject and maybe a more focused
dedication into pushing these sets of applications plugins upstream or
downstream would help. Currently trunk covers most of the default
applications that ship with Ubuntu. We intend to package these sets of
dataproviders for Ubuntu soon.

On Fri, Sep 24, 2010 at 6:36 PM, Mark Shuttleworth <
<email address hidden>> wrote:

>
> Reporter, we agree that seeing the world though a Zeitgeist filter is
> risky, and causes problems when apps are not integrated, directly or
> indirectly, with ZG.
>
> Mikkel, would you agree it's better to close this by aiming for more
> complete Zeitgeist coverage (or .xbel coverage) than by re-architecting
> for some sort of periodic Tracker like sweep of the home dir?
>
> If so, I recommend this bug be turned into a tracking bug for ZG
> integration, with tasks on each of the apps that, in time, need to be
> integrated.
>
> However, I wonder if we aren't going to *have* to do some sort of
> forcing function eventually, so that we can be sure that any content
> *does* eventually get picked up, one way or the other. Because it seems
> weird that there might always be a hole in our search.
>
> Mark
>
> --
> not all files show up in files-place
> https://bugs.launchpad.net/bugs/646724
> You received this bug notification because you are subscribed to unity-
> place-files.
>
> Status in Unity: Triaged
> Status in Unity Files Place: Triaged
>
> Bug description:
> The Unity interface, with its files-place feature for user access to
> files/documents etc. is really great.
>
> However there is a problem since the search feature relies wholly on
> zeitgeist (AFAIK), in the zeitgeist is not tracking everything.
>
> E.g. in openoffice, I create a new file (or open an existing one) from
> within openoffice itself. This file does not show up, presumably because
> openoffice does not (yet) push its activity to zeitgeist.
>
> I guess any number of other programs could suffer the same problem.
>
> For the average user, this is surely going to be confusing. They will
> create documents via legitimate means ('New ...' buttons in their
> applications), and then expect to be able to search for them in the Unity
> files-place interface.
>
> To avoid this confusion I would guess there really needs to be a filesystem
> watcher on the home dir (or key folders within it) so that zeitgeist is
> aware of activity caused by non-zeitgeist-aware applications.
>
>
>

--
This is me doing some advertisement for my blog http://seilo.geekyogre.com

Seif Lotfy (seif)
Changed in zeitgeist-dataproviders:
status: New → In Progress
importance: Undecided → High
Revision history for this message
Seif Lotfy (seif) wrote :

As for OpenOffice specifically we can look into replacing the current zeitgeist-datahub with the vala version with libzeitgeist. I think the new zeitgeist-datahub covers that.

Revision history for this message
Mark Shuttleworth (sabdfl) wrote : Re: [Bug 646724] Re: not all files show up in files-place

That's very useful insight, thanks Seif!

Revision history for this message
Siegfried Gevatter (rainct) wrote :

I've just pushed a fix for zeitgeist-datahub (so that it'll find OpenOffice.org files) to lp:zeitgeist. I think Didier Roche wanted to get an updated Zeitgeist package into Maverick; I'll e-mail him about including this change.

--------------------------------------------------------------------------------------
revision <email address hidden> (1608)
...
Branch: zeitgeist-trunk

    recent.py: add workaround for OpenOffice.org files

    Backport the workaround used in the Vala datahub, which special-cases
    OpenOffice.org documents so that we can identify their .desktop file by
    looking at the mime-type in addition to the executable name.
--------------------------------------------------------------------------------------

Changed in zeitgeist:
importance: Undecided → Low
milestone: none → 0.5.2
status: New → Fix Committed
Changed in zeitgeist:
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unity-place-files - 0.5.30-0ubuntu1

---------------
unity-place-files (0.5.30-0ubuntu1) maverick; urgency=low

  * New upstream release:
    - take files from OOo with zg (LP: #646724)
    - fix "Files & Folders" tooltips not translatable (LP: #644215)
 -- Didier Roche <email address hidden> Mon, 27 Sep 2010 17:19:46 +0200

Changed in unity-place-files (Ubuntu):
status: New → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package zeitgeist - 0.5.2-0ubuntu1

---------------
zeitgeist (0.5.2-0ubuntu1) maverick; urgency=low

  * New upstream release:
    - Added a workaround so that events concerning OpenOffice.org are logged
      correctly (LP: #646724).
 -- Didier Roche <email address hidden> Mon, 27 Sep 2010 16:51:11 +0200

Changed in zeitgeist (Ubuntu):
status: New → Fix Released
Revision history for this message
Mikkel Kamstrup Erlandsen (kamstrup) wrote :

I think the right approach is to stick with Zeitgeist and then figure out ways to access/discover contents that you don't have in your Zeitgeist log.

For Natty I see two things we can do to improve the situation:

 a) Ship more dataproviders for Zeitgeist. In other words: Make sure apps either update .recently-used.xbel or that they log stuff directly in ZG.

 b) Integrate other means of discovery. For example:
   b1) Listing music files via the MPRIS2 API
   b2) Integrating removable devices (USB sticks) directly in the places (they just show up in the launcher now)
   b3) U1 integration
   bN) ...

If we combine these things I think we can have a very light and flexible system which we can tailor just to our liking.

Revision history for this message
Mark Shuttleworth (sabdfl) wrote :

RIght, Mikkel, in the latter category we might conceive a "smart bot"
which nosies through your data very occasionally (and in a way we
believe will minimise the negative user effects of updatedb and friends)
and lets ZG know about interesting stuff.

Mark

Revision history for this message
KarlRelton (karllinuxtest-relton) wrote :

Thanks guys. Alot more of my Openoffice work is now showing up and is searchable.

Regarding #11, I agree that an occasional bot that snoops through the data will still be necessary as a catch-all.

For the record, what are the 'negative user effects of updatedb and friends'?

Two use cases:

1) you can currently select to filter by folders, and then do a search (on folders). Currently the search will not find many of the folders that exist because zeitgeist has never seen them.

2) Inevitably people will run commands without dataloggers (e.g. command line tool, un-packing an archive) which will create legitimate files. The result of these need to searchable within a reasonable period of time.

Keep up the good work

Revision history for this message
Mark Shuttleworth (sabdfl) wrote :

 On 29/09/10 13:27, KarlRelton wrote:
> For the record, what are the 'negative user effects of updatedb and
> friends'?

"Why is my system like treacle?"

Revision history for this message
Erin (tuxgirl) wrote :

The case I have is that I scp'd a folder of files from another box into my home directory. Now, I can't seem to figure out any way to reach those files without the terminal unless I switch to ubuntu-desktop. That seems really silly! The files are in my home dir -- not hidden off in the middle of nowhere!

Revision history for this message
Ruben Grimm (rubengrimm-deactivatedaccount) wrote :

I think you can't avoid serving a search, that searches for folders and files in a given folder like the good ole' find command for the shell.

ZG is a great thing. It offers a search that is performant and at the same time adjusted to the files the user often needs. But as long as you can't find EVERY and I mean EVERY folder and file (at least) in your home folder it shouldn't displace, but complete the good ole' way of searching.

Both search forms could be done at the same time and be presented in one window. An example: The user searches for the word "thunder". ZG instantly shows AC/DCs Thunderstruck since the user already got the Rhythmbox-ZG-Dataprovider installed, but he can see that the search isn't done yet. He can stop the search if he wishes to, but he can also wait until the folder ".thunderbird" is found, which he was looking for.

I don't see why one shouldn't serve both ways. I use nautlilus-elementary on maverick which also comes with a great ZG search, but since maverick I happen to use the find command a lot...

Neil J. Patel (njpatel)
Changed in unity:
status: Triaged → Fix Released
Changed in unity-place-files:
status: Triaged → Fix Released
Revision history for this message
Mikkel Kamstrup Erlandsen (kamstrup) wrote :

Looks like this bug was mistakenly marked as Fix Released when it's really not. Probably the action of some rogue script ;-) So reopening.

Changed in unity:
status: Fix Released → Triaged
Changed in unity-place-files:
status: Fix Released → Triaged
Changed in unity-place-files (Ubuntu):
status: Fix Released → Triaged
Revision history for this message
Matteo Pagliazzi (paglia) wrote :

Maybe i'm wrong but synapse uses zeigeist, right?

And it shows also pdf, doc, .php, html and so on type of files, why we can't show that type of files in the files lens?

Revision history for this message
Manish Sinha (मनीष सिन्हा) (manishsinha) wrote :

Sorry for closing it. There is absolutelty no limit to the number of data-providers we can write.

If you need any specific dataprovider please file a bug against http://launchpad.net/zeitgeist-dataproviders

I know the status of this is very tricky since this bug does affect zeitgeist-dataproviders, but we are helpless, we cannot write dataproviders for every application.

Please comment if you disagree. Will reopen it against zeitgeist-dataproviders

Changed in zeitgeist-dataproviders:
importance: High → Low
status: In Progress → Invalid
Revision history for this message
KarlRelton (karllinuxtest-relton) wrote :

I still find comments #10, #11, and #15 sum up the situ well - if 'the main way users are expected to search' is purely zeitgeist powered, and zeitgeist only ever relies on data-providers, then those searches are going to fall short of users expectations.

So one way or another the user needs access to a directory-crawler, either:
- one which populates the zeitgeist log, or
- the dash/lenses consulting zeitgeist PLUS some other source (tracker, customer updatedb, or custom-something-else)

Revision history for this message
Manish Sinha (मनीष सिन्हा) (manishsinha) wrote :

Yeah. Right now we have dataproviders. The aim is to write more dataproviders esp the one for which are default on ubuntu. Then those dataproviders should also be shipped by default. In that case it would make sense.

Revision history for this message
Paddy Landau (paddy-landau) wrote :

Post #15 has to be the way to go. It merges the best of flexible, fast, and thorough, in a user-friendly way.

You cannot prevent people from installing programs that have no knowledge about ZG. Therefore, you always will need a thorough search. Having a "locate" every now and then is not good enough, because the user could well search for a file before "locate" has picked it up.

However, if ZG can link into the file system itself (or the file system can notify ZG of changes), ZG would be instantly aware of any change. Is that possible?

Revision history for this message
Seif Lotfy (seif) wrote :

It is possible for the filesystem to notify Zeitgeist if a change happens.
Sadly it is very costly memory wise (kernel side - inotify). We are looking
into an alternative to use gio. We already did the changes needed on the ZG
side. We still need to solve the issue where we capture events on the
filesystem.

On Fri, May 6, 2011 at 10:46 AM, Paddy Landau <email address hidden>wrote:

> Post #15 has to be the way to go. It merges the best of flexible, fast,
> and thorough, in a user-friendly way.
>
> You cannot prevent people from installing programs that have no
> knowledge about ZG. Therefore, you always will need a thorough search.
> Having a "locate" every now and then is not good enough, because the
> user could well search for a file before "locate" has picked it up.
>
> However, if ZG can link into the file system itself (or the file system
> can notify ZG of changes), ZG would be instantly aware of any change. Is
> that possible?
>
> --
> You received this bug notification because you are subscribed to unity-
> place-files.
> https://bugs.launchpad.net/bugs/646724
>
> Title:
> not all files show up in files-place
>
> Status in Unity:
> Triaged
> Status in Unity Files Place:
> Triaged
> Status in Zeitgeist Framework:
> Fix Released
> Status in Zeitgeist Data-Providers:
> Invalid
> Status in “unity-place-files” package in Ubuntu:
> Triaged
> Status in “zeitgeist” package in Ubuntu:
> Fix Released
>
> Bug description:
> The Unity interface, with its files-place feature for user access to
> files/documents etc. is really great.
>
> However there is a problem since the search feature relies wholly on
> zeitgeist (AFAIK), in the zeitgeist is not tracking everything.
>
> E.g. in openoffice, I create a new file (or open an existing one) from
> within openoffice itself. This file does not show up, presumably
> because openoffice does not (yet) push its activity to zeitgeist.
>
> I guess any number of other programs could suffer the same problem.
>
> For the average user, this is surely going to be confusing. They will
> create documents via legitimate means ('New ...' buttons in their
> applications), and then expect to be able to search for them in the
> Unity files-place interface.
>
> To avoid this confusion I would guess there really needs to be a
> filesystem watcher on the home dir (or key folders within it) so that
> zeitgeist is aware of activity caused by non-zeitgeist-aware
> applications.
>

Revision history for this message
Christoph Buchner (bilderbuchi) wrote :

Is it possible to look to beagle and/or tracker for how to approach this problem? I've used both in the past, and it worked reasonably well (Impact of bugs aside). Definitely better than the current situation.
I also agree comment #15 seems to be a very reasonable approach.

Revision history for this message
vanadium (ftack) wrote :

Needs are more modest than what Tracker or Beagle do. There is only a need to index file and directory names, not their contents.

Revision history for this message
KarlRelton (karllinuxtest-relton) wrote :

Wrt to comment #23, I remember over a year ago in blog-posts or wherever some communication between Seif of Zeitgeist fame and the Tracker people about integration/linking together. There was talk of one of them pushing events to the other.

Certainly if Tracker pushed an event to Zeitgeist everytime it 'discovered' a new file, that would certainly help Zeitgeist to be aware of alot more of the user's filesystem. If the user is using Tracker, then resource-wise the user would be no worse off, since Tracker is already doing the crawling & monitoring anyway.

However, of course Tracker itself is configured by the user to NOT crawl their whole filesystem, so for some users it is arguably still incomplete (though it would be good enough for my own uses).

Revision history for this message
Seif Lotfy (seif) wrote :

I am on it already :)

On Tue, May 10, 2011 at 10:57 AM, KarlRelton <
<email address hidden>> wrote:

> Wrt to comment #23, I remember over a year ago in blog-posts or wherever
> some communication between Seif of Zeitgeist fame and the Tracker people
> about integration/linking together. There was talk of one of them
> pushing events to the other.
>
> Certainly if Tracker pushed an event to Zeitgeist everytime it
> 'discovered' a new file, that would certainly help Zeitgeist to be aware
> of alot more of the user's filesystem. If the user is using Tracker,
> then resource-wise the user would be no worse off, since Tracker is
> already doing the crawling & monitoring anyway.
>
> However, of course Tracker itself is configured by the user to NOT crawl
> their whole filesystem, so for some users it is arguably still
> incomplete (though it would be good enough for my own uses).
>
> --
> You received this bug notification because you are subscribed to unity-
> place-files.
> https://bugs.launchpad.net/bugs/646724
>
> Title:
> not all files show up in files-place
>
> Status in Unity:
> Triaged
> Status in Unity Files Place:
> Triaged
> Status in Zeitgeist Framework:
> Fix Released
> Status in Zeitgeist Data-Providers:
> Invalid
> Status in “unity-place-files” package in Ubuntu:
> Triaged
> Status in “zeitgeist” package in Ubuntu:
> Fix Released
>
> Bug description:
> The Unity interface, with its files-place feature for user access to
> files/documents etc. is really great.
>
> However there is a problem since the search feature relies wholly on
> zeitgeist (AFAIK), in the zeitgeist is not tracking everything.
>
> E.g. in openoffice, I create a new file (or open an existing one) from
> within openoffice itself. This file does not show up, presumably
> because openoffice does not (yet) push its activity to zeitgeist.
>
> I guess any number of other programs could suffer the same problem.
>
> For the average user, this is surely going to be confusing. They will
> create documents via legitimate means ('New ...' buttons in their
> applications), and then expect to be able to search for them in the
> Unity files-place interface.
>
> To avoid this confusion I would guess there really needs to be a
> filesystem watcher on the home dir (or key folders within it) so that
> zeitgeist is aware of activity caused by non-zeitgeist-aware
> applications.
>

Revision history for this message
Christoph Buchner (bilderbuchi) wrote :

wrt to tracker: I actually didn't want to push this discussion towards tracker/beagle integration itself, I just wanted to point out that obviously, indexing the relevant files _and their contents_ is already possible and reasonable performance-wise. Consequently, I don't think it should be too much of a performance hog to maintain a proper index (without file's contents, too boot) for search in ZG; that's what the tracker/beagle people have been doing for years, after all.

Revision history for this message
Seif Lotfy (seif) wrote :

There is a difference between Tracker and Zeitgeist:
Tracker indexes stuff in the XDG directories only per default which means it
will only index stuff in Music, Documents, etc...
It will not cover a Directory "Foo" or anything in it. (it makes optimal for
people with lots of files in the default directories)
Zeitgeist indexes stuff you touch. Which means it does not index stuff you
never interacted with (it makes it optimal for fresh installs). Plus
Zeitgeist allows you to sort through receny, frequency, relevancy and
resemblance to the search string.

I am creating a zeitgeist extension that if Tracker is installed it will ask
Tracker for results and attach them to the zeitgeist results and return them
to the client. We have a working prototype. I am not happy with the solution
but its fast enough...
Stay tuned.

On Tue, May 10, 2011 at 11:48 AM, Christoph Buchner <
<email address hidden>> wrote:

> wrt to tracker: I actually didn't want to push this discussion towards
> tracker/beagle integration itself, I just wanted to point out that
> obviously, indexing the relevant files _and their contents_ is already
> possible and reasonable performance-wise. Consequently, I don't think it
> should be too much of a performance hog to maintain a proper index
> (without file's contents, too boot) for search in ZG; that's what the
> tracker/beagle people have been doing for years, after all.
>
> --
> You received this bug notification because you are subscribed to unity-
> place-files.
> https://bugs.launchpad.net/bugs/646724
>
> Title:
> not all files show up in files-place
>
> Status in Unity:
> Triaged
> Status in Unity Files Place:
> Triaged
> Status in Zeitgeist Framework:
> Fix Released
> Status in Zeitgeist Data-Providers:
> Invalid
> Status in “unity-place-files” package in Ubuntu:
> Triaged
> Status in “zeitgeist” package in Ubuntu:
> Fix Released
>
> Bug description:
> The Unity interface, with its files-place feature for user access to
> files/documents etc. is really great.
>
> However there is a problem since the search feature relies wholly on
> zeitgeist (AFAIK), in the zeitgeist is not tracking everything.
>
> E.g. in openoffice, I create a new file (or open an existing one) from
> within openoffice itself. This file does not show up, presumably
> because openoffice does not (yet) push its activity to zeitgeist.
>
> I guess any number of other programs could suffer the same problem.
>
> For the average user, this is surely going to be confusing. They will
> create documents via legitimate means ('New ...' buttons in their
> applications), and then expect to be able to search for them in the
> Unity files-place interface.
>
> To avoid this confusion I would guess there really needs to be a
> filesystem watcher on the home dir (or key folders within it) so that
> zeitgeist is aware of activity caused by non-zeitgeist-aware
> applications.
>

Revision history for this message
Christoph Buchner (bilderbuchi) wrote :

Ah ic, thanks for the clarification.
Would it be possible to create a feature where one manually points ZG, say, to a directory (or, e.g. a separate data partition), and ZG would index the given directory's contents once? Thus, ZG would circumvent the problem of not knowing about stuff you haven't accessed yet (think fresh install, but separate data partition), and would be able to deliver more meaningful results more quickly.

Revision history for this message
Mikkel Kamstrup Erlandsen (kamstrup) wrote :

The reason that we are not using tools like Tracker and Beagle is that we didn't find them "reasonable performance-wise", nor providing the quality of results, or fine control of queries we wanted. And working "reasonably well" is just not good enough in my book.

That said, there's nothing saying we must stick with ZG, it's just that it is an easy practical solution.

My personal check list for an indexing tool is something like:

 a) Doesn't crawl you home dir on login (to install inotify watches or check for new stuff)
 b) High quality query relevancy ranking
 c) Able to sort by usage frequency or recency
 d) (wishlist: direct access to index api to inspect term stats etc without the need for DBus roundtrips)

Solving a) requires something like btrfs or fsnotify. To my knowledge both Tracker and Beagle rely on inotify (possibly indirectly via GIO). And Tracker fell short on b) + c) + d) last I checked. Solving b), c) , and d) seems to inevitably require something like Zeitgeist combined with Xapian or Lucene (much like we do now :-)).

My currently preferred solution would be to:

 i) Extend ZG with a new range of data providers (music, u1, telepathy, contacts, etc)
 ii) Write a Nautilus extension that makes ZG index all files and folders you ever see in the file browser (this may require some filering logic in order not to spam your log too badly)

Revision history for this message
vanadium (ftack) wrote :

The issue is that the current implementation of "files and folders" is unacceptable. It is unacceptable that it does not show up the files the user knows are out there. This effectively breaks the function from the perspective of the user. Thus, this issue should be corrected with the highest priority, or the function should be removed if it cannot be technically implemented in a satisfactory way.

Additional problem: inconsistency. "Applications" finds all applications on the computer, and even Applications *not* on the computer. "Files and Folders" on the other hand, does not even find several the user's own files that are on the local computer.

Currently, this functions as an adequate "Recent documents" tool. Rename it to "Recent documents" and the current issue is solved. Then, a "Files and folders" lens could be a feature request for unity.

Extending ZG with a new range of data providers does not seem like a priority right know, except if aims to make sure that at least, a user can retrieve his files. Let the rest follow once this fundamental flaw is solved.

Revision history for this message
Seif Lotfy (seif) wrote :
Download full text (3.5 KiB)

BTW just for you info there is no perfect solution right now. Even Tracker
that "crawls your desktop" doesn't find anything that is NOT in the
following folders:
- Desktop
- Documents
- Downloads
- Music
- Public
- Pictures Templates
- Videos

While this sounds good alot of files do not reside there. So currently there
is no perfect solution. I am personally working on a hybrid search extension
that allows you to search Tracker + Zeitgeist results. Problem here is the
speed of the search in Tracker do a technical decision as well as the merge
of search results.

I think removing a functionality because its not working properly is a bad
idea. Because it works for the default and normal users. Anyone who uses the
default Ubuntu apps doesnt have problems finding his stuff since "once you
touch the file" with a standard Ubuntu app it will be covered by Zeitgeist.

On Thu, May 12, 2011 at 11:17 AM, vanadium <email address hidden>wrote:

> The issue is that the current implementation of "files and folders" is
> unacceptable. It is unacceptable that it does not show up the files the
> user knows are out there. This effectively breaks the function from the
> perspective of the user. Thus, this issue should be corrected with the
> highest priority, or the function should be removed if it cannot be
> technically implemented in a satisfactory way.
>
> Additional problem: inconsistency. "Applications" finds all applications
> on the computer, and even Applications *not* on the computer. "Files and
> Folders" on the other hand, does not even find several the user's own
> files that are on the local computer.
>
> Currently, this functions as an adequate "Recent documents" tool. Rename
> it to "Recent documents" and the current issue is solved. Then, a "Files
> and folders" lens could be a feature request for unity.
>
> Extending ZG with a new range of data providers does not seem like a
> priority right know, except if aims to make sure that at least, a user
> can retrieve his files. Let the rest follow once this fundamental flaw
> is solved.
>
> --
> You received this bug notification because you are subscribed to unity-
> place-files.
> https://bugs.launchpad.net/bugs/646724
>
> Title:
> not all files show up in files-place
>
> Status in Unity:
> Triaged
> Status in Unity Files Place:
> Triaged
> Status in Zeitgeist Framework:
> Fix Released
> Status in Zeitgeist Data-Providers:
> Invalid
> Status in “unity-place-files” package in Ubuntu:
> Triaged
> Status in “zeitgeist” package in Ubuntu:
> Fix Released
>
> Bug description:
> The Unity interface, with its files-place feature for user access to
> files/documents etc. is really great.
>
> However there is a problem since the search feature relies wholly on
> zeitgeist (AFAIK), in the zeitgeist is not tracking everything.
>
> E.g. in openoffice, I create a new file (or open an existing one) from
> within openoffice itself. This file does not show up, presumably
> because openoffice does not (yet) push its activity to zeitgeist.
>
> I guess any number of other programs could suffer the same problem.
>
> For the average user, this is surely going to be confusing. They will
> create doc...

Read more...

Revision history for this message
Christoph Buchner (bilderbuchi) wrote :

So, who's up to writing a script that "touches" each file in a desired directory? :-P this, as well as my comment #29, would neatly circumvent this problem, no?
Also, what exactly does "touching" mean? I can neatly touch all files on my harddisk if I let baobab (the disk usage analyzer) run over it, but that won't work, will it? What would the cheapest (performance-wise) application be to do this?

btw, Seif, are you sure about this location limitation for tracker? It's been a while since I last used it, but I seem to remember that I could add user-chosen locations with one visit to preferences, to be indexed by tracker.

Revision history for this message
Seif Lotfy (seif) wrote :

It can add a location for you to index but it gets more and more expensive.
Again I touching mean you do something with the file (open/close/modify). I
am working on a hybrid solution as i said. Sadly if you replace Zeitgeist
you also get drawbacks like sorting, constant directory monitoring (very
expensive over time) ans more, less sorting options, slower searching
(sqlite searches are too slow as compared to xapian powered ones).

So the only solution is either to improve the current of find a hybrid. But
replacing with a new solution will break things more...
:(

On Thu, May 12, 2011 at 1:18 PM, Christoph Buchner <
<email address hidden>> wrote:

> So, who's up to writing a script that "touches" each file in a desired
> directory? :-P this, as well as my comment #29, would neatly circumvent this
> problem, no?
> Also, what exactly does "touching" mean? I can neatly touch all files on my
> harddisk if I let baobab (the disk usage analyzer) run over it, but that
> won't work, will it? What would the cheapest (performance-wise) application
> be to do this?
>
> btw, Seif, are you sure about this location limitation for tracker? It's
> been a while since I last used it, but I seem to remember that I could
> add user-chosen locations with one visit to preferences, to be indexed
> by tracker.
>
> --
> You received this bug notification because you are subscribed to unity-
> place-files.
> https://bugs.launchpad.net/bugs/646724
>
> Title:
> not all files show up in files-place
>
> Status in Unity:
> Triaged
> Status in Unity Files Place:
> Triaged
> Status in Zeitgeist Framework:
> Fix Released
> Status in Zeitgeist Data-Providers:
> Invalid
> Status in “unity-place-files” package in Ubuntu:
> Triaged
> Status in “zeitgeist” package in Ubuntu:
> Fix Released
>
> Bug description:
> The Unity interface, with its files-place feature for user access to
> files/documents etc. is really great.
>
> However there is a problem since the search feature relies wholly on
> zeitgeist (AFAIK), in the zeitgeist is not tracking everything.
>
> E.g. in openoffice, I create a new file (or open an existing one) from
> within openoffice itself. This file does not show up, presumably
> because openoffice does not (yet) push its activity to zeitgeist.
>
> I guess any number of other programs could suffer the same problem.
>
> For the average user, this is surely going to be confusing. They will
> create documents via legitimate means ('New ...' buttons in their
> applications), and then expect to be able to search for them in the
> Unity files-place interface.
>
> To avoid this confusion I would guess there really needs to be a
> filesystem watcher on the home dir (or key folders within it) so that
> zeitgeist is aware of activity caused by non-zeitgeist-aware
> applications.
>

Revision history for this message
Christoph Buchner (bilderbuchi) wrote :

Just to be clear, I don't want to replace ZG, I didn't even intend to pull tracker into this (it was just meant as a technology reference, I shouldn't have mentioned it at all maybe). I'm just looking for a feasible solution to enable users to be able to find all files they would most probably search for, using unity. Seeing that this feature is powered by ZG, I see 3 alternatives:

a) manually open all files, or wait until user has interacted with all files without her being able to use unity search for said interaction in the meantime (this is the status now, and I think we all agree it's not very enticing).

b) Add an option to ZG to point to some directory (e.g. stuff in the users' extra data partition), whose contents will then be indexed _once_ by ZG. (Comment #29) This is basically an automated version of a), something like a "fast-forward" button.
The state of the index should then be not much worse than after normal usage for a long time, assuming the user will stumble over most of the files in said directory during normal usage, over time. Wildcard exceptions could be added to minimize cruft being added to the db (e.g. *~ files)
Is this a workable idea? If not, why? Are there negative performance impacts on ZG and/or unity-place-files I don't see?
Pro: tracker or similar technology does not have to be involved.

c) Write a script which, when pointed to a directory, accesses all the files sequentially, thus adding them to the ZG index as desired by the user. (comment #31) Admittedly the most hackish workaround, and just saves you the tedium of opening and closing files for a day or so, to be able to find them with unity-place-files.

To me at least, b) sounds reasonable. thoughts? alternatives?

Revision history for this message
Seif Lotfy (seif) wrote :
Download full text (3.4 KiB)

I am writing a little one time process then crawl your folders and emits an
open event to zeitgeist... :)
Thanks for the idea

On Thu, May 12, 2011 at 3:30 PM, Christoph Buchner <
<email address hidden>> wrote:

> Just to be clear, I don't want to replace ZG, I didn't even intend to
> pull tracker into this (it was just meant as a technology reference, I
> shouldn't have mentioned it at all maybe). I'm just looking for a
> feasible solution to enable users to be able to find all files they
> would most probably search for, using unity. Seeing that this feature is
> powered by ZG, I see 3 alternatives:
>
> a) manually open all files, or wait until user has interacted with all
> files without her being able to use unity search for said interaction in
> the meantime (this is the status now, and I think we all agree it's not
> very enticing).
>
> b) Add an option to ZG to point to some directory (e.g. stuff in the users'
> extra data partition), whose contents will then be indexed _once_ by ZG.
> (Comment #29) This is basically an automated version of a), something like a
> "fast-forward" button.
> The state of the index should then be not much worse than after normal
> usage for a long time, assuming the user will stumble over most of the files
> in said directory during normal usage, over time. Wildcard exceptions could
> be added to minimize cruft being added to the db (e.g. *~ files)
> Is this a workable idea? If not, why? Are there negative performance
> impacts on ZG and/or unity-place-files I don't see?
> Pro: tracker or similar technology does not have to be involved.
>
> c) Write a script which, when pointed to a directory, accesses all the
> files sequentially, thus adding them to the ZG index as desired by the
> user. (comment #31) Admittedly the most hackish workaround, and just
> saves you the tedium of opening and closing files for a day or so, to be
> able to find them with unity-place-files.
>
> To me at least, b) sounds reasonable. thoughts? alternatives?
>
> --
> You received this bug notification because you are subscribed to unity-
> place-files.
> https://bugs.launchpad.net/bugs/646724
>
> Title:
> not all files show up in files-place
>
> Status in Unity:
> Triaged
> Status in Unity Files Place:
> Triaged
> Status in Zeitgeist Framework:
> Fix Released
> Status in Zeitgeist Data-Providers:
> Invalid
> Status in “unity-place-files” package in Ubuntu:
> Triaged
> Status in “zeitgeist” package in Ubuntu:
> Fix Released
>
> Bug description:
> The Unity interface, with its files-place feature for user access to
> files/documents etc. is really great.
>
> However there is a problem since the search feature relies wholly on
> zeitgeist (AFAIK), in the zeitgeist is not tracking everything.
>
> E.g. in openoffice, I create a new file (or open an existing one) from
> within openoffice itself. This file does not show up, presumably
> because openoffice does not (yet) push its activity to zeitgeist.
>
> I guess any number of other programs could suffer the same problem.
>
> For the average user, this is surely going to be confusing. They will
> create documents via legitimate means ('New ...' buttons in their
> appli...

Read more...

Revision history for this message
Christoph Buchner (bilderbuchi) wrote :

Awesome! Always happy to help, even if it's just ideas, not code. :-)

Revision history for this message
Seif Lotfy (seif) wrote :

We are almost done :) w00t w00t :)

Changed in zeitgeist:
status: Fix Released → In Progress
Revision history for this message
linusND (christian-remboldt) wrote :

@Seif, ive got this bug bookmarked now. cant wait to try your script. If it works maybe it could be added to future distro releases. Running once on install seems very reasonable.

Revision history for this message
Siegfried Gevatter (rainct) wrote :

I'm closing the Zeitgeist task on this since it is more of a data-source/integration issue. Please open separate bugs against the project in case you have any concrete features you're missing to implement this (and feel free to discuss any ideas on our mailing list or IRC).

Changed in zeitgeist:
milestone: 0.5.2 → none
status: In Progress → Invalid
Revision history for this message
Christoph Buchner (bilderbuchi) wrote :

So, will you put this into activity-log-manager, Seif?

Revision history for this message
Seif Lotfy (seif) wrote : Re: [Zeitgeist] [Bug 646724] Re: not all files show up in files-place

Very good idea. Maybe a button saying "dig up the past" could be a good
start :)
Thanks for the idea

On Thu, May 19, 2011 at 12:31 PM, Christoph Buchner <
<email address hidden>> wrote:

> So, will you put this into activity-log-manager, Seif?
>
> --
> You received this bug notification because you are a member of Zeitgeist
> Framework Team, which is subscribed to Zeitgeist Data-Sources.
> https://bugs.launchpad.net/bugs/646724
>
> Title:
> not all files show up in files-place
>
> Status in Unity:
> Triaged
> Status in Unity Files Place:
> Triaged
> Status in Zeitgeist Framework:
> Invalid
> Status in Zeitgeist Data-Providers:
> Invalid
> Status in “unity-place-files” package in Ubuntu:
> Triaged
> Status in “zeitgeist” package in Ubuntu:
> Fix Released
>
> Bug description:
> The Unity interface, with its files-place feature for user access to
> files/documents etc. is really great.
>
> However there is a problem since the search feature relies wholly on
> zeitgeist (AFAIK), in the zeitgeist is not tracking everything.
>
> E.g. in openoffice, I create a new file (or open an existing one) from
> within openoffice itself. This file does not show up, presumably
> because openoffice does not (yet) push its activity to zeitgeist.
>
> I guess any number of other programs could suffer the same problem.
>
> For the average user, this is surely going to be confusing. They will
> create documents via legitimate means ('New ...' buttons in their
> applications), and then expect to be able to search for them in the
> Unity files-place interface.
>
> To avoid this confusion I would guess there really needs to be a
> filesystem watcher on the home dir (or key folders within it) so that
> zeitgeist is aware of activity caused by non-zeitgeist-aware
> applications.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~zeitgeist
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~zeitgeist
> More help : https://help.launchpad.net/ListHelp
>

Revision history for this message
Sean Fitzpatrick (sean-fitzpatrick) wrote :

Sorry to chime in since it sounds like you guys have got this thing figured out; I just commented on bug #773204, which is probably the same issue (after trying to answer somebody on Ask Ubuntu who wanted to know how to reproduce the functionality in classic gnome of using Alt+F2 + path-to-file to open a document). The discussion above seems to confirm my guess that the dash only sees recent documents. (Which, as you've also already noticed, is annoying if you've ported your old files to a clean install.) So I guess the right answer for the Ask Ubuntu guy is, 'use the Super key, but it's not 100% there yet'? (BTW, Alt+F2 no longer works for this purpose.)

Revision history for this message
Christoph Buchner (bilderbuchi) wrote :

Yes, I'd think so. Better yet, "use the files&folders lens, but it's not 100% there yet".

Changed in unity-2d:
status: New → Triaged
Revision history for this message
vanadium (ftack) wrote :

There is more going on than the file lens only seeing recent files. I have a project folder 6 levels deep under / which I use regularly, and the folder itself *never* showed up in the files lens. To me, the file lens is completely useless in its current state.

Revision history for this message
Seif Lotfy (seif) wrote : Re: [Bug 646724] Re: not all files show up in files-place

Which app do u use to edit the files in that folder ?

On Thu, Jun 30, 2011 at 6:58 PM, vanadium <email address hidden> wrote:

> There is more going on than the file lens only seeing recent files. I
> have a project folder 6 levels deep under / which I use regularly, and
> the folder itself *never* showed up in the files lens. To me, the file
> lens is completely useless in its current state.
>
> --
> You received this bug notification because you are subscribed to unity-
> place-files.
> https://bugs.launchpad.net/bugs/646724
>
> Title:
> not all files show up in files-place
>
> Status in Unity:
> Triaged
> Status in Unity 2D:
> Triaged
> Status in Unity Files Place:
> Triaged
> Status in Zeitgeist Framework:
> Invalid
> Status in Zeitgeist Data-Sources:
> Invalid
> Status in “unity-place-files” package in Ubuntu:
> Triaged
> Status in “zeitgeist” package in Ubuntu:
> Fix Released
>
> Bug description:
> The Unity interface, with its files-place feature for user access to
> files/documents etc. is really great.
>
> However there is a problem since the search feature relies wholly on
> zeitgeist (AFAIK), in the zeitgeist is not tracking everything.
>
> E.g. in openoffice, I create a new file (or open an existing one) from
> within openoffice itself. This file does not show up, presumably
> because openoffice does not (yet) push its activity to zeitgeist.
>
> I guess any number of other programs could suffer the same problem.
>
> For the average user, this is surely going to be confusing. They will
> create documents via legitimate means ('New ...' buttons in their
> applications), and then expect to be able to search for them in the
> Unity files-place interface.
>
> To avoid this confusion I would guess there really needs to be a
> filesystem watcher on the home dir (or key folders within it) so that
> zeitgeist is aware of activity caused by non-zeitgeist-aware
> applications.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/unity/+bug/646724/+subscriptions
>

Revision history for this message
vanadium (ftack) wrote :

Some files opened from that folder and below do show up. It is the project folder itself that cannot be retrieved through the file lens. In some cases, there is a need to organize related files in folders. Easy access to the folder would then be a benefit. The "Files & Folders" lens could be very useful to easily retrieve folder just by typing part of the name. Working on different project, it is not always practical to remember the exact name of a document (e.g. meeting reports), so one wants to see the folder.

Another issue with how things currently work. There is a subfolder with pictures. Once you've been looking at the pictures, the search term for the folder causes the lens to be flooded with these pictures. Retrieval of e.g. word processing documents using the folder name as search term then becomes very cumbersome.

Revision history for this message
Christoph Buchner (bilderbuchi) wrote :

Regarding the folders not showing, this is another bug: https://bugs.launchpad.net/unity/+bug/749566

Revision history for this message
Bernhard (b.a.koenig) wrote :

The users should somehow be made aware that the files listed in the dash are not *all* files in their home folder, but that it is just a selection made by zeitgeist. There should be some explanation about what are the "recent files" and what are the others. And even more importantly, make the user aware that the "search for files" (gnome-search-tool) gives them a more complete list of the files in the home directory.

Revision history for this message
Krister Swenson (thekswenson) wrote :

> access to the folder would then be a benefit. The "Files & Folders" lens
> could be very useful to easily retrieve  folder just by typing part of
> the name.

Yes! Why isn't there a hotkey to search only folders...
   I want to do this every day.

Revision history for this message
DavidBriscoe (idbrii) wrote :

@Krister: It's not built-in, but you could create your own hotkey to search folders: http://askubuntu.com/questions/49133/are-there-any-hotkeys-in-dash-for-choosing-among-search-term-matches/50113#50113

Revision history for this message
Christoph Buchner (bilderbuchi) wrote :

Seif, what happened to the "dig up the past" button you mentioned further up? Is this coming? I remember seeing a mock-up screenshot somewhere, but I don't have it in activity-log-manager 0.8.0 from the zeitgeist ppa.

Revision history for this message
Seif Lotfy (seif) wrote :

You can get it from lp:activity-log-manager (trunk)
it works like charm here. Can you test it?
make sure you backup your zeitgeist directory "cp ~/.local/share/zeitgeist
~/.local/share/zeitgeist-backup"
give us feedback on #zeitgeist so we can finish this.
Cheers
Seif

On Wed, Aug 17, 2011 at 11:14 AM, Christoph Buchner <
<email address hidden>> wrote:

> Seif, what happened to the "dig up the past" button you mentioned
> further up? Is this coming? I remember seeing a mock-up screenshot
> somewhere, but I don't have it in activity-log-manager 0.8.0 from the
> zeitgeist ppa.
>
> --
> You received this bug notification because you are subscribed to unity-
> place-files.
> https://bugs.launchpad.net/bugs/646724
>
> Title:
> not all files show up in files-place
>
> Status in Unity:
> Triaged
> Status in Unity 2D:
> Triaged
> Status in Unity Files Lens:
> Triaged
> Status in Zeitgeist Framework:
> Invalid
> Status in Zeitgeist Data-Sources:
> Invalid
> Status in “unity-place-files” package in Ubuntu:
> Triaged
> Status in “zeitgeist” package in Ubuntu:
> Fix Released
>
> Bug description:
> The Unity interface, with its files-place feature for user access to
> files/documents etc. is really great.
>
> However there is a problem since the search feature relies wholly on
> zeitgeist (AFAIK), in the zeitgeist is not tracking everything.
>
> E.g. in openoffice, I create a new file (or open an existing one) from
> within openoffice itself. This file does not show up, presumably
> because openoffice does not (yet) push its activity to zeitgeist.
>
> I guess any number of other programs could suffer the same problem.
>
> For the average user, this is surely going to be confusing. They will
> create documents via legitimate means ('New ...' buttons in their
> applications), and then expect to be able to search for them in the
> Unity files-place interface.
>
> To avoid this confusion I would guess there really needs to be a
> filesystem watcher on the home dir (or key folders within it) so that
> zeitgeist is aware of activity caused by non-zeitgeist-aware
> applications.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/unity/+bug/646724/+subscriptions
>

Revision history for this message
Manish Sinha (मनीष सिन्हा) (manishsinha) wrote :

To check the history, you can install activity-log-manager from the PPA ppa:zeitgeist/ppa
https://launchpad.net/~zeitgeist/+archive/ppa

Changed in unity-lens-files (Ubuntu):
status: New → Triaged
Revision history for this message
Heimen Stoffels (vistaus) wrote :

When I search for music in 11.10 using the Music-lens, it finds only recently played files and not even every file of it. Therefore, this bug is still present.

Revision history for this message
AndreK (andre-k) wrote :

unity is clearly not ready for prime time yet - I cannot believe 11.10 got released with this huge problem.

Revision history for this message
Bernhard (b.a.koenig) wrote :

I never understood why the file lens is depending on zeitgeist logs. When I search for files, I usually want to search them all, not just my recent files.

Revision history for this message
Manish Sinha (मनीष सिन्हा) (manishsinha) wrote :

On Oct 16, 2011 4:35 PM, "Bernhard" <email address hidden> wrote:
>
> I never understood why the file lens is depending on zeitgeist logs.
> When I search for files, I usually want to search them all, not just my
> recent files.

As per you how shall it be done? I know the present way has shortcomings.
Please provide a solution if available. It will be surely implemented

Revision history for this message
Krister Swenson (thekswenson) wrote :

The program locate/slocate keeps a database of all files so that
searching is quick, for example.

2011/10/16 Manish Sinha (मनीष सिन्हा <email address hidden>:
> On Oct 16, 2011 4:35 PM, "Bernhard" <email address hidden> wrote:
>>
>> I never understood why the file lens is depending on zeitgeist logs.
>> When I search for files, I usually want to search them all, not just my
>> recent files.
>
> As per you how shall it be done? I know the present way has shortcomings.
> Please provide a solution if available. It will be surely implemented
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/646724
>
> Title:
>  not all files show up in files-place
>
> Status in Unity:
>  Triaged
> Status in Unity 2D:
>  Triaged
> Status in Unity Files Lens:
>  Triaged
> Status in Zeitgeist Framework:
>  Invalid
> Status in Zeitgeist Data-Sources:
>  Invalid
> Status in “unity-lens-files” package in Ubuntu:
>  Triaged
> Status in “unity-place-files” package in Ubuntu:
>  Triaged
> Status in “zeitgeist” package in Ubuntu:
>  Fix Released
>
> Bug description:
>  The Unity interface, with its files-place feature for user access to
>  files/documents etc. is really great.
>
>  However there is a problem since the search feature relies wholly on
>  zeitgeist (AFAIK), in the zeitgeist is not tracking everything.
>
>  E.g. in openoffice, I create a new file (or open an existing one) from
>  within openoffice itself. This file does not show up, presumably
>  because openoffice does not (yet) push its activity to zeitgeist.
>
>  I guess any number of other programs could suffer the same problem.
>
>  For the average user, this is surely going to be confusing. They will
>  create documents via legitimate means ('New ...' buttons in their
>  applications), and then expect to be able to search for them in the
>  Unity files-place interface.
>
>  To avoid this confusion I would guess there really needs to be a
>  filesystem watcher on the home dir (or key folders within it) so that
>  zeitgeist is aware of activity caused by non-zeitgeist-aware
>  applications.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/unity/+bug/646724/+subscriptions
>

Revision history for this message
DJ (ke7mbz) wrote :

I think the dash is revolutionary compared to the old way of finding files and applications. It presents the user with a much simpler and more intuitive way to get to what they're looking for. But it's useless to me if I can't really use it to find stuff, or if I can only find a small subset of my home content. Right now it's really nothing more than a novelty, and not very useful for files and folders. I hope that changes in the near future. IMHO, the concept is so far ahead of what MAC and Windows currently offer. It's really unity's biggest selling point., but it has to actually work.

The solution is really quite simple, if you can take your dev hat off for a second and see it from a user's standpoint. Show recent files and folders first, with a button to press to do a deeper search. The button needs to be obvious and visible, not hidden away on the "filter results" menu. In fact, it could be a few buttons, e.g. "one week, one month, 6 months, one year, show all." Much like a browser history. You could put them all in a drop down menu.

You could wait to implement everything through Zeitgeist, but why not offer a temporary fix using Linux's built in search functions. Use locate first, and then keep going with find. If locate finds what the user's looking for, the search stops when they click on it. That way, potential users won't just dismiss Unity as beta software and go elsewhere. They may not be willing to give it another try for 12.04. Especially after they spend 1-2k on a Macbook. I have several potential users lined up, but I'm hesitant because of stuff like this. The idea is to wow them with how convenient and intuitive it is, rather than to disappoint and frustrate.

Revision history for this message
Christoph Buchner (bilderbuchi) wrote :

@Seif, re comment #53: Any estimate when this will land in a release and/or PPA?

Revision history for this message
KarlRelton (karllinuxtest-relton) wrote :

Re #53, I have compiled from source the a-l-m and tried the 'retrieve from past' feature.

Unfortunately no matter what I do it says '0 events inserted', and no extra files then seem to be visible.

I then tried the 'history.py' script that Seif kindly referenced on a blog post of his a while back, which has the same basic crawler code but packaged to be run on a directory of the users choice from the command line. This does insert a number of events (almost as many events as there are files in the directory concerned) - but then nothing can seem to be able to find them: neither Unity Dash of gaj display them.

So although there is now crawler code out there in the wild, it doesn't seem to be helping for some reason.

Revision history for this message
SRoesgen (s-roesgen) wrote :

What if have been asking myself now for a time is the reason why there is no nautilus data source/nautilus data provider?

I suppose I do not get the problem entirely, because otherwise it would have been implemented already.

After you installed a fresh system you could simply copy back all files from a backup hard drive and they should show up then when nautilus notifies Zeitgeist about the copying process of the files.

The only problem would be that you still have to find a way to inform Zeitgeist about all the files which have already been there after you did a system upgrade instead of a fresh install. What, thus, makes the "dig up the past" option still necessary. Anyway I see one big problem with the "dig up the past" action: a normal user will not know that it exists. And therefore a normal user will still think that search in Ubuntu is broken if using the dash to find certain files.

A nautilus data provider would also solve a different problem I recently discovered. Whenever you change the name or location of a file which has already been "indexed" by Zeitgeist, then suddenly the dash won't show it to you anymore. So instead of informing Zeitgeist of the fact that the file has been changed (and this providing Zeitgeist with the new location or the new name of of the file) the file is completely ignored from then on.
Try it. Change a name of a file and then search for it in the dash. Even though it had been found before the change, it won't be found after the renaming.

I know that there is/was the GtkRecentManager and nautilus GIO plugin. But seemingly it does not work the way that I expected.

Revision history for this message
KarlRelton (karllinuxtest-relton) wrote :

Hacked a bit more ...

The 'crawler' code in a-l-m has a bug comparing a timestamp in the form of a string with another in the form of a float. Thats why it always inserts 0 events (timestamp test always fails). It also has a potential divide by zero bug if there are zero 'valid_uris'.

Correcting those will make it insert events based on the files mtime. However they still don't show up in gaj or dash.

Watching the zeitgeist-daemon running in a terminal one can see the events being inserted, but still no sign of them showing up in any queries.

description: updated
Revision history for this message
Pavol Klačanský (pavolzetor-deactivatedaccount) wrote :

and if you have file like "gwibber_multiple.png", you can search it like ".png" or "gwibber..." but if I enter "multiple", I get no result

Omer Akram (om26er)
Changed in unity:
importance: High → Medium
Changed in unity-2d:
importance: Undecided → Medium
Changed in unity-lens-files:
importance: High → Medium
Changed in unity-lens-files (Ubuntu):
importance: Undecided → Medium
Changed in unity-place-files (Ubuntu):
importance: Undecided → Medium
Revision history for this message
AndreK (andre-k) wrote :

reducing the importance of the bug does not make it less annoying.

Revision history for this message
Pablo Almeida (pabloalmeidaff9) wrote :

But does make it less likely to be fixed.

2012/2/13 AndreK <email address hidden>

> reducing the importance of the bug does not make it less annoying.
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (884594).
> https://bugs.launchpad.net/bugs/646724
>
> Title:
> not all files show up in files-place
>
> Status in Unity:
> Triaged
> Status in Unity 2D:
> Triaged
> Status in Unity Files Lens:
> Triaged
> Status in Zeitgeist Framework:
> Invalid
> Status in Zeitgeist Data-Sources:
> Invalid
> Status in “unity-lens-files” package in Ubuntu:
> Triaged
> Status in “unity-place-files” package in Ubuntu:
> Triaged
> Status in “zeitgeist” package in Ubuntu:
> Fix Released
>
> Bug description:
> The Unity interface, with its files-place feature for user access to
> files/documents etc. is really great.
>
> However there is a problem since the search feature relies wholly on
> zeitgeist (AFAIK), in the zeitgeist is not tracking everything.
>
> E.g. in openoffice, I create a new file (or open an existing one) from
> within openoffice itself. This file does not show up, presumably
> because openoffice does not (yet) push its activity to zeitgeist.
>
> I guess any number of other programs could suffer the same problem.
>
> For the average user, this is surely going to be confusing. They will
> create documents via legitimate means ('New ...' buttons in their
> applications), and then expect to be able to search for them in the
> Unity files-place interface.
>
> To avoid this confusion I would guess there really needs to be a
> filesystem watcher on the home dir (or key folders within it) so that
> zeitgeist is aware of activity caused by non-zeitgeist-aware
> applications.
>
> (See also bug 897393, about warning users that the search doesn't
> search all files.)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/unity/+bug/646724/+subscriptions
>

--
Pablo Almeida
http://www.google.com/profiles/pabloalmeidaff9

Revision history for this message
Krister Swenson (thekswenson) wrote :
Download full text (3.9 KiB)

ouch.

On Tue, Feb 14, 2012 at 8:57 AM, Pablo Almeida
<email address hidden> wrote:
> But does make it less likely to be fixed.
>
> 2012/2/13 AndreK <email address hidden>
>
>> reducing the importance of the bug does not make it less annoying.
>>
>> --
>> You received this bug notification because you are subscribed to a
>> duplicate bug report (884594).
>> https://bugs.launchpad.net/bugs/646724
>>
>> Title:
>>  not all files show up in files-place
>>
>> Status in Unity:
>>  Triaged
>> Status in Unity 2D:
>>  Triaged
>> Status in Unity Files Lens:
>>  Triaged
>> Status in Zeitgeist Framework:
>>  Invalid
>> Status in Zeitgeist Data-Sources:
>>  Invalid
>> Status in “unity-lens-files” package in Ubuntu:
>>  Triaged
>> Status in “unity-place-files” package in Ubuntu:
>>  Triaged
>> Status in “zeitgeist” package in Ubuntu:
>>  Fix Released
>>
>> Bug description:
>>  The Unity interface, with its files-place feature for user access to
>>  files/documents etc. is really great.
>>
>>  However there is a problem since the search feature relies wholly on
>>  zeitgeist (AFAIK), in the zeitgeist is not tracking everything.
>>
>>  E.g. in openoffice, I create a new file (or open an existing one) from
>>  within openoffice itself. This file does not show up, presumably
>>  because openoffice does not (yet) push its activity to zeitgeist.
>>
>>  I guess any number of other programs could suffer the same problem.
>>
>>  For the average user, this is surely going to be confusing. They will
>>  create documents via legitimate means ('New ...' buttons in their
>>  applications), and then expect to be able to search for them in the
>>  Unity files-place interface.
>>
>>  To avoid this confusion I would guess there really needs to be a
>>  filesystem watcher on the home dir (or key folders within it) so that
>>  zeitgeist is aware of activity caused by non-zeitgeist-aware
>>  applications.
>>
>>  (See also bug 897393, about warning users that the search doesn't
>>  search all files.)
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/unity/+bug/646724/+subscriptions
>>
>
>
> --
> Pablo Almeida
> http://www.google.com/profiles/pabloalmeidaff9
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/646724
>
> Title:
>  not all files show up in files-place
>
> Status in Unity:
>  Triaged
> Status in Unity 2D:
>  Triaged
> Status in Unity Files Lens:
>  Triaged
> Status in Zeitgeist Framework:
>  Invalid
> Status in Zeitgeist Data-Sources:
>  Invalid
> Status in “unity-lens-files” package in Ubuntu:
>  Triaged
> Status in “unity-place-files” package in Ubuntu:
>  Triaged
> Status in “zeitgeist” package in Ubuntu:
>  Fix Released
>
> Bug description:
>  The Unity interface, with its files-place feature for user access to
>  files/documents etc. is really great.
>
>  However there is a problem since the search feature relies wholly on
>  zeitgeist (AFAIK), in the zeitgeist is not tracking everything.
>
>  E.g. in openoffice, I create a new file (or open an existing one) from
>  within openoffice itself. This file does not show up, presumably
>  because open...

Read more...

Revision history for this message
Raffaele Zippo (rzippo) wrote :

Excuse me for not reading all the discussion up there..

From the first time i used the dash, i found it extremely handy and useful; but for what are my necessities zeigeist's way to search files is not suited for me, so that i'm only using tracker.

So I ask: can you integrate tracker, obviously as an option and not as default, with the dash search lens?

Revision history for this message
Mikkel Kamstrup Erlandsen (kamstrup) wrote :

@Raffaele: Yes. It should be most easy to integrate Tracker as a lens on its own or simply as a scope in the files lens. Honestly, with all the buzz, I am surprised that no one has done so yet. My guess would be that it could be done in less than 100 lines of Python

Revision history for this message
KarlRelton (karllinuxtest-relton) wrote :

@Raffaele & @Mikkel: Someone already has - see http://hoheinzollern.wordpress.com/2011/11/21/unity-tracker-lens/
Oh for this to be released and integrated!

Revision history for this message
Arthur Blair (adblair) wrote :

I think using the locate command as an intermediate solution to this problem is sensible. However, the locate command does not find files within an encrypted home directory (one of the many things that encrypted homes break), so this will continue to be a problem for many users.

Changed in unity:
status: Triaged → Fix Released
Changed in unity-2d:
status: Triaged → Fix Released
no longer affects: unity-place-files (Ubuntu)
Changed in unity-lens-files:
status: Triaged → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unity-lens-files - 5.6.0-0ubuntu1

---------------
unity-lens-files (5.6.0-0ubuntu1) precise; urgency=low

  * New upstream release.
    - Content of hidden folders shown in unity-lens-files (LP: #878052)
    - not all files show up in files-place (LP: #646724)
 -- Didier Roche <email address hidden> Mon, 12 Mar 2012 12:04:25 +0100

Changed in unity-lens-files (Ubuntu):
status: Triaged → Fix Released
Revision history for this message
Jakob (jodsalz2000-launchpad) wrote :

Will tracker be used instead of locate in future? I'd need it, too, according to comment #72.

Revision history for this message
KarlRelton (karllinuxtest-relton) wrote :

Jakob

There is a trade-off between 'locate' and 'tracker'.

locate:
  + will actually find more files than other solutions, including files not in your home directory (and not owned by you)
  - Issue with encrypted home directory (?)
  - relies on a daemon which by default runs just once a day (though you could tweak this up yourself if needed)
  - searching can be slower

tracker:
  - will only find files in directories that it watches (a subset of places)
  - some worry about the background performance penalty running tracker imposes
  ++ can search on both filenames, their metadata content, and their real text content

Its that last ++ that is so attractive to me. I did actually get the prototype tracker lens of #71 compiled and running in 12.04 (does need mods to work with latest Unity API) and it works reasonably well.

Unfortunately I don't have the time myself to get it properly working and release worthy, but if someone could be persuaded to at least polish the current code it could then be installed along with Tracker for users who want it.

The next step would then be to write a combo tracker/zeitgeist lens that combines the search power of tracker with the relevancy of zeitgeist. Such a beast should be possible, and for those users who like to install Tracker would give them a top class search on their tracker-indexed directories.

Revision history for this message
Sean Fitzpatrick (sean-fitzpatrick) wrote :

I noticed an issue today related to the problem of indexing files to make them searchable in Unity: I have a laptop at work and a desktop at home that are both synced via Ubuntu One. If I decide to reorganise files on one computer, Ubuntu One will execute the same reorganisation on the other computer, but on the second one, I'll lose access to the files via the Unity Dash. (In my case I have documents for a course I'm teaching right now - tests, assignments, handouts, etc. - and again in the next semester, so I've added a date hierarchy to my folder structure.)
What's the current state of affairs? Will Unity update its file indexing, or will these files only make their way back in as I access them again from Nautilus? It would be nice to be able to move files around on synced computers without breaking the ability to search via the Unity Dash. (Now that I'm used to the dash, it feels like a huge pain to have to open up Nautilus and click my way through the folders!)

Revision history for this message
Marius B. Kotsbak (mariusko) wrote :

Sean, please open a new bug report about that issue as this one is now closed.

Changed in unity (Ubuntu):
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.