lp:~robru/friends/lp_1135038

Created by Robert Bruce Park and last modified
Get this branch:
bzr branch lp:~robru/friends/lp_1135038
Only Robert Bruce Park can upload to this branch. If you are Robert Bruce Park please log in for upload directions.

Branch merges

Related bugs

Related blueprints

Branch information

Owner:
Robert Bruce Park
Project:
Friends
Status:
Merged

Recent revisions

168. By Robert Bruce Park

Don't raise an exception for userActionFinished error 10 (LP: #1135038)

167. By Robert Bruce Park

[ Robert Bruce Park ]
* Automatic snapshot from revision 145 (bootstrap)
* Friends re-publishing and re-notifying about stale tweets. (LP:
  #1152417)
* The dbus-invocation stuff that just landed broke Twitter's rate
  limiter. (LP: #1118878)
[ Ubuntu daily release ]
* Automatic snapshot from revision 160

166. By Robert Bruce Park

* Stop deduplicating messages across protocols, simplifying model schema.
* Add schema columns for latitude, longitude, and location name.
* Fix 'likes' column from gdouble to guint64.
* Add geotagging support from foursquare.

165. By Robert Bruce Park

Add a location name string to the model schema, because that just makes sense.

164. By Robert Bruce Park

Add lat/long support to Foursquare. That was easy!

163. By Robert Bruce Park

While we're at it, why are likes being stored as a double?

That doesn't even make sense. Go home Gwibber, you're drunk.

162. By Robert Bruce Park

Add latitude and longitude columns to the schema.

Because schema changes are so painful that we should do them all at once.

161. By Robert Bruce Park

Give up on the idea of deduplication across protocols.

We used to have a *ton* of corner case logic that attempted to merge
messages that had the same content, but were from distinct social
networks. That made some sense back in the old gwibber days but it's
becoming an increasingly irrelevant corner case that is needlessly
complicating a lot of our application logic.

This commit removes that logic, and also greatly simplifies the model
schema accordingly, allowing for much easier filtering and sorting of
the model by it's consumers (such as qml-friends).

There's a big impact on the test suites, as this changes *huge*
portions of our fundamental architectural assumptions. In the process
of updating it, I also discovered that Base._unpublish_all was
poorly/indirectly tested, so I wrote some better, explicit tests for it.

160. By Robert Bruce Park

    Start using since_id= on Twitter API requests. (LP: #1152417)

    This was accomplished by implementing two new classes, and ended up
    simplifying some of the RateLimiter code as a side effect.

    The first new class is called JsonCache. It is a subclass of dict,
    which attempts to populate it's initial state by reading in a json
    text file at a configurable location, and also adds a new "write()"
    method that dumps the json back out to the same location. This class
    was a generalization of what we were already doing inside the
    RateLimiter, so it should not be considered a "new feature" if we are
    going to butt heads with today's feature freeze.

    The second new class is a subclass of JsonCache, which enforces that:

    A) keys may not contain slashes, to avoid it getting polluted with
    every search term ever searched for, or every message that's ever been
    replied to ever, it only observes the values of the "main" streams,
    such as "messages", "mentions" and "private", although those values
    are not hardcoded so it's flexible to adapt to new streams in the
    future.

    B) values must be ints (tweet_ids), and values can only be
    incremented. This is so that we can easily just throw every observed
    tweet_id into the cache, and it only records the largest (newest) one.

    The end result is that we now have two new files located at
    ~/.cache/friends/twitter_ids.json and
    ~/.cache/friends/identica_ids.json which track the newest tweet_id
    that we have ever seen for each of the streams that we publish to.
    These values are then consulted to form the since_id= argument to
    several of Twitter's API endpoints, which solves bug #1152417.

    As an added bonus, this also greatly reduces our network usage because
    we are no longer redownloading duplicate messages over and over, so if
    there are no new messages, Twitter is now returning an empty list of
    Tweets rather than a large list of stale tweets.

    This commit includes full test coverage for all new code. Fixes: https://bugs.launchpad.net/bugs/1152417.

Approved by Ken VanDine, PS Jenkins bot.

159. By Ken VanDine

set back to UNRELEASED

Branch metadata

Branch format:
Branch format 7
Repository format:
Bazaar repository format 2a (needs bzr 1.16 or later)
Stacked on:
lp:friends
This branch contains Public information 
Everyone can see this information.