Merge lp:~abreu-alexandre/oxide/donottrack into lp:~oxide-developers/oxide/oxide.trunk

[Alexandre Abreu]

Handle do not track

Handle do not track

[lp:~abreu-alexandre/oxide/donottrack]

1126. By Chris Coulson on 2015-06-12

Bump Chromium rev to 45.0.2427.7

1127. By Chris Coulson on 2015-06-12

Persist media device permissions for the life of WebContext

1128. By Chris Coulson on 2015-06-12

Make ContentBrowserClient / ContentRendererClient memebers of ContentMainDelegate like ContentClient already is

1129. By Chris Coulson on 2015-06-12

Pass the application locale to ui::ResourceBundle::InitSharedInstanceWithLocale. Because we don't build with glib support, it's unable to detect the system locale and depends on a preferred one being passed

1130. By Chris Coulson on 2015-06-12

Convert UserScriptMaster in to a KeyedService

1131. By Chris Coulson on 2015-06-12

Create a single TestWebContext in tst_WebView_save_restore_state.qml

1132. By Chris Coulson on 2015-06-12

Don't allow QObjects to be passed to WebContextDelegateWorker

1133. By Chris Coulson on 2015-06-17

Add justifications around less-obvious uses of base::Unretained

1134. By Olivier Tilloy on 2015-06-17

Fix save/restore unit tests.

1135. By Chris Coulson on 2015-06-17

Bump Chromium rev to 45.0.2431.0

1136. By Chris Coulson on 2015-06-17

Fix an intermittent test failure

1137. By Chris Coulson on 2015-06-18

Ensure we have a WebScopedMicrotaskSuppression instance on the stack when we call in to V8

1138. By Chris Coulson on 2015-06-18

Add a basic editorconfig file - you can get editorconfig plugins for various editors from http://editorconfig.org/. This avoids having to turn on support for modelines, which is disabled in some editors (eg, vim) for security reasons

[Chris Coulson (chrisccoulson)]

Thanks, I've left some comments inline.

Also, the test is missing a few bits:

- It should verify that the default is off.
- It should check that the notify signal is emitted when you toggle it.
- What happens to window.navigator.doNotTrack if you change the setting without reloading the page? Does it update automatically? If it does, then that should be tested.

review: Needs Fixing

[lp:~abreu-alexandre/oxide/donottrack]

1139. By Chris Coulson on 2015-06-19

Some updates for the content script messaging:
- The payload is no longer converted to a string for the wire - instead, we use base::Value directly.
- As a consequence, on the renderer side, we now use content::V8ValueConverter to convert the payload from v8::Value to base::Value. This means we can get rid of the hacky script we compile for each script context, just to call JSON.stringify.
- On the browser-side, we now convert from base::Value directly to QVariant using a custom converter. There is no longer a dependency on QJsonDocument.
- As a hardening measure, our converter on the browser side protects against excessive recursion that would otherwise exhaust stack space.
- The "args" parameter for script messages is deprecated - use "payload" instead. "args" never really made sense as it implies it's always a list, and is there as a consequence of this API being put together in haste to get unit tests working.
- Message payloads no longer have to be JSON arrays or objects - they can be any primitive that can be represented in JSON.
- The script message dispatcher on the browser side is no longer a BrowserMessageFilter running on the IO thread, but instead a WebContentsObserver. We can do that now since it has an API to expose the source RenderFrameHost.

1140. By Chris Coulson on 2015-06-19

Use the preferred allow() method for the geolocation permission tests (we still retain one test for the deprecated accept() method though)

1141. By Chris Coulson on 2015-06-19

Store TestApiHost in TestWebView, keyed by WebFrame

1142. By Chris Coulson on 2015-06-22

Add TestUtils.waitFor, so that we can stop duplicating the same function for tests that don't use TestWebView

1143. By Chris Coulson on 2015-06-23

Bump depot_tools to c1ae89ecd635abfea2d94e5b49c7d92f49f28f22

1144. By Chris Coulson on 2015-06-24

Improve script messaging tests

1145. By Chris Coulson on 2015-06-24

LP: #1466208

1146. By Chris Coulson on 2015-06-25

Remove a spurious console.log

1147. By Chris Coulson on 2015-06-25

Correctly detect when we are running on Mir/Mesa

1148. By Chris Coulson on 2015-06-25

Assume that the display is already initialized

1149. By Chris Coulson on 2015-06-26

Bump Chromium rev to 45.0.2438.3

1150. By Chris Coulson on 2015-06-29

Add WebContext.userAgentOverrides API, which is a replacement for WebContext.userAgentOverrideDelegate. The new API allows applications to provide a list of regular expressions and user agent overrides as opposed to providing a JS worker that we call in to to ask for a user agent string. This allows us to stop relying on synchronous IPC to the browser

1151. By Chris Coulson on 2015-06-30

We can actually lazyily create UserAgentSettings. RenderProcessInitializer will create it when the first render process starts anyway

1152. By Alexandre Abreu on 2015-07-02

Fixes

[Alexandre Abreu (abreu-alexandre)]

All comments have been addressed

[Chris Coulson (chrisccoulson)]

review: Approve

=== modified file 'qt/core/browser/oxide_qt_web_context.cc'
--- qt/core/browser/oxide_qt_web_context.cc	2015-07-01 09:56:23 +0000
+++ qt/core/browser/oxide_qt_web_context.cc	2015-07-02 14:03:46 +0000
@@ -144,9 +144,15 @@
         session_cookie_mode(content::CookieStoreConfig::EPHEMERAL_SESSION_COOKIES),
         popup_blocker_enabled(true),
         devtools_enabled(false),
+<<<<<<< TREE
         devtools_port(-1),
         legacy_user_agent_override_enabled(false) {}
 
+=======
+        devtools_port(kDefaultDevtoolsPort),
+        legacy_user_agent_override_enabled(false),
+        do_not_track(false) {}
+>>>>>>> MERGE-SOURCE
   std::string product;
   std::string user_agent;
   base::FilePath data_path;
@@ -164,6 +170,7 @@
   std::string default_video_capture_device_id;
   std::vector<UserAgentSettings::UserAgentOverride> user_agent_overrides;
   bool legacy_user_agent_override_enabled;
+  bool do_not_track;
 };
 
 class SetCookiesContext : public base::RefCounted<SetCookiesContext> {
@@ -550,6 +557,7 @@
 
   context_->SetCookiePolicy(construct_props_->cookie_policy);
   context_->SetIsPopupBlockerEnabled(construct_props_->popup_blocker_enabled);
+  context_->SetDoNotTrack(construct_props_->do_not_track);
 
   MediaCaptureDevicesContext* dc =
       MediaCaptureDevicesContext::Get(context_.get());
@@ -1074,5 +1082,21 @@
   client_->DefaultVideoCaptureDeviceChanged();
 }
 
+bool WebContext::doNotTrack() const {
+    if (IsInitialized()) {
+    return context_->GetDoNotTrack();
+  }
+
+  return construct_props_->do_not_track;
+}
+
+void WebContext::setDoNotTrack(bool dnt) {
+  if (IsInitialized()) {
+    context_->SetDoNotTrack(dnt);
+  } else {
+    construct_props_->do_not_track = dnt;
+  }
+}
+
 } // namespace qt
 } // namespace oxide

=== modified file 'qt/core/browser/oxide_qt_web_context.h'
--- qt/core/browser/oxide_qt_web_context.h	2015-06-29 17:52:40 +0000
+++ qt/core/browser/oxide_qt_web_context.h	2015-07-02 14:03:46 +0000
@@ -148,6 +148,9 @@
   void clearTemporarySavedPermissionStatuses() override;
   void setLegacyUserAgentOverrideEnabled(bool enabled) override;
 
+  bool doNotTrack() const override;
+  void setDoNotTrack(bool dnt) override;
+
   // oxide::MediaCaptureDevicesContextClient implementation
   void DefaultAudioDeviceChanged() override;
   void DefaultVideoDeviceChanged() override;

=== modified file 'qt/core/glue/oxide_qt_web_context_proxy.h'
--- qt/core/glue/oxide_qt_web_context_proxy.h	2015-07-01 09:56:23 +0000
+++ qt/core/glue/oxide_qt_web_context_proxy.h	2015-07-02 14:03:46 +0000
@@ -138,6 +138,9 @@
   virtual void clearTemporarySavedPermissionStatuses() = 0;
 
   virtual void setLegacyUserAgentOverrideEnabled(bool enabled) = 0;
+
+  virtual bool doNotTrack() const = 0;
+  virtual void setDoNotTrack(bool dnt) = 0;
 };
 
 } // namespace qt

=== modified file 'qt/quick/api/oxideqquickwebcontext.cc'
--- qt/quick/api/oxideqquickwebcontext.cc	2015-07-01 09:56:23 +0000
+++ qt/quick/api/oxideqquickwebcontext.cc	2015-07-02 14:03:46 +0000
@@ -1119,6 +1119,7 @@
   // if the actual device is removed
 }
 
+
 QVariantList OxideQQuickWebContext::userAgentOverrides() const {
   Q_D(const OxideQQuickWebContext);
 
@@ -1133,6 +1134,12 @@
   return rv;
 }
 
+bool OxideQQuickWebContext::doNotTrack() const {
+  Q_D(const OxideQQuickWebContext);
+
+  return d->proxy()->doNotTrack();
+}
+
 void OxideQQuickWebContext::setUserAgentOverrides(
     const QVariantList& overrides) {
   Q_D(OxideQQuickWebContext);
@@ -1160,4 +1167,16 @@
   emit userAgentOverridesChanged();
 }
 
+void OxideQQuickWebContext::setDoNotTrack(bool dnt) {
+  Q_D(OxideQQuickWebContext);
+
+  if (doNotTrack() == dnt) {
+    return;
+  }
+
+  d->proxy()->setDoNotTrack(dnt);
+
+  emit doNotTrackEnabledChanged();
+}
+
 #include "moc_oxideqquickwebcontext_p.cpp"

=== modified file 'qt/quick/api/oxideqquickwebcontext_p.h'
--- qt/quick/api/oxideqquickwebcontext_p.h	2015-06-29 16:47:46 +0000
+++ qt/quick/api/oxideqquickwebcontext_p.h	2015-07-02 14:03:46 +0000
@@ -73,6 +73,8 @@
 
   Q_PROPERTY(QVariantList userAgentOverrides READ userAgentOverrides WRITE setUserAgentOverrides NOTIFY userAgentOverridesChanged REVISION 3)
 
+  Q_PROPERTY(bool doNotTrackEnabled READ doNotTrack WRITE setDoNotTrack NOTIFY doNotTrackEnabledChanged REVISION 3)
+
   Q_ENUMS(CookiePolicy)
   Q_ENUMS(SessionCookieMode)
 
@@ -169,6 +171,9 @@
   QVariantList userAgentOverrides() const;
   void setUserAgentOverrides(const QVariantList& overrides);
 
+  bool doNotTrack() const;
+  void setDoNotTrack(bool dnt);
+
  Q_SIGNALS:
   void productChanged();
   void userAgentChanged();
@@ -191,6 +196,7 @@
   Q_REVISION(3) void defaultAudioCaptureDeviceIdChanged();
   Q_REVISION(3) void defaultVideoCaptureDeviceIdChanged();
   Q_REVISION(3) void userAgentOverridesChanged();
+  Q_REVISION(3) void doNotTrackEnabledChanged();
 
  private:
   Q_PRIVATE_SLOT(d_func(), void userScriptUpdated());

=== added file 'qt/tests/qmltests/api/tst_WebContext_doNotTrack.py'
--- qt/tests/qmltests/api/tst_WebContext_doNotTrack.py	1970-01-01 00:00:00 +0000
+++ qt/tests/qmltests/api/tst_WebContext_doNotTrack.py	2015-07-02 14:03:46 +0000
@@ -0,0 +1,18 @@
+from cStringIO import StringIO
+
+def handler(request):
+  request.send_response(200)
+  request.send_header("Content-type", "text/html")
+
+  html = StringIO()
+  for header in request.headers:
+    print header
+  html.write(
+      "<html><body><div id='dnt'>{}</div></body></html>".format(
+          "1" if 'DNT' in request.headers and request.headers['DNT'] == '1' else "0"))
+
+  request.send_header("Content-Length", html.tell())
+  request.send_header("Cache-Control", "no-cache")
+  request.end_headers()
+
+  request.wfile.write(html.getvalue())

=== added file 'qt/tests/qmltests/api/tst_WebContext_doNotTrack.qml'
--- qt/tests/qmltests/api/tst_WebContext_doNotTrack.qml	1970-01-01 00:00:00 +0000
+++ qt/tests/qmltests/api/tst_WebContext_doNotTrack.qml	2015-07-02 14:03:46 +0000
@@ -0,0 +1,74 @@
+import QtQuick 2.0
+import QtTest 1.0
+import com.canonical.Oxide 1.9
+import com.canonical.Oxide.Testing 1.0
+
+TestWebView {
+  id: webView
+
+  focus: true
+
+  width: 200
+  height: 200
+
+  SignalSpy {
+    id: dntSignalSpy
+    target: webView.context
+    signalName: "doNotTrackEnabledChanged"
+  }
+
+  TestCase {
+    id: test
+    name: "WebContext_doNotTrack"
+    when: windowShown
+
+    function init() {
+      dntSignalSpy.clear();
+    }
+ 
+    function test_dnt_data() {
+      return [
+        {expectedHeaderDnt: "0", expectedNavigatorDnt: "0"}, /* default */
+        {expectedHeaderDnt: "0", expectedNavigatorDnt: "0", dnt: false},
+        {expectedHeaderDnt: "1", expectedNavigatorDnt: "1", dnt: true}
+      ]
+    }
+
+    function test_dnt(data) {
+      if (data.dnt != null) {
+        webView.context.doNotTrackEnabled = data.dnt
+      }
+
+      webView.url = "http://testsuite/tst_WebContext_doNotTrack.py";
+      verify(webView.waitForLoadSucceeded(),
+             "Timed out waiting for successful load");
+
+      compare(webView.getTestApi().evaluateCode(
+              "document.querySelector(\"#dnt\").innerHTML"),
+              data.expectedHeaderDnt);
+      compare(webView.getTestApi().evaluateCode(
+              "window.navigator.doNotTrack && window.navigator.doNotTrack === '1' ? '1' : '0'"),
+              data.expectedNavigatorDnt);
+    }
+
+    function test_dntNavigatorUpdate(data) {
+      webView.context.doNotTrackEnabled = false
+
+      webView.url = "http://testsuite/tst_WebContext_doNotTrack.py";
+      verify(webView.waitForLoadSucceeded(),
+             "Timed out waiting for successful load");
+
+      compare(webView.getTestApi().evaluateCode(
+              "window.navigator.doNotTrack && window.navigator.doNotTrack === '1' ? '1' : '0'"),
+              "0");
+
+      webView.context.doNotTrackEnabled = true;
+
+      compare(webView.getTestApi().evaluateCode(
+              "window.navigator.doNotTrack && window.navigator.doNotTrack === '1' ? '1' : '0'"),
+              "1");
+
+      compare(dntSignalSpy.count, 2, "Should have had 1 doNotTrackEnabledChanged signal");
+    }
+  }
+}

=== modified file 'shared/browser/oxide_browser_context.cc'
--- shared/browser/oxide_browser_context.cc	2015-07-01 12:25:41 +0000
+++ shared/browser/oxide_browser_context.cc	2015-07-02 14:03:46 +0000
@@ -191,7 +191,8 @@
         session_cookie_mode(params.session_cookie_mode),
         popup_blocker_enabled(true),
         host_mapping_rules(params.host_mapping_rules),
-        user_agent_settings(new UserAgentSettingsIOData(context)) {}
+        user_agent_settings(new UserAgentSettingsIOData(context)),
+        do_not_track(false) {}
 
   mutable base::Lock lock;
 
@@ -207,6 +208,8 @@
 
   scoped_ptr<UserAgentSettingsIOData> user_agent_settings;
 
+  bool do_not_track;
+
   scoped_refptr<BrowserContextDelegate> delegate;
 };
 
@@ -343,6 +346,12 @@
   return max_cache_size_hint;
 }
 
+bool BrowserContextIOData::GetDoNotTrack() const {
+  const BrowserContextSharedIOData& data = GetSharedData();
+  base::AutoLock lock(data.lock);
+  return data.do_not_track;
+}
+
 URLRequestContext* BrowserContextIOData::CreateMainRequestContext(
     content::ProtocolHandlerMap& protocol_handlers,
     content::URLRequestInterceptorScopedVector request_interceptors) {
@@ -869,10 +878,33 @@
   return io_data()->cookie_store_;
 }
 
+
 TemporarySavedPermissionContext*
 BrowserContext::GetTemporarySavedPermissionContext() const {
   DCHECK(CalledOnValidThread());
   return io_data()->GetTemporarySavedPermissionContext();
 }
 
+bool BrowserContext::GetDoNotTrack() const {
+  DCHECK(CalledOnValidThread());
+  return io_data()->GetSharedData().do_not_track;
+}
+
+void BrowserContext::SetDoNotTrack(bool dnt) {
+  DCHECK(CalledOnValidThread());
+
+  BrowserContextSharedIOData& data = io_data()->GetSharedData();
+  base::AutoLock lock(data.lock);
+  data.do_not_track = dnt;
+
+  FOR_EACH_OBSERVER(BrowserContextObserver,
+                    GetOriginalContext()->observers_,
+                    NotifyDoNotTrackChanged());
+  if (HasOffTheRecordContext()) {
+    FOR_EACH_OBSERVER(BrowserContextObserver,
+                      GetOffTheRecordContext()->observers_,
+                      NotifyDoNotTrackChanged());
+  }
+}
+
 } // namespace oxide

=== modified file 'shared/browser/oxide_browser_context.h'
--- shared/browser/oxide_browser_context.h	2015-07-01 09:56:23 +0000
+++ shared/browser/oxide_browser_context.h	2015-07-02 14:03:46 +0000
@@ -84,6 +84,8 @@
   base::FilePath GetCachePath() const;
   int GetMaxCacheSizeHint() const;
 
+  bool GetDoNotTrack() const;
+
   virtual bool IsOffTheRecord() const = 0;
 
   URLRequestContext* CreateMainRequestContext(
@@ -208,6 +210,9 @@
 
   const std::vector<std::string>& GetHostMappingRules() const;
 
+  bool GetDoNotTrack() const;
+  void SetDoNotTrack(bool dnt);
+
   // from content::BrowserContext
   content::ResourceContext* GetResourceContext() override;
 

=== modified file 'shared/browser/oxide_browser_context_observer.h'
--- shared/browser/oxide_browser_context_observer.h	2014-06-02 15:06:30 +0000
+++ shared/browser/oxide_browser_context_observer.h	2015-07-02 14:03:46 +0000
@@ -30,6 +30,8 @@
 
   virtual void NotifyPopupBlockerEnabledChanged() {}
 
+  virtual void NotifyDoNotTrackChanged() {}
+
  protected:
   BrowserContextObserver();
   BrowserContextObserver(BrowserContext* context);

=== modified file 'shared/browser/oxide_network_delegate.cc'
--- shared/browser/oxide_network_delegate.cc	2015-06-29 17:14:55 +0000
+++ shared/browser/oxide_network_delegate.cc	2015-07-02 14:03:46 +0000
@@ -26,6 +26,10 @@
 
 namespace oxide {
 
+namespace {
+const char kDoNotTrackHeaderName[] = "DNT";
+}
+
 int NetworkDelegate::OnBeforeURLRequest(
     net::URLRequest* request,
     const net::CompletionCallback& callback,
@@ -35,6 +39,12 @@
     return net::OK;
   }
 
+  bool do_not_track = context_->GetDoNotTrack();
+  if (do_not_track) {
+    request->SetExtraRequestHeaderByName(
+        kDoNotTrackHeaderName, "1", true);
+  }
+
   return delegate->OnBeforeURLRequest(request, callback, new_url);
 }
 

=== modified file 'shared/browser/oxide_web_view_contents_helper.cc'
--- shared/browser/oxide_web_view_contents_helper.cc	2015-06-03 09:51:37 +0000
+++ shared/browser/oxide_web_view_contents_helper.cc	2015-07-02 14:03:46 +0000
@@ -59,6 +59,19 @@
   UpdateWebPreferences();
 }
 
+void WebViewContentsHelper::NotifyDoNotTrackChanged() {
+  content::RendererPreferences* renderer_prefs =
+      web_contents_->GetMutableRendererPrefs();
+  renderer_prefs->enable_do_not_track = context_->GetDoNotTrack();
+
+  // Send the new override string to the renderer.
+  content::RenderViewHost* rvh = web_contents_->GetRenderViewHost();
+  if (!rvh) {
+    return;
+  }
+  rvh->SyncRendererPrefs();
+}
+
 WebViewContentsHelper::WebViewContentsHelper(content::WebContents* contents,
                                              content::WebContents* opener)
     : BrowserContextObserver(
@@ -73,6 +86,7 @@
   content::RendererPreferences* renderer_prefs =
       web_contents_->GetMutableRendererPrefs();
   renderer_prefs->browser_handles_non_local_top_level_requests = true;
+  renderer_prefs->enable_do_not_track = context_->GetDoNotTrack();
 
   content::RenderViewHost* rvh = web_contents_->GetRenderViewHost();
   if (rvh) {

=== modified file 'shared/browser/oxide_web_view_contents_helper.h'
--- shared/browser/oxide_web_view_contents_helper.h	2015-06-03 09:51:37 +0000
+++ shared/browser/oxide_web_view_contents_helper.h	2015-07-02 14:03:46 +0000
@@ -62,6 +62,7 @@
 
   // BrowserContextObserver implementation
   void NotifyPopupBlockerEnabledChanged() final;
+  void NotifyDoNotTrackChanged() final;
 
   // WebPreferencesObserver implementation
   void WebPreferencesValueChanged() final;