Merge lp:~serge-hallyn/serverguide/cgroups into lp:serverguide/trunk

[Serge Hallyn]

This is a first draft for a new section on control groups, written mainly to document the cgmanager.

[Doug Smythies (dsmythies)]

Hi Serge,
Thanks for another chunk for really great stuff.
I am not a subject matter expert on this, and defer to you on that part of it.

In lines 37, 38, and 39 in the below diff, the word "Section" needs to be deleted. Why? Because the PDF compile automatically puts it in so it then reads as "Section Section 1, “Cgroups overview” [p. 345] will describe cgroups" (for example). In my opinion, the HTML reads fine without the word "Section", as the HTML doesn't really have that concept anyhow (at least under the "new" theme). I can make this change if you are busy.

For the Resources item 5: Do we really want to refer to kernel V3.14- rc2? Is there not some more generic way to make the link?

Peter M: This addition will juggle the wiki page numbering. I'll fix it, but perhaps not right away.

review: Needs Fixing

[Doug Smythies (dsmythies)]

For the link, I think this is more generic:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/cgroups

[lp:~serge-hallyn/serverguide/cgroups]

193. By Serge Hallyn on 2014-03-07

use more general link for the kernel cgroup doc page

[Serge Hallyn (serge-hallyn)]

Quoting Doug Smythies (<email address hidden>):
> For the link, I think this is more generic:
>
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/cgroups

Good point! updated that, thanks. (haven't addressed the rest of the
feedback yet)

[Doug Smythies (dsmythies)]

Serge, Thanks.
I'll make the other change.

review: Approve

[Doug Smythies (dsmythies)]

Oh Crap: I forgot that for consistency throughout the Serverguide, we don't name sections within chapters with the chapter name. I'm saying that, for example, the title for Chapter 21 Section 1 should be "Overview" instead of "Cgroups Overview". I'll fix it.

[Serge Hallyn (serge-hallyn)]

Quoting Doug Smythies (<email address hidden>):
> Oh Crap: I forgot that for consistency throughout the Serverguide, we don't name sections within chapters with the chapter name. I'm saying that, for example, the title for Chapter 21 Section 1 should be "Overview" instead of "Cgroups Overview". I'll fix it.

Oh, thanks! Sorry I had every intention of making the other changes on
Monday.

=== added file 'serverguide/C/cgroups.xml'
--- serverguide/C/cgroups.xml	1970-01-01 00:00:00 +0000
+++ serverguide/C/cgroups.xml	2014-03-07 23:59:35 +0000
@@ -0,0 +1,242 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN"
+"http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd" [
+<!ENTITY % globalent SYSTEM "../../libs/global.ent">
+%globalent;
+<!ENTITY % xinclude SYSTEM "../../libs/xinclude.mod">
+%xinclude;
+<!ENTITY language "&EnglishAmerican;">
+]>
+<chapter id="cgroups" status="review">
+  <title>Control Groups</title>
+
+  <para>
+Control groups (cgroups) are a kernel mechanism for grouping, tracking,
+and limiting the resource usage of tasks.  The kernel-provided administration
+interface is through a virtual filesystem.  Higher level cgroup
+administration tools have been developed, including libcgroup and
+lmctfy.  Additionally, there is guidance at freedesktop.org
+for how applications can best cooperate using the cgroup filesystem
+interface (see Resources).
+  </para>
+
+  <para>
+As of Ubuntu 14.04, the cgroup manager (cgmanager) is available as
+another cgroup administion interface.  It's goal is to respond to dbus
+requests from any user, allowing him to administer only those cgroups
+which have been delegated to him.
+  </para>
+
+  <para>
+Section <xref linkend="cgroups-overview"/> will describe cgroups in more detail.
+Section <xref linkend="cgroups-fs"/> will describe the long-standing cgroups filesystem
+interface.  Section <xref linkend="cgroups-manager"/> will describe the cgroup
+manager.
+  </para>
+
+  <sect1 id="cgroups-overview" status="review">
+    <title>Cgroups overview</title>
+
+  <para>
+Cgroups are the generalized feature for grouping tasks.  The actual
+resource tracking and limits are implemented by subsystems.  A
+hierarchy is a set of subsystems mounted together.  For instance,
+if the memory and devices subsystems are mounted together under
+/sys/fs/cgroups/set1, then any task which is in "/child1" will
+be subject to the corresponding limits of both subsystems.
+  </para>
+
+  <para>
+Each set of mounted subsystems consittutes a 'hierarchy'.  With
+exceptions, cgroups which are children of "/child1" will be
+subject to all limits placed on "/child1", and their resource
+usage will be accounted to "/child1".
+  </para>
+
+  <para>
+The existing subsystems include:
+  </para>
+
+  <itemizedlist>
+<listitem><para><emphasis>cpusets</emphasis>: fascilitate assigning a set of
+CPUS and memory nodes to cgroups.
+  Tasks in a cpuset cgroup may only be scheduled on CPUS assigned to that
+  cpuset.</para></listitem>
+<listitem><para><emphasis> blkio </emphasis>: limits per-cgroup block io.</para></listitem>
+<listitem><para><emphasis> cpuacct </emphasis>: provides per-cgroup cpu usage accounting.</para></listitem>
+<listitem><para><emphasis> devices </emphasis>: controls the ability of tasks to create or use devices nodes
+  using either a blacklist or whitelist.</para></listitem>
+<listitem><para><emphasis> freezer </emphasis>: provides a way to 'freeze' and 'thaw' whole cgroups.  Tasks
+  in the cgroup will not be scheduled while they are frozen.</para></listitem>
+<listitem><para><emphasis> hugetlb </emphasis>: fascilitates limiting hugetlb usage per cgroup.</para></listitem>
+<listitem><para><emphasis> memory </emphasis>: allows memory, kernel memory, and swap usage to be tracked
+  and limited.</para></listitem>
+<listitem><para><emphasis> net_cls </emphasis>: provides an interface for tagging packets based on the
+  sender cgroup.  These tags can then be used by tc (traffic controller)
+  to assign priorities.</para></listitem>
+<listitem><para><emphasis> net_prio </emphasis>: allows setting network traffic priority on a per-cgroup
+  basis.</para></listitem>
+<listitem><para><emphasis> cpu </emphasis>: enables setting of scheduling preferences on per-cgroup basis.</para></listitem>
+<listitem><para><emphasis> perf_event </emphasis>: enables per-cpu mode to monitor only threads in certain
+  cgroups.</para></listitem>
+  </itemizedlist>
+
+  <para>
+In addition, named cgroups can be created with no bound
+subsystems for the sake of process tracking.  As an example,
+systemd does this to track services and user sessions.
+  </para>
+
+  </sect1>
+
+  <sect1 id="cgroups-fs" status="review">
+    <title>Cgroup filesystem</title>
+
+  <para>
+A hierarchy is created by mounting an instance of the cgroup filesystem
+with each of the desired subsystems listed as a mount option.  For instance,
+  </para>
+
+<screen><command>
+mount -t cgroup -o devices,memory,freezer cgroup /cgroup1
+</command></screen>
+
+  <para>
+would instantiate a hierarchy with the devices and memory cgroups comounted.
+A child cgroup "child1" can be created using 'mkdir'
+  </para>
+
+<screen><command>
+mkdir /cgroup1/child1
+</command></screen>
+
+  <para>
+and tasks can be moved into the new child cgroup by writing their process
+ids into the 'tasks' or 'cgroup.procs' file:
+  </para>
+
+<screen><command>
+sleep 100
+echo $! > /cgroup1/child1/cgroup.procs
+</command></screen>
+
+  <para>
+Other administration is done through files in the cgroup directories.  For
+instance, to freeze all tasks in child1,
+  </para>
+
+<screen><command>
+echo FROZEN > /cgroup1/child1/freezer.state
+</command></screen>
+
+  <para>
+A great deal of information about cgroups and its subsystems can be found
+under the cgroups documentation directory in the kernel source tree (see
+Resources).
+  </para>
+
+  </sect1>
+
+  <sect1 id="cgroups-delegation" status="review">
+    <title>Cgroups Delegation</title>
+
+  <para>
+Cgroup files and directories can be owned by non-root users, enabling
+delegation of cgroup administration.  In general, the kernel enforces
+the hierarchical constraints on limits, so that for instance if
+devices cgroup <filename>/child1</filename> cannot access a disk drive, then
+<filename>/child1/child2</filename> cannot give itself those rights.
+  </para>
+
+  <para>
+As of Ubuntu 14.04, users are automatically placed in a set of cgroups
+which they own, safely allowing them to contrain their own jobs using child
+cgroups.  This feature is relied upon, for instance, for unprivileged
+container creation in lxc.
+  </para>
+
+  </sect1>
+
+  <sect1 id="cgroups-manager" status="review">
+    <title>Cgroup Manager</title>
+
+  <para>
+The cgroup manager (cgmanager) provides a D-Bus service allowing
+programs and users to administer cgroups without needing direct
+knowledge of or access to the cgroup filesystem.  For requests
+from tasks in the same namespaces as the manager, the manager can
+directly perform the needed security checks to ensure that requests
+are legitimate.  For other requests - such as those from a task in
+a container - enhanced D-Bus requests must be made, where process-,
+user- and group-ids are passed as SCM_CREDENTIALS, so that the kernel
+maps the identifiers to their global host values.
+  </para>
+
+  <para>
+To fascilitate the use of simple D-Bus calls from all users, a
+'cgroup manager proxy' (cgproxy) is automatically started when in
+a container.  The proxy accepts standard D-Bus requests from tasks
+in the same namespaces as itself, and converts them to 
+SCM-enhanced D-Bus requests which it passes on to the cgmanager.
+  </para>
+
+  <para>
+A simple example of creating a new cgroup in which to run a
+cpu-intensive compile would look like:
+  </para>
+
+<screen><command>
+dbus-send --print-reply --address=unix:path=/sys/fs/cgroup/cgmanager/sock \
+    --type=method_call /org/linuxcontainers/cgmanager \
+    org.linuxcontainers.cgmanager0_0.Create string:'cpuset' string:"build1"
+dbus-send --print-reply --address=unix:path=/sys/fs/cgroup/cgmanager/sock \
+	--type=method_call /org/linuxcontainers/cgmanager \
+	org.linuxcontainers.cgmanager0_0.MovePid string:'cpuset' \
+	string:"build1" int32:$$
+dbus-send --print-reply --address=unix:path=/sys/fs/cgroup/cgmanager/sock \
+	--type=method_call /org/linuxcontainers/cgmanager \
+	org.linuxcontainers.cgmanager0_0.SetValue string:'cpuset' \
+	string:"build1" string:"cpuset.cpus" string:"1"
+make
+</command></screen>
+
+  <para>
+The above can also be done much more simply by using lmctfy or
+cgroup-bin, once they are converted to use the cgmanager.
+  </para>
+
+  </sect1>
+
+    <sect1 id="cgroups-resources" status="review">
+      <title>Resources</title>
+
+      <itemizedlist>
+        <listitem>
+          <para>Manual pages referenced above can be found at:</para>
+          <screen>
+<ulink url="http://manpages.ubuntu.com/manpages/en/man5/cgconfig.conf.5.html">cgconfig.conf</ulink>
+<ulink url="http://manpages.ubuntu.com/manpages/en/man8/cgmanager.8.html">cgmanager</ulink>
+<ulink url="http://manpages.ubuntu.com/manpages/en/man8/cgproxy.8.html">cgproxy</ulink>
+</screen>
+        </listitem>
+
+        <listitem>
+          <para>The upstream cgmanager project is hosted at <ulink
+          url="http://cgmanager.linuxcontainers.org">linuxcontainers.org</ulink>.</para>
+        </listitem>
+
+        <listitem>
+          <para>The upstream kernel documentation page on cgroups can be seen <ulink
+	  url="https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/cgroups">here
+	  </ulink>.</para>
+        </listitem>
+
+        <listitem>
+          <para>The freedesktop.org control group usage guidelines can be seen <ulink
+	  url="http://www.freedesktop.org/wiki/Software/systemd/PaxControlGroups/">here</ulink>.</para>
+        </listitem>
+
+      </itemizedlist>
+    </sect1>
+
+</chapter>
 
=== modified file 'serverguide/C/serverguide.xml'
--- serverguide/C/serverguide.xml	2013-02-09 00:40:31 +0000
+++ serverguide/C/serverguide.xml	2014-03-07 23:59:35 +0000
@@ -41,6 +41,7 @@
   <xi:include href="samba.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
   <xi:include href="backups.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
   <xi:include href="virtualization.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
+  <xi:include href="cgroups.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
   <xi:include href="clustering.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
   <xi:include href="vpn.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
   <xi:include href="other-apps.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>