add NX 842 hw compression patches

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1454687/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

Waiting on these to land in Linus' tree.

------- Comment From <email address hidden> 2015-06-23 11:02 EDT-------
Patches are in Linus' tree:

ed70b47 crypto: nx - add hardware 842 crypto comp alg
b8e04187 crypto: nx - simplify pSeries nx842 driver
99182a42 crypto: nx - add PowerNV platform NX-842 driver
959e665 crypto: nx - add nx842 constraints
7011a12 crypto: nx - add NX-842 platform frontend driver
fdd05e4 crypto: nx - rename nx-842.c to nx-842-pseries.c
2062c5b crypto: 842 - change 842 alg to use software
2da572c lib: add software 842 compression/decompression
edc424f powerpc: Add ICSWX instruction
b130e7c powerpc: export of_get_ibm_chip_id function

additionally, these patches further improve NX-842 and are also in Linus' tree:

c47d630 crypto: nx - add LE support to pSeries platform driver
2c6f6ea crypto: nx - replace NX842_MEM_COMPRESS with function
32be6d3 crypto: nx - move include/linux/nx842.h into drivers/crypto/nx/nx-842.h
3154de7 crypto: nx - fix nx-842 pSeries driver minimum buffer size
3e648cb crypto: nx - prevent nx 842 load if no hw driver
9358eac crypto: nx - remove 842-nx null checks
ca7fc7e lib: correct 842 decompress for 32 bit
f7ead7b lib: make lib/842 decompress functions static

------- Comment From <email address hidden> 2015-06-23 16:17 EDT-------
Also, to enable NX-842 and zswap (which is the primary user of NX-842 in our case), please set:

CONFIG_CRYPTO_DEV_NX=y
CONFIG_CRYPTO_DEV_NX_COMPRESS=y
CONFIG_CRYPTO_DEV_NX_COMPRESS_POWERNV=y
CONFIG_CRYPTO_DEV_NX_COMPRESS_PSERIES=y
CONFIG_CRYPTO_DEV_NX_COMPRESS_CRYPTO=y
CONFIG_CRYPTO_ALGAPI=y
CONFIG_842_DECOMPRESS=y
CONFIG_ZSWAP=y
CONFIG_ZPOOL=y
CONFIG_ZBUD=y
CONFIG_ZSMALLOC=y

All patches cleanly cherry-pick into 3.19.

The following config options are already set in Ubuntu:

CONFIG_ZSWAP=y
CONFIG_ZPOOL=y
CONFIG_ZBUD=y
CONFIG_ZSMALLOC=y
CONFIG_CRYPTO_ALGAPI=y
CONFIG_CRYPTO_DEV_NX=y
CONFIG_CRYPTO_DEV_NX_COMPRESS=m

I'll add the following:
+CONFIG_842_COMPRESS=m
+CONFIG_842_DECOMPRESS=m
+CONFIG_CRYPTO_DEV_NX_COMPRESS_CRYPTO=m
+CONFIG_CRYPTO_DEV_NX_COMPRESS_POWERNV=m
+CONFIG_CRYPTO_DEV_NX_COMPRESS_PSERIES=m

The only changes are build these as modules, and add 842_COMPRESS.

Does this work?

Thanks,

------- Comment From <email address hidden> 2015-06-25 13:13 EDT-------
(In reply to comment #11)
> All patches cleanly cherry-pick into 3.19.
>
> The following config options are already set in Ubuntu:
>
> CONFIG_ZSWAP=y
> CONFIG_ZPOOL=y
> CONFIG_ZBUD=y
> CONFIG_ZSMALLOC=y
> CONFIG_CRYPTO_ALGAPI=y
> CONFIG_CRYPTO_DEV_NX=y
> CONFIG_CRYPTO_DEV_NX_COMPRESS=m
>
> I'll add the following:
> +CONFIG_842_COMPRESS=m
> +CONFIG_842_DECOMPRESS=m
> +CONFIG_CRYPTO_DEV_NX_COMPRESS_CRYPTO=m
> +CONFIG_CRYPTO_DEV_NX_COMPRESS_POWERNV=m
> +CONFIG_CRYPTO_DEV_NX_COMPRESS_PSERIES=m
>
> The only changes are build these as modules, and add 842_COMPRESS.
>
> Does this work?

yep that's fine.

The downside of them being modules is that to use NX 842 with zswap - which requires configuration to all be done at boot time (currently, I have patches pending to change that) - the initramfs has to be rebuilt with the NX 842 modules. But that's not a big deal, just creating a dracut.d conf file and rerunning dracut.

In Ubuntu we use initramfs-tools to update the initramfs. In this package there is a 'auto_add_modules' hook-function. So we can add the modules here. Can you give me a specific list of which modules are required?

------- Comment From <email address hidden> 2015-06-25 18:38 EDT-------
(In reply to comment #13)
> In Ubuntu we use initramfs-tools to update the initramfs.

ah ok.

> In this package
> there is a 'auto_add_modules' hook-function. So we can add the modules
> here. Can you give me a specific list of which modules are required?

Assuming it's ok that the modules won't always be present - i.e. they're only built on powerpc - the modules will be:

nx-compress
nx-compress-crypto
nx-compress-platform
nx-compress-pseries
nx-compress-powernv
842-decompress

the 842-compress module actually isn't needed; the 842-compress and 842-decompress modules are only the "reference implementation" in software, and due to the specifics of the 842 format they are very painfully slow; 842 is really only suited to be done in custom hardware.

The 842-decompress module is needed, because the 842 hardware driver falls back to using it if the hardware fails or is disabled, so that decompression never fails.

but including the 842-compress module doesn't hurt anything.

You can review my pull request here:
git://kernel.ubuntu.com/arges/ubuntu-vivid.git lp1454687

Working on patches for initramfs.

Attached is a patch to initramfs for wily. Once the kernel patches are in I'll build a test package for ppc64el.
Is there an easy way to test functionality? And can I do this on a power8/powernv system?

This bug was fixed in the package initramfs-tools - 0.103ubuntu16

---------------
initramfs-tools (0.103ubuntu16) wily; urgency=medium

  * hook-functions: Add nx-* and 842-* modules in order to enable PowerPC NX
    Crypto Coprocessor. (LP: #1454687)

 -- Chris J Arges <email address hidden> Thu, 25 Jun 2015 13:59:55 -0500

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-vivid' to 'verification-done-vivid'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug was fixed in the package linux - 3.19.0-23.24

---------------
linux (3.19.0-23.24) vivid; urgency=low

  [ Luis Henriques ]

  * Release Tracking Bug
    - LP: #1472346

  [ Chris J Arges ]

  * SAUCE: Don't use atomic read in evlist.c
    - LP: #1410673

linux (3.19.0-23.23) vivid; urgency=low

  [ Brad Figg ]

  * Release Tracking Bug
    - LP: #1472048

  [ Chris J Arges ]

  * [Config] Add CRYPTO_DEV_NX_*, 842_* as modules
    - LP: #1454687

  [ Lu, Han ]

  * SAUCE: i915_bpo: drm/i915/audio: add codec wakeup override
    enabled/disable callback
    - LP: #1460674

  [ Timo Aaltonen ]

  * SAUCE: Backport I915_OVERLAY_DISABLE_DEST_COLORKEY
    - LP: #1460674
  * SAUCE: i915_bpo: Rebase to drm-intel-next-fixes-2015-05-29
    - LP: #1460674
  * SAUCE: i915_bpo: Revert "drm/i915: Implement the intel_dp_autotest_edid
    function for DP EDID complaince tests"
    - LP: #1460674
  * SAUCE: i915_bpo: Revert "drm/i915: Add debugfs test control files for
    Displayport compliance testing"
    - LP: #1460674
  * SAUCE: Load i915_bpo from the hda driver on SKL/CHV
    - LP: #1460674
  * SAUCE: i915_bpo: Don't try to support BXT
    - LP: #1460674
  * SAUCE: i915_bpo: drm/i915/skl: Fix DMC API version.

  [ Upstream Kernel Changes ]

  * Revert "usb: dwc2: add bus suspend/resume for dwc2"
    - LP: #1471252
  * Revert "HID: logitech-hidpp: support combo keyboard touchpad TK820"
    - LP: #1471252
  * Revert "KVM: x86: drop fpu_activate hook"
    - LP: #1471252
  * Revert "libceph: clear r_req_lru_item in __unregister_linger_request()"
    - LP: #1471252
  * drm/i915: add component support
    - LP: #1460661
  * ALSA: hda: export struct hda_intel
    - LP: #1460661
  * ALSA: hda: pass intel_hda to all i915 interface functions
    - LP: #1460661
  * ALSA: hda: add component support
    - LP: #1460661
  * drm/atomic-helpers: Fix documentation typos and wrong copy&paste
    - LP: #1460674
  * drm/atomic: Rename drm_atomic_helper_commit_pre_planes() state argument
    - LP: #1460674
  * drm/atomic-helper: Rename commmit_post/pre_planes
    - LP: #1460674
  * drm/atomic-helpers: make mode_set hooks optional
    - LP: #1460674
  * drm/atomic-helper: Fix kerneldoc for prepare_planes
    - LP: #1460674
  * drm: Complete moving rotation property to core
    - LP: #1460674
  * drm: Share plane pixel format check code between legacy and atomic
    - LP: #1460674
  * drm/atomic: Constify a bunch of functions pointer structs
    - LP: #1460674
  * drm: Fix some typo mistake of the annotations
    - LP: #1460674
  * drm: change connector to tmp_connector
    - LP: #1460674
  * drm: atomic: Expose CRTC active property
    - LP: #1460674
  * drm: atomic: Allow setting CRTC active property
    - LP: #1460674
  * drm/atomic-helpers: Properly avoid full modeset dance
    - LP: #1460674
  * drm/atomic: Add helpers for state-subclassing drivers
    - LP: #1460674
  * drm: Fix some typos
    - LP: #1460674
  * drm/atomic: Add for_each_{connector,crtc,plane}_in_state helper macros
    - LP: #1460674
  * drm/atomic-helper: Don't call atomic_update_plane when it stays off
    - LP: #1460674
  * drm/atomic-helper: Really recover pre-atomic plane/cursor behavior
 ...

------- Comment From <email address hidden> 2015-07-23 16:25 EDT-------
There are actually some more NX 842 patches that should be in the 4.3 kernel. They're currently all in Herbert's tree:
git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git

In the usual descending order:

03952d9 crypto: nx - make platform drivers directly register with crypto
174d66d crypto: nx - rename nx-842-crypto.c to nx-842.c
d31581a crypto: nx - merge nx-compress and nx-compress-crypto
20fc311 crypto: nx - use common code for both NX decompress success cases
ee781b7 crypto: nx - don't register pSeries driver if ENODEV
7f6e3aa crypto: nx - move kzalloc() out of spinlock
90fd73f crypto: nx - remove pSeries NX 'status' field
039af96 crypto: nx - remove __init/__exit from VIO functions
23ad69a crypto: nx/842 - Fix context corruption
2b93f7e crypto: nx - reduce chattiness of platform drivers
7abd75b crypto: nx - do not emit extra output if status is disabled
ec13bcb crypto: nx - rename nx842_{init, exit} to nx842_pseries_{init, exit}
fa9a9a0 crypto: nx - nx842_OF_upd_status should return ENODEV if device is not 'okay'

Should I open a new bug for these additional patches, or can you add them using this bug?

------- Comment From <email address hidden> 2015-07-29 20:03 EDT-------
> Should I open a new bug for these additional patches, or can you add them
> using this bug?
Please reopen a new bug for this request, since the current bug is closed on Canonical side and this one should be closed also.

------- Comment From <email address hidden> 2015-08-27 13:22 EDT-------
> Haren, you may want to follow up with a new Ubuntu bug with the patches from
> comment 21.
Haren, did you opened a new feature for the inclusion of the other patches?

------- Comment From <email address hidden> 2015-08-27 15:41 EDT-------
Hello. Haren has opened it and it was mirrored to https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1488495