Merge lp:~ac-ies/fluidity/binary-vtk into lp:fluidity

[IES developer]

This change means that VTU files written by Fluidity have field data in a compressed, raw binary format, rather than the encoded Base64 format used currently.

This means files become smaller - roughly 66% of their previous size.

Code has been tested with Paraview and resuming from checkpoints.

This change means that VTU files written by Fluidity have field data in a compressed, raw binary format, rather than the encoded Base64 format used currently.

This means files become smaller - roughly 66% of their previous size.

Code has been tested with Paraview and resuming from checkpoints.

[Tim Greaves (tim-greaves)]

This is now green on buildbot:

 http://buildbot.ese.ic.ac.uk:8080/builders/binary-vtk/builds/0

One question Sam asked at the dev meeting: will Fluidity as it is after this change be able to pick up from checkpoints dropped before this change? I think this was answered briefly at the meeting but would be good to have it on record here.

[IES developer (ac-ies)]

Hi Tim

Excellent news.

In answer to Sam's query -- I've not tested it, but I'd be very surprised if it doesn't. I've only changed the code that writes VTK files, not the code that reads it.

Cheers
--
Dr Angus Creech

Research Fellow
Institute of Energy Systems
University of Edinburgh
Scotland

On 15 May 2013, at 18:56, Tim Greaves <email address hidden> wrote:

> This is now green on buildbot:
>
> http://buildbot.ese.ic.ac.uk:8080/builders/binary-vtk/builds/0
>
> One question Sam asked at the dev meeting: will Fluidity as it is after this change be able to pick up from checkpoints dropped before this change? I think this was answered briefly at the meeting but would be good to have it on record here.
> --
> https://code.launchpad.net/~ac-ies/fluidity/binary-vtk/+merge/164008
> You are the owner of lp:~ac-ies/fluidity/binary-vtk.
>

--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

[Tim Greaves (tim-greaves)]

Also pertinent here - Angus confirms this code was written when he was between jobs, therefore is in a position to sign off his own copyright in this instance.

[Stephan Kramer (s-kramer)]

This sounds very useful! Only (small) worry I have is that it'd be good if we could test this on intel. We've had issues with reading gmsh files and it would be quite disruptive if all of a sudden checkpointing on cx1/cx2 would break. Angus: do you have access to an intel compiler? If so, could you try to run make test with it?

[IES developer (ac-ies)]

Hi Stephan

I'm afraid I don't..

If it's any reassurance, all I'm doing is adding a few calls to the VTK libraries to switch on the binary compressed data format. I'd be very surprised if that broke under Intel.

Cheers
--
Dr Angus Creech

Research Fellow
Institute of Energy Systems
University of Edinburgh
Scotland

On 16 May 2013, at 11:10, Stephan Kramer <email address hidden> wrote:

> This sounds very useful! Only (small) worry I have is that it'd be good if we could test this on intel. We've had issues with reading gmsh files and it would be quite disruptive if all of a sudden checkpointing on cx1/cx2 would break. Angus: do you have access to an intel compiler? If so, could you try to run make test with it?
> --
> https://code.launchpad.net/~ac-ies/fluidity/binary-vtk/+merge/164008
> You are the owner of lp:~ac-ies/fluidity/binary-vtk.
>

--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

[Stephan Kramer (s-kramer)]

It's not so much the writing that I'd expect to break, but the reading back in of data (the written vtus are used in checkpoints) in another format than we'd been using before. But on second thought the reading itself also happens inside the vtk library so would not be hampered by ifort's fortran i/o brokenness. So afaic: good to go!

review: Approve

=== modified file 'libvtkfortran/vtkfortran.cpp'
--- libvtkfortran/vtkfortran.cpp	2011-04-14 15:40:24 +0000
+++ libvtkfortran/vtkfortran.cpp	2013-05-15 17:13:54 +0000
@@ -665,6 +665,11 @@
 #endif    
     writer->SetFileName( fl_vtkFileName.c_str() );
  
+    // Set to true binary format (not encoded as base 64)
+    writer->SetDataModeToAppended();
+    writer->EncodeAppendedDataOff();
+
+
     writer->SetInput(dataSet);
   
     writer->SetCompressor(compressor);
@@ -716,6 +721,11 @@
     writer->SetCompressor(compressor);
     
     compressor->Delete();
+
+    // Set to true binary format (not encoded as base 64)
+    writer->SetDataModeToAppended();
+    writer->EncodeAppendedDataOff();
+
     
     writer->Write();
     writer->Delete();