Bug #1023329 “cdo does not do what it is supposed to do” : Bugs : cdo package : Ubuntu

Revision history for this message

Javier López (javier-lopez) wrote on 2012-11-11:

#1

Thank you for taking the time to report this bug and helping to make Ubuntu better. Please answer this question:

* What specific steps should we take to recreate this bug?

I've read both reports (this and the one you made in upstream) and still don't know what specific steps I need to do to recreate it, could you provide the so called 'GRIB 1 datafile'?

I've seen that you used:

$ cdo sellonlatbox,23,31,-25,-31 t2m_ei_1979.grb tagliato_t2m_ei_1979.grb

To test cdo, could you provide both the wrong and right output?, or any other hint to detect that the program has run successfully?

The latest version in Ubuntu of the cdo program is 1.5.6.1+dfsg.1-2ubuntu2, and in other ubuntu releases:

       cdo | 1.5.3.dfsg.1-2 precise
       cdo | 1.5.4+dfsg.1-4build1 quantal
       cdo | 1.5.6.1+dfsg.1-2ubuntu2 raring

So there is a good change that in the latest Ubuntu release it works as expected, if that's the case (we'll know when we can verify the error) we'll need to cherry pick the solution to add it to the Ubuntu precise version (SRU [0])

Again, thanks for your time invested in Ubuntu, have a nice day.

[0] https://wiki.ubuntu.com/StableReleaseUpdates

---
Ubuntu Bug Squad volunteer triager
http://wiki.ubuntu.com/BugSquad

Revision history for this message

Benjamin Kerensa (bkerensa) wrote on 2012-11-12:

#2

Marked incomplete pending more information from Reporter.

Changed in cdo (Ubuntu):
status:	New → Incomplete

Revision history for this message

marcofgalli (marcofgalli) wrote on 2012-11-12:

#3

Download full text (8.8 KiB)

Dear Javier, thank you for the tests on this bug.

I will post hereafter a "short and quick" way to reproduce the bug, and a more detailed one. First of all, in both cases, it is necessary to get the input example data file to process. This is here:

https://code.zmaw.de/attachments/3103/t2m_ei_1979.grb

In order to let the bug come out, change the working directory to the one where you downloared t2m_ei_1979.grb and type:

cdo sellonlatbox,23,31,-25,-31 t2m_ei_1979.grb tagliato_t2m_ei_1979.grb

This command produces, in the same working directory, the file tagliato_t2m_ei_1979.grb, which has the wrong indication of the time in the metadata section of the data it contains. The only above command enlights the misfunctioning.

In order to understand why this behaviour of cdo is wrong, I report here more details about the problem I wanted to solve, what I expected and what I got in turn.

The input file, t2m_ei_1979.grb, contains data in grib 1 format (see, for instance, http://en.wikipedia.org/wiki/GRIB and http://www.wmo.int/pages/prog/www/WDM/Guides/Guide-binary-2.html). I think that the difference between grib 1 and grib 2 data formats are not relevant in this context, however I'm 100% sure that the data file used here is in grib 1 (for further details, ask). The data refer to the year 1979, and they are all of two metres temperature on a spatial latitude - longitude grid over all the state of South Africa and some surrounding areas (lat -20.00 to -40.00 by 0.25, long 15.00 to 35.00 by 0.25).

They are structured as follow: at 00 and at 12 UTC of each day of the year, a forecast of two metre temperature is initialised (this procedure is called analysis, a jargon useful to interpret how cdo misfunctions); after each analysis, a forecast of two metres temperature is performed, and the data are written in the grib file every three hours up to twelve hours for each day and initialisation. Only the forecasts, and not the initial values, are reported in the grib file (so, in the original file t2m_ei_1979.grb, for each day in 1979, contains the forecasts valid at 03, 06, 09 and 12 after the initialisation at 00, and the forecasts valid at 15, 18, 21 and 24 after the initialisation at 12). It is important to underline that all this time information is contained in a special descriptive section within the grib data file itself, written, basically, as the initialisation time (called analysis time, accordingly), and forecast time (to be added up to the analysis time to get the validity time).

I typed the command

cdo sellonlatbox,23,31,-25,-31 t2m_ei_1979.grb tagliato_t2m_ei_1979.grb

because I needed to work on a smaller area than the original one. As referred in the appropriate section of the cdo user guide,

https://code.zmaw.de/embedded/cdo/1.5.5/cdo.html#x1-1070002.3.3

the command I reported above means to cut spatially the input datafile (t2m_ei_1979.grb) on a smaller area and to put the results in the output data file (tagliato_t2m_ei_1979.grb ; "tagliato" is the Italian for "it has been cut"), leaving the input untouched.

The trusted and well assessed utility command wgrib is an alternative tool to get some information for grib 1 files....

Dear Javier, thank you for the tests on this bug.

I will post hereafter a "short and quick" way to reproduce the bug, and a more detailed one. First of all, in both cases, it is necessary to get the input example data file to process. This is here:

https://code.zmaw.de/attachments/3103/t2m_ei_1979.grb

In order to let the bug come out, change the working directory to the one where you downloared t2m_ei_1979.grb and type:

cdo sellonlatbox,23,31,-25,-31 t2m_ei_1979.grb tagliato_t2m_ei_1979.grb

This command produces, in the same working directory, the file tagliato_t2m_ei_1979.grb, which has the wrong indication of the time in the metadata section of the data it contains. The only above command enlights the misfunctioning.

In order to understand why this behaviour of cdo is wrong, I report here more details about the problem I wanted to solve, what I expected and what I got in turn.

The input file, t2m_ei_1979.grb, contains data in grib 1 format (see, for instance, http://en.wikipedia.org/wiki/GRIB   and   http://www.wmo.int/pages/prog/www/WDM/Guides/Guide-binary-2.html). I think that the difference between grib 1 and grib 2 data formats are not relevant in this context, however I'm 100% sure that the data file used here is in grib 1 (for further details, ask). The data refer to the year 1979, and they are all of two metres temperature on a spatial latitude - longitude grid over all the state of South Africa and some surrounding areas (lat  -20.00 to -40.00 by 0.25, long 15.00 to 35.00 by 0.25).

They are structured as follow: at 00 and at 12 UTC of each day of the year, a forecast of two metre temperature is initialised (this procedure is called analysis, a jargon useful to interpret how cdo misfunctions); after each analysis, a forecast of two metres temperature is performed, and the data are written in the grib file every three hours up to twelve hours for each day and initialisation. Only the forecasts, and not the initial values, are reported in the grib file (so, in the original file t2m_ei_1979.grb, for each day in 1979, contains the forecasts valid at 03, 06, 09 and 12 after the initialisation at 00, and the forecasts valid at 15, 18, 21 and 24 after the initialisation at 12). It is important to underline that all this time information is contained in a special descriptive section within the grib data file itself, written, basically, as the initialisation time (called analysis time, accordingly), and forecast time (to be added up to the analysis time to get the validity time).

I typed the command

cdo sellonlatbox,23,31,-25,-31 t2m_ei_1979.grb tagliato_t2m_ei_1979.grb

because I needed to work on a smaller area than the original one. As referred in the appropriate section of the cdo user guide,

https://code.zmaw.de/embedded/cdo/1.5.5/cdo.html#x1-1070002.3.3

the command I reported above means to cut spatially the input datafile (t2m_ei_1979.grb) on a smaller area and to put the results in the output data file (tagliato_t2m_ei_1979.grb ; "tagliato" is the Italian for "it has been cut"), leaving the input untouched.

The trusted and well assessed utility command wgrib is an alternative tool to get some information for grib 1 files. It is available in ubuntu within the grads package, or it is possible to download it from the developers' source:

ftp://ftp.cpc.ncep.noaa.gov/wd51we/wgrib/wgrib.c

and compile it just with gcc, i. e.

gcc -o wgrib wgrib.c

There are many ways to assess that wgrib does very well what it is supposed to do, so I think that this utility should not be investigated.

Without entering in the details of the usage of wgrib, it is important to know that the command (select only the first 8 lines, corresponding to the first day, rather than having all the output for 8 data per day per one year)

wgrib -v t2m_ei_1979.grb | head -n 8

gives in output

1:0:D=1979010100:2T:sfc:kpds=167,1,0:3hr fcst:type=9:winds are N/S:"2 metre temperature [K]
2:13230:D=1979010100:2T:sfc:kpds=167,1,0:6hr fcst:type=9:winds are N/S:"2 metre temperature [K]
3:26460:D=1979010100:2T:sfc:kpds=167,1,0:9hr fcst:type=9:winds are N/S:"2 metre temperature [K]
4:39690:D=1979010100:2T:sfc:kpds=167,1,0:12hr fcst:type=9:winds are N/S:"2 metre temperature [K]
5:52920:D=1979010112:2T:sfc:kpds=167,1,0:3hr fcst:type=9:winds are N/S:"2 metre temperature [K]
6:66150:D=1979010112:2T:sfc:kpds=167,1,0:6hr fcst:type=9:winds are N/S:"2 metre temperature [K]
7:79380:D=1979010112:2T:sfc:kpds=167,1,0:9hr fcst:type=9:winds are N/S:"2 metre temperature [K]
8:92610:D=1979010112:2T:sfc:kpds=167,1,0:12hr fcst:type=9:winds are N/S:"2 metre temperature [K]

For the present purposes, it is sufficient to consider this list as a colon separated list of informations about the different records within the grib file. Each line refers to a specific data set within the grib files (the first record of each line, ranging, in this example, from 1 to 8). The third record of each line is very important in this context, because it is about the analysis time (expressed as YYYYMMDDHH); the seventh is the other important one, because it tells the forecast time (3hr fcst, 6hr fcst...).

If one installs cdo from the Ubuntu repositories, and types the command

cdo sellonlatbox,23,31,-25,-31 t2m_ei_1979.grb tagliato_t2m_ei_1979.grb

then checks the output file tagliato_t2m_ei_1979.grb with wgrib, this is the result:

wgrib -v tagliato_t2m_ei_1979.grb | head -n 8

1:0:D=1979010100:2T:sfc:kpds=167,1,0:anl:type=analysis:winds are N/S:"2 metre temperature [K]
2:1758:D=1979010100:2T:sfc:kpds=167,1,0:anl:type=analysis:winds are N/S:"2 metre temperature [K]
3:3516:D=1979010100:2T:sfc:kpds=167,1,0:anl:type=analysis:winds are N/S:"2 metre temperature [K]
4:5274:D=1979010100:2T:sfc:kpds=167,1,0:anl:type=analysis:winds are N/S:"2 metre temperature [K]
5:7032:D=1979010112:2T:sfc:kpds=167,1,0:anl:type=analysis:winds are N/S:"2 metre temperature [K]
6:8790:D=1979010112:2T:sfc:kpds=167,1,0:anl:type=analysis:winds are N/S:"2 metre temperature [K]
7:10548:D=1979010112:2T:sfc:kpds=167,1,0:anl:type=analysis:winds are N/S:"2 metre temperature [K]
8:12306:D=1979010112:2T:sfc:kpds=167,1,0:anl:type=analysis:winds are N/S:"2 metre temperature [K]

This is wrong because all the information about the forecast time is lost (instead of 3hr fcst, 6hr fcst... it is reported "anl", wich stands for "analysis", so 0 hours forecast time).

It is obvious, in my opinion, that just cutting spatially the data grid should not modify its time information.

If one uses cdo on a Debian system (I tried this on a Debian 6.0.6 squeeze), after installing it from the Debian official repositories, the result is correct, and it is, as expected

wgrib -v tagliato_t2m_ei_1979.grb | head -n 8

1:0:D=1979010100:2T:sfc:kpds=167,1,0:3hr fcst:winds are N/S:"2 metre temperature [K]
2:1736:D=1979010100:2T:sfc:kpds=167,1,0:6hr fcst:winds are N/S:"2 metre temperature [K]
3:3472:D=1979010100:2T:sfc:kpds=167,1,0:9hr fcst:winds are N/S:"2 metre temperature [K]
4:5208:D=1979010100:2T:sfc:kpds=167,1,0:12hr fcst:winds are N/S:"2 metre temperature [K]
5:6944:D=1979010112:2T:sfc:kpds=167,1,0:3hr fcst:winds are N/S:"2 metre temperature [K]
6:8680:D=1979010112:2T:sfc:kpds=167,1,0:6hr fcst:winds are N/S:"2 metre temperature [K]
7:10416:D=1979010112:2T:sfc:kpds=167,1,0:9hr fcst:winds are N/S:"2 metre temperature [K]
8:12152:D=1979010112:2T:sfc:kpds=167,1,0:12hr fcst:winds are N/S:"2 metre temperature [K]

The same happens if one downloads cdo from its developers' site, as I wrote in my previous post, and compiles the program from its source code. The problem is that Ubuntu packager probably disabled the cdo internal grib 1 library by setting the configure option "--disable-cgribex" (which is not cdo default). Then, the Ubuntu compilation uses the grib_api library both for grib 1 and grib 2, but for grib 1 it is not supported (and in fact, it ends up in wrong results).

This should answer both the questions on how to reproduce the bug, and how to get hints if cdo works well or not (by checking its results with wgrib).

I'm not attaching nor the expected correct output file, nor the wrong one, because, trivially, I cannot find a tool to upload them in here. However, despite this has been got with an older version of cdo, this is the correct output (I can confirm it personally, I tried that later with the release of cdo shipping with Ubuntu), and, as I found, the problems are not related to the cdo release, rather on its compilation configuration:

https://code.zmaw.de/attachments/3104/tagliato_t2m_ei_1979_cdo-1.4.6.grb

And this one is the wrong one (same observations as above apply):

https://code.zmaw.de/attachments/3105/tagliato_t2m_ei_1979_cdo-1.5.3.grb

You can check the correctness of the time informations about the above two datafiles yourself, after my brief wgrib tutoring :) !

That should be all. I underline that I found this bug for
       cdo | 1.5.3.dfsg.1-2 precise
       cdo | 1.5.4+dfsg.1-4build1 quantal
and I did not yeat give raring a chance.

Faithfully,
Marco Galli

Revision history for this message

Javier López (javier-lopez) wrote on 2012-11-14:

#4

Hi Marco,

Thanks for the long and clear explanation, after reading it I was able to reproduce it in raring this way:

$ wget https://code.zmaw.de/attachments/3103/t2m_ei_1979.grb
$ sudo apt-get install cdo
$ cdo sellonlatbox,23,31,-25,-31 t2m_ei_1979.grb tagliato_t2m_ei_1979.grb
$ wget ftp://ftp.cpc.ncep.noaa.gov/wd51we/wgrib/wgrib.c
$ gcc -o wgrib wgrib.c
$ ./wgrib -v t2m_ei_1979.grb | head -n 8
1:0:D=1979010100:2T:sfc:kpds=167,1,0:anl:type=analysis:winds are N/S:"2 metre temperature [K]
...

With the 'anl' definition after kpds=167,1,0:

After recompiling with the --enable-cgribex option it behaves as expected, with the following output:

$ ./wgrib -v t2m_ei_1979.grb | head -n 8
1:0:D=1979010100:2T:sfc:kpds=167,1,0:3hr:type=analysis:winds are N/S:"2 metre temperature [K]

I looked into the changelog and cgribex was disabled in the first place for this report:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=633934

So, I'd like to know if that error is still happening with this version, I've uploaded precise/quantal/raring patched cdo versions to my ppa so you can test them. Specifically the part that says 'Error (decodeIS) : GRIB version 2 unsupported!'

$ sudo add-apt-repository ppa:chilicuil/pmx
$ sudo apt-get update && sudo apt-get upgrade
$ rm tagliato_t2m_ei_1979.grb
$ cdo sellonlatbox,23,31,-25,-31 t2m_ei_1979.grb tagliato_t2m_ei_1979.grb
$ ./wgrib -v t2m_ei_1979.grb | head -n 8

I was able to test successfully the steps in the three ubuntu versions:

cdo - 1.5.6.1+dfsg.1-2ubuntu3 - raring
cdo - 1.5.4+dfsg.1-4build2 - quantal
cdo - 1.5.3.dfsg.1-2ubuntu1 - precise

If the patched version doesn't work with GRIB version 2 I suspect it may be difficult to be accepted in Ubuntu, probably GRIB2 will have preference over GRIB1

Have a nice day
---
Ubuntu Bug Squad volunteer triager
http://wiki.ubuntu.com/BugSquad

Changed in cdo (Ubuntu):
status:	Incomplete → Confirmed

Revision history for this message

marcofgalli (marcofgalli) wrote on 2012-11-14:

#5

Test file with data expressed in grib2 Edit (5.2 KiB, application/octet-stream)

Download full text (8.1 KiB)

Dear Javier,
thanks for continuing in giving me attention about this problem.

I made some tests and I succesfully made cdo work in the following versions:

cdo 1.5.6.1 (source package from the developers' site; https://code.zmaw.de/attachments/download/3605/cdo-1.5.6.1.tar.gz)
cdo 1.5.4 (since this is no more available on the developers' site, I downloaded it from http://archive.ubuntu.com/ubuntu/pool/universe/c/cdo/cdo_1.5.4+dfsg.1.orig.tar.gz ; I hope this is is conformal to the developers' original one).

I made it work on Ubuntu Quantal, it seems reasonable to me that these conclusion should hold also for other Ubuntu releases (but I'm not an IT scientist, and I won't dare state this sentence too loud! :) ).

I run cdo succesfully both on grib1 and grib2 data, getting the correct results with both versions of cdo.

I compiled the two above packages with the following compilation commands:

export CC=mpicc
configure --prefix=${HOME}/src/cdo-1.5.6.1 --with-grib-api=yes --with-netcdf=yes --with-hdf5=yes
make
make install

Of course, a different prefix value for version 1.5.4, so I did not override the resulting executable. I made the above choice of the configure options in order to stick as closer as possible to the cdo compiled for the Ubuntu Quantal repositories, but ensuring that the resulting executable was working. Of course, in order to compile cdo with those options, the following packages are needed in the compiling environment:

libhdf5-openmpi-dev
libnetcdf-dev

Actually, I do not remember why I have the MPI parallelised version of the libhdf5 package installed on my system, maybe for compatibility with some other software (maybe the same cdo). However, it peacefully coexhists with the Ubuntu repositories version of cdo, while the serial one does not.

I'm attaching to this report a sample grib 2 file, with analysis and forecasts of 2 metre temperature packaged in the same file. Unfortunately, the area of the data I had available for this report is not the same (Europe for the grib 2 data, versus the South African framework I used to rise the problem for grib 1), so the spatial cutting command must be different in the two cases.

For both versions of the cdo compiled software, I run (I am reporting only the test of version 1.5.6.1 for brevity, but I performed the same operations also for version 1.5.4, getting the same results):

${HOME}/src/cdo-1.5.6.1/bin/cdo sellonlatbox,10,15,50,40 t2m.grib2 tagliato_t2m.grib2
${HOME}/src/cdo-1.5.6.1/bin/cdo sellonlatbox,23,31,-25,-31 t2m_ei_1979.grb tagliato_t2m_ei_1979.grb
wgrib2 -v tagliato_t2m.grib2 # See further notes
wgrib -v tagliato_t2m_ei_1979.grb | head -n 8

Till this moment, I only posted the commands rather than their output. It is important to note that for grib 2 data files the wgrib utility mentioned in my previous post is not helpful anymore. Rather, there is a specific utility for grib 2 data that is the wgrib2.

The wgrib2 is downloadable from
http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/
and, more specifically, in that quite messy page, from
http://www.ftp.cpc.ncep.noaa.gov/wd51we/wgrib2/wgrib2.tgz.v1.9.7
In order to use the wgrib2 it must be compiled. Unpack the ...

Dear Javier,
thanks for continuing in giving me attention about this problem.

I made some tests and I succesfully made cdo work in the following versions:

cdo 1.5.6.1 (source package from the developers' site; https://code.zmaw.de/attachments/download/3605/cdo-1.5.6.1.tar.gz)
cdo 1.5.4 (since this is no more available on the developers' site, I downloaded it from http://archive.ubuntu.com/ubuntu/pool/universe/c/cdo/cdo_1.5.4+dfsg.1.orig.tar.gz ; I hope this is is conformal to the developers' original one).

I made it work on Ubuntu Quantal, it seems reasonable to me that these conclusion should hold also for other Ubuntu releases (but I'm not an IT scientist, and I won't dare state this sentence too loud! :) ).

I run cdo succesfully both on grib1 and grib2 data, getting the correct results with both versions of cdo.

I compiled the two above packages with the following compilation commands:

export CC=mpicc
configure --prefix=${HOME}/src/cdo-1.5.6.1 --with-grib-api=yes --with-netcdf=yes --with-hdf5=yes
make
make install

Of course, a different prefix value for version 1.5.4, so I did not override the resulting executable. I made the above choice of the configure options in order to stick as closer as possible to the cdo compiled for the Ubuntu Quantal repositories, but ensuring that the resulting executable was working. Of course, in order to compile cdo with those options, the following packages are needed in the compiling environment:

libhdf5-openmpi-dev
libnetcdf-dev

Actually, I do not remember why I have the MPI parallelised version of the libhdf5 package installed on my system, maybe for compatibility with some other software (maybe the same cdo). However, it peacefully coexhists with the Ubuntu repositories version of cdo, while the serial one does not.

I'm attaching to this report a sample grib 2 file, with analysis and forecasts of 2 metre temperature packaged in the same file. Unfortunately, the area of the data I had available for this report is not the same (Europe for the grib 2 data, versus the South African framework I used to rise the problem for grib 1), so the spatial cutting command must be different in the two cases.

For both versions of the cdo compiled software, I run (I am reporting only the test of version 1.5.6.1 for brevity, but I performed the same operations also for version 1.5.4, getting the same results):

${HOME}/src/cdo-1.5.6.1/bin/cdo sellonlatbox,10,15,50,40 t2m.grib2 tagliato_t2m.grib2
${HOME}/src/cdo-1.5.6.1/bin/cdo sellonlatbox,23,31,-25,-31 t2m_ei_1979.grb tagliato_t2m_ei_1979.grb
wgrib2 -v tagliato_t2m.grib2   # See further notes
wgrib -v tagliato_t2m_ei_1979.grb | head -n 8

Till this moment, I only posted the commands rather than their output. It is important to note that for grib 2 data files the wgrib utility mentioned in my previous post is not helpful anymore. Rather, there is a specific utility for grib 2 data that is the wgrib2.

The wgrib2 is downloadable from
http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/
and, more specifically, in that quite messy page, from
http://www.ftp.cpc.ncep.noaa.gov/wd51we/wgrib2/wgrib2.tgz.v1.9.7
In order to use the wgrib2 it must be compiled. Unpack the archive, change directory to "grib2" then edit the makefile as the following:
from lines 89 on, my makefile reads as

# often enviroment variable FC=fortran compiler, is set to f77, need f90 compiler
FC=gfortran
CC=gcc

(I uncommented the declaration of the FC and CC compilers variables)

then it is necessary to execute make twice (the first one gives a compilation error; the second time, everything works fine and the executable is created, and it works)

make
make

The executable of wgrib2 is named wgrib2 and it is in the directory wgrib2 (considering the directory "grib2" where there is the makefile as a reference point).

The usage of wgrib2, for the present purpose, is the same as wgrib. The grib2 file I posted as an example has less than 8 time records, so it is not necessary to select only a few lines of output piping it into the head command used in the other case. A difference between the wgrib and the wgrib2 utilities is the format of the on screen output

wgrib2 -v tagliato_t2m.grib2
results, correctly, in

1:0:d=2010113000:TMP:2 m above ground:anl:
2:292:d=2010113000:TMP:2 m above ground:6 hour fcst:
3:584:d=2010113000:TMP:2 m above ground:12 hour fcst:
4:876:d=2010113000:TMP:2 m above ground:18 hour fcst:
5:1168:d=2010113000:TMP:2 m above ground:24 hour fcst:
6:1460:d=2010113000:TMP:2 m above ground:30 hour fcst:
7:1752:d=2010113000:TMP:2 m above ground:36 hour fcst:

#########################################

wgrib -v tagliato_t2m_ei_1979.grb | head -n 8
results, correctly, in:

1:0:D=1979010100:2T:sfc:kpds=167,1,0:3hr fcst:winds are N/S:"2 metre temperature [K]
2:1736:D=1979010100:2T:sfc:kpds=167,1,0:6hr fcst:winds are N/S:"2 metre temperature [K]
3:3472:D=1979010100:2T:sfc:kpds=167,1,0:9hr fcst:winds are N/S:"2 metre temperature [K]
4:5208:D=1979010100:2T:sfc:kpds=167,1,0:12hr fcst:winds are N/S:"2 metre temperature [K]
5:6944:D=1979010112:2T:sfc:kpds=167,1,0:3hr fcst:winds are N/S:"2 metre temperature [K]
6:8680:D=1979010112:2T:sfc:kpds=167,1,0:6hr fcst:winds are N/S:"2 metre temperature [K]
7:10416:D=1979010112:2T:sfc:kpds=167,1,0:9hr fcst:winds are N/S:"2 metre temperature [K]
8:12152:D=1979010112:2T:sfc:kpds=167,1,0:12hr fcst:winds are N/S:"2 metre temperature [K]

This demonstrates that the compilation I posted at the beginning of this comment produces on Ubuntu Quantal for the two versions of cdo under examination two good executables, that they actually do well what they are supposed to do.

There is in fact one difference between the cdo available in the Ubuntu repositories and the ones I tested to write this post. The command
cdo -V
gives some information about the cdo executable creation. For the official Ubuntu repositories the output of this command is:

Climate Data Operators version 1.5.4 (http://code.zmaw.de/projects/cdo)
Compiler: mpicc -std=gnu99 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -fPIC
version: gcc: error: unrecognized command line option '-V'
with: PTHREADS Z JASPER PROJ.4
Compiled: by buildd on roseapple (i686-pc-linux-gnu) Sep  8 2012 18:02:35
CDI library version : 1.5.4 of Sep  8 2012 18:02:29
GRIB_API library version : 1.9.16
netCDF library version : 4.1.3 of May  1 2012 04:15:00 $
HDF5 library version : 1.8.8
SERVICE library version : 1.3.0 of Sep  8 2012 18:02:13
EXTRA library version : 1.3.0 of Sep  8 2012 18:02:08
IEG library version : 1.3.0 of Sep  8 2012 18:02:12
FILE library version : 1.7.1 of Sep  8 2012 18:02:08

while for the one I compiled it is

Climate Data Operators version 1.5.4 (http://code.zmaw.de/projects/cdo)
Compiler: mpicc -std=gnu99 -g -O2
version: gcc: error: unrecognized command line option '-V'
with: PTHREADS NC4 Z
Compiled: by myself on pcubuntu (i686-pc-linux-gnu) Nov 14 2012 12:17:31
CDI library version : 1.5.4 of Nov 14 2012 12:17:19
CGRIBEX library version : 1.5.1 of Aug 29 2011 20:30:27
GRIB_API library version : 1.9.16
netCDF library version : 4.1.3 of May  1 2012 04:15:00 $
HDF5 library version : 1.8.8
SERVICE library version : 1.3.0 of Nov 14 2012 12:16:49
EXTRA library version : 1.3.0 of Nov 14 2012 12:16:41
IEG library version : 1.3.0 of Nov 14 2012 12:16:47
FILE library version : 1.7.1 of Nov 14 2012 12:16:42

The only significative difference is the presence, in my configuration, of the CGRIBEX library, that is the one dealing with grib 1 data files. Now I cannot understand why, for my case, the --enable-cgribex option was not necessary to make things go well, while in your case it was necessary to specify it explicitely at configuration time.

At the moment I did not have the possibility to test the packages on your ppa; I prefer to do that on a virtual machine before harming my system. However, I will do that as soon as possible, with both grib1 data files, grib2 data files and maybe also netCDF files. At the moment I found that the procedure I outlined in this and in the previous post produces something working at least on my system.

I will let you know of my forthcoming double check on your ppa. However, if you have any further questions or comments, let me know.

Best regards
Marco

Revision history for this message

Javier López (javier-lopez) wrote on 2012-11-15:

#6

Download full text (4.2 KiB)

Hi Marco,

Thank you for your grib2 file and full explanation about how to test it.

I was able to pass the two testcases (grib1 - grib2) with cdo in precise/quantal/raring with the ppa version. Now let me give you my opinion about some of your questions.

> cdo 1.5.4 (since this is no more available on the developers' site, I downloaded it from http://archive.ubuntu.com/ubuntu/pool/universe/c/cdo/cdo_1.5.4+dfsg.1.orig.tar.gz ; I hope this is is conformal to the developers' original one).

Yes, the orig.* files are the upstream unmodified sources

> There is in fact one difference between the cdo available in the Ubuntu repositories and the ones I tested to write this post.

This is the 'cdo -V' output of the ppa versions:

raring:
cdo -V
Climate Data Operators version 1.5.6.1 (http://code.zmaw.de/projects/cdo)
Compiler: gcc -std=gnu99 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -fPIC -pthread
version: gcc (Ubuntu/Linaro 4.7.2-5ubuntu5) 4.7.2
    with: PTHREADS Z JASPER PROJ.4
Compiled: by buildd on mercury (i686-pc-linux-gnu) Nov 14 2012 01:16:49
     CDI library version : 1.5.6 of Nov 14 2012 01:16:37
CGRIBEX library version : 1.5.3 of Jun 14 2012 13:22:45
GRIB_API library version : 1.9.16
  netCDF library version : 4.1.3 of May 1 2012 04:15:00 $
    HDF5 library version : 1.8.8
SERVICE library version : 1.3.0 of Nov 14 2012 01:16:12
   EXTRA library version : 1.3.0 of Nov 14 2012 01:16:05
     IEG library version : 1.3.0 of Nov 14 2012 01:16:10
    FILE library version : 1.8.0 of Nov 14 2012 01:16:05

quantal:
cdo -V
Climate Data Operators version 1.5.4 (http://code.zmaw.de/projects/cdo)
Compiler: mpicc -std=gnu99 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -fPIC
version: gcc: error: unrecognized command line option '-V'
    with: PTHREADS Z JASPER PROJ.4
Compiled: by buildd on actinium (i686-pc-linux-gnu) Nov 14 2012 06:51:52
     CDI library version : 1.5.4 of Nov 14 2012 06:51:44
CGRIBEX library version : 1.5.1 of Aug 29 2011 20:30:27
GRIB_API library version : 1.9.16
  netCDF library version : 4.1.3 of May 1 2012 04:15:00 $
    HDF5 library version : 1.8.8
SERVICE library version : 1.3.0 of Nov 14 2012 06:51:23
   EXTRA library version : 1.3.0 of Nov 14 2012 06:51:17
     IEG library version : 1.3.0 of Nov 14 2012 06:51:22
    FILE library version : 1.7.1 of Nov 14 2012 06:51:1

precise:
cdo -V
Climate Data Operators version 1.5.3 (http://code.zmaw.de/projects/cdo)
Compiler: mpicc -std=gnu99 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -fPIC
version: gcc: error: unrecognized option '-V'
    with: PTHREADS Z JASPER PROJ.4
Compiled: by buildd on platinum (i686-pc-linux-gnu) Nov 14 2012 04:36:09
     CDI library version : 1.5.3 of Nov 14 2012 04:35:59
CGRIBEX library version : 1.5.1 of Aug 29 2011 20:30:27
GRIB_API library version : 1.9.9
  netCDF library version : 4.1.1 of Nov 7 2011 11:30:05 $
    HDF5 library version : 1.8.4
SERVICE library version : 1.3.0 of Nov 14 2012 04:35:34
   EXTRA library version : 1.3.0 of Nov 14 2012 04:35:28
     IEG library version : 1.3.0 of Nov 14 2012 0...

	Status	Importance	Assigned to
cdo (Debian)	Fix Released	Unknown	debbugs #695009
cdo (Ubuntu)	Fix Released	Undecided	Unassigned
Precise	Won't Fix	Undecided	Unassigned

Ubuntu
cdo package

cdo does not do what it is supposed to do

Bug Description

Related branches

Other bug subscribers

Bug attachments

Remote bug watches

Changed in cdo (Ubuntu):
status:	Confirmed → Fix Released

Changed in cdo (Ubuntu Precise):
status:	New → Won't Fix

Ubuntucdo package

cdo does not do what it is supposed to do

Bug Description

Related branches

Other bug subscribers

Bug attachments

Remote bug watches

Ubuntu
cdo package