Add new mime type "video/mp2t" to totem, xine and vlc

This bug probably covers the lack of support for AVCHD HD video file extensions - as generated by many HD camcorders.

.MTS and .M2TS file extensions should resolve to mime type video/MP2T

*** Bug 24690 has been marked as a duplicate of this bug. ***

Patch for video/mp2t attached to Bug 24690.

I just tested the new patch attached to http://bugs.freedesktop.org/show_bug.cgi?id=24690#c1

My .ts-Files indeed all start with 0x47. But after the patch is applied, all files that start with 0x47 are "video/mp2t", even if their filenames do not end with ".ts". I have here .ts-files that come together with meta files with same name but different suffix. Some of these meta files incidentally start with 0x47, and hence are treated as videos too.

I'd prefer that files are "video/mp2t" only if they start with 0x47 *AND* their filename ends with '.ts' (or '.mp2t' or '.mpts' or '.mpg' or other suffix that might be common for mpeg transport stream files).

Can you test with this pattern please?

<match type="big32" mask="0xff5fff1f" value="0x47400010" offset="0"/>

Regards, Daniel

This does not match my .ts. They were recorded on a digital cable receiver and start like this:

00000000 47 40 1f 10 00 7f 80 24 00 00 01 00 00 00 00 27 |G@.....$.......'|
...

If I remove some more bits from your mask then it matches my .ts too:
(I did this without knowledge about mpeg stream standards!)

  <match type="big32" mask="0xff5fe01f" value="0x47400010" offset="0"/>

Created an attachment (id=33748)
Adds support for video/mp2t (MPEG-2 transport streams)

I have here the patch to recognize video/mp2t (MPEG-2 transport streams). It's an enhanced version of https://bugs.freedesktop.org/show_bug.cgi%3Fid%3D24690%23c1. Naming comes from http://www.rfc-editor.org/rfc/rfc3555.txt (page 38). The glob patterns come from http://en.wikipedia.org/wiki/MPEG_transport_stream and are all not yet mentioned in the mime database, except *.ts and *.m2t.

To control matching order of magic patterns one may use "magic priority=", but glob patterns match in order of their appearance in the database file, so:

*.ts is currently recognized as application/x-linguist but without any magic pattern. If we position video/mp2t AFTER it, we can control with the magic pattern if a *.ts becomes video/mp2t or not. I carefully chose the magic pattern to be exactly as strict as necessary to match all valid video/mp2t.

*.m2t is currently recognized as video/mpeg which is wrong according to Wikipedia and e.g. https://bugs.launchpad.net/bugs/89543. So I chose to position video/mp2t BEFORE video/mpeg.

Maybe this is be better:

+ <_comment>MPEG-2 Transport Stream</_comment>
+ <acronym>MPEG-2 TS</acronym>
+ <expanded-acronym>Moving Picture Experts Group 2 Transport Stream</expanded-acronym>

It is not wise to use the same acronym for two different MIME types.

Created an attachment (id=33767)
Adds support for video/mp2t (MPEG-2 transport streams) + unique acronym

@Stanislav: Thanks! I don't know where these acronyms are used, but it is definitely better to keep them unique. So I replaced <acronym> and <expanded-acronym>.

The lower-case letters in <_comment> were chosen intentionally, because other mime types are like this. Probably it does not matter - this attribute is later mapped to a translated string anyway.

Please verify patch.

Created an attachment (id=34447)
Patch to add support for video/mp2t (MPEG-2 transport streams) + unique acronym + unscrambled only

Patch has been improved. I added another 2 bits to the magic mask to only match if a transport stream is not scrambled. This is ok because scrambled streams could not be handled by common desktop apps anyway. With this mask GIMP resources don't match anymore. (They start with the 4 bytes "GIMP" which is a valid MP2T header)

If you own MP2T files (from a camcorder or a settop box), please test if they match and leave a comment! (Feel free to ask me how to apply the patch to your system)

For details about successful tests see comment 7+8 on https://bugs.launchpad.net/ubuntu/+source/shared-mime-info/+bug/502642

Please add this patch to shared-mime-info. Otherwise other bugs are blocked:
https://bugzilla.gnome.org/show_bug.cgi?id=614422
http://trac.videolan.org/vlc/ticket/3485
http://bugs.xine-project.org/show_bug.cgi?id=341

The patch has been successfully applied for Ubuntu 10.04 (see URL in bug details)

Current git now has magic for linguist files, could you check if that causes any needs for changes to this patch?

A (small as sanely possible) test file would be welcome for the test suite, do you happen to have one available?

I checked new magic for linguist type in current git and saw no problem. The new magic alone does not prevent video/mp2t files to match linguist type. But it does not interfere with my patch either. Together with my patch video/mp2t still are recognized reliably.

I did not check if true application/x-linguist files are affected by my patch, because I have none. But I am pretty sure that this is not the case, because the magic in my patch is quite restrictive. And to verify this I pulled the source of shared-mime-info (including all its test-files) into an Ubuntu Live-CD and then I did:

  $ sudo find /bin /boot /etc /home /lib /opt /root /sbin /tmp /usr /var \
     -xdev -type f -exec mimetype \{\} \; | grep video/mp2t

This showed no false positives.

For a small video/mp2t test file see https://bugs.launchpad.net/ubuntu/+source/shared-mime-info/+bug/502642/comments/7.

There's a linguist test file in git. Adding this to tests/list (at end of the video files section) and the mp2t test file in place makes the test suite pass:

test-mp2t.ts video/mp2t xo

The test file seems to be taken from copyrighted material. It's just a couple of seconds but I suppose it'd be better to have a non-copyrighted one there if someone can produce one e.g. with a camcorder.

Created an attachment (id=34779)
non-copyrighted mpeg2 transport stream recorded by a settopbox

Thanks for testing. I agree that only non-copyrighted streams should be used as test files. I recorded a video/mp2t snippet with black picture and silent audio. This attachment should be ok to be included in shared-mime-info.

Created an attachment (id=35043)
non-copyrighted mpeg2 transport stream recorded by a settopbox

The previous attachment was sub-optimal. It shows is black image sequence but indeed is a non-decrypted content. Now I found a test channel with a nice test pattern (color stripes without any letters or logos). Therefore I submit a new attachment which is IMHO well-suited for inclusion into the source tree of shared-mime-info.

commit e88b56dd9836c36124baf69d88ae22be8ef776b1
Author: Oliver Joos <email address hidden>
Date: Fri May 21 20:06:31 2010 +0300

    Add video/mp2t.

    http://bugs.freedesktop.org/show_bug.cgi?id=14276