mythtranscode doesn't exit on completion

Bug #481104 reported by Steven Ellis
34
This bug affects 6 people
Affects Status Importance Assigned to Milestone
MythTV
Fix Released
Unknown
mythtv (Ubuntu)
Fix Released
Medium
Unassigned

Bug Description

Binary package hint: mythtv-transcode-utils

System is running Mythbuntu 9.10 with mythtv-transcode-utils = 0.22.0+fixes22594-0ubuntu1

I'm running a bunch of lossless transcodes on DVB-S MPEG2 captures. The transcode completes but the process doesn't exit which uses up memory and leaves files locked.

Output in mythtvbackend.log as follows

2009-11-08 17:08:40.365 JobQueue: Transcode Starting for Go Girls: High Quality (1.9 GB)
2009-11-08 17:08:40.382 transcode: Transcode Starting: Go Girls: High Quality (1.9 GB)
2009-11-08 17:08:43.858 Using runtime prefix = /usr
2009-11-08 17:08:43.863 Using configuration directory = /home/mythtv/.mythtv
2009-11-08 17:08:43.878 Empty LocalHostName.
2009-11-08 17:08:43.883 Using localhost value of mythtv
2009-11-08 17:08:43.974 New DB connection, total: 1
2009-11-08 17:08:43.995 Connected to database 'mythconverg' at host: localhost
2009-11-08 17:08:43.995 Closing DB connection named 'DBManager0'
2009-11-08 17:08:43.996 Connected to database 'mythconverg' at host: localhost
2009-11-08 17:08:43.998 Enabled verbose msgs: important
2009-11-08 17:08:43.999 New DB connection, total: 2
2009-11-08 17:08:44.000 Connected to database 'mythconverg' at host: localhost
2009-11-08 17:08:44.018 Transcoding from /myth/tv/5907_20090514203000.mpg to /myth/tv/5907_20090514203000.mpg.tmp
2009-11-08 17:08:44.078 MythContext: Connecting to backend server: 192.168.0.17:6543 (try 1 of 1)
2009-11-08 17:08:44.083 Using protocol version 50
2009-11-08 17:08:44.084 MainServer::ANN Monitor
2009-11-08 17:08:44.085 adding: mythtv as a client (events: 0)
2009-11-08 17:08:44.086 MainServer::ANN Monitor
2009-11-08 17:08:44.087 adding: mythtv as a client (events: 1)
2009-11-08 17:08:44.238 AFD: Opened codec 0x9627a20, id(MPEG2VIDEO) type(Video)
2009-11-08 17:08:44.239 AFD: codec MP2 has 2 channels
2009-11-08 17:08:44.239 AFD: Opened codec 0x962f8b0, id(MP2) type(Audio)
2009-11-08 17:08:44.252 Honoring the cutlist while transcoding
2009-11-08 17:08:44.260 Cutlist : 0-3048,17785-23799,37633-43525,53764-59897,71044-76693,93301-104808
2009-11-08 17:08:44.260 Original Length: 104808 frames
2009-11-08 17:08:44.261 New Length : 66565 frames
2009-11-08 17:08:44.273 SampleRate: Attempted to add a rate 32000 Hz, which is not in the list of allowed rates.
009-11-08 17:08:44.275 Switching to MPEG-2 transcoder.
2009-11-08 17:08:44.298 Opening /myth/tv/5907_20090514203000.mpg
2009-11-08 17:08:44.309 Input #0, mpegts, from '/myth/tv/5907_20090514203000.mpg':
2009-11-08 17:08:44.309 Duration: 01:09:53.54, start: 77827.738189, bitrate: 3845 kb/s
2009-11-08 17:08:44.311 Stream #0.0[0x204]: Video: mpeg2video, yuv420p, 720x576 [PAR 64:45 DAR 16:9], 15000 kb/s, 25 tbr, 90k tbn, 50 tbc
2009-11-08 17:08:44.311 Stream #0.1[0x244]: Data: 0x0101
2009-11-08 17:08:44.312 Stream #0.2[0x28e]: Audio: mp2, 48000 Hz, 2 channels, s16, 192 kb/s
2009-11-08 17:08:44.313 Stream #0.3[0x3e9]: Data: 0x0000
2009-11-08 17:08:44.313 Skipping unsupported codec 2 on stream 1
2009-11-08 17:08:44.314 Skipping unsupported codec 2 on stream 3
2009-11-08 17:08:44.318 #0 PTS:21:37:08.359 Delta: 0.0ms queue: 14
2009-11-08 17:08:44.318 #2 PTS:21:37:08.337 Delta: 21.5333ms queue: 2
2009-11-08 17:08:46.136 Converting frame #18 from P to I
Mux rate: 15.44 Mbit/s
2009-11-08 17:09:00.885 Converting frame #0 from B to I
2009-11-08 17:09:00.905 Converting frame #1 from B to I
2009-11-08 17:09:05.160 Converting frame #1 from P to I
2009-11-08 17:09:21.234 Converting frame #0 from B to I
2009-11-08 17:09:21.252 Converting frame #1 from B to I
2009-11-08 17:09:23.969 Converting frame #0 from B to I
2009-11-08 17:09:23.988 Converting frame #1 from B to I
2009-11-08 17:09:36.549 Converting frame #0 from B to I
2009-11-08 17:09:36.561 Converting frame #1 from B to I
2009-11-08 17:09:39.757 Converting frame #18 from P to I
2009-11-08 17:09:56.071 Converting frame #0 from B to I
2009-11-08 17:09:56.082 Converting frame #1 from B to I
2009-11-08 17:10:02.874 Converting frame #1 from P to I
2009-11-08 17:10:31.763 Converting frame #0 from B to I
2009-11-08 17:10:31.789 Converting frame #1 from B to I
2009-11-08 17:10:44.565 Converting frame #0 from B to I
2009-11-08 17:10:44.585 Converting frame #1 from B to I
2009-11-08 17:10:44.602 Generating Keyframe Index
2009-11-08 17:10:44.603 Opening /myth/tv/5907_20090514203000.mpg.tmp
2009-11-08 17:10:44.676 Input #0, mpeg, from '/myth/tv/5907_20090514203000.mpg.tmp':
2009-11-08 17:10:44.677 Duration: 00:44:23.48, start: 0.301533, bitrate: 3403 kb/s
2009-11-08 17:10:44.677 Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 720x576 [PAR 64:45 DAR 16:9], 15000 kb/s, 25 tbr, 90k tbn, 50 tbc
2009-11-08 17:10:44.678 Stream #0.1[0x1c0]: Audio: mp2, 48000 Hz, 2 channels, s16, 192 kb/s
2009-11-08 17:11:33.769 Transcode Completed
2009-11-08 17:11:35.330 Transcoding /myth/tv/5907_20090514203000.mpg done
2009-11-08 17:11:35.385 Error Truncating '/myth/tv/5907_20090514203000.mpg.old' could not determine size of the file, unlinking immediately.
2009-11-08 17:11:35.444 transcode: Transcode Finished: Go Girls: High Quality (1.1 GB)
2009-11-08 17:11:35.450 JobQueue: Transcode Finished: Go Girls: High Quality (1.1 GB)

Revision history for this message
Steven Ellis (steven-openmedia) wrote :
Download full text (13.6 KiB)

Running lsof on the process shows it has the following files locked.
{{{
lsof -p 20042
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
mythtrans 20042 mythtv cwd DIR 8,17 4096 2 /
mythtrans 20042 mythtv rtd DIR 8,17 4096 2 /
mythtrans 20042 mythtv txt REG 8,17 375928 25105 /usr/bin/mythtranscode
mythtrans 20042 mythtv mem REG 8,17 659232 25624 /usr/lib/libmythavformat-0.22.so.0.22.0
mythtrans 20042 mythtv mem REG 8,17 720468 30618 /usr/lib/libmythupnp-0.22.so.0.22.0
mythtrans 20042 mythtv mem REG 8,17 622240 30611 /usr/lib/libmythdb-0.22.so.0.22.0
mythtrans 20042 mythtv mem REG 8,17 59264 26407 /usr/lib/libraw1394.so.11.0.1
mythtrans 20042 mythtv mem REG 8,17 13680 26419 /usr/lib/librom1394.so.0.3.0
mythtrans 20042 mythtv mem REG 8,17 42268 30610 /usr/lib/libmythavutil-0.22.so.0.22.0
mythtrans 20042 mythtv mem REG 8,17 3177332 25595 /usr/lib/libmyth-0.22.so.0.22.0
mythtrans 20042 mythtv mem REG 8,17 810700 30614 /usr/lib/libmythlivemedia-0.22.so.0.22.0
mythtrans 20042 mythtv mem REG 8,17 83608 201767 /lib/libz.so.1.2.3.3
mythtrans 20042 mythtv mem REG 8,17 50612 26137 /usr/lib/libiec61883.so.0.1.1
mythtrans 20042 mythtv mem REG 8,17 17808 25737 /usr/lib/libavc1394.so.0.3.0
mythtrans 20042 mythtv mem REG 8,17 6528 25665 /usr/lib/libXinerama.so.1.0.0
mythtrans 20042 mythtv mem REG 8,17 13872 25685 /usr/lib/libXvMC.so.1.0.0
mythtrans 20042 mythtv mem REG 8,17 55100 30613 /usr/lib/libmythhdhomerun-0.22.so.0.22.0
mythtrans 20042 mythtv mem REG 8,17 60024 24750 /usr/lib/libXext.so.6.4.0
mythtrans 20042 mythtv mem REG 8,17 17848 25683 /usr/lib/libXv.so.1.0.0
mythtrans 20042 mythtv mem REG 8,17 30064 25673 /usr/lib/libXrandr.so.2.2.0
mythtrans 20042 mythtv mem REG 8,17 17784 25687 /usr/lib/libXvMCW.so.1.0.0
mythtrans 20042 mythtv mem REG 8,17 9736 218634 /lib/tls/i686/cmov/libdl-2.10.1.so
mythtrans 20042 mythtv mem REG 8,17 3656 24972 /usr/lib/libvdpau.so.185.18.36
mythtrans 20042 mythtv mem REG 8,17 9564 25642 /usr/lib/libXau.so.6.0.0
mythtrans 20042 mythtv mem REG 8,17 113320 201611 /lib/ld-2.10.1.so
mythtrans 20042 mythtv mem REG 8,17 1625872 30617 /usr/lib/libmythui-0.22.so.0.22.0
mythtrans 20042 mythtv mem REG 8,17 552860 30612 /usr/lib/libmythfreemheg-0.22.so.0.22.0
mythtrans 20042 mythtv mem REG 8,17 513472 25944 /usr/lib/libfreetype.so.6.3.20
mythtrans 20042 mythtv mem REG 8,17 257012 25002 /usr/lib/libpulse.so.0.12.0
mythtrans 20042 mythtv mem REG 8,17 116920 218645 /lib/tls/i686/cmov/libpthread-2.10.1.so
mythtrans 20042 mythtv mem REG 8,17 456380 24818 /usr/lib/libGLU.so.1.3.070600
mythtrans 20042 mythtv mem REG 8,17 34412 25551 /usr/lib/libXrender.so.1.3.0
mythtrans 20042 mythtv mem REG 8,17 30080 25634 /u...

Revision history for this message
Steven Ellis (steven-openmedia) wrote :

Running an strace shows that the process is waiting on something

strace -p 20042
Process 20042 attached - interrupt to quit
futex(0x9569fb4, FUTEX_WAIT_PRIVATE, 1, NULL

Revision history for this message
MarcRandolph (mrand) wrote :

Howdy, and thank you for helping to improve Mythbuntu by opening this ticket. I noticed you say you are running a "bunch" of these... do all fail, most fail, or is this the only failure? Thank you.

Changed in mythtv (Ubuntu):
status: New → Incomplete
Revision history for this message
Steven Ellis (steven-openmedia) wrote :

It looks like it happens every time judging by the number of mythtranscode processes I could see on the system.

A simple kill will force it to exit and doesn't appear to break anything as the actual transcode has finished.

Revision history for this message
Martin F. (martin1fink) wrote :

Hello, my system is affected by the same bug. I'm running Mythbuntu 9.10 upgraded from 9.04 two weeks ago with the same version of mythtv-transcode-utils. I'm also using DVB-S doeing lossless transcodes. In mythfrontend everything seems to work fine, but after the file is transcoded, I can see that a job like

/usr/bin/mythtranscode -j 5875 -V 3 -p autodetect -l

is still running but using no cpu-time and doeing nothing. This happens everytime I am transcoding a recording an every process uses memory and prevents mythtv from automatically shuting down the system (that's why i realized the problem). The "Error Truncating" messages in mythbackend.log and the output of strace are also the same for me.
However I noticed that

mythtranscode --mpeg2 --buildindex --allkeys --showprogress --infile $filename

which I sometimes have to do for some recordings exits just fine, so maybe it's a problem of the communication mythtranscode an mythbackend.

Revision history for this message
Brendan Cully (bcully) wrote :

I'm seeing this too.

Revision history for this message
Brent N. (brent-nesbitt) wrote :

Yes, I have this same problem. The non-exiting mythtranscode processes are preventing my backend from auto-powering off. I can kill them, and all is back to normal.

For the time being I have set up a cron job to killall mythtranscode every evening, to allow the system to shut down, but of course, there COULD be valid transcodes going on - and the system would be shut down most of the day too, if these jobs exited correctly.

Revision history for this message
Fisslefink (erin-simonds) wrote :

I have the same problem with a script designed to export shows for viewing on my iPod touch. Like Brent, the major problem for me is that the stalled mythtranscode processes prevent the system from shutting down automatically.

Output of 'mythshutdown -s;echo $?' is 1 (Transcoding)

I am running Mythbuntu 9.10 with latest updates. I do have a custom package of ffmpeg (with libx264 support) installed, following Post #1 on this thread: http://ubuntuforums.org/showthread.php?t=786095

Revision history for this message
Fisslefink (erin-simonds) wrote :

I have attached a script which can be run as a cron job as "root" or as the "mythtv" user. The script will kill all mythtranscode processes with CPU usage less than 1%. Active transcoding processes tend to have high CPU usage, so this should avoid killing active jobs before they are done transcoding.

This is still just a hack to regain normal functionality. This bug needs to be fixed by a maintainer who can convince mythtranscode to exit properly when finished!

Revision history for this message
Fisslefink (erin-simonds) wrote :

I have attached a script which can be run as a cron job as "root" or as the "mythtv" user. The script will kill all mythtranscode processes with CPU usage less than 1%. Active transcoding processes tend to have high CPU usage, so this should avoid killing active jobs before they are done transcoding.

This is still just a hack to regain normal functionality. This bug needs to be fixed by a maintainer who can convince mythtranscode to exit properly when finished!

Revision history for this message
Dave (davem99) wrote :

I have this problem too with Mythbuntu 9.10 and mythtv 0.22. The backend is prevented from shutting down automatically and if I kill the phantom transcode job, everything proceeds as it should.

Revision history for this message
robbak (robbak) wrote :

This is a known issue in mythtv. The mythtv bug ticket is http://svn.mythtv.org/trac/ticket/7315

Fixes in svn have been reported as working for some, but others are still reporting the issue. A workaround seems to be turning off 'Delete files slowly' (database name settings:TruncateDeletesSlowly), until the svn fix is complted and sent out. However, further work seems to have stopped.

Changed in mythtv (Ubuntu):
status: Incomplete → Triaged
importance: Undecided → Medium
Changed in mythtv:
status: Unknown → New
Revision history for this message
Dave (davem99) wrote :

Disabling "Delete files slowly" fixes this issue for me.

Revision history for this message
Steven Ellis (steven-openmedia) wrote : Re: [Bug 481104] Re: mythtranscode doesn't exit on completion

Dave wrote:
> Disabling "Delete files slowly" fixes this issue for me.
>
>
Sadly that then impacts general performance.

Changed in mythtv:
status: New → Fix Released
Dave Walker (davewalker)
Changed in mythtv (Ubuntu):
status: Triaged → Fix Committed
Revision history for this message
Brent N. (brent-nesbitt) wrote :

I have upgraded my mythtv system to Ubuntu 10.04 beta-1, and the problem still exists in that version. I hope this fix is pushed out before the final release of 10.04.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package mythtv - 0.23.0+fixes24133-0ubuntu1

---------------
mythtv (0.23.0+fixes24133-0ubuntu1) lucid; urgency=low

  [ Dave Walker (Daviey) ]
  * New upstream snapshot (r24133)
    - Fixes crash of mythfrontend when upconverting audio to 5.1 (LP: #550562)
    - Improve Jamu handling of invalid database data (LP: #549352)
    - Remove mythtranscode's internal copy of the slow-delete code and instead
      send a request to the backend to delete the file. (LP: #481104)
    - LiveTV & recording with V4L cards on 64-bit backend fixed. (LP: #562690)
  * debian/mcc-mirobridge/*: New icon, UI alignment changes and description
    text added. Courtesy of R. Doug Vaughan.
  * debian/rules: Added cxx option "-maltivec" to PowerPC specifc configure
    option to fix FTBFS for that architecture. (LP: #559614)

  [ Mario Limonciello ]
  * 40_libudev.path:
    - Add -ludev to the library check.
  * debian/control:
    - Drop linux-libc-dev build-dep.
  * Actually disable halt and reboot commands by default.
 -- Dave Walker (Daviey) <email address hidden> Thu, 15 Apr 2010 00:04:14 +0100

Changed in mythtv (Ubuntu):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.