Merge lp:~thomas-voss/media-hub/move-playbin-implementation-to-cpp-file into lp:media-hub

Proposed by Thomas Voß
Status: Merged
Approved by: Ricardo Mendoza
Approved revision: 109
Merged at revision: 118
Proposed branch: lp:~thomas-voss/media-hub/move-playbin-implementation-to-cpp-file
Merge into: lp:media-hub
Prerequisite: lp:~thomas-voss/media-hub/introduce-recorder-observer-interface
Diff against target: 1177 lines (+608/-495)
4 files modified
src/core/media/CMakeLists.txt (+2/-0)
src/core/media/gstreamer/engine.cpp (+1/-1)
src/core/media/gstreamer/playbin.cpp (+556/-0)
src/core/media/gstreamer/playbin.h (+49/-494)
To merge this branch: bzr merge lp:~thomas-voss/media-hub/move-playbin-implementation-to-cpp-file
Reviewer Review Type Date Requested Status
PS Jenkins bot continuous-integration Approve
Jim Hodapp (community) code Approve
Review via email: mp+242890@code.launchpad.net

Commit message

Move gstreamer::Playbin implementation to its own cpp file, thus internalizing the Hybris setup portions.
Make sure that media-hub-service knows about media::Player::Error::OutOfProcessBufferStreamingNotSupported by linking with media-hub-client.

Description of the change

Move gstreamer::Playbin implementation to its own cpp file, thus internalizing the Hybris setup portions.
Make sure that media-hub-service knows about media::Player::Error::OutOfProcessBufferStreamingNotSupported by linking with media-hub-client.

To post a comment you must log in.
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
104. By Thomas Voß

Remerge prereq branch.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Jim Hodapp (jhodapp) wrote :

Looks good

review: Approve (code)
105. By Thomas Voß

[ Jim Hodapp ]
* Resubmitting with prerequisite branch (LP: #1331041)
[ Justin McPherson ]
* Resubmitting with prerequisite branch (LP: #1331041)

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
106. By Thomas Voß

[ Jim Hodapp ]
* Error reporting all the way up to the app level from the playbin
  pipeline.
[ Ubuntu daily release ]
* New rebuild forced
[ Jim Hodapp ]
* Don't auto-resume playback of videos after a phone call ends. (LP:
  #1411273)
[ Ubuntu daily release ]
* New rebuild forced
[ Ricardo Salveti de Araujo ]
* service_implementation: adding debug for call started/ended signals.
  Make sure account and connection are available when setting up
  account manager (patch from Gustavo Boiko). call_monitor: don't
  check caps when hooking up on/off signals, until bug 1409125 is
  fixed. Enable parallel building . (LP: #1409125)
[ Jim Hodapp ]
* Pause playback when recording begins. (LP: #1398047)
[ Ricardo Salveti de Araujo ]
* call_monitor.cpp: waiting for bridge to be up, and also protecting
  the on_change call (LP: #1408137)

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
107. By Thomas Voß

Merge prereq branch.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Francis Ginther (fginther) wrote :

The jenkins node for the i386 build failed, I've restarted a new ci run.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
108. By Thomas Voß

Merge prereq branch.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
109. By Thomas Voß

* debian/control:
  - Removing pre-depends that are not required
  - Bumping standards-version to 3.9.6
[ Ricardo Salveti de Araujo ]
* Migrating tests to use ogg instead of mp3/avi removed:
  tests/h264.avi tests/test.mp3 added: tests/test-audio-1.ogg
  tests/test-video.ogg tests/test.mp3 renamed: tests/test.ogg =>
  tests/test-audio.ogg

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'src/core/media/CMakeLists.txt'
--- src/core/media/CMakeLists.txt 2015-03-12 11:42:10 +0000
+++ src/core/media/CMakeLists.txt 2015-03-12 11:42:10 +0000
@@ -93,6 +93,7 @@
93 recorder_observer.cpp93 recorder_observer.cpp
94 hybris_recorder_observer.cpp94 hybris_recorder_observer.cpp
95 gstreamer/engine.cpp95 gstreamer/engine.cpp
96 gstreamer/playbin.cpp
9697
97 player_skeleton.cpp98 player_skeleton.cpp
98 player_implementation.cpp99 player_implementation.cpp
@@ -105,6 +106,7 @@
105target_link_libraries(106target_link_libraries(
106 media-hub-service107 media-hub-service
107108
109 media-hub-client
108 media-hub-common110 media-hub-common
109 call-monitor111 call-monitor
110 ${DBUS_LIBRARIES}112 ${DBUS_LIBRARIES}
111113
=== modified file 'src/core/media/gstreamer/engine.cpp'
--- src/core/media/gstreamer/engine.cpp 2015-03-12 11:42:10 +0000
+++ src/core/media/gstreamer/engine.cpp 2015-03-12 11:42:10 +0000
@@ -333,7 +333,7 @@
333333
334bool gstreamer::Engine::open_resource_for_uri(const media::Track::UriType& uri)334bool gstreamer::Engine::open_resource_for_uri(const media::Track::UriType& uri)
335{335{
336 d->playbin.set_uri(uri);336 d->playbin.set_uri(uri, core::ubuntu::media::Player::HeadersType{});
337 return true;337 return true;
338}338}
339339
340340
=== added file 'src/core/media/gstreamer/playbin.cpp'
--- src/core/media/gstreamer/playbin.cpp 1970-01-01 00:00:00 +0000
+++ src/core/media/gstreamer/playbin.cpp 2015-03-12 11:42:10 +0000
@@ -0,0 +1,556 @@
1/*
2 * Copyright © 2013 Canonical Ltd.
3 *
4 * This program is free software: you can redistribute it and/or modify it
5 * under the terms of the GNU Lesser General Public License version 3,
6 * as published by the Free Software Foundation.
7 *
8 * This program is distributed in the hope that it will be useful,
9 * but WITHOUT ANY WARRANTY; without even the implied warranty of
10 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11 * GNU Lesser General Public License for more details.
12 *
13 * You should have received a copy of the GNU Lesser General Public License
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *
16 * Authored by: Thomas Voß <thomas.voss@canonical.com>
17 */
18
19#include <core/media/gstreamer/playbin.h>
20
21#include <core/media/gstreamer/engine.h>
22
23#if defined(MEDIA_HUB_HAVE_HYBRIS_MEDIA_COMPAT_LAYER)
24#include <hybris/media/surface_texture_client_hybris.h>
25#include <hybris/media/media_codec_layer.h>
26
27namespace
28{
29void setup_video_sink_for_buffer_streaming(GstElement* video_sink)
30{
31 // Get the service-side BufferQueue (IGraphicBufferProducer) and associate it with
32 // the SurfaceTextureClientHybris instance
33 IGBPWrapperHybris igbp = decoding_service_get_igraphicbufferproducer();
34 SurfaceTextureClientHybris stc = surface_texture_client_create_by_igbp(igbp);
35 // Because mirsink is being loaded, we are definitely doing * hardware rendering.
36 surface_texture_client_set_hardware_rendering (stc, TRUE);
37 g_object_set (G_OBJECT (video_sink), "surface", static_cast<gpointer>(stc), static_cast<char*>(NULL));
38}
39}
40#else // MEDIA_HUB_HAVE_HYBRIS_MEDIA_COMPAT_LAYER
41namespace
42{
43void setup_video_sink_for_buffer_streaming(GstElement*)
44{
45 throw core::ubuntu::media::Player::Error::OutOfProcessBufferStreamingNotSupported{};
46}
47}
48#endif // MEDIA_HUB_HAVE_HYBRIS_MEDIA_COMPAT_LAYER
49
50namespace
51{
52bool is_mir_video_sink()
53{
54 return g_strcmp0(::getenv("CORE_UBUNTU_MEDIA_SERVICE_VIDEO_SINK_NAME"), "mirsink") == 0;
55}
56}
57// Uncomment to generate a dot file at the time that the pipeline
58// goes to the PLAYING state. Make sure to export GST_DEBUG_DUMP_DOT_DIR
59// before starting media-hub-server. To convert the dot file to something
60// other image format, use: dot pipeline.dot -Tpng -o pipeline.png
61//#define DEBUG_GST_PIPELINE
62
63namespace media = core::ubuntu::media;
64namespace video = core::ubuntu::media::video;
65
66const std::string& gstreamer::Playbin::pipeline_name()
67{
68 static const std::string s{"playbin"};
69 return s;
70}
71
72void gstreamer::Playbin::about_to_finish(GstElement*, gpointer user_data)
73{
74 auto thiz = static_cast<Playbin*>(user_data);
75 thiz->signals.about_to_finish();
76}
77
78void gstreamer::Playbin::source_setup(GstElement*,
79 GstElement *source,
80 gpointer user_data)
81{
82 if (user_data == nullptr)
83 return;
84
85 static_cast<Playbin*>(user_data)->setup_source(source);
86}
87
88gstreamer::Playbin::Playbin()
89 : pipeline(gst_element_factory_make("playbin", pipeline_name().c_str())),
90 bus{gst_element_get_bus(pipeline)},
91 file_type(MEDIA_FILE_TYPE_NONE),
92 video_sink(nullptr),
93 on_new_message_connection(
94 bus.on_new_message.connect(
95 std::bind(
96 &Playbin::on_new_message,
97 this,
98 std::placeholders::_1))),
99 is_seeking(false),
100 player_lifetime(media::Player::Lifetime::normal)
101{
102 if (!pipeline)
103 throw std::runtime_error("Could not create pipeline for playbin.");
104
105 // Add audio and/or video sink elements depending on environment variables
106 // being set or not set
107 setup_pipeline_for_audio_video();
108
109 g_signal_connect(
110 pipeline,
111 "about-to-finish",
112 G_CALLBACK(about_to_finish),
113 this
114 );
115
116 g_signal_connect(
117 pipeline,
118 "source-setup",
119 G_CALLBACK(source_setup),
120 this
121 );
122}
123
124gstreamer::Playbin::~Playbin()
125{
126 if (pipeline)
127 gst_object_unref(pipeline);
128}
129
130void gstreamer::Playbin::reset()
131{
132 std::cout << "Client died, resetting pipeline" << std::endl;
133 // When the client dies, tear down the current pipeline and get it
134 // in a state that is ready for the next client that connects to the
135 // service
136
137 // Don't reset the pipeline if we want to resume
138 if (player_lifetime != media::Player::Lifetime::resumable) {
139 reset_pipeline();
140 }
141 // Signal to the Player class that the client side has disconnected
142 signals.client_disconnected();
143}
144
145void gstreamer::Playbin::reset_pipeline()
146{
147 std::cout << __PRETTY_FUNCTION__ << std::endl;
148 auto ret = gst_element_set_state(pipeline, GST_STATE_NULL);
149 switch(ret)
150 {
151 case GST_STATE_CHANGE_FAILURE:
152 std::cout << "Failed to reset the pipeline state. Client reconnect may not function properly." << std::endl;
153 break;
154 case GST_STATE_CHANGE_NO_PREROLL:
155 case GST_STATE_CHANGE_SUCCESS:
156 case GST_STATE_CHANGE_ASYNC:
157 break;
158 default:
159 std::cout << "Failed to reset the pipeline state. Client reconnect may not function properly." << std::endl;
160 }
161 file_type = MEDIA_FILE_TYPE_NONE;
162}
163
164void gstreamer::Playbin::on_new_message(const Bus::Message& message)
165{
166 switch(message.type)
167 {
168 case GST_MESSAGE_ERROR:
169 signals.on_error(message.detail.error_warning_info);
170 break;
171 case GST_MESSAGE_WARNING:
172 signals.on_warning(message.detail.error_warning_info);
173 break;
174 case GST_MESSAGE_INFO:
175 signals.on_info(message.detail.error_warning_info);
176 break;
177 case GST_MESSAGE_TAG:
178 {
179 gchar *orientation;
180 if (gst_tag_list_get_string(message.detail.tag.tag_list, "image-orientation", &orientation))
181 {
182 // If the image-orientation tag is in the GstTagList, signal the Engine
183 signals.on_orientation_changed(orientation_lut(orientation));
184 g_free (orientation);
185 }
186
187 signals.on_tag_available(message.detail.tag);
188 }
189 break;
190 case GST_MESSAGE_STATE_CHANGED:
191 signals.on_state_changed(message.detail.state_changed);
192 break;
193 case GST_MESSAGE_ASYNC_DONE:
194 if (is_seeking)
195 {
196 // FIXME: Pass the actual playback time position to the signal call
197 signals.on_seeked_to(0);
198 is_seeking = false;
199 }
200 break;
201 case GST_MESSAGE_EOS:
202 signals.on_end_of_stream();
203 default:
204 break;
205 }
206}
207
208gstreamer::Bus& gstreamer::Playbin::message_bus()
209{
210 return bus;
211}
212
213void gstreamer::Playbin::setup_pipeline_for_audio_video()
214{
215 gint flags;
216 g_object_get (pipeline, "flags", &flags, nullptr);
217 flags |= GST_PLAY_FLAG_AUDIO;
218 flags |= GST_PLAY_FLAG_VIDEO;
219 flags &= ~GST_PLAY_FLAG_TEXT;
220 g_object_set (pipeline, "flags", flags, nullptr);
221
222 if (::getenv("CORE_UBUNTU_MEDIA_SERVICE_AUDIO_SINK_NAME") != nullptr)
223 {
224 auto audio_sink = gst_element_factory_make (
225 ::getenv("CORE_UBUNTU_MEDIA_SERVICE_AUDIO_SINK_NAME"),
226 "audio-sink");
227
228 std::cout << "audio_sink: " << ::getenv("CORE_UBUNTU_MEDIA_SERVICE_AUDIO_SINK_NAME") << std::endl;
229
230 g_object_set (
231 pipeline,
232 "audio-sink",
233 audio_sink,
234 NULL);
235 }
236
237 if (::getenv("CORE_UBUNTU_MEDIA_SERVICE_VIDEO_SINK_NAME") != nullptr)
238 {
239 video_sink = gst_element_factory_make (
240 ::getenv("CORE_UBUNTU_MEDIA_SERVICE_VIDEO_SINK_NAME"),
241 "video-sink");
242
243 std::cout << "video_sink: " << ::getenv("CORE_UBUNTU_MEDIA_SERVICE_VIDEO_SINK_NAME") << std::endl;
244
245 g_object_set (
246 pipeline,
247 "video-sink",
248 video_sink,
249 NULL);
250 }
251}
252
253void gstreamer::Playbin::create_video_sink(uint32_t)
254{
255 if (not video_sink) throw std::logic_error
256 {
257 "No video sink configured for the current pipeline"
258 };
259
260 setup_video_sink_for_buffer_streaming(video_sink);
261}
262
263void gstreamer::Playbin::set_volume(double new_volume)
264{
265 g_object_set (pipeline, "volume", new_volume, NULL);
266}
267
268/** Translate the AudioStreamRole enum into a string */
269std::string gstreamer::Playbin::get_audio_role_str(media::Player::AudioStreamRole audio_role)
270{
271 switch (audio_role)
272 {
273 case media::Player::AudioStreamRole::alarm:
274 return "alarm";
275 break;
276 case media::Player::AudioStreamRole::alert:
277 return "alert";
278 break;
279 case media::Player::AudioStreamRole::multimedia:
280 return "multimedia";
281 break;
282 case media::Player::AudioStreamRole::phone:
283 return "phone";
284 break;
285 default:
286 return "multimedia";
287 break;
288 }
289}
290
291media::Player::Orientation gstreamer::Playbin::orientation_lut(const gchar *orientation)
292{
293 if (g_strcmp0(orientation, "rotate-0") == 0)
294 return media::Player::Orientation::rotate0;
295 else if (g_strcmp0(orientation, "rotate-90") == 0)
296 return media::Player::Orientation::rotate90;
297 else if (g_strcmp0(orientation, "rotate-180") == 0)
298 return media::Player::Orientation::rotate180;
299 else if (g_strcmp0(orientation, "rotate-270") == 0)
300 return media::Player::Orientation::rotate270;
301 else
302 return media::Player::Orientation::rotate0;
303}
304
305/** Sets the new audio stream role on the pulsesink in playbin */
306void gstreamer::Playbin::set_audio_stream_role(media::Player::AudioStreamRole new_audio_role)
307{
308 GstElement *audio_sink = NULL;
309 g_object_get (pipeline, "audio-sink", &audio_sink, NULL);
310
311 std::string role_str("props,media.role=" + get_audio_role_str(new_audio_role));
312 std::cout << "Audio stream role: " << role_str << std::endl;
313
314 GstStructure *props = gst_structure_from_string (role_str.c_str(), NULL);
315 if (audio_sink != nullptr && props != nullptr)
316 g_object_set (audio_sink, "stream-properties", props, NULL);
317 else
318 {
319 std::cerr <<
320 "Warning: couldn't set audio stream role - couldn't get audio_sink from pipeline" <<
321 std::endl;
322 }
323
324 gst_structure_free (props);
325}
326
327void gstreamer::Playbin::set_lifetime(media::Player::Lifetime lifetime)
328{
329 player_lifetime = lifetime;
330}
331
332uint64_t gstreamer::Playbin::position() const
333{
334 int64_t pos = 0;
335 gst_element_query_position (pipeline, GST_FORMAT_TIME, &pos);
336
337 // FIXME: this should be int64_t, but dbus-cpp doesn't seem to handle it correctly
338 return static_cast<uint64_t>(pos);
339}
340
341uint64_t gstreamer::Playbin::duration() const
342{
343 int64_t dur = 0;
344 gst_element_query_duration (pipeline, GST_FORMAT_TIME, &dur);
345
346 // FIXME: this should be int64_t, but dbus-cpp doesn't seem to handle it correctly
347 return static_cast<uint64_t>(dur);
348}
349
350void gstreamer::Playbin::set_uri(
351 const std::string& uri,
352 const core::ubuntu::media::Player::HeadersType& headers = core::ubuntu::media::Player::HeadersType())
353{
354 reset_pipeline();
355
356 g_object_set(pipeline, "uri", uri.c_str(), NULL);
357 if (is_video_file(uri))
358 file_type = MEDIA_FILE_TYPE_VIDEO;
359 else if (is_audio_file(uri))
360 file_type = MEDIA_FILE_TYPE_AUDIO;
361
362 request_headers = headers;
363}
364
365void gstreamer::Playbin::setup_source(GstElement *source)
366{
367 if (source == NULL || request_headers.empty())
368 return;
369
370 if (request_headers.find("Cookie") != request_headers.end()) {
371 if (g_object_class_find_property(G_OBJECT_GET_CLASS(source),
372 "cookies") != NULL) {
373 gchar ** cookies = g_strsplit(request_headers["Cookie"].c_str(), ";", 0);
374 g_object_set(source, "cookies", cookies, NULL);
375 g_strfreev(cookies);
376 }
377 }
378
379 if (request_headers.find("User-Agent") != request_headers.end()) {
380 if (g_object_class_find_property(G_OBJECT_GET_CLASS(source),
381 "user-agent") != NULL) {
382 g_object_set(source, "user-agent", request_headers["User-Agent"].c_str(), NULL);
383 }
384 }
385}
386
387std::string gstreamer::Playbin::uri() const
388{
389 gchar* data = nullptr;
390 g_object_get(pipeline, "current-uri", &data, nullptr);
391
392 std::string result((data == nullptr ? "" : data));
393 g_free(data);
394
395 return result;
396}
397
398bool gstreamer::Playbin::set_state_and_wait(GstState new_state)
399{
400 static const std::chrono::nanoseconds state_change_timeout
401 {
402 // We choose a quite high value here as tests are run under valgrind
403 // and gstreamer pipeline setup/state changes take longer in that scenario.
404 // The value does not negatively impact runtime performance.
405 std::chrono::milliseconds{5000}
406 };
407
408 auto ret = gst_element_set_state(pipeline, new_state);
409 bool result = false; GstState current, pending;
410 switch(ret)
411 {
412 case GST_STATE_CHANGE_FAILURE:
413 result = false; break;
414 case GST_STATE_CHANGE_NO_PREROLL:
415 case GST_STATE_CHANGE_SUCCESS:
416 result = true; break;
417 case GST_STATE_CHANGE_ASYNC:
418 result = GST_STATE_CHANGE_SUCCESS == gst_element_get_state(
419 pipeline,
420 &current,
421 &pending,
422 state_change_timeout.count());
423 break;
424 }
425
426 // We only should query the pipeline if we actually succeeded in
427 // setting the requested state.
428 if (result && new_state == GST_STATE_PLAYING)
429 {
430 // Get the video height/width from the video sink
431 try
432 {
433 signals.on_video_dimensions_changed(get_video_dimensions());
434 }
435 catch (const std::exception& e)
436 {
437 std::cerr << "Problem querying video dimensions: " << e.what() << std::endl;
438 }
439 catch (...)
440 {
441 std::cerr << "Problem querying video dimensions." << std::endl;
442 }
443
444#ifdef DEBUG_GST_PIPELINE
445 std::cout << "Dumping pipeline dot file" << std::endl;
446 GST_DEBUG_BIN_TO_DOT_FILE((GstBin*)pipeline, GST_DEBUG_GRAPH_SHOW_ALL, "pipeline");
447#endif
448 }
449
450 return result;
451}
452
453bool gstreamer::Playbin::seek(const std::chrono::microseconds& ms)
454{
455 is_seeking = true;
456 return gst_element_seek_simple(
457 pipeline,
458 GST_FORMAT_TIME,
459 (GstSeekFlags)(GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT),
460 ms.count() * 1000);
461}
462
463core::ubuntu::media::video::Dimensions gstreamer::Playbin::get_video_dimensions() const
464{
465 if (not video_sink || not is_mir_video_sink())
466 throw std::runtime_error
467 {
468 "Missing video sink or video sink does not support query of width and height."
469 };
470
471 // Initialize to default value prior to querying actual values from the sink.
472 uint32_t video_width = 0, video_height = 0;
473 g_object_get (video_sink, "height", &video_height, nullptr);
474 g_object_get (video_sink, "width", &video_width, nullptr);
475 // TODO(tvoss): We should probably check here if width and height are valid.
476 return core::ubuntu::media::video::Dimensions
477 {
478 core::ubuntu::media::video::Height{video_height},
479 core::ubuntu::media::video::Width{video_width}
480 };
481}
482
483std::string gstreamer::Playbin::get_file_content_type(const std::string& uri) const
484{
485 if (uri.empty())
486 return std::string();
487
488 std::string filename(uri);
489 std::cout << "filename: " << filename << std::endl;
490 size_t pos = uri.find("file://");
491 if (pos != std::string::npos)
492 filename = uri.substr(pos + 7, std::string::npos);
493 else
494 // Anything other than a file, for now claim that the type
495 // is both audio and video.
496 // FIXME: implement true net stream sampling and get the type from GstCaps
497 return std::string("audio/video/");
498
499
500 GError *error = nullptr;
501 std::unique_ptr<GFile, void(*)(void *)> file(
502 g_file_new_for_path(filename.c_str()), g_object_unref);
503 std::unique_ptr<GFileInfo, void(*)(void *)> info(
504 g_file_query_info(
505 file.get(), G_FILE_ATTRIBUTE_STANDARD_FAST_CONTENT_TYPE ","
506 G_FILE_ATTRIBUTE_ETAG_VALUE, G_FILE_QUERY_INFO_NONE,
507 /* cancellable */ NULL, &error),
508 g_object_unref);
509 if (!info)
510 {
511 std::string error_str(error->message);
512 g_error_free(error);
513
514 std::cout << "Failed to query the URI for the presence of video content: "
515 << error_str << std::endl;
516 return std::string();
517 }
518
519 std::string content_type(g_file_info_get_attribute_string(
520 info.get(), G_FILE_ATTRIBUTE_STANDARD_FAST_CONTENT_TYPE));
521
522 return content_type;
523}
524
525bool gstreamer::Playbin::is_audio_file(const std::string& uri) const
526{
527 if (uri.empty())
528 return false;
529
530 if (get_file_content_type(uri).find("audio/") == 0)
531 {
532 std::cout << "Found audio content" << std::endl;
533 return true;
534 }
535
536 return false;
537}
538
539bool gstreamer::Playbin::is_video_file(const std::string& uri) const
540{
541 if (uri.empty())
542 return false;
543
544 if (get_file_content_type(uri).find("video/") == 0)
545 {
546 std::cout << "Found video content" << std::endl;
547 return true;
548 }
549
550 return false;
551}
552
553gstreamer::Playbin::MediaFileType gstreamer::Playbin::media_file_type() const
554{
555 return file_type;
556}
0557
=== modified file 'src/core/media/gstreamer/playbin.h'
--- src/core/media/gstreamer/playbin.h 2015-03-12 11:42:10 +0000
+++ src/core/media/gstreamer/playbin.h 2015-03-12 11:42:10 +0000
@@ -19,12 +19,11 @@
19#ifndef GSTREAMER_PLAYBIN_H_19#ifndef GSTREAMER_PLAYBIN_H_
20#define GSTREAMER_PLAYBIN_H_20#define GSTREAMER_PLAYBIN_H_
2121
22#include <core/media/player.h>
23
22#include "bus.h"24#include "bus.h"
23#include "../mpris/player.h"25#include "../mpris/player.h"
2426
25#include <hybris/media/surface_texture_client_hybris.h>
26#include <hybris/media/media_codec_layer.h>
27
28#include <gio/gio.h>27#include <gio/gio.h>
29#include <gst/gst.h>28#include <gst/gst.h>
3029
@@ -37,8 +36,6 @@
37// other image format, use: dot pipeline.dot -Tpng -o pipeline.png36// other image format, use: dot pipeline.dot -Tpng -o pipeline.png
38//#define DEBUG_GST_PIPELINE37//#define DEBUG_GST_PIPELINE
3938
40namespace media = core::ubuntu::media;
41
42namespace gstreamer39namespace gstreamer
43{40{
44struct Playbin41struct Playbin
@@ -57,508 +54,66 @@
57 MEDIA_FILE_TYPE_VIDEO54 MEDIA_FILE_TYPE_VIDEO
58 };55 };
5956
60 static const std::string& pipeline_name()57 static std::string get_audio_role_str(core::ubuntu::media::Player::AudioStreamRole audio_role);
61 {58
62 static const std::string s{"playbin"};59 static const std::string& pipeline_name();
63 return s;60
64 }61 static void about_to_finish(GstElement*, gpointer user_data);
65
66 static void about_to_finish(GstElement*,
67 gpointer user_data)
68 {
69 auto thiz = static_cast<Playbin*>(user_data);
70 thiz->signals.about_to_finish();
71 }
7262
73 static void source_setup(GstElement*,63 static void source_setup(GstElement*,
74 GstElement *source,64 GstElement *source,
75 gpointer user_data)65 gpointer user_data);
76 {66
77 if (user_data == nullptr)67 Playbin();
78 return;68 ~Playbin();
7969
80 static_cast<Playbin*>(user_data)->setup_source(source);70 void reset();
81 }71 void reset_pipeline();
8272
83 Playbin()73 void on_new_message(const Bus::Message& message);
84 : pipeline(gst_element_factory_make("playbin", pipeline_name().c_str())),74
85 bus{gst_element_get_bus(pipeline)},75 gstreamer::Bus& message_bus();
86 file_type(MEDIA_FILE_TYPE_NONE),76
87 video_sink(nullptr),77 void setup_pipeline_for_audio_video();
88 on_new_message_connection(78
89 bus.on_new_message.connect(79 void create_video_sink(uint32_t texture_id);
90 std::bind(80
91 &Playbin::on_new_message,81 void set_volume(double new_volume);
92 this,82
93 std::placeholders::_1))),83 void set_lifetime(core::ubuntu::media::Player::Lifetime);
94 is_seeking(false),84 core::ubuntu::media::Player::Orientation orientation_lut(const gchar *orientation);
95 player_lifetime(media::Player::Lifetime::normal)
96 {
97 if (!pipeline)
98 throw std::runtime_error("Could not create pipeline for playbin.");
99
100 // Add audio and/or video sink elements depending on environment variables
101 // being set or not set
102 setup_pipeline_for_audio_video();
103
104 g_signal_connect(
105 pipeline,
106 "about-to-finish",
107 G_CALLBACK(about_to_finish),
108 this
109 );
110
111 g_signal_connect(
112 pipeline,
113 "source-setup",
114 G_CALLBACK(source_setup),
115 this
116 );
117
118 }
119
120 ~Playbin()
121 {
122 if (pipeline)
123 gst_object_unref(pipeline);
124 }
125
126 void reset()
127 {
128 std::cout << "Client died, resetting pipeline" << std::endl;
129 // When the client dies, tear down the current pipeline and get it
130 // in a state that is ready for the next client that connects to the
131 // service
132
133 // Don't reset the pipeline if we want to resume
134 if (player_lifetime != media::Player::Lifetime::resumable) {
135 reset_pipeline();
136 }
137
138 // Signal to the Player class that the client side has disconnected
139 signals.client_disconnected();
140 }
141
142 void reset_pipeline()
143 {
144 std::cout << __PRETTY_FUNCTION__ << std::endl;
145 auto ret = gst_element_set_state(pipeline, GST_STATE_NULL);
146 switch(ret)
147 {
148 case GST_STATE_CHANGE_FAILURE:
149 std::cout << "Failed to reset the pipeline state. Client reconnect may not function properly." << std::endl;
150 break;
151 case GST_STATE_CHANGE_NO_PREROLL:
152 case GST_STATE_CHANGE_SUCCESS:
153 case GST_STATE_CHANGE_ASYNC:
154 break;
155 default:
156 std::cout << "Failed to reset the pipeline state. Client reconnect may not function properly." << std::endl;
157 }
158 file_type = MEDIA_FILE_TYPE_NONE;
159 }
160
161 void on_new_message(const Bus::Message& message)
162 {
163 switch(message.type)
164 {
165 case GST_MESSAGE_ERROR:
166 signals.on_error(message.detail.error_warning_info);
167 break;
168 case GST_MESSAGE_WARNING:
169 signals.on_warning(message.detail.error_warning_info);
170 break;
171 case GST_MESSAGE_INFO:
172 signals.on_info(message.detail.error_warning_info);
173 break;
174 case GST_MESSAGE_TAG:
175 {
176 gchar *orientation;
177 if (gst_tag_list_get_string(message.detail.tag.tag_list, "image-orientation", &orientation))
178 {
179 // If the image-orientation tag is in the GstTagList, signal the Engine
180 signals.on_orientation_changed(orientation_lut(orientation));
181 g_free (orientation);
182 }
183
184 signals.on_tag_available(message.detail.tag);
185 }
186 break;
187 case GST_MESSAGE_STATE_CHANGED:
188 signals.on_state_changed(message.detail.state_changed);
189 break;
190 case GST_MESSAGE_ASYNC_DONE:
191 if (is_seeking)
192 {
193 // FIXME: Pass the actual playback time position to the signal call
194 signals.on_seeked_to(0);
195 is_seeking = false;
196 }
197 break;
198 case GST_MESSAGE_EOS:
199 signals.on_end_of_stream();
200 default:
201 break;
202 }
203 }
204
205 gstreamer::Bus& message_bus()
206 {
207 return bus;
208 }
209
210 void setup_pipeline_for_audio_video()
211 {
212 gint flags;
213 g_object_get (pipeline, "flags", &flags, nullptr);
214 flags |= GST_PLAY_FLAG_AUDIO;
215 flags |= GST_PLAY_FLAG_VIDEO;
216 flags &= ~GST_PLAY_FLAG_TEXT;
217 g_object_set (pipeline, "flags", flags, nullptr);
218
219 if (::getenv("CORE_UBUNTU_MEDIA_SERVICE_AUDIO_SINK_NAME") != nullptr)
220 {
221 auto audio_sink = gst_element_factory_make (
222 ::getenv("CORE_UBUNTU_MEDIA_SERVICE_AUDIO_SINK_NAME"),
223 "audio-sink");
224
225 std::cout << "audio_sink: " << ::getenv("CORE_UBUNTU_MEDIA_SERVICE_AUDIO_SINK_NAME") << std::endl;
226
227 g_object_set (
228 pipeline,
229 "audio-sink",
230 audio_sink,
231 NULL);
232 }
233
234 if (::getenv("CORE_UBUNTU_MEDIA_SERVICE_VIDEO_SINK_NAME") != nullptr)
235 {
236 video_sink = gst_element_factory_make (
237 ::getenv("CORE_UBUNTU_MEDIA_SERVICE_VIDEO_SINK_NAME"),
238 "video-sink");
239
240 std::cout << "video_sink: " << ::getenv("CORE_UBUNTU_MEDIA_SERVICE_VIDEO_SINK_NAME") << std::endl;
241
242 g_object_set (
243 pipeline,
244 "video-sink",
245 video_sink,
246 NULL);
247 }
248 }
249
250 void create_video_sink(uint32_t texture_id)
251 {
252 std::cout << "Creating video sink for texture_id: " << texture_id << std::endl;
253
254 if (::getenv("CORE_UBUNTU_MEDIA_SERVICE_VIDEO_SINK_NAME") != nullptr)
255 {
256 g_object_get (pipeline, "video_sink", &video_sink, NULL);
257
258 // Get the service-side BufferQueue (IGraphicBufferProducer) and associate it with
259 // the SurfaceTextureClientHybris instance
260 IGBPWrapperHybris igbp = decoding_service_get_igraphicbufferproducer();
261 SurfaceTextureClientHybris stc = surface_texture_client_create_by_igbp(igbp);
262 // Because mirsink is being loaded, we are definitely doing * hardware rendering.
263 surface_texture_client_set_hardware_rendering (stc, TRUE);
264 g_object_set (G_OBJECT (video_sink), "surface", static_cast<gpointer>(stc), static_cast<char*>(NULL));
265 }
266 }
267
268 void set_volume(double new_volume)
269 {
270 g_object_set (pipeline, "volume", new_volume, NULL);
271 }
272
273 /** Translate the AudioStreamRole enum into a string */
274 static std::string get_audio_role_str(media::Player::AudioStreamRole audio_role)
275 {
276 switch (audio_role)
277 {
278 case media::Player::AudioStreamRole::alarm:
279 return "alarm";
280 break;
281 case media::Player::AudioStreamRole::alert:
282 return "alert";
283 break;
284 case media::Player::AudioStreamRole::multimedia:
285 return "multimedia";
286 break;
287 case media::Player::AudioStreamRole::phone:
288 return "phone";
289 break;
290 default:
291 return "multimedia";
292 break;
293 }
294 }
295
296 media::Player::Orientation orientation_lut(const gchar *orientation)
297 {
298 if (g_strcmp0(orientation, "rotate-0") == 0)
299 return media::Player::Orientation::rotate0;
300 else if (g_strcmp0(orientation, "rotate-90") == 0)
301 return media::Player::Orientation::rotate90;
302 else if (g_strcmp0(orientation, "rotate-180") == 0)
303 return media::Player::Orientation::rotate180;
304 else if (g_strcmp0(orientation, "rotate-270") == 0)
305 return media::Player::Orientation::rotate270;
306 else
307 return media::Player::Orientation::rotate0;
308 }
30985
310 /** Sets the new audio stream role on the pulsesink in playbin */86 /** Sets the new audio stream role on the pulsesink in playbin */
311 void set_audio_stream_role(media::Player::AudioStreamRole new_audio_role)87 void set_audio_stream_role(core::ubuntu::media::Player::AudioStreamRole new_audio_role);
312 {88
313 GstElement *audio_sink = NULL;89 uint64_t position() const;
314 g_object_get (pipeline, "audio-sink", &audio_sink, NULL);90 uint64_t duration() const;
31591
316 std::string role_str("props,media.role=" + get_audio_role_str(new_audio_role));92 void set_uri(const std::string& uri, const core::ubuntu::media::Player::HeadersType& headers);
317 std::cout << "Audio stream role: " << role_str << std::endl;93 std::string uri() const;
31894
319 GstStructure *props = gst_structure_from_string (role_str.c_str(), NULL);95 void setup_source(GstElement *source);
320 if (audio_sink != nullptr && props != nullptr)96
321 g_object_set (audio_sink, "stream-properties", props, NULL);97 bool set_state_and_wait(GstState new_state);
322 else98 bool seek(const std::chrono::microseconds& ms);
323 {99
324 std::cerr <<100 core::ubuntu::media::video::Dimensions get_video_dimensions() const;
325 "Warning: couldn't set audio stream role - couldn't get audio_sink from pipeline" <<101
326 std::endl;102 std::string get_file_content_type(const std::string& uri) const;
327 }103
328104 bool is_audio_file(const std::string& uri) const;
329 gst_structure_free (props);105 bool is_video_file(const std::string& uri) const;
330 }106
331107 MediaFileType media_file_type() const;
332 void set_lifetime(media::Player::Lifetime lifetime)
333 {
334 player_lifetime = lifetime;
335 }
336
337 uint64_t position() const
338 {
339 int64_t pos = 0;
340 gst_element_query_position (pipeline, GST_FORMAT_TIME, &pos);
341
342 // FIXME: this should be int64_t, but dbus-cpp doesn't seem to handle it correctly
343 return static_cast<uint64_t>(pos);
344 }
345
346 uint64_t duration() const
347 {
348 int64_t dur = 0;
349 gst_element_query_duration (pipeline, GST_FORMAT_TIME, &dur);
350
351 // FIXME: this should be int64_t, but dbus-cpp doesn't seem to handle it correctly
352 return static_cast<uint64_t>(dur);
353 }
354
355 void set_uri(const std::string& uri,
356 const core::ubuntu::media::Player::HeadersType& headers = core::ubuntu::media::Player::HeadersType())
357 {
358 reset_pipeline();
359
360 g_object_set(pipeline, "uri", uri.c_str(), NULL);
361 if (is_video_file(uri))
362 file_type = MEDIA_FILE_TYPE_VIDEO;
363 else if (is_audio_file(uri))
364 file_type = MEDIA_FILE_TYPE_AUDIO;
365
366 request_headers = headers;
367 }
368
369 void setup_source(GstElement *source)
370 {
371 if (source == NULL || request_headers.empty())
372 return;
373
374 if (request_headers.find("Cookie") != request_headers.end()) {
375 if (g_object_class_find_property(G_OBJECT_GET_CLASS(source),
376 "cookies") != NULL) {
377 gchar ** cookies = g_strsplit(request_headers["Cookie"].c_str(), ";", 0);
378 g_object_set(source, "cookies", cookies, NULL);
379 g_strfreev(cookies);
380 }
381 }
382
383 if (request_headers.find("User-Agent") != request_headers.end()) {
384 if (g_object_class_find_property(G_OBJECT_GET_CLASS(source),
385 "user-agent") != NULL) {
386 g_object_set(source, "user-agent", request_headers["User-Agent"].c_str(), NULL);
387 }
388 }
389 }
390
391 std::string uri() const
392 {
393 gchar* data = nullptr;
394 g_object_get(pipeline, "current-uri", &data, nullptr);
395
396 std::string result((data == nullptr ? "" : data));
397 g_free(data);
398
399 return result;
400 }
401
402 bool set_state_and_wait(GstState new_state)
403 {
404 static const std::chrono::nanoseconds state_change_timeout
405 {
406 // We choose a quite high value here as tests are run under valgrind
407 // and gstreamer pipeline setup/state changes take longer in that scenario.
408 // The value does not negatively impact runtime performance.
409 std::chrono::milliseconds{5000}
410 };
411
412 auto ret = gst_element_set_state(pipeline, new_state);
413 bool result = false; GstState current, pending;
414 switch(ret)
415 {
416 case GST_STATE_CHANGE_FAILURE:
417 result = false; break;
418 case GST_STATE_CHANGE_NO_PREROLL:
419 case GST_STATE_CHANGE_SUCCESS:
420 result = true; break;
421 case GST_STATE_CHANGE_ASYNC:
422 result = GST_STATE_CHANGE_SUCCESS == gst_element_get_state(
423 pipeline,
424 &current,
425 &pending,
426 state_change_timeout.count());
427 break;
428 }
429
430 // The state change has to have been successful to make
431 // sure that we indeed reached the requested new state.
432 if (result && new_state == GST_STATE_PLAYING)
433 {
434 // Get the video height/width from the video sink
435 if (has_video_sink_with_height_and_width())
436 signals.on_video_dimensions_changed(get_video_dimensions());
437#ifdef DEBUG_GST_PIPELINE
438 std::cout << "Dumping pipeline dot file" << std::endl;
439 GST_DEBUG_BIN_TO_DOT_FILE((GstBin*)pipeline, GST_DEBUG_GRAPH_SHOW_ALL, "pipeline");
440#endif
441 }
442
443 return result;
444 }
445
446 bool seek(const std::chrono::microseconds& ms)
447 {
448 is_seeking = true;
449 return gst_element_seek_simple(
450 pipeline,
451 GST_FORMAT_TIME,
452 (GstSeekFlags)(GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT),
453 ms.count() * 1000);
454 }
455
456 bool has_video_sink_with_height_and_width()
457 {
458 return video_sink != nullptr && g_strcmp0(::getenv("CORE_UBUNTU_MEDIA_SERVICE_VIDEO_SINK_NAME"), "mirsink") == 0;
459 }
460
461 core::ubuntu::media::video::Dimensions get_video_dimensions()
462 {
463 if (not has_video_sink_with_height_and_width())
464 throw std::runtime_error{"Could not get the height/width of each video frame"};
465
466 uint32_t video_height = 0, video_width = 0;
467 g_object_get (video_sink, "height", &video_height, nullptr);
468 g_object_get (video_sink, "width", &video_width, nullptr);
469 std::cout << "video_height: " << video_height << ", video_width: " << video_width << std::endl;
470
471 return core::ubuntu::media::video::Dimensions
472 {
473 core::ubuntu::media::video::Height{video_height},
474 core::ubuntu::media::video::Width{video_width}
475 };
476 }
477
478 std::string get_file_content_type(const std::string& uri) const
479 {
480 if (uri.empty())
481 return std::string();
482
483 std::string filename(uri);
484 std::cout << "filename: " << filename << std::endl;
485 size_t pos = uri.find("file://");
486 if (pos != std::string::npos)
487 filename = uri.substr(pos + 7, std::string::npos);
488 else
489 // Anything other than a file, for now claim that the type
490 // is both audio and video.
491 // FIXME: implement true net stream sampling and get the type from GstCaps
492 return std::string("audio/video/");
493
494
495 GError *error = nullptr;
496 std::unique_ptr<GFile, void(*)(void *)> file(
497 g_file_new_for_path(filename.c_str()), g_object_unref);
498 std::unique_ptr<GFileInfo, void(*)(void *)> info(
499 g_file_query_info(
500 file.get(), G_FILE_ATTRIBUTE_STANDARD_FAST_CONTENT_TYPE ","
501 G_FILE_ATTRIBUTE_ETAG_VALUE, G_FILE_QUERY_INFO_NONE,
502 /* cancellable */ NULL, &error),
503 g_object_unref);
504 if (!info)
505 {
506 std::string error_str(error->message);
507 g_error_free(error);
508
509 std::cout << "Failed to query the URI for the presence of video content: "
510 << error_str << std::endl;
511 return std::string();
512 }
513
514 std::string content_type(g_file_info_get_attribute_string(
515 info.get(), G_FILE_ATTRIBUTE_STANDARD_FAST_CONTENT_TYPE));
516
517 return content_type;
518 }
519
520 bool is_audio_file(const std::string& uri) const
521 {
522 if (uri.empty())
523 return false;
524
525 if (get_file_content_type(uri).find("audio/") == 0)
526 {
527 std::cout << "Found audio content" << std::endl;
528 return true;
529 }
530
531 return false;
532 }
533
534 bool is_video_file(const std::string& uri) const
535 {
536 if (uri.empty())
537 return false;
538
539 if (get_file_content_type(uri).find("video/") == 0)
540 {
541 std::cout << "Found video content" << std::endl;
542 return true;
543 }
544
545 return false;
546 }
547
548 MediaFileType media_file_type() const
549 {
550 return file_type;
551 }
552108
553 GstElement* pipeline;109 GstElement* pipeline;
554 gstreamer::Bus bus;110 gstreamer::Bus bus;
555 MediaFileType file_type;111 MediaFileType file_type;
556 SurfaceTextureClientHybris stc_hybris;
557 GstElement* video_sink;112 GstElement* video_sink;
558 core::Connection on_new_message_connection;113 core::Connection on_new_message_connection;
559 bool is_seeking;114 bool is_seeking;
560 core::ubuntu::media::Player::HeadersType request_headers;115 core::ubuntu::media::Player::HeadersType request_headers;
561 media::Player::Lifetime player_lifetime;116 core::ubuntu::media::Player::Lifetime player_lifetime;
562 struct117 struct
563 {118 {
564 core::Signal<void> about_to_finish;119 core::Signal<void> about_to_finish;

Subscribers

People subscribed via source and target branches