`

Media Capture in Mobile Browsers

    博客分类:
  • js
 
阅读更多

ntroduction

Media Capture is one of the most interesting features in web applications, especially for mobile devices. Surprisingly capturing media on the spot is quite a new thing for browsers in general, until recently always being delegated to browser plugins such as Flash or Silverlight. In this article we will explore how to use the Media Capture APIs, their compatibility across mobile browsers and the current state of the W3C specifications that define them.

At the moment the W3C Device APIs Working Group is working on two different Media Capture APIs:

  • HTML Media Capture
  • Media Capture and Streams

The HTML Media Capture specification defines an HTML form extension, while the Media Capture and Streams specification specification defines a set of JavaScript APIs.

HTML Media Capture

Since 9 May 2013, the HTML Media Capture specification is a W3C Candidate Recommendation. It basically extends <input> elements with a capture attribute.

The capture attribute is a boolean attribute that, if specified, indicates that the capture of media directly from the device’s environment using a media capture mechanism is preferred.

Here is the HTML for image capturing:

<input type="file" accept="image/*" capture>

Capturing video is quite similar; you just need to set the accept attribute accordingly.

<input type="file" accept="video/*" capture>

And the story is the same for capturing audio:

<input type="file" accept="audio/*" capture>

For example, if you want to take a photo using the device camera and upload the image using an HTML form, this is all the code you need.

<form action="upload.htm" method="post" enctype="multipart/form-data">
	<input type="file" accept="image/*" capture>
	<input type="submit" value="Upload">
</form>

Easy, isn’t it?

Android OS 3.0 was the first platform to provide HTML Media Capture support, via its default Android Webkit browser. Now HTML Media Capture is also supported by:

  • Safari and Chrome Mobile for iOS 6+
  • Chrome Mobile for Android OS 3+
  • Firefox Mobile for Android OS 3+
  • Opera 16 for Android OS 3+

Nonetheless some of them only partially implement the specification or implement an older W3C specification, that makes the code above slightly different:

<input type="file" accept="image/*" capture="camera">
<input type="file" accept="video/*" capture="camcorder">
<input type="file" accept="audio/*" capture="microphone">

Check mobilehtml5.org for current compatibility information.

I have written a simple HTML Media Capture demo, which can be used to test and review HTML Media Capture on your favorite mobile browser. Here are some screenshots taken from Opera Mobile 16.

    

Here are some considerations, which I would hope to see changed in the future:

  • First of all, I would expect different default labels from Choose File and No file chosen, and the possibility to customize them.
  • I would expect specifying the capture attribute to show the camera directly, rather than having to navigate through the dialog boxes above.
  • Actually, it seems that current implementations don’t rely on the capture attribute at all, but only on the type and accept attributes: the browser displays a dialog box in which the user can choose where the file has to be taken, and the capture attribute is not taken into consideration. For example, iOS Safari relies on the accept attribute (not capture) for images and videos (not audio). Even if you don’t use the accept attribute, the browser will let you choose between “Take Photo or Video” and “Choose Existing” (thanks to @firt for pointing this out).
  • Audio capture does not trigger the sound recorder with Firefox Mobile (I tried versions 20 and 21 beta for Android OS 4) and with Safari and Chrome for iOS 6 (I tried version 26)

Another interesting aspect to consider is that before releasing iOS 6 Safari didn’t supporttype="file" at all; that is, it did not allow access to the filesystem. It’s only with iOS 6, released in September 2012, that Apple decided to open the doors to filesystem and media capturing.

It’s also worth mentioning that if the browser does not support media capture, it will ask the user to select a media object from the file system instead. Graceful degradation for the win!

Media Capture and Streams

The Media Capture and Streams specification is a W3C Working Draft, a joint deliverable created by the WebRTC and Device APIs Working Groups. Many refer to this specification asgetUserMedia. It is a pretty new feature, which is partially supported only by Opera Mobile Classic at the moment.

Using this technology, accessing multimedia streams (video, audio, or both) from local devices (video cameras, microphones, web cams) should now be possible without browser plugins, which will allow easy real time communication between anyone with a supporting web browser.

The code is quite easy to understand.

navigator.getUserMedia(
	constraints,
	successCallback,
	errorCallback
);

The getUserMedia method takes 3 arguments:

  • constraints, a JavaScript object that defines the tracks to be used (audio, video)
  • successCallback, the function called if the user allows access to the media input device requested by the web application. It takes a LocalMediaStream object as an argument, which represents streams of media data, and provides a stop method, used to stop the stream.
  • errorCallback, the function that is invoked if the user denies permission or if there is a failure in finding valid tracks.

After calling navigator.getUserMedia the user is asked for permission to let the browser access the camera or the microphone.

Now let’s look at a quite simple example in which the stop() method is used.

var constraints = {
	video: true,
	audio: true
}

var successCallback = function(mediaStream) {
	var button = document.querySelector('button');
	button.addEventListener('click', function() {
		mediaStream.stop();
	}, false);
};

var errorCallback = function() {
	console.log('failure to get media');
};

navigator.getUserMedia(
	constraints,
	successCallback,
	errorCallback
);

I simply added an event listener to a button inside the success callback, which activates thestop() method, providing an easy way to stop the stream.

Taking a photo

In the following example I have used getUserMedia to take a photo. To do this, I had to capture a video from the device’s camera, load the stream in an HTML5 <video> element, and then wire up a button with image capture functionality.

Let’s look at the code in detail. This is the HTML: there’s a <video> element with the autoplayattribute set, an empty <img> tag into which I will put the photo, a <canvas>, and the Capture button.

<video autoplay></video>
<img src="">
<canvas style="display:none"></canvas>
<button>Capture</button>

The capture function is called when the DOM is ready; its parameters are the JavaScript DOM objects of the tags defined above.

function capture(video, canvas, image, snapshotButton) {

	var constraints = {
		video: true
	}

	var successCallback = function(mediaStream) {
		video.src = window.URL.createObjectURL(mediaStream);
		video.addEventListener("loadedmetadata", function(e) {
			snapshotButton.onclick = function() {
				takePhoto();
			}

		}
	};

	var errorCallback = function() {
		console.log('failure to get media');
	};

	var takePhoto = function () {
		var ctx = canvas.getContext('2d');
		ctx.drawImage(video,0,0);
		showImage();
	};

	var showImage = function () {
		image.src = canvas.toDataURL('image/webp');
	};

	navigator.getUserMedia(constraints, successCallback, errorCallback);

}

If the user grants access to the camera, a blob URL is created from the stream and used as thesrc attribute of the <video> element, so that the HTML5 video object starts reproducing the stream from the camera. By clicking on the snapshotButton, the takePhoto function is called, resulting in our <canvas> capturing a still image from the video. The image is then converted to adataUrl in order to fill the src attribute of the <img> element and display the photo.

A live demo of the Media Capture and Streams example can be found on my web site. More advanced getUserMedia examples can be found at shinydemos.com.

In order to natively support the use case of taking a photo via JavaScript, the W3C Media Capture Task Force started approaching this scenario as follows:

A common usage scenario of local device capture is to simply “take a picture”. The hardware and optics of many camera-devices often support video in addition to photos, but can be set into a specific “camera mode” where the possible capture resolutions are significantly larger than their maximum video resolution.

The advantage to having a photo-mode is to be able to capture these very high-resolution images (versus the post-processing scenarios that are possible with still-frames from a video source).

Capturing a picture is strongly tied to the “video” capability because a video preview is often an important component to setting up the scene and getting the right shot.

Because photo capabilities are somewhat different from those of regular video capabilities, devices that support a specific “photo” mode, should likely provide their “photo” capabilities separately from their “video” capabilities.

Many of the considerations that apply to video capture also apply to taking a picture.

Issues

  • What are the implications on the device mode switch on video captures that are in progress? Will there be a pause? Can this problem be avoided?
  • Should a “photo mode” be a type of user media that can be requested via getUserMedia?

In March 2012, a simple image capture API that extends getUserMedia’s functionality was proposed. In February 2013, an alternative proposal based on a new ImageCapture objectpopped up and in April, after some discussions, the need for a dedicated Image Capture specification has been recognized, which would allow us to take a picture without needing to use a video and a <canvas> element! On 9 July 2013 the W3C finally published the Mediastream Image Capture’s First Public Working Draft.

This is how the code would appear.

function capture(captureButton, image) {

	var constraints = {
		video: true
	}

	var successCallback = function(mediaStream) {
		var videoStreamTrack = mediastream.getTrackByID()[0];
		var imageCapture = new ImageCapture(videoDevice);

		captureButton.onclick = function() {
			imageCapture.takePhoto();
		}

		pictureDevice.onphoto = showImage;
		pictureDevice.onphotoerror = photoError;
	};

	var errorCallback = function() {
		console.log('failure to get media');
	};

	var showImage = function (blobevent) {
		image.src = URL.createObjectURL(blobevent.data);
	};

	var photoError = function() {
		console.log('failure to take photo');
	};

	navigator.getUserMedia(constraints, successCallback, errorCallback);
}

This way, it would be enough to instantiate an ImageCapture object and call the takephotomethod on it. A photo event would be fired with a BlobEvent containing the captured image, so that my showImage function would create a URL directly from the blob object and display the image.

In the W3C Media Capture Task Force, there’s also an interesting debate about photo settings which likely will be part of another article, along with the MediaStream Recording API.

One thing to bear in mind is that the syntax is still under discussion, so it’s likely to change in the near future.

Summary

Media Capture opens the door to a number of scenarios for web applications, especially for mobile devices. Think of capturing, adding effects and uploading media on the fly. Think of real-time communication, recording, surveillance. Think of augmented reality! There’s still a lot of work to do with W3C specifications and mobile browser compatibility, but I’m sure we will see the light at the end of the tunnel very soon!

分享到:
评论

相关推荐

    深入探索Media Capture and Streams API:JavaScript中的媒体捕获与流处理

    Media Capture and Streams API(媒体捕获和流API)是一个强大的Web API,它允许Web应用程序直接访问硬件设备(如摄像头和麦克风)来捕获音频和视频数据。这个API为开发者提供了一种在网页上实现音视频捕获、处理和...

    MobiMB Mobile Media Browser

    轻松浏览手机内存 主要用于修改手机内部文件

    手机资源管理器MobiMB Mobile Media Browser_v3.3.8 汉化版+注册码.rar

    《手机资源管理器MobiMB Mobile Media Browser:深入解析与应用指南》 手机资源管理器MobiMB Mobile Media Browser是一款专为移动设备设计的强大媒体管理工具,它以其高效、便捷的特性深受用户喜爱。该软件的最新...

    深入探索JavaScript中的Media Capture and Streams API:实现音视频捕获与处理

    Media Capture and Streams API(简称MediaStream API)是一个允许网页访问音频和视频输入设备的Web API。通过这个API,开发者可以创建丰富的多媒体应用,如视频会议、直播、屏幕共享等。本文将详细介绍MediaStream ...

    MobiMB Mobile Media Browser V3.1

    《MobiMB Mobile Media Browser V3.1:诺基亚手机媒体管理利器》 MobiMB Mobile Media Browser V3.1是一款专为Nokia手机设计的PC端连接工具,它允许用户深入到手机的隐藏根目录,进行文件的查看、编辑和管理。这款...

    Mobile.Browser 手机浏览器自适应文件

    Mobile.Browser 文件主要关注的是手机浏览器的自适应性技术,这是现代移动互联网体验中的关键要素。自适应设计允许网页和应用程序根据用户设备的屏幕尺寸、操作系统以及浏览器特性进行优化显示,提供良好的用户体验...

    cordova-plugin-media-capture, Apache Cordova插件媒体捕获镜像.zip

    cordova-plugin-media-capture, Apache Cordova插件媒体捕获镜像 标题说明媒体捕获捕获音频,视频和图像。AppVeyor Travis CI cordova-plugin-media-capture这里插件提供对设备。图像和视频捕获功能的音频访问。警告...

    Media Piracy in Emerging Economies

    Media Piracy in Emerging Economies

    Media-capture-capture-device-.zip_Capture

    本项目“Media-capture-capture-device-.zip_Capture”聚焦于微软Windows 8系统中应用商店应用的简单录音功能,这为开发者提供了一个理解和实践媒体捕获功能的实例。 首先,我们要了解Windows 8的媒体捕捉框架。这...

    Propagation of Sound in Porous Media

    developments in the domain of wave propagation in porous media and practical numerical prediction methods that are widley used by researchers and engineers. Chapters 1 to 3 dealing with sound ...

    Media Browser on Android Platform

    根据给定的文件信息,我们可以提炼出以下关于“Media Browser on Android Platform”的详细知识点: ### 一、项目背景与目标 - **项目发起方**:报告由乌梅奥大学计算机科学系硕士研究生Malin Berggren撰写,指导...

    Responsive Media in HTML5

    Rapid growth in the use of mobile devices to access the Internet has forced designers to adapt to creating content that is easily viewable on a wide range of different devices. The key is to make ...

    react-mediacapture:React Media Capture模块

    React Media Capture 模块是专为React应用设计的一个强大组件,它允许开发者集成多媒体捕捉功能,包括音频和视频记录,从而提升用户交互体验。在现代Web应用中,媒体捕获功能已经成为很多应用场景的关键部分,例如...

    Media Browser (Windows) for BMP

    "Media Browser (Windows) for BMP" 是一个专为BMP(Bitmap Image)格式设计的多媒体浏览工具,尤其适用于Windows操作系统。这款软件利用DLNA(Digital Living Network Alliance)技术,允许用户在家庭网络环境中...

    Media Browser RIA Tutorial

    Media Browser RIA Tutorial来自javafx官方网站

    Responsive.Media.in.HTML5

    Rapid growth in the use of mobile devices to access the Internet has forced designers to adapt to creating content that is easily viewable on a wide range of different devices. The key is to make ...

    诺基亚WAP开发工具箱 (Nokia Mobile Internet Toolkit 4.1) part4

    Nokia Mobile Internet Toolkit facilitates the creation of different mobile media content via a comprehensive set of editors and wizards. These include Web page creation (editors for creating WML, ...

    mbserver_Media_Browser_server

    【标题】"mbserver_Media_Browser_server" 指的是Media Browser服务器,这是一个用于管理和流媒体个人多媒体库的应用程序。Media Browser(现已更名为Emby Server)是一个强大的媒体中心解决方案,允许用户在本地...

Global site tag (gtag.js) - Google Analytics