Java Media Framework本地玩转摄像头

1、简介
The JavaTM Media Framework (JMF) is an application programming interface (API) for incorporating media data such as audio and video into
Java applications and applets. It is specifically designed to take advantage of Java platform features.

这个框架是SUN公司出的，最新更新至2003年，用JMF来处理图像和音频设备并直接应用到JAVA应用程序中，虽然年代有点久远，但却还是可以用的。

 

The JMF 2.1.1 Reference Implementation supports SunVideo / SunVideoPlus capture devices on Solaris. On Windows, most capture devices
 that have VFW drivers are supported. On Linux, devices that have a Video4Linux driver are expected to work, but not extensively tested.
今天小试了下，自己的laptop也有个摄像头，录制了一段视频到本地，第一次玩这个东西：）

 

2、模型

JMF的模型如下：

基本上是摄像头数据、文件数据、网络媒体流通过process输出到设备、文件或网络中。

 

Design Goals for JMF：

1、Be easy to program

2、Support capturing media data

3、Enable the development of media streaming and conferencing applications in Java

4、Enable advanced developers and technology providers to implement custom solutions based on the existing API and easily integrate new features with the existing framework

5、Provide access to raw media data

6、Enable the development of custom, downloadable demultiplexers, codecs, effects processors, multiplexers, and renderers (JMF plug-ins)

 

Design Goals for RTP：（RTP是JMF数据流在网络上的传输协议）

1、Enable the development of media streaming and conferencing applications in Java

2、Support media data reception and transmission using RTP and RTCP

3、Support custom packetizer and depacketizer plug-ins through the JMF 2.0 plug-in architecture.

4、Be easy to program

 

 3、媒体数据展现

 Streaming Media

 媒体流，可以存储为多种文件格式或通过HTTP,RTP等协议传输在网络上。

 

Media Presentation

Most time-based media is audio or video data that can be presented through output devices such as speakers and monitors. Such devices are the

most common destination for media data output. Media streams can also be sent to other destinations--for example, saved to a file or transmitted

across the network. An output destination for media data is sometimes referred to as a data sink.

 

Media Data Storage and Transmission

 
A DataSink is used to read media data from a DataSource and render the media to some destination--generally a destination other

than a presentation device. A particular DataSink might write data to a file, write data across the network, or function as an RTP broadcaster.

 

DataSink用来做数据存储和传输。

 

开始presenting time-based media

 

Presenting Time-Based Media

To present time-based media such as audio or video with JMF, you use a Player. Playback can be controlled programmatically, or you can display a control-panel component that enables the user to control playback interactively. If you have several media streams that you want to play,

you need to use a separate Player for each one. to play them in sync, you can use one of the Player objects to control the operation of the others.

 

基本上是说播放实时多媒体需要用到Player，且每个单独的媒体需要单独的Player。

代码如下：

// create processor   
        ProcessorModel processorModel = new ProcessorModel(mixedDataSource, outputFormat, outputType);    
// 创建model，类似JTable控件的 JModel，The ProcessorModel defines the input and output requirements for the Processor    
        Processor processor = null;    
        try   
        {   
            processor = Manager.createRealizedProcessor(processorModel);// 创建model，类似JTable控件的 JModel
        }   
        catch (IOException e) { Stdout.logAndAbortException(e); }    
        catch (NoProcessorException e) { Stdout.logAndAbortException(e); }    
        catch (CannotRealizeException e) { Stdout.logAndAbortException(e); } 

 

创建processor后，可以装进不同的compont里来present。

 

For example, you can call getControls to determine if a Player supports the CachingControl interface.

 

Control[] controls = player.getControls(); 

for (int i = 0; i < controls.length; i++) 
{ 
  if (controls[i] instanceof CachingControl) 
   { cachingControl = (CachingControl) controls[i]; 
   } 
}

 

JMF提供了很多“播放器”，当然也提供了基本的控制，比如播放、停止以及播放信息等等。

控制产生事件。

Responding to Media Events

ControllerListener is an asynchronous interface for handling events generated by Controller objects.

for example:

if (event instanceof EventType){   
 ...   
 } else if (event instanceof OtherEventType){    
 ...   
 }  
if (event instanceof EventType){ ... } else if (event instanceof OtherEventType){ ... } 

  JMF提供ControllerAdapter：

player.addControllerListener(new ControllerAdapter() {
    public void endOfMedia(EndOfMediaEvent e) {
       Controller controller = e.getSource();
       controller.stop();
       controller.setMediaTime(new Time(0));
       controller.deallocate();
    }
 })

 

ControllerAdapter automatically dispatches the event to the appropriate event method, filtering out the events that you're not

 interested in.自动为您分发事件并过滤不需要的事件。

 

There is a movie in Applet...展现到applet

import java.applet.*;   
import java.awt.*;    
import java.net.*;    
import javax.media.*;    
  
public class PlayerApplet extends Applet implements ControllerListener {    
   Player player = null;    
   public void init() {    
      setLayout(new BorderLayout());    
      String mediaFile = getParameter("FILE");    
      try {    
         URL mediaURL = new URL(getDocumentBase(), mediaFile);    
         player = Manager.createPlayer(mediaURL);   
         player.addControllerListener(this);    
      }    
      catch (Exception e) {    
         System.err.println("Got exception "+e);    
      }   
   }   
   public void start() {    
      player.start();   
   }   
   public void stop() {    
      player.stop();   
      player.deallocate();   
   }   
   public void destroy() {    
      player.close();   
   }   
   public synchronized void controllerUpdate(ControllerEvent event) {    
      if (event instanceof RealizeCompleteEvent) {    
         Component comp;   
         if ((comp = player.getVisualComponent()) != null)    
            add ("Center", comp);     
         if ((comp = player.getControlPanelComponent()) != null)    
            add ("South", comp);             
         validate();   
      }   
   }   
}  

 

 

 Presenting RTP Media Streams

Creating a Player for RTP session.

 

String url= "rtp://224.144.251.104:49150/audio/1";   
  
        MediaLocator mrl= new MediaLocator(url);    
           
        if (mrl == null) {    
            System.err.println("Can't build MRL for RTP");    
            return false;    
        }   
           
        // Create a player for this rtp session    
        try {    
            player = Manager.createPlayer(mrl);   
        } catch (NoPlayerException e) {    
            System.err.println("Error:" + e);    
            return false;    
        } catch (MalformedURLException e) {    
            System.err.println("Error:" + e);    
            return false;    
        } catch (IOException e) {    
            System.err.println("Error:" + e);    
            return false;    
        }  

 

4、媒体数据处理

Converting Media Data from One Format to Another

你可以调整媒体的格式。

Specifying the Media Destination

你可以指定录像保存的方式，比如输出到文件， 输出到另一个player。

 

 Saving captured media data to a file

 

DataSink sink;   
 MediaLocator dest = new MediaLocator("file://newfile.wav");    
 try {    
     sink = Manager.createDataSink(p.getDataOutput(), dest);    
     sink.open();   
     sink.start();   
 } catch (Exception) {}  

 

 

5、扩展JMF

 

    Custom JMF plug-ins can be used seamlessly with Processors that support the plug-in API. After you implement your plug-in, you need to install it and register it with the PlugInManager to make it available to plug-in compatible Processors.

 

Implementing a Codec or Effect Plug-In

    Codec plug-ins are used to decode compressed media data, convert media data from one format to another, or encode raw media data into a compressed format.

    用户可以自行扩展压缩格式，以及格式转换。
Implementing a Renderer Plug-In

It is a single-input processing component with no output. Renderer plug-ins read data from a DataSource and typically present the media data to the user, but can also be used to provide access to the processed media data for use by another application or device.


To make a custom plug-in available to a Processor through the TrackControl interface, you need to register it with the PlugInManager

 

 // Name of the new plugin
 string GainPlugin = new String("COM.mybiz.media.GainEffect");
 
 // Supported input Formats
 Format[] supportedInputFormats = new Format[] {
 	     new AudioFormat(
 	         AudioFormat.LINEAR,
                 Format.NOT_SPECIFIED,
                 16,
                 Format.NOT_SPECIFIED,
                 AudioFormat.LITTLE_ENDIAN,
                 AudioFormat.SIGNED,
                 16,
                 Format.NOT_SPECIFIED,
                 Format.byteArray

 	     )
 };
 
 // Supported output Formats 
 Format[] supportedOutputFormats = new Format[] {
 	     new AudioFormat(
 	         AudioFormat.LINEAR,
                 Format.NOT_SPECIFIED,
                 16,
                 Format.NOT_SPECIFIED,
                 AudioFormat.LITTLE_ENDIAN,
                 AudioFormat.SIGNED,
                 16,
                 Format.NOT_SPECIFIED,
                 Format.byteArray
 	     )
 };
 
 // Add the new plug-in to the plug-in registry
 PlugInManager.addPlugIn(GainPlugin, supportedInputFormats, 
                          supportedOutputFormats, EFFECT);
 
 // Save the changes to the plug-in registry
 PlugInManager.commit();

 

 

Implementing a Protocol Data Source

A DataSource is an abstraction of a media protocol-handler. You can implement new types of DataSources to support additional protocols by extending PullDataSource,


Implementing a DataSink

JMF provides a default DataSink that can be used to write data to a file. Other types of DataSink classes can be implemented to facilitate writing data to the network or to other destinations.

 

6、事例代码

    this test program will capture the video and audio stream from your USB camera for 10 seconds and stores it on a file, named "testcam.avi".