Monday, February 18, 2008

Processing videos in Java

In this post, I would like to share with the visitors on how to

  1. How to load videos into Java using JMF
  2. How to extract frames from a video file like avi,mpg,etc using Java Media Framework.

The class I developed for this purpose is called VideoUtility which is similar to the Image Processing utility I wrote in Java. In order to process Videos in Java, you need to use Java Media Framework, which can be obtained from the Sun Website and should be installed. Note that the JMF sometimes can be installed as a platform dependant package which makes it more efficient than the platform-independent version. Anyway once you download the JMF library, add it to the application classpath. Refer to the code shown below. I appreciate if you keep my copyright intact and if you find proper use of this class, please let me know.

   1: import java.awt.Image;


   2: import java.io.File;


   3: import java.io.FileNotFoundException;


   4: import java.io.IOException;


   5: import java.util.ArrayList;


   6: import javax.media.Buffer;


   7: import javax.media.Manager;


   8: import javax.media.MediaLocator;


   9: import javax.media.NoPlayerException;


  10: import javax.media.Player;


  11: import javax.media.Time;


  12: import javax.media.control.FrameGrabbingControl;


  13: import javax.media.control.FramePositioningControl;


  14: import javax.media.format.VideoFormat;


  15: import javax.media.util.BufferToImage;


  16: /** @author Krishna Vangapandu **/


  17: public class VideoUtility


  18: {


  19:     @SuppressWarnings("deprecation") 


  20: /** * videoFile - path to the video File. */ 


  21: public static Player getPlayer(String videoFile) 


  22:     throws NoPlayerException, IOException


  23:     {


  24:         File f = new File(videoFile);


  25:         if (!f.exists()) throw new FileNotFoundException("File doesnt exist");


  26:         MediaLocator ml = new MediaLocator(f.toURL());


  27:         Player player = Manager.createPlayer(ml);


  28:         player.realize();


  29:         while (player.getState() != Player.Realized);


  30:         return player;


  31:     }


  32:     public static float getFrameRate(Player player)


  33:     {


  34:         return (float)noOfFrames(player)/(float)player.getDuration().getSeconds();


  35:     }


  36:     public static int noOfFrames(Player player)


  37:     {


  38:         FramePositioningControl fpc = (FramePositioningControl)player.getControl("javax.media.control.FramePositioningControl");


  39:         Time duration = player.getDuration();


  40:         int i = fpc.mapTimeToFrame(duration);


  41:         if (i != FramePositioningControl.FRAME_UNKNOWN) return i;


  42:         else return -1;


  43:     }


  44:     /** * * @param player - the player from which you want to get the image 


  45:     * @param frameNumber - the framenumber you want to extract 


  46:     * @return Image at the current frame position */ 


  47:   public static Image getImageOfCurrentFrame(Player player, int frameNumber)


  48:     {


  49:         FramePositioningControl fpc = (FramePositioningControl) player .getControl("javax.media.control.FramePositioningControl");


  50:         FrameGrabbingControl fgc = (FrameGrabbingControl) player .getControl("javax.media.control.FrameGrabbingControl");


  51:         return getImageOfCurrentFrame(fpc, fgc, frameNumber);


  52:     }


  53:  


  54:     public static Image getImageOfCurrentFrame(FramePositioningControl fpc, FrameGrabbingControl fgc, int frameNumber)


  55:     {


  56:         fpc.seek(frameNumber);


  57:         Buffer frameBuffer = fgc.grabFrame();


  58:         BufferToImage bti = new BufferToImage((VideoFormat) frameBuffer .getFormat());


  59:         return bti.createImage(frameBuffer);


  60:     }


  61:  


  62:     public static FramePositioningControl getFPC(Player player)


  63:     {


  64:         FramePositioningControl fpc = (FramePositioningControl) player .getControl("javax.media.control.FramePositioningControl");


  65:         return fpc;


  66:     }


  67:  


  68:     public static FrameGrabbingControl getFGC(Player player)


  69:     {


  70:         FrameGrabbingControl fgc = (FrameGrabbingControl) player .getControl("javax.media.control.FrameGrabbingControl");


  71:         return fgc;


  72:     }


  73:  


  74:     public static ArrayList<Image> getAllImages(Player player)


  75:     {


  76:         ArrayList<Image> imageSeq = new ArrayList<Image>();


  77:         int numberOfFrames = noOfFrames(player);


  78:         FramePositioningControl fpc = getFPC(player);


  79:         FrameGrabbingControl fgc = getFGC(player);


  80:         for (int i = 0;i <= numberOfFrames;i++)


  81:         {


  82:             Image img = getImageOfCurrentFrame(fpc, fgc, i);


  83:             if(img!=null) imageSeq.add(img);


  84:         }


  85:         return imageSeq;


  86:     }


  87:  


  88:     public static ArrayList<Image> getAllImages(String fileName) throws NoPlayerException,IOException


  89:     {


  90:         Player player = getPlayer(fileName);


  91:         ArrayList<Image> img = getAllImages(player);


  92:         player.close();


  93:         return img;


  94:     }


  95: }



The main method of interest here is the "getAllImages()" method which returns all the frames as an arraylist of Images. For the specified filename it obtains a Player that is used to retrieve all the images. The reason I posted this article is that I find quiet a few visits to my blog to my article on Image Processing. I am not sure on how useful that post was but I thought may be there is someone out there who might need to get started to extract frames from videos. I put in a lot of effort and looked up a lot of material to actually come up with some code that makes sense.

5 comments:

Prashant Chandrakar said...

Hi krishna,

I tried your code.It working fine for me.But i have a lot of problem with many of compression type.

When i try to open MPEG1,MPEG2, Cinepack compressed files it is giving me error.

Please can you help me in that issue.

I am crating a media player which supports many of the format.
After opening particular video i want to take image array from movies.
And i have to drawn my image on JPanel.

JMF support many format for me.
I tried same cineack ,MPEG1 and MPEG2 videos with JMF studio and it works fine.

Indies tester said...

it is not working with .mp4 and generate error like

javax.media.NoPlayerException: Cannot find a Player for :file:/C:/Users/Desktop/test.mp4
at javax.media.Manager.createPlayerForContent(Manager.java:1412)
at javax.media.Manager.createPlayer(Manager.java:417)
at javax.media.Manager.createPlayer(Manager.java:332)
at com.toocoolproductions.model.service.impl.UserSettingsServiceImpl.getVideoThumbnailSmall(UserSettingsServiceImpl.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

Indies tester said...

Hi..

Nice explanation but it is not working with .mp4 and generate error like

javax.media.NoPlayerException: Cannot find a Player for :file:/C:/Users/indies-22/Desktop/test.mp4
at javax.media.Manager.createPlayerForContent(Manager.java:1412)
at javax.media.Manager.createPlayer(Manager.java:417)
at javax.media.Manager.createPlayer(Manager.java:332)
at com.toocoolproductions.model.service.impl.UserSettingsServiceImpl.getVideoThumbnailSmall(UserSettingsServiceImpl.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

Indies tester said...

Hi..

Nice explanation but it is not working with .mp4 and generate error like

javax.media.NoPlayerException: Cannot find a Player for :file:/C:/Users/indies-22/Desktop/test.mp4
at javax.media.Manager.createPlayerForContent(Manager.java:1412)
at javax.media.Manager.createPlayer(Manager.java:417)
at javax.media.Manager.createPlayer(Manager.java:332)
at com.toocoolproductions.model.service.impl.UserSettingsServiceImpl.getVideoThumbnailSmall(UserSettingsServiceImpl.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

Indies tester said...

Hi..

Nice explanation but it is not working with .mp4 and generate error like

javax.media.NoPlayerException: Cannot find a Player for :file:/C:/Users/indies-22/Desktop/test.mp4
at javax.media.Manager.createPlayerForContent(Manager.java:1412)
at javax.media.Manager.createPlayer(Manager.java:417)
at javax.media.Manager.createPlayer(Manager.java:332)
at com.toocoolproductions.model.service.impl.UserSettingsServiceImpl.getVideoThumbnailSmall(UserSettingsServiceImpl.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)