Extracting JPGs from MJPG

I have a camera that serves MJPG, and I want to display the live image(s) in a "PictureBox".

Here is what I have so far:

private void button1_Click(object sender, System.EventArgs e)
{
      HttpWebRequest httpRequest;
      HttpWebResponse httpResponse;
      Stream responseStream;
      Int32 bytes;
      httpRequest = (HttpWebRequest) WebRequest.Create("http://24.116.133.73/axis-cgi/mjpg/video.cgi?resolution=320x240");
      httpResponse = (HttpWebResponse) httpRequest.GetResponse();
      responseStream = httpResponse.GetResponseStream();

      for(int i = 0; i<httpResponse.Headers.Count; i++)
      {
            Debug.WriteLine(httpResponse.Headers.GetKey(i) + " " + httpResponse.Headers.Get(i));
      }

      Debug.WriteLine("BEGIN TRANSFER");

      while(true)
      {
            Byte[] RecvBytes = new Byte[Byte.MaxValue];
            bytes = responseStream.Read(RecvBytes, 0, RecvBytes.Length);
            Debug.WriteLine(Encoding.ASCII.GetString(RecvBytes, 0, RecvBytes.Length));
            if (bytes<=0) break;
            //MemoryStream ms = new MemoryStream(RecvBytes, 0, RecvBytes.Length);
            //pictureBox1.Image = Image.FromStream(ms); <---- Throws exception
      }      
}

I set up a test camera for this post, and the MJPG stream is here: "http://24.116.133.73/axis-cgi/mjpg/video.cgi?resolution=320x240"
To view the motion, you need Mozilla, since MJPG is not supported in IE.
When run I get Debug information similiar to this:

-----BEGIN-----

--myboundary
Content-Type: image/jpeg
Content-Length: 3159

(BINARY)

--myboundary
Content-Type: image/jpeg
Content-Length: 3162

(BINARY)

--myboundary
Content-Type: image/jpeg
Content-Length: 3151

(BINARY)

-----END-----

I need to get this byte array to a state that can be loaded into the PictureBox, from there I hope to use the same concept to store in DB, or save to disk.

Thanks,

Joel
j_ollerAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

cupawntaeCommented:
Well one problem is this:

new Byte[Byte.MaxValue];

Byte.MaxValue is 255 so you're always trying to read 255 bytes. Next:

new MemoryStream(RecvBytes, 0, RecvBytes.Length);

You're always trying to create a MemoryStream of 255 bytes, no matter how many were read.

So unless your frames happen to be 255 bytes long this won't work.

You need to read and parse the Content-Length header in each section, and read that number of bytes from the stream. You also need a much bigger Byte array to hold this. And if your Read() returns less than the expected number of bytes, you need to read again to fill the buffer.

Finally, if you have a while (true) loop in an event handler, you will hang your app and the pictureBox will never get to show its image.

I'll post a code snippet shortly!
cupawntaeCommented:
Try this (note there's no error handling and it's pretty yukky - if your image has less than 1000 bytes or more than 9999 it'll fail etc.) but it should give you the idea...

private void button1_Click(object sender, System.EventArgs e) {
    System.Threading.Thread thread;
    thread=new System.Threading.Thread(new System.Threading.ThreadStart(this.goFetch));
    thread.Start();
}

private void goFetch() {
    HttpWebRequest httpRequest;
    HttpWebResponse httpResponse;
    Stream responseStream;
    Int32 bytes;
    httpRequest = (HttpWebRequest) WebRequest.Create("http://24.116.133.73/axis-cgi/mjpg/video.cgi?resolution=320x240");
    httpResponse = (HttpWebResponse) httpRequest.GetResponse();
    responseStream = httpResponse.GetResponseStream();

    int loops=10;
    while(loops-->0) {
        Byte[] RecvBytes = new Byte[64];
        bytes = responseStream.Read(RecvBytes, 0, RecvBytes.Length);
   
        string preamble=Encoding.ASCII.GetString(RecvBytes, 0, RecvBytes.Length);
        string lenStr=preamble.Substring(preamble.LastIndexOf(' ')+1).Trim();
        int bytesToRead=Convert.ToInt32(lenStr);

        RecvBytes=new Byte[bytesToRead];
        while (bytesToRead>0) {
            bytes = responseStream.Read(RecvBytes, RecvBytes.Length-bytesToRead, bytesToRead);
            bytesToRead-=bytes;
        }
        MemoryStream ms = new MemoryStream(RecvBytes, 0, RecvBytes.Length);
        pictureBox1.Image = Image.FromStream(ms);

        responseStream.Read(RecvBytes,0,2); // CRLF after bytes
    }
}
cupawntaeCommented:
PS Your linksys AP is a bit boring, you could have pointed the camera out the window or something ;-)
OWASP: Threats Fundamentals

Learn the top ten threats that are present in modern web-application development and how to protect your business from them.

cupawntaeCommented:
PPS just changed that loop to infinite and re-ran it - it breaks fairly quickly when the first Read doesn't fetch the whole "preamble" in one go - so you'll need a similar loop to make sure you get the whole thing. You'll probably want to read that differently anyway to better handle variations in the headers.
cupawntaeCommented:
PPPS you should allocate your buffer outside the loop (and adjust the length parameter to the MemoryStream constructor) to avoid creating thousands of byte arrays that need to be garbage collected

Also, you probably want to avoid creating multiple threads doing the same thing. And provide for some way of stopping the thread when you're done.

:-}
j_ollerAuthor Commented:
cupawntae, wow, that was quick, and gave me an good direction, so thanks for the snippets. Your right when it comes to allocating the buffer outside of the loop...

I put it in an endless loop just for testing, I agree it is sloppy, and it should have a way to stop, and I will work on that when I get the basics down, but thats the least of my worries at this point.

I may be wrong, but for testing, I think a Application.DoEvents() will make the pictureBox1 update.

I am having a hard time visualizing the code, when it comes to handling variations in the headers, and how the buffer should be declared, I am looking forward to some more golden code, that will get me to where I need to be...

Thanks,

Joel
cupawntaeCommented:
Yeah, Application.DoEvents() will do this - I'm just not sure about having the main event loop's thread infinitely executing your button's event handler code... Could be fine, just feels a little wrong to me!

So you're really going to make me work for these points, yeah?

I'll see what I can do...
cupawntaeCommented:
By the way, did you try out that code? It actually does work... For a while...
j_ollerAuthor Commented:
No, I am at work, I have no way to try it... I am looking forward to hopefully getting home and having a production quality working solution...

I am open to perhaps awarding someone more points, like maybe create another post, and giving another 500 points, on top of the points for this post, for the production quality code, after I have had a chance to test it...

cupawntae, that means 1000 points for clean production code, if you are the only one who posts decent code...
cupawntaeCommented:
Here ya go...

private void button1_Click(object sender, System.EventArgs e) {
    System.Threading.Thread thread;
    thread=new System.Threading.Thread(new System.Threading.ThreadStart(this.goFetch));
    thread.Start();
}

private static int findLength(Stream stream)  {
    int b;
    string line="";
    int result=-1;
    bool atEOL=false;

    while ((b=stream.ReadByte())!=-1) {
        if (b==10) continue; // ignore LF char
        if (b==13) { // CR
            if (atEOL) {  // two blank lines means end of header
                stream.ReadByte(); // eat last LF
                return result;
            }
            if (line.StartsWith("Content-Length:")) {
                result=Convert.ToInt32(line.Substring("Content-Length:".Length).Trim());
            } else {
                line="";
            }
            atEOL=true;
        } else {
            atEOL=false;
            line+=(char)b;
        }
    }
    return -1;
}

private void goFetch() {
    HttpWebRequest httpRequest;
    HttpWebResponse httpResponse;
    Stream responseStream;
    httpRequest = (HttpWebRequest) WebRequest.Create("http://24.116.133.73/axis-cgi/mjpg/video.cgi?resolution=320x240");
    httpResponse = (HttpWebResponse) httpRequest.GetResponse();
    responseStream = new BufferedStream(httpResponse.GetResponseStream());

    const int MAX_JPG_SIZE=65536; // 64k

    Byte [] JpegData = new Byte[MAX_JPG_SIZE];

    while(true) {
        int bytesToRead=findLength(responseStream);

        if (bytesToRead==-1) {
            MessageBox.Show("End of stream");
            return;
        }

        int leftToRead=bytesToRead;

        while (leftToRead>0) {
            leftToRead -= responseStream.Read(JpegData, bytesToRead-leftToRead, leftToRead);
        }

        MemoryStream ms = new MemoryStream(JpegData, 0, bytesToRead);
        pictureBox1.Image = Image.FromStream(ms);

        responseStream.ReadByte(); // CR after bytes
        responseStream.ReadByte(); // LF after bytes
    }
}

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
cupawntaeCommented:
Erp, meant to put a check in there beside  if (bytesToRead==-1) - you should check if it's greater than MAX_JPG_SIZE and if so display a message (or allocate a bigger buffer).
j_ollerAuthor Commented:
This seems to be working good. The first thing that catches my eye is:

        responseStream.ReadByte(); // CR after bytes
        responseStream.ReadByte(); // LF after bytes

why are there two "responseStream.ReadByte();" in a row?

I added:

      if (bytesToRead>JpegData.Length)
      {
            JpegData = new Byte[bytesToRead];
      }

right above

        if (bytesToRead==-1) {
            MessageBox.Show("End of stream");
            return;
        }

in case the buffer needs to be resized...

Is this what you had in mind?
cupawntaeCommented:
Looks good. The two reads in a row are to get two characters (CR followed by LF).

I'll look at the other posts in a bit, off to work now!
daniel142005Commented:
Just in case anyone is using VLC as their MJPEG source, here is the code I used to read it since VLC provides the content as a octet-stream with very limited header information. It's probably not perfect, but it seems to be working.
private bool stopRequested = false;
        public Form1()
        {
            InitializeComponent();
            Debug.WriteLine("Starting");
                Debug.Flush();
        }
 
        private void button1_Click(object sender, System.EventArgs e)
        {
            stopRequested = false;
            System.Threading.Thread thread;
            thread = new System.Threading.Thread(new System.Threading.ThreadStart(this.goFetch));
            thread.Start();
        }
 
        private void button2_Click(object sender, EventArgs e)
        {
            stopRequested = true;
        }
        private static int findLength(Stream stream)
        {
            int b;
            string line = "";
            int result = -1;
            bool atEOL = false;
 
            while ((b = stream.ReadByte()) != -1)
            {
                if (b == 10) continue; // ignore LF char
                if (b == 13)
                { // CR
                    if (atEOL)
                    {  // two blank lines means end of header
                        stream.ReadByte(); // eat last LF
                        return result;
                    }
                    if (line.StartsWith("Content-Length:"))
                    {
                        result = Convert.ToInt32(line.Substring("Content-Length:".Length).Trim());
                    }
                    else
                    {
                        line = "";
                    }
                    atEOL = true;
                }
                else
                {
                    atEOL = false;
                    line += (char)b;
                }
            }
            return -1;
        }
 
        private void goFetch()
        {
            HttpWebRequest httpRequest;
            HttpWebResponse httpResponse;
            Stream responseStream;
            Int32 bytes;
            httpRequest = (HttpWebRequest)WebRequest.Create("http://192.168.0.199:4570/");
            httpResponse = (HttpWebResponse)httpRequest.GetResponse();
            responseStream = new BufferedStream(httpResponse.GetResponseStream());
            for (int i = 0; i < httpResponse.Headers.Count; i++)
            {
                Debug.WriteLine(httpResponse.Headers.GetKey(i) + " " + httpResponse.Headers.Get(i));
            }
 
            const int MAX_JPG_SIZE = 65536 * 10; // 64k
 
            Byte[] JpegBuffer = new Byte[MAX_JPG_SIZE];
            Int32 totBytes = 0;
            while (true)
            {
                if (stopRequested == true)
                {
                    Debug.WriteLine("Exit");
                    break;
                }
                Byte[] RecvBytes = new Byte[2];
                bytes = responseStream.Read(RecvBytes, 0, RecvBytes.Length);
                if (bytes <= 0)
                {
                    Debug.WriteLine("Exit");
                    break;
                }
                if (RecvBytes[0] == 255 && RecvBytes[1] == 216)
                {
                    if (totBytes > 0)
                    {
                        // We have an image, do something with it.
                        MemoryStream ms = new MemoryStream(JpegBuffer, 0, totBytes);
                        pictureBox1.Image = Image.FromStream(ms);
                    }
                    Debug.WriteLine("");
                    Debug.WriteLine("NEW IMAGE! Last Size: " + totBytes);
                    JpegBuffer = new Byte[MAX_JPG_SIZE];
                    totBytes = 0;
                }
                JpegBuffer[totBytes] = RecvBytes[0];
                JpegBuffer[totBytes + 1] = RecvBytes[1];
                totBytes += 2;
            }
        }

Open in new window

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C#

From novice to tech pro — start learning today.