com_exception when sending message (<2MB) to MSMQ using PHP

I'm trying to send messages to MSMQ using PHP. When the message is smaller than approximately 1.1 MB the message is sent. Otherwise, I get exception "com_exception: There are insufficient resources to perform this operation." The message is smaller than the limit 4 MB. This is what I've found out so far:

- file_get_contents and file_put_contents with PHP works with no problem, so upload limit should be fine
- sending same message not with php works with no problem, so it is somehow PHP-related
- purging messages from queue does not help
- no dead-letter messages

php.ini settings:

upload_max_filesize=20M
memory_limit=512M

There is no limit on message size in msmq properties.
Kvapka KvaplaAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

gr8gonzoConsultantCommented:
This isn't really a PHP issue. You might be using PHP to send the data, but the error that's coming back is coming from MSMQ, which is notorious for the "insufficient resources" error message.

There's several possible reasons for MSMQ to return that message, but the best write-up I've seen on it is this blog article:
https://blogs.msdn.microsoft.com/johnbreakwell/2006/09/18/insufficient-resources-run-away-run-away/

That said, my gut feeling is that this is due to a character encoding issue. Here's the corresponding excerpt from that blog article:
6    The message size is too large.

MSMQ (up to and including 3.0) can only send messages as large as 4MB. Anything larger throws an error. Note that this is 4 million bytes and NOT 4 million characters - UNICODE characters are double-byte representations and so you can only send 2 million characters from a standard Windows application. MSMQ logging (as documented in the MSMQ FAQ) allows us to check if the message size exceeds the 4MB limit.  Log entries similar to the following will occur where the Data indicates the hex size of the message:

0x488> Wed Apr 02 11:34:19 2003: DRIVER Error: heap/1235,  HR: 0xc000009a, Data-0x560600
0x478> Wed Apr 02 11:34:19 2003: DRIVER Error: Packet/150, HR: 0xc000009a, Data-0x0


0xC000009A means INSUFFICIENT RESOURCES
0x560600 = 5,637,632 bytes in decimal

You said "When the message is smaller than approximately 1.1 MB  the message is sent." So the fact that a smaller message can be sent will eliminate several reasons, and could mean that you might be miscalculating the message size. For example, if you're working with UTF-16-encoded data, then each character will usually take up 2 bytes (on rare occasions 3 bytes). If you're working with UTF-8 data, then most "ASCII" bytes will take up 1 byte and most special characters will take up 2 bytes, and you'll occasionally have a 3-byte character. In other words, a 2-million character string might end up taking over 4 MB.

That said, that blog article I quoted shows you how to confirm whether this is the problem. Check that "Data" parameter in the log messages of the failed entries and see if it shows you a value that exceeds 4 MB.
1

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Kvapka KvaplaAuthor Commented:
Thank you very much for your response and detailed explanation. The messages has sometimes more then 2-million characters indeed. I'd like to set message encoding to ASCII - I'd do base64 encoding and then it should be fine to send it as ASCII. The problem is that I haven't find a way to set MSMQ message encoding to ASCII in PHP.
 For C# you can set the encoding. I've tried to get all the variables for the COM class like this

 $oExplorer = new COM("MSMQ.MSMQMessage");
        com_print_typeinfo($oExplorer, 'IMSMQMessage4');

Open in new window


But there is no such attribute as
BodyEncoding

Open in new window

. Do you think there is any way how to achieve it?
0
NerdsOfTechTechnology ScientistCommented:
Another important note: With BASE64 encoding you will increase the message length by ~133%
given an input of n bytes, the output will be (n + 2 - ((n + 2) % 3)) / 3 * 4 bytes long, so that the number of output bytes per input byte converges to 4 / 3 or 1.33333 for large n.
-Wikipedia

Also, I'm not sure what you mean by going from BASE64 to ASCII as they are not interchangeable data-wise.
0
gr8gonzoConsultantCommented:
I'd like to set message encoding to ASCII - I'd do base64 encoding and then it should be fine to send it as ASCII.

That's a bad idea. As NerdsOfTech pointed out, Base64 encoding has an even higher amount of overhead.

For example, take the string: "My two ¢"

The Experts Exchange site (and most web sites) uses UTF-8, and the above string ends with the Unicode character "¢" which takes up 2 bytes in UTF-8. So that string has a total of 8 characters, but takes up 9 bytes. All of the other characters only take up 1 byte.

Now if we base64-encode our 8-character/9-byte string to make it all ASCII, the result is "TXkgdHdvIMKi", which is 12 bytes.

Remember, Base64 is trying to represent a the FULL range of bytes using ONLY a limited set of characters. Imagine you have to give a speech, but you have trouble pronouncing a particular letter, like "R", so whenever you have that letter in a word, you replace it with a letter that you CAN pronounce, like "L". However, you wouldn't know when an "L" actually meant an "L" or an "R", so you'd have to come up with a sort of way to rewrite the whole speech so it was consistently using ONLY the characters you could pronounce, and in a way that could be read so the original words would still be said. You'd sort of have to rewrite the whole speech in a new language that only had the letters you could pronounce. That's what Base64 encoding is doing. It's taking a speech that is in a language that has 256 letters and rewriting it using a language that only has 64 letters. And since it has LESS available characters than the original, it has to use MORE of the 64 letters to be able to represent the full original word. So Base64 is ALWAYS going to be bigger than the original and it's almost always going to be exactly 1/3 bigger.

For encoding, UTF-8 is your best bet if most of your characters in your message are already within the typical "ASCII" range, like numbers and letters within the standard English alphabet. With UTF-8 encoding, your byte length will often be the same as the number of characters. You'll only end up taking MORE bytes if you actually have a multibyte Unicode character in there.

If the majority of your characters in your message are Unicode multibyte characters, then you're likely better off with UTF-16. The UTF-16 character set uses 2 bytes for every character, but it almost never uses 3 bytes (except in very rare circumstances).

Either way, you need to have code that checks and controls the length of your message. You can use strlen() to check the number of BYTES that a string has (regardless of characters). If it exceeds the 4 MB message size limit, then it's up to you how you want to proceed from there.

Also, if your string is currently UTF-16 and you want to convert it to UTF-8, the simplest method is to use the mbstring extension's "mb_convert_encoding" function:
http://php.net/manual/en/function.mb-convert-encoding.php

If you find Unicode and UTF-8 and UTF-16 confusing, check out my "beginner's" article on it:
https://www.experts-exchange.com/articles/25999/Unicode-UTF-8-and-Multibyte-in-Plain-English.html
1
NerdsOfTechTechnology ScientistCommented:
Encoding issue can be resolved using gr8gonzo's solution; assist points for BASE64 explanation.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Chat / IM

From novice to tech pro — start learning today.