WinHTTP and canonicalize parameter string

Posted on 2017-07-14
Medium Priority
Last Modified: 2017-07-31
We just moved from WinINET to WinHTTP. The latter does not have an equivalent InternetCanonicalizeUrl function.

The WinHTTP docs say that WinHTTPOpenRequest does this automatically but there is nothing there that specifically addresses this.

According to this reference you can use WinHTTPCreateURL which uses a URL_COMPONENTS structure to define the various URL parts. In the latter the lpszExtraInfo property is where you put the ?param=xyz& ... part.

Here is my problem - one of the parameters in the paramter string needs to be canonicalized as it could contain non-URL friendly characters (Example &).

With the WinINET version we used InternetCanonicalizeUrl on the specific URL parts which were then appended to the URL string.

If we have a URL now that looks like this
mydomain.com?param='Name & Surname','Description'

Open in new window

And we pass that to WinHTTPOpenRequest - it does not know that the & in Name & Surname is data and not a parameter separator.

How do we canonicalize individual parameters in the URL with WinHTTP
Question by:Julian Hansen
  • 3
  • 3
LVL 31

Expert Comment

ID: 42215958
Hi Julian Hansen,

in the linked document about WinHttpCreateUrl (https://msdn.microsoft.com/en-us/library/windows/desktop/aa384093(v=vs.85).aspx) there's a description of the passed parameter DWORD dwFalgs:

Converts all unsafe characters to their corresponding escape sequences in the path string pointed to by the lpszUrlPath member and in lpszExtraInfo the extra-information string pointed to by the member of the URL_COMPONENTS structure pointed to by the lpUrlComponents parameter.
I think this is the same as using InternetCanonicalizeUrl.

Hope this helps,

LVL 60

Author Comment

by:Julian Hansen
ID: 42216504
Thanks Zoppo but as I mentioned in my post what if my lpszExtra is

groups='Group 1 & 2','Users','Group 3 & 4'&user=fred&ip=

I want that to come out as

Open in new window

Not as

Open in new window

We are taking the same code we had working for WinINET that had a step to run the different parameters through InternetCanonicalizeUrl and passed that to WinHTTP but the query is failing - when we look at the logs it is because WinHTTP is seeing the & - data as parameter separators
LVL 35

Assisted Solution

sarabande earned 400 total points
ID: 42219024
you could replace the & and = by sequences like "[amb]" and "[eq]" and after WinHTTPCreateURL undo the replacement.

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

LVL 31

Accepted Solution

Zoppo earned 1600 total points
ID: 42219099
It's a bit strange - I just used the sample code from https://msdn.microsoft.com/de-de/library/windows/desktop/aa384093(v=vs.85).aspx, added the ICU_ESCAPE in both calls to WinHttpCreateUrl and did some minor changes to use the string you posted above with the sample URL, i.e.:
void foo()
	LPCWSTR pwszUrl1 = L"http://search.msn.com/results.asp?info=test";
	DWORD dwUrlLen = 0;

	// Initialize the URL_COMPONENTS structure.
	ZeroMemory( &urlComp, sizeof( urlComp ) );
	urlComp.dwStructSize = sizeof( urlComp );

	// Set required component lengths to non-zero, 
	// so that they are cracked.
	urlComp.dwSchemeLength = (DWORD)-1;
	urlComp.dwHostNameLength = (DWORD)-1;
	urlComp.dwUrlPathLength = (DWORD)-1;
	urlComp.dwExtraInfoLength = (DWORD)-1;

	// Crack the URL.
	if ( !WinHttpCrackUrl( pwszUrl1, (DWORD)wcslen( pwszUrl1 ), 0, &urlComp ) )
		printf( "Error %u in WinHttpCrackUrl.\n", GetLastError() );
		// Change the search data. New data is the same length.
		urlComp.lpszExtraInfo = L"?groups='Group 1 & 2','Users','Group 3 & 4'&user=fred&ip=";
		urlComp.dwExtraInfoLength = wcslen( urlComp.lpszExtraInfo );

		// Obtain the size of the new URL and allocate memory.
		WinHttpCreateUrl( &urlComp, ICU_ESCAPE, NULL, &dwUrlLen );
		LPWSTR pwszUrl2 = new WCHAR[dwUrlLen];

		// Create a new URL.
		if ( !WinHttpCreateUrl( &urlComp, ICU_ESCAPE, pwszUrl2, &dwUrlLen ) )
			printf( "Error %u in WinHttpCreateUrl.\n", GetLastError() );
			// Show both URLs.
			printf( "Old URL:  %S\nNew URL:  %S\n", pwszUrl1, pwszUrl2 );

		// Free allocated memory.
		delete[] pwszUrl2;

Open in new window

When I execute this the output is:
Old URL:  http://search.msn.com/results.asp?info=test
New URL:  http://search.msn.com/results.asp?groups='Group%201%20&%202','Users','Group%203%20&%204'&user=fred&ip=

Open in new window

For any reason this differs from the result you posted above.

LVL 60

Author Comment

by:Julian Hansen
ID: 42219176
Interesting - our app throws out very verbose logs and it definitely shows that the URL was not encoded.

I am going to match your code to ours and see what is different.

Thanks for this - appreciate the second pair of eyes and the input.
LVL 31

Expert Comment

ID: 42219222
No problem, you know, that's what we're here for :o)
LVL 60

Author Closing Comment

by:Julian Hansen
ID: 42236167
I have not had time to implement but Zoppo is pointing me in the right direction.

Sarabande's solution also has some merit - I will look into that as a fallback.

Thank you both for responding - apologies for the late closing of the question.

Featured Post

Nothing ever in the clear!

This technical paper will help you implement VMware’s VM encryption as well as implement Veeam encryption which together will achieve the nothing ever in the clear goal. If a bad guy steals VMs, backups or traffic they get nothing.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this modest contribution, I want to share with the IT community (especially system administrators, IT Support Engineers and IT Help Desks) about Windows crashes/hangs and how to deal with these particular problems.
Most folks would know the basics of how Dropbox works, so that’s not the purpose of this article. Security is what it’s all about, so here I’ll share how I choose to secure my Dropbox Account and the Data it contains.
The viewer will learn how to clear a vector as well as how to detect empty vectors in C++.
Windows 8 came with a dramatically different user interface known as Metro. Notably missing from that interface was a Start button and Start Menu. Microsoft responded to negative user feedback of the Metro interface, bringing back the Start button a…
Suggested Courses

850 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question