How big can IMAP accounts be and outlook still works OK?

IMAP and Exchange are very different. They are similar only in the idea that the mail is stored on an outside server as opposed to a local computer. The protocols themselves are totally different.

Unfortunately IMAP is a bit clunky. The main issue is that it needs to download and parse all of the headers in an (almost) synchronous manner.

When Outlook is the bottleneck (e.g. with PST files or Exchange mailboxes), the limitation depends on the version of Outlook. Outlook 2003 is ~4GB or 3000 items. Outlook 2007/2010 is closer to 20GB / 100,000 items before the performance degradation is unbearable. With big mailboxes Outlook is usually unbearably unresponsive until a complete copy of the mailbox is cached locally. This usually means letting Outlook sit overnight as it caches the Exchange mailbox.

However IMAP imposes it's own limits. In general if there are more than about 4000-5000 items in any particular folder it tends to bog down.  Outlook 2007/2010 is much better at telling you about the progress of downloading headers. Outlook 2003 tends to just hang while it is working which can be frustrating for users.

I do find that the main thing that causes IMAP to mess up is when the connection to the IMAP server fails for whatever reason - latency, flaky internet, etc. A good solid internet connection is important to having a good IMAP experience as it doesn't recover from a connection failure well. If you've been sitting at 25% for the last half an hour you probably have experienced a connection failure. Closing and re-opening Outlook will most likely let it continue.

Also, if your internet connection will allow, have Outlook configured to sync headers first, then full items including attachments. Having Outlook cache a complete local copy of the mailbox will reduce the lagging when switching from one message to another.