Infopath/Sharepoint Performance issue

Hello. I had setup a simple Infopath 2010 web browser form that retrieves data from SQL database. It has 2 fields to search data by and displays results in the table, no workflow setup on the form. Connection is converted to ucdx file, that is uploaded to Sharepoint; form (full trust) is also uploaded to Sharepoint as Content Type. It works perfectly fast in Infopath; no records are loaded on opening of the form and once search is performed - it returns anywhere from 1 to 250 records. There are almost 20K records in the SQL table. In Sharepoint I used Infopath web part to load this Infopath form. It works really inconsistent in Sharepoint - initial search could take anywhere from 2 to 20 seconds, but all subsequent searches take 1-3 seconds. Only 2 search fields have setting of postback, only when necessary, table with results is read only and postback set to 'never' for all those fields. I had read all articles I could find about best practices and couldn't find any stable resolution. As I mentioned - sometimes it could take 2 second to pull up the results and sometimes up to 20 seconds. It doesn't matter if initial search returns 1 record or 200 - same delay. We had checked Sharepoint server - no errors or performance issues are recorded. Also experimented with changing SQL query to have only few records, to see if 20K records is a problem, but performance was the same - 1 to 20 seconds for initial search and 1-3 seconds for next search. Please advise!