Any way of increasing a mysql view performance when you can't use the merge algorithm.

Posted on 2011-09-29
Last Modified: 2012-05-12
Long story short, I created a view that utilizes a UNION ALL and does some math operations.  I am aware that this means I cannot use the Merge algorithm for the view, but the performance is pretty bad using the temptable algorithm.  

I really need to use this view because it makes the coding so much easier for our developers.  Are there any suggestions on improving the performance or possibly an alternative to using the temptable algorithm?  I would be open to switching database platforms (postgre maybe?) or change the server configuration, adding resources to the server, etc.

Any ideas?
Question by:gr1z
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 5

Expert Comment

ID: 36815397

first I need to know:

- how big are your tables and your indexes (tables used in your query)
- hardware specs (processor, ram)
- operating sistem
- platform (32bit or 64bit)
- contents of your mysql configuration file (my.cnf)

With this info I will provide a solution for sure.

There a are 3 possible solutions:

1. mysql configuration changes (easy solution)
2. increase Memory (RAM) (medium solution)
3. change platform (hard way)

Please provide more information of what data form which tables do you want to query, maybe I will give you alternative solutions.


Author Comment

ID: 36815876
1. how big are the tables - approx 100,000 rows

2. hardware specs - dedicated mysql server, rackspace cloud server with 1 gb ram, not sure how they rate the processor on those.

3. OS - Ubuntu

4. platform - 64 bit

5. contents of my.cnf - attached



Expert Comment

ID: 36816333

- 1 GB ram is not enough for a mysql server (it is not enough to run decent even a desktop computer with Windows 7)
- key_buffer            = 16M it is very small - try increasing step by step that value and check for speed improve.

Here are some values from a running mysql server with 96 GB ram.

key_buffer_size = 8G
query_cache_size = 1G
query_cache_limit = 88G
join_buffer_size = 32M
tmp_table_size = 16G
max_heap_table_size = 2G
thread_cache_size = 16
table_cache = 204800
innodb_buffer_pool_size = 1G
sort_buffer_size = 16M
thread_concurrency = 16

This server have 100 GB of data ( and growing ) that needs to be extracted every second and it is lighting fast.

You cand tune your mysql server with this little tool written in perl:


Let me know the results of and if there is a speed increase after changing key_buffer to 64M (for example)

Remember that I don't have any server with 1 GB ram - even my desktop computer have 4 GB ram.

Let's not forget that your queryes must use indexes a lot if you want speed, if your queries is not optimised they will be slow on any hardware so please paste those queries here and some table info.


More Than Just A Video Library

Train for your certification. Learn the latest DevOps tools. Grow your skillset to do better work.

At Linux Academy, we release new training modules every week so you'll always be up to date on the latest tech.


Author Comment

ID: 36816405
thanks, i will give these a try and report back.  i think part of the problem is when you use a view with a UNION it has to is the temptable algorithm, and this does not allow indexes.  

Expert Comment

ID: 36816675
That is why I want to see exactly what you want to do, maybe we can find another way, a better way.


Author Comment

ID: 36816716
I basically have 2 tables.  1 table is for hours (billable time) and one tables is for expenses and etc.  i am joining these tables together so it is easier to grab a log of all the relevant hours / expenses for a certain project, employee, etc.

here is my query:

		_utf8 'time' AS `type`,
		`logHours`.`hourId` AS `typeId`,
		`logHours`.`proId` AS `proId`,
		`logHours`.`loginId` AS `loginId`,
		`logHours`.`coId` AS `coId`,
		`logHours`.`coName` AS `coName`,
		`logHours`.`proName` AS `proName`,
		`logHours`.`date` AS `thisDate`,
		`logHours`.`item` AS `item`,
		`logHours`.`amount` AS `quantity`,
		`logHours`.`loginBillable` AS `rate`,
			`logHours`.`loginBillable` * `logHours`.`amount`
		)AS `total`,
		`logHours`.`RealParentId` AS `RealParentId`,
		`logHours`.`proParentId` AS `proParentId`,
		`logHours`.`isBillable` AS `isBillable`,
		`logHours`.`timesheet` AS `timesheet`,
		`logHours`.`notes` AS `notes`,
		`logHours`.`ParentName` AS `parentName`
		_utf8 'expense' AS `type`,
		`logExpense`.`expId` AS `typeId`,
		`logExpense`.`proId` AS `proID`,
		`logExpense`.`expLoginId` AS `loginId`,
		`logExpense`.`coId` AS `coId`,
		`logExpense`.`coName` AS `coName`,
		`logExpense`.`proName` AS `proName`,
		`logExpense`.`date` AS `thisDate`,
		`logExpense`.`item` AS `item`,
		`logExpense`.`expQuantity` AS `quantity`,
		`logExpense`.`expRate` AS `rate`,
			`logExpense`.`expRate` * `logExpense`.`expQuantity`
		)AS `total`,
		`logExpense`.`RealParentId` AS `RealParentId`,
		`logExpense`.`proParentId` AS `proParentId`,
		`logExpense`.`isBillable` AS `isBillable`,
		`logExpense`.`timesheet` AS `timesheet`,
		`logExpense`.`notes` AS `notes`,
		`logExpense`.`ParentName` AS `parentName`

Open in new window


Accepted Solution

Adrianc earned 500 total points
ID: 36816796

Please try without UNION and see the results.

Make two separate queries and join the results programmatically, I think it will be faster because there is no need for creating a temp table.


Author Comment

ID: 36823849
Not sure I follow what you mean by join programmatically?  

do you mean have a separate view for each table and then do the union on them in my scripting language (like php or coldfusion)?

if this isnt what you mean can you provide a quick example?

thanks for the help

Author Comment

ID: 36830432
some quick testing, making two separate views and then doing the union on those in coldfusion makes quite a difference in speed.

quick question,

if i do a query like this:

SELECT * FROM test_hours WHERE loginId = 1 AND proId = 641 AND isBillable = 'yes' AND thisDate = '04/09/11' AND type = 'time'
SELECT * FROM test_exp WHERE loginId = 1 AND proId = 641 AND isBillable = 'yes' AND thisDate = '04/09/11' AND type = 'time'

Open in new window

is there a better way to specify the where clause so I only have to define it once?  not a huge deal just cleans up the code a little bit.

also if this example isn't what you were suggesting please let me know, thanks!

Expert Comment

ID: 36889968
Hi, by joining them programmatically what I meant is:

PHP code:

$res = mysql_query("SELECT query number 1");
while ($row = mysql_fetch_assoc($res))
    //do the calculations here and store them in an array

$res = mysql_query("SELECT query number 2");
while ($row = mysql_fetch_assoc($res))
    //do the calculations here and store them in the same array and build your report

Open in new window

Of course this is just an ideea, maybe it is too hard to implement in your project OR maybe there will be a decrease of performance because we all know that PHP is not good for heavy calculations and memory allocation.

Do some tests and let me know.

If you are using UNION keep the where clause in the subquery.


Featured Post

Ransomware: The New Cyber Threat & How to Stop It

This infographic explains ransomware, type of malware that blocks access to your files or your systems and holds them hostage until a ransom is paid. It also examines the different types of ransomware and explains what you can do to thwart this sinister online threat.  

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Containers like Docker and Rocket are getting more popular every day. In my conversations with customers, they consistently ask what containers are and how they can use them in their environment. If you’re as curious as most people, read on. . .
This post contains step-by-step instructions for setting up alerting in Percona Monitoring and Management (PMM) using Grafana.
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

729 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question