Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win


Any way of increasing a mysql view performance when you can't use the merge algorithm.

Posted on 2011-09-29
Medium Priority
Last Modified: 2012-05-12
Long story short, I created a view that utilizes a UNION ALL and does some math operations.  I am aware that this means I cannot use the Merge algorithm for the view, but the performance is pretty bad using the temptable algorithm.  

I really need to use this view because it makes the coding so much easier for our developers.  Are there any suggestions on improving the performance or possibly an alternative to using the temptable algorithm?  I would be open to switching database platforms (postgre maybe?) or change the server configuration, adding resources to the server, etc.

Any ideas?
Question by:gr1z
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 5

Expert Comment

ID: 36815397

first I need to know:

- how big are your tables and your indexes (tables used in your query)
- hardware specs (processor, ram)
- operating sistem
- platform (32bit or 64bit)
- contents of your mysql configuration file (my.cnf)

With this info I will provide a solution for sure.

There a are 3 possible solutions:

1. mysql configuration changes (easy solution)
2. increase Memory (RAM) (medium solution)
3. change platform (hard way)

Please provide more information of what data form which tables do you want to query, maybe I will give you alternative solutions.


Author Comment

ID: 36815876
1. how big are the tables - approx 100,000 rows

2. hardware specs - dedicated mysql server, rackspace cloud server with 1 gb ram, not sure how they rate the processor on those.

3. OS - Ubuntu

4. platform - 64 bit

5. contents of my.cnf - attached



Expert Comment

ID: 36816333

- 1 GB ram is not enough for a mysql server (it is not enough to run decent even a desktop computer with Windows 7)
- key_buffer            = 16M it is very small - try increasing step by step that value and check for speed improve.

Here are some values from a running mysql server with 96 GB ram.

key_buffer_size = 8G
query_cache_size = 1G
query_cache_limit = 88G
join_buffer_size = 32M
tmp_table_size = 16G
max_heap_table_size = 2G
thread_cache_size = 16
table_cache = 204800
innodb_buffer_pool_size = 1G
sort_buffer_size = 16M
thread_concurrency = 16

This server have 100 GB of data ( and growing ) that needs to be extracted every second and it is lighting fast.

You cand tune your mysql server with this little tool written in perl:



Let me know the results of mysqltunner.pl and if there is a speed increase after changing key_buffer to 64M (for example)

Remember that I don't have any server with 1 GB ram - even my desktop computer have 4 GB ram.

Let's not forget that your queryes must use indexes a lot if you want speed, if your queries is not optimised they will be slow on any hardware so please paste those queries here and some table info.


What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.


Author Comment

ID: 36816405
thanks, i will give these a try and report back.  i think part of the problem is when you use a view with a UNION it has to is the temptable algorithm, and this does not allow indexes.  

Expert Comment

ID: 36816675
That is why I want to see exactly what you want to do, maybe we can find another way, a better way.


Author Comment

ID: 36816716
I basically have 2 tables.  1 table is for hours (billable time) and one tables is for expenses and etc.  i am joining these tables together so it is easier to grab a log of all the relevant hours / expenses for a certain project, employee, etc.

here is my query:

		_utf8 'time' AS `type`,
		`logHours`.`hourId` AS `typeId`,
		`logHours`.`proId` AS `proId`,
		`logHours`.`loginId` AS `loginId`,
		`logHours`.`coId` AS `coId`,
		`logHours`.`coName` AS `coName`,
		`logHours`.`proName` AS `proName`,
		`logHours`.`date` AS `thisDate`,
		`logHours`.`item` AS `item`,
		`logHours`.`amount` AS `quantity`,
		`logHours`.`loginBillable` AS `rate`,
			`logHours`.`loginBillable` * `logHours`.`amount`
		)AS `total`,
		`logHours`.`RealParentId` AS `RealParentId`,
		`logHours`.`proParentId` AS `proParentId`,
		`logHours`.`isBillable` AS `isBillable`,
		`logHours`.`timesheet` AS `timesheet`,
		`logHours`.`notes` AS `notes`,
		`logHours`.`ParentName` AS `parentName`
		_utf8 'expense' AS `type`,
		`logExpense`.`expId` AS `typeId`,
		`logExpense`.`proId` AS `proID`,
		`logExpense`.`expLoginId` AS `loginId`,
		`logExpense`.`coId` AS `coId`,
		`logExpense`.`coName` AS `coName`,
		`logExpense`.`proName` AS `proName`,
		`logExpense`.`date` AS `thisDate`,
		`logExpense`.`item` AS `item`,
		`logExpense`.`expQuantity` AS `quantity`,
		`logExpense`.`expRate` AS `rate`,
			`logExpense`.`expRate` * `logExpense`.`expQuantity`
		)AS `total`,
		`logExpense`.`RealParentId` AS `RealParentId`,
		`logExpense`.`proParentId` AS `proParentId`,
		`logExpense`.`isBillable` AS `isBillable`,
		`logExpense`.`timesheet` AS `timesheet`,
		`logExpense`.`notes` AS `notes`,
		`logExpense`.`ParentName` AS `parentName`

Open in new window


Accepted Solution

Adrianc earned 2000 total points
ID: 36816796

Please try without UNION and see the results.

Make two separate queries and join the results programmatically, I think it will be faster because there is no need for creating a temp table.


Author Comment

ID: 36823849
Not sure I follow what you mean by join programmatically?  

do you mean have a separate view for each table and then do the union on them in my scripting language (like php or coldfusion)?

if this isnt what you mean can you provide a quick example?

thanks for the help

Author Comment

ID: 36830432
some quick testing, making two separate views and then doing the union on those in coldfusion makes quite a difference in speed.

quick question,

if i do a query like this:

SELECT * FROM test_hours WHERE loginId = 1 AND proId = 641 AND isBillable = 'yes' AND thisDate = '04/09/11' AND type = 'time'
SELECT * FROM test_exp WHERE loginId = 1 AND proId = 641 AND isBillable = 'yes' AND thisDate = '04/09/11' AND type = 'time'

Open in new window

is there a better way to specify the where clause so I only have to define it once?  not a huge deal just cleans up the code a little bit.

also if this example isn't what you were suggesting please let me know, thanks!

Expert Comment

ID: 36889968
Hi, by joining them programmatically what I meant is:

PHP code:

$res = mysql_query("SELECT query number 1");
while ($row = mysql_fetch_assoc($res))
    //do the calculations here and store them in an array

$res = mysql_query("SELECT query number 2");
while ($row = mysql_fetch_assoc($res))
    //do the calculations here and store them in the same array and build your report

Open in new window

Of course this is just an ideea, maybe it is too hard to implement in your project OR maybe there will be a decrease of performance because we all know that PHP is not good for heavy calculations and memory allocation.

Do some tests and let me know.

If you are using UNION keep the where clause in the subquery.


Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In part one, we reviewed the prerequisites required for installing SQL Server vNext. In this part we will explore how to install Microsoft's SQL Server on Ubuntu 16.04.
Lotus Notes has been used since a very long time as an e-mail client and is very popular because of it's unmatched security. In this article we are going to learn about  RRV Bucket corruption and understand various methods to Fix "RRV Bucket Corrupt…
In this video, Percona Solution Engineer Rick Golba discuss how (and why) you implement high availability in a database environment. To discuss how Percona Consulting can help with your design and architecture needs for your database and infrastr…
In this video, Percona Solutions Engineer Barrett Chambers discusses some of the basic syntax differences between MySQL and MongoDB. To learn more check out our webinar on MongoDB administration for MySQL DBA: https://www.percona.com/resources/we…
Suggested Courses

636 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question