• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 656
  • Last Modified:

Long Running UPDATE Query - plpgsql

I have an UPDATE query that updates one int4 column of a table with around 220,000 rows in it. The subquery to pull the new value of this column executes in approximately 10 seconds. When I run the UPDATE query:

UPDATE mytable
SET mycol = sq.mycol
FROM (
      ... ) sq
WHERE mytable.id = sq.id;

It takes somewhere in the area of 5 minutes to execute. I think when Postgres gets around to running COMMIT is where locking begins to occur, and all of my SQL statements begin backing up.

This table is pretty heavily hit on a live website - so things get backed up pretty quickly - and if it gets too bad, I have no choice but to restart the database.

I am running this statement from with in plpgsql function, and I have even tried looping through running smaller UPDATE statements:
FOR row IN EXECUTE SQL1 LOOP
   UPDATE ...
END LOOP;

This seems to run better, as it slowly degrades performance, whereas the first one would run fairly quickly, and then at one point everything would start backing up. I attempted to explicitly COMMIT smaller blocks of records, but plpgsql doesn't allow for this.

A few of questions, any of which might lead me to a good solution:
1) Is there just an overall better way to approach this, in plpgsql?
2) Is there some way to tell the function's execution to pause for a number of seconds to allow some backed up queries to complete execution?
3) Is there a way to use a READ UNCOMMITED isolation level or disable locking for this execution? I don't care about any issues associated with dirty/phantom reads while dealing with this table.

Thanks in advance!
0
InterWorks
Asked:
InterWorks
3 Solutions
 
m1tk4Commented:
run

EXPLAIN UPDATE mytable
SET mycol = sq.mycol
FROM (
      ... ) sq
WHERE mytable.id = sq.id;

and post the results here.

Also, are mytable.id and sq.id of the same type, and what's their type?

READ UNCOMMITED won't help: " The SQL standard defines two additional levels, READ UNCOMMITTED and REPEATABLE READ. In PostgreSQL READ UNCOMMITTED is treated as READ COMMITTED, while REPEATABLE READ is treated as SERIALIZABLE." (http://www.postgresql.org/docs/8.1/interactive/sql-set-transaction.html)
0
 
gheistCommented:
Do joined columns have INDEX associated?
0
 
earth man2Commented:
Are you encountering deadlock, rather than bottleneck.
0
 
InterWorksAuthor Commented:
Sorry for lack of response. Yes they're of the same type and it's an indexed btree primary key on mytable.

We brought in a high level postgres guy to tune the configuration. Apparently a simple UPDATE on this table inserts an entirely new row and removes the old one - we were running out of workspace memory - so it resorted to running most of this query in swapfiles on disk. That has been corrected, but this query still takes too long and locks too much. We're breaking the query up into smaller blocks with an outside language so we can use smaller transactions, from what I've heard, PL/PGSQL cannot do this.

All points brought up were valid, so I'll split the points. Thanks for the responses.
0

Featured Post

Get your problem seen by more experts

Be seen. Boost your question’s priority for more expert views and faster solutions

Tackle projects and never again get stuck behind a technical roadblock.
Join Now