Waiting for idle_blocker which we cannot find

We have been seeing long wait times at random intervals throughout the day.  When we dig into these waits, it always seems they are being caused by an idle blocker (as we have discovered through DPA as well as some extended events.)  The problem is, once we realize this has happened, it's already too late.  The SPID that was causing the idle blocking is closed and we cannot determine what was running.  We have some extended events set up that give us a bit of information about the SPID (such as the call stack, duration, wait_resource, and wait_type) but none of this tells me exactly what stored procedure, query, or other process was running.  

Does anyone have a recommended way to store all information about past SPIDs, or any other advice on how to deal with this going forward?  At this point, I'm not worried about necessarily finding out what's happened in the past (though that is welcomed as well,) but how to get this information stored going forward so I can refer back to it.
Cyprexx ITAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

dbaSQLCommented:
This is a very good piece from Brent Ozar on identifying the blocking processes with a server side trace.  See the section that begins with:
  Scripts to use the Blocked Process Report

https://www.brentozar.com/sql/locking-and-blocking-in-sql-server/

You could put it into place now, to help identify the problem, and then re-enable the trace in the future, if any blocking related problems recur.
Cyprexx ITAuthor Commented:
I have actually read through this article, and it does have some really good info.  I have turned on the Blocked Process Report, but the article itself is only useful if you know the blocking is going to occur, which I never do.  It specifically tells you to make sure you turn off the blocked process report at the end, so the scripts in here won't really help me since I have to know when it's happening to utilize this.
dbaSQLCommented:
Yes, I've read that, too, but have also used it successfully when leaving it running throughout the day -- to identify the blocking problem, as you are trying to do.  The server side traces are much lighter than the profiler, and when they are defined explicitly to only collect on the block events, then the overhead is minimal.

I do understand your concerns, though, so here are two more that I've used that are defined w/extended events.  Again, they are defined solely for the blocked process events, so the overhead is minimal.

https://www.sqlskills.com/blogs/erin/capture-blocking-information-with-extended-events-and-the-blocked-process-report/
https://blogs.msdn.microsoft.com/nav/2015/01/16/using-sql-server-extended-events-to-produce-a-blocked-process-report/

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

Cyprexx ITAuthor Commented:
Excellent, I will set up the Extended Event and leave it on.  As long as the overhead is minimal (which I agree with you, it certainly should be, or no worse that the other extended events I have running currently,) it shouldn't be a problem.  I was just concerned about the fact the first article was explicit on the fact it should be turned off in the end.  

As far as our situation, it seems to occur once every 10-14 days.  If it was daily, or hourly, I would have had no issue with leaving it on, but I wasn't sure about leaving it on for several weeks.  Once I get some information on what's causing this, I will be sure to turn it off again.  Thank you!
Cyprexx ITAuthor Commented:
Excellent responses.  Thank you!
dbaSQLCommented:
Completely my pleasure.  In a similar situation recently, I have left blocking process session on for weeks at a time, and had no related problems.  Just be sure to check status and output intermittently, and be sure nothing is getting away from you.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Extended Events

From novice to tech pro — start learning today.