I had a very interesting talk with a vendor this morning. Ok, it wasn’t really interesting as much as maddening. I swear, it’s guys like this that are going to be the death of me. I know I shouldn’t let them get to me but sometimes they do. You know how it goes, you try to explain something and it just escalates until you’re upset. Anyway…
OK, so they were having blocking issues in the DB. The vendor was trying to convince the app team to turn off the SQL backups because they were interfering with their processes. That’s when they called me. Frankly, I can’t believe they called because the vendor was very adamant and this is the kind of thing our app groups do on their own without consulting us at all, so I’m more shocked than anything. But I knew this was going to be a bad call the second my app guy told me what was going on.
Here’s the conversation as it took place just a couple hrs ago. It took me this long to regain my strength.
V = vendor
M = me
M: Ok, I’m here now. So explain to me what’s going on.
V: Well, we’re having severe blocking issues in the DB and after looking at it we noticed it’s the SQL backups causing it. So what we need to do is stop the SQL backups.
M: Backups don’t hold locks in the DB so there’s no way for it to be causing blocking.
V: Of course backups hold locks. Everything holds locks.
M: No, they don’t hold locks. There’s never been a case where a backup has blocked a process.
V: We see it all the time. And you’ll just have to trust me because I do this for a living. I’m very familiar with this app and this is how we always fix this issue.
M: You always fix your poor coding issues by having people not backup their data?
V: You can backup your data, but you have to schedule downtime every week to do it… to keep it from blocking.
M: I really don’t want to finish this conversation, but I’m developing this morbid curiosity about where this is leading.
V: What does that mean?
M: Well, you clearly know nothing about SQL and you’re doing your customers such a huge disservice by making them turn off their backups when all you have to do is fix your query.
V: What do you mean I know nothing about SQL, you’re the one who thinks backups don’t cause blocks when I see that all the time.
M: No, what you see all the time is your process causing blocks and people running backups. Because I’m on your server now and I can see the blocking you’re referring to and it has nothing to do with the log backup, it has to do with your process on spid 201.
V: I thought you were a DBA. When there’s severe blocking in a system SQL Server sometimes reports the wrong spid. Our tech lead assures us that this is coming from the backup.
M: Well, if you’re just going to insult me we can end this call right now because I really don’t have time for this. I can’t teach every moron who writes an app for SQL everything they need to know to support it.
V: Now who’s being insulting?
M: Oh, that would still be you. Because you know nothing about SQL and you hop on the phone and start calling me an idiot because you don’t know how to troubleshoot a simple blocked process. Since we’ve been on the phone, I’ve looked at the sp that’s causing this and the problem is simple; you’ve got a hardcoded TABLOCK in 2 of your queries. That’s what causing the blocking. Have you ever bothered looking at this process to see what it’s doing?
V: No, because it’s the backup causing it. There’s nothing wrong with the code. We run this at several customer sites and nobody ever complains.
M: Ok, let me try something different here… if SQL’s reporting a different spid for the blocking process, how does your tech lead know that this incorrect spid points back to the backup? Why couldn’t it be pointing to one of the many other processes running right now?
V: He’s got a script that can tie them together.
M: I would love to see that script. Maybe I could learn something, huh? But if SQL itself gets confused about the spid and doesn’t report it right in sysprocesses, then how can he tie it to the right process in his script? Wouldn’t SQL tie it to the right spid and present it to us the right way to begin with?
V: He won’t show anyone the script because he says it’s too advanced and we don’t need to know how to do those types of things.
M: Wow, that’s convenient isn’t it? Let me try something different here. Because now this is like a train wreck I just can’t turn away from. When you kill the backups at other client sites, does the process clear up right away?
V: No, unfortunately, the database can’t recover from something that’s been blocked by a backup so we have to kill our process to and then things work fine.
M: My, you’ve just built your own little world in there haven’t you? You’ve got excuses for everything.
V: They’re not excuses. It’s how databases work. Where did they find you anyway? Have you ever been to any SQL Server classes? Maybe they should send you to some.
M: No, I’ve never been to any SQL classes. I’ve never needed them. And if you call me an idiot again you’re gonna find out what else I can do. In fact, I’m a hair away from calling the CEO of your company and letting him know what kind of stupid, snide little ass he’s got working for him. So you’d better watch your tone with me boy and remember I”M the customer. So you don’t tell me what to do and you don’t get on the phone with me and call me an idiot. And here’s something else for you. Before I even got on this call I ripped your access from the box because I don’t want you touching anything on our server. Nobody’s turning off our backups, and nobody’s ever touching our server. And I’m actually done with you. I’m not going to troubleshoot this with you one more second. Here’s what I want. I want you to escalate this to your team lead and I’ll fight this out with him. I want a call with him at the next available opportunity. You are out of this issue.
V: Look, I’m sorry if you thought I was being
S: Shut up. I’m done with you. The next call I get had better be from your team lead. Good-bye.
I’m telling you guys, if voodoo were real I’d be sticking pins in this guy’s ass for the next month. And the sad thing is I doubt it’s his fault. He’s got some jackass yanking his chain and this is probably just some kid who they’ve taught to be this way. He sounded like he was probably in his early 20′s. I still haven’t heard from his tech lead, btw.
The best database career advice you’ve never heard!
Become a DBA. Become a BETTER DBA. Use the Roadmap.
The DBA Roadmap Seminar is 7 MP3 tracks (over 5 hours!) of insider guidance on your database career. We'll teach you how and what to study as a DBA, weigh in on controversial resume debates, teach you to recognize a worthy recruiter, and discuss the new professionalism of interviews. Also some bonus materials, PDF companion guides, and really spiffy intro music!
Once your $99 PayPal payment is submitted, you'll get the download link in e-mail! (Download is a 370Mb ZIP file.)
Visit www.DBARoadmap.com for info, forums, and more!
TagsArchitecture blunders Camtasia Career Coding Standards CPU data generator DBA Development disaster firewall Idiots Inside SQL Server Interview ITBookworm Jobs Kalen Delaney Katmai Ken Henderson LiteSpeed MidnightDBA PASS Summit 2011 Pinnacle politics Powershell Powershell Hero Query techniques Red-Gate reporting restore SQL Express 1433 SCM SQL Server SQLServerDVD.com SSIS SSRS Training troubleshooting TSQL Tuesday tuning Tutorials Video post videos Yukon
Interview: Kalen Delaney!
Need help? Got an emergency? Write us at Support@MidnightDBA.com!
We can schedule time to help with your backup/restore issues, high availability and disaster recovery setup, performance problems, and a great deal more. Very often, we're even available on the moment for downtime issues and emergencies.
- Guest Review: Microsoft Wireless Sculpt Ergonomic Keyboard
- Quick Review: Anker Vertical Ergonomic Optical USB Wired Mouse
- Reading: Microsoft SQL Server 2012 High-Performance T-SQL Using Window Functions by Itzik Ben-Gan
- Repost: “Problems with my new book” by Grant Fritchey
- Announcing SQL Server 2012 Query Performance Tuning by Grant Fritchey