Category Archives: Admin

Why is adopting Powershell so hard?

One of the biggest battles I fight when I teach PS or show it to different companies is adoption.  DBAs and managers just don’t get why they should be using PS to manage their environments.  And the reason so many of them are slow to adopt is because the right message is very hard to come by.  In fact, I see the wrong message all the time.  This was driven home again just now as I received my “SQL Server Pro UPDATE” email (read it here).  The editorial was about why DBAs should care about PS.  It was written by Michael Otey (pronounced Otee) and I’m sorry to say that as the Sr. Technical Director he’s completely missed the mark.  He’s managed to relegate the most important management tool DBAs have to a mere sideline player that allows you to combine SQL work with OS-level work.  Dude, you’ve completely missed the boat!!!

I’ve said this a number of times and I’ll say it again here.  Let’s get the right message out about PS.  Let’s start telling everyone that truth about it and that truth is that you’re going to get left behind if you don’t know PS and you’re not going to be able to provide true value to your company.

I’m a working production DBA in a very large shop.  And I can tell you with absolute authority that PS is more than just a way to give you some OS-level functionality in your SQL processes.  PS is vital to the management of my shop.  The primary role of PS is to allow you to manage hundreds of objects at once with just a few lines of code.  As you’ve seen in my videos again and again, and in my classes again and again, you just can’t manage any decently sized environment by hand.  It’s just too much and it’s accident-prone.

Now, a large environment doesn’t have to mean hundreds of servers.  Oh no, it can mean dozens of DBs on a single server.  It can mean dozens of schemas in a single DB, or dozens of tables, SPs, views, etc.  Or it can even mean dozens of jobs or logins.  It doesn’t matter what you’re cycling through.  The point is that PS is much more efficient at managing these objects than T-SQL will ever be.  There may be a built-in SP for running a T-SQL statement against all the DBs on your server, and there’s even one for running something on all the tables I believe.  But there’s nothing for all the jobs, or SPs, or views, or logins, or credentials, or any of the other many objects you can manage natively with PS.  With just a few characters of code you can manage all the tables in your DB.  And with just a few more characters you can do that same operation against all the DBs on your server.  And with yet just a few more characters of code, you can do that same operation against all the DBs across as many servers as you like.  THAT’S THE POWER OF POWERSHELL.  It’s more than just a way to manage files while you’re doing some T-SQL operations.

A perfect example is adding user accounts to DBs on several boxes.  We quite often get contractors who come in and do specific work for us and they need to be given rights to all of the environments for a given application.  So this one we have has 500+ DBs on each server.  There are 7 prod servers, 3 QA servers, and 5 dev boxes.  All of them have 500+ DBs on them.  So that’s 15 servers, and at least 7500 DBs that I have to add these users to.  And using windows groups will only take you so far because when the next contractor comes in he may need different rights so you still find yourself modifying the perms regardless.  The point is I go through this exercise about once every 2mos and PS has made it a trivial matter.  And that’s just ONE example.

The time has come to stop downplaying the role of Powershell in SQL Server.  DBAs need to be running to it.  We’re being asked to do more and more with less time and fewer resources.  And there are too many courses being taught, too many SQLSaturday sessions, too many videos (FREE on MidnightDBA.com) for you to remain in the dark any longer.  I’ve managed 900+ servers by myself using mainly PS.  I’d say it goes quite a bit deeper than being a helper to keep you from using xp_cmdshell.

Cluster Upgrade Error: 19019

So we upgraded a cluster from 2008 to R2 tonight.  We upgraded the inactive node to R2 SP2 and then went to failover the cluster so we could upgrade the other node.  But when we tried to failover it failed.  The only thing the critical events for the resource said was that it couldn’t failover.  So no help there.  The only other error was in the event viewer and it said that the shared component upgrade failed… even though nothing in the upgrade process failed.  There actually were other errors in the event viewer but I’ll let the link below cover those.  The big important error was Error:  19019.

I remembered something from a few yrs ago that involved a registry fix so I started looking at the registry.  After opening regedit I went to the most logical place:

HKLM\Software\Microsoft\Microsoft SQL Server\MSSQL10_50.SQL2008

Once there I saw the Setup folder, which seemed like a good place to start.  Inside there I saw a key called SQLDataRoot and its value started with E:\

Well, I have a pretty standardized environment so I know the SQL root was supposed to be on D:.  I changed the location to D:\ and the SQL Server resource came online instantly. 

So again, that registry key is:

HKLM\Software\Microsoft\Microsoft SQL Server\MSSQL10_50.SQL2008\Setup\SQLDataRoot

This is not only an interesting fix, it’s also a testament to what having a highly standardized environment can do for you.  I could have compared this entry with node1 to see if they matched, but it may not have even been flagged as an issue if we  didn’t always install SQL on D:\.

So once I had the fix in place, I had something to search for, so I went to google to see if I could find anything about it out there.  I came across this great blog post that even gives more detail on this error, so rather than try to reproduce the entire thing here, I’ll just point you to that post. 

http://blogs.msdn.com/b/jorgepc/archive/2010/10/14/onlinethread-sql-cluster-shared-data-upgrade-failed-error-installing-cumulative-update-in-sql-server.aspx

 So this is why I call myself the bumbling DBA.  I don’t actually know anything, but I manage to find my way through solutions somehow.  And I know I’ll get lots of comments on this with everyone telling me they knew this cause they’ve seen it 100x and all I can say is bully for you.  I’ve only seen it once and it was a few yrs ago.  I don’t get out much.

Powershell till you drop

Ok, well I’ve been very busy again and released 3 new PS vids today. 

There’s one on dropping tables.  I use regex to make this happen, so even if you’re not interested in dropping tables, you can come learn how to do a simple regex.

http://midnightdba.itbookworm.com/VidPages/PowershellDropTables/PowershellDropTables.aspx
http://midnightdba.itbookworm.com/VidPages/PowershellDropTables/PowershellDropTables.wmv

 

The next one is on truncating tables.  Here I just limit it by a specific schema.

http://midnightdba.itbookworm.com/VidPages/PowershellTruncateTables/PowershellTruncateTables.aspx
http://midnightdba.itbookworm.com/VidPages/PowershellTruncateTables/PowershellTruncateTables.wmv

And the last one is just damn good.  I show you how to get sp_configure functionality in PS.  There’s a trick to it so don’t discard it cause you think you can figure it out on your own. 

http://midnightdba.itbookworm.com/VidPages/PowershellServerConfigSettings/PowershellServerConfigSettings.aspx
http://midnightdba.itbookworm.com/VidPages/PowershellServerConfigSettings/PowershellServerConfigSettings.wmv

 

Get IP and DNS in Powershell

Hey guys, I posted a new video last night on how to get IP and DNS info from your servers.  I know there are more ways to do it so if you guys have a way you like better send it to me and I’ll make another vid.

http://midnightdba.itbookworm.com/VidPages/PowershellGetIPandDNS/PowershellGetIPandDNS.aspxhttp://midnightdba.itbookworm.com/VidPages/PowershellGetIPandDNS/PowershellGetIPandDNS.aspx

 

Hey, how about some new Powershell vids?

I’ve been a busy little guy this week.  I’ve posted 4 new videos.

The first one is on cycling the SQL error log from Powershell.  Well, not really, you’re really deploying the solution to other boxes using Powershell.
You can see it here:
http://midnightdba.itbookworm.com/VidPages/PowershellCycleErrorLog/PowershellCycleErrorLog.aspx

The next one is about reading the SQL error log from Powershell.

You can see it here:  http://midnightdba.itbookworm.com/VidPages/PowershellReadErrorLogs/PowershellReadErrorLogs.aspx

The next 2 are a short series on changing DB permissions in Powershell.
You can see them here:

http://midnightdba.itbookworm.com/VidPages/PowershellChangeSQLPermissions/PowershellChangeSQLPermissions.aspx
http://midnightdba.itbookworm.com/VidPages/PowershellChangeSQLPermissions2/PowershellChangeSQLPermissions2.aspx

Red-Gate is Badass!!

Ok, so the other night I uninstalled my R2 install on my box and put on the new SQL2012.  I know upgrade exists, but I just wanted the thing on there and didn’t feel like messing with anything stupid going wrong.  So anyway, I was half way through my uninstall when I remembered, Oh Crap, I’m gonna lose SQLPrompt cause SQL2012 just came out.  I was pretty upset because I use that tool all the time.

So While Denali was installing, I went out to Red-Gate to see if they had it listed anywhere when SQLPrompt would support Denali… and you know what?  It already does.  I installed the newest version and it works beautifully in SQL2012.  And that’s not all.  They’ve got this kewl new thing they’re doing where they let you see beta features.  They call it Experimental Features and it’s at the bottom of the SQL Prompt 5 menu.  Here’s what it looks like.

These are beta features and you’re welcome to install them and try them out as you wish.  Personally I love this because I quite often want stuff like this and even if there are a couple bugs it’s still better than not having it at all.

So Red-Gate, I salute you for having this ready in time for the release and for doing such a good job with presenting us with these new features.  I want to have your babies.

Changing Job Step Properties in Powershell

I read a blog today from my good friend @SirSQL (Nic Cain) where he was talking about how to change retry attempts on all the jobs on a server. Well, technically it’s the job steps that have the retry attempts, not the jobs themselves. And whenever I see something like this, my fabulously huge MCM brain tries to turn it into Powershell. So I put my fingers to the keys and like 10mins later I had a workable script. And while Nic’s solution is perfectly good T-SQL, it’s another one of those things that highlights the power of the shell because I only need 3 lines of code to do it.

The easiest way to do this is through SSMS. Just start PS from the Jobs node.

And that’ll give you a blank SQLPS window that’s already in the jobs node.

Since we’re in SQLPS we don’t need to load the assembly, but I’ll do it anyway to show you how cause you might not be in SQLPS when you do it. It won’t hurt anything to load it again. But that’s one of the things that SQLPS does for you; It loads these assemblies.

Here’s how you load the assembly and set a variable to the JobStep object type.

[reflection.assembly]::loadwithpartialname("microsoft.sqlserver.smo")
$js = new-object microsoft.sqlserver.management.smo.agent.jobstep

Now we have our new $js var (stands for JobStep) we can fill it with the steps of all the jobs on the server. However, first, let’s take a look at the members of our new var.

$js | gm

Now you’ll get a good listing of all the members so you’ll know what you can and can’t do with it.

Towards the bottom you’ll see the property we’re interested in here: RetryAttempts. Now we just have to go through all the steps and change the retries to what we want. Here I’ll change it to 5 just because it’s a nice round number. You’ll be surprised how easy it is to do this. I’ll go ahead and tack it onto the partial script above and this will become our entire script.

[reflection.assembly]::loadwithpartialname("microsoft.sqlserver.smo")
$js = new-object microsoft.sqlserver.management.smo.agent.jobstep

$js = dir | %{$_.enumjobstepsbyid()}
$js | %{$_.RetryAttempts = 5;$_.alter()}

Ok, that’s all we need to do to change the RetryAttempts property for all the jobs on the server. But we’re not done talking about this… not even by a longshot.

First, in line 4 notice I call the EnumJobStepsByID() method on each item of the dir. This is how I populate the $js var with all of the job steps. What this line says is list all of the jobs (using dir) and then for each one, get a list of its steps and put it in $js.

Line 5 runs through each of the job steps in $js and actually performs the work of setting the RetryAttempts to our new value. And remember, jobs have an Alter() method, and typically whenever something in PS has an alter method it likes you to use it. if you don’t the changes will take effect in your PS session only and will not be pushed to the server. So call the Alter() method.

Now, I know what you’re saying… how would you do it for only some of the jobs? Because so far we’ve assumed that you want to apply the changes to every job on the server. Well, there are 2 ways to do that and they’re incredibly easy.

First, you can limit the data from the dir cmd in line 4. It could look like this:

[reflection.assembly]::loadwithpartialname("microsoft.sqlserver.smo")
$js = new-object microsoft.sqlserver.management.smo.agent.jobstep

$js = dir | ?{$_.Name -match "maint"} | %{$_.enumjobstepsbyid()}
$js | %{$_.RetryAttempts = 5;$_.alter()}

Take note of the new line 4. I’ve just added a where clause to the pipeline so now only jobs with the word “maint” in their names will be in the list.
The 2nd way is just as easy, but you do it at the job step level. if you remember from above when we looked at the methods for $js there was a property “Parent”. This is the parent job name for the step. So all you have to do is add the where clause to the $js instead and you’ll achieve the same thing.

[reflection.assembly]::loadwithpartialname("microsoft.sqlserver.smo")
$js = new-object microsoft.sqlserver.management.smo.agent.jobstep

$js = dir | %{$_.EnumJobStepsById()}
$js | ?{$_.Parent -match "maint"} | %{$_.RetryAttempts = 5;$_.alter()}

Now the new line 5 reflects our easy change.

There are so many uses for this code it’s incredible. There are plenty of properties to change and so many ways to limit the result set. Let’s say you have a maint routine on all of your servers and you want to alter the command that the 1st job step runs. That’s very easy. Instead of changing the RetryAttempts property, just change it to the Command property like this:

[reflection.assembly]::loadwithpartialname("microsoft.sqlserver.smo")
$js = new-object microsoft.sqlserver.management.smo.agent.jobstep

$js = dir | %{$_.enumjobstepsbyid()}
$js | ?{$_.Parent -match "maint"} | %{$_.Command = "My new code.";$_.alter()}

There’s nothing here Nic couldn’t change his script to do pretty easily, but this is much shorter and easier to read I think. It also has one huge advantage… I can run it on multiple servers without much effort. I’m not going to go down that road in this blog because I’ve posted many solutions with that code in it already so it wouldn’t be anything for you to add that to this script.

So anyway, thanks Nic for letting me use you as the base for this post. Your solution is fine, I just prefer the look and feel of 4 lines of code.

I’ve also got a companion video for this post:

http://midnightdba.itbookworm.com/VidPages/PowershellChangeJobStepProperties/PowershellChangeJobStepProperties.aspx

SSRS Videos

I posted 2 new vids this week on SSRS.  The first is a data-driven subscription primer that’ll get you up and running with not only what it takes to set it up, but the different methods you have for sending the results exactly the way you want them.  I even go out into Profiler to show you the effect it has on the system.  It’s about 30mins, and you can find it here:  http://midnightdba.itbookworm.com/VidPages/SSRSDataDrivenSubscription/SSRSDataDrivenSubscription.aspx

The 2nd vid is about deploying reports from VS.  This is a common question I get from noobs and because it’s not obvious how it works, I went ahead and filmed it.  I also cover one of the common problems you have when deploying reports using this method and how to solve it.  It’s about 4mins and can be found here:
http://midnightdba.itbookworm.com/VidPages/SSRSDeployReports/SSRSDeployReports.aspx

 

 

Mystery of the missing files

About 3mos ago I had my work box rebuilt.  The encryption software glitched and it just wouldn’t boot.  So the helpdesk girl was able to slave the drive to another box and copy my files to a NAS.  I looked and all the folders were where they should be.  Great.

She then rebuilt the box and when I got it back I copied all the folders back.  I let it rest and I was very pleased that I was able to preserve all my scripts, etc.  A few days passed and I needed a script, so I went to that folder and it was empty.  WTx dude?  I looked and everything was still out on the NAS so I tried to get it off of there but they were missing there too.    OK, now I’m upset.  I’ve just lost most everything I cared about on my box.  It took me a while to accept it but I finally did.

Well, today (this is now several wks later) I finally went to replace one of the things I lost by downloading it again.  When I went to save it though, it said that it already existed and did I want to replace it.  See, I saved it to the same folder as before because I’m OCD that way.  Needless to say it took me about 2secs to figure out what had happened.  Normally I setup my box the way I want right away, but this time I got busy and just forgot.  And as it turns out my files were there, just hidden.  And I didn’t have ‘show hidden files’ checked in folder properties.  Now that I know the problem, how do I fix it because there’s a lot of files and folders to go through.  The answer of course is Powershell.  I got it right on the first try and the code is so small, and so compact, and so easy to come up with, it just screams PS.  And it is the perfect example of what PS is all about.  So here’s the query I used… enjoy

dir -recurse -force | %{$_.attributes = 'Archive'}

Now, here’s a brief explanation of a couple of the flags.

-recurse — means to run it against all the subfolders too.
-force — this is a good one because it tells PS to show you all the files.  If they’re hidden you won’t see them unless you use this flag.

So that’s it really.  A single line of code to unhide hundreds of files in dozens of folders and subfolders.  And since this is my first day back after vacation, I’d say I’m starting it off on a decent note.