SSIS: Deploy Packages, Part 1

This is the companion text to the video SSIS: Deploy Packages.  This will be a two- or three-part series, I think.

There are two ways to deploy SIS packages, and the difference becomes important when you have a lot of packages. The first way is just to individually deploy a package directly to SQL (or the file system). The second way is to create a manifest, which allows you to deploy all the packages at once, instead of having to do one at a time. Today we’re talking about the manifest.

A quick side note: When you create a new Integration Services (SSIS) package in Visual Studio, it's named Package1.dtsx by default.  Always rename your package to something meaningful, and select Yes in answer to "Do you want to rename the package object as well?" See the blog titled SSIS Packages: Rename the Package Object as Well? for more information.

Deploy with Create Deployment Utility

When you have a package ready in Visual Studio,

  1. Go to the Project menu and select Properties.
  2. Under “Deployment Utility”, Set "Create Deployment Utility" to True, which creates a manifest file when you build the project.  (By the way, does anyone know how to set this by default?)
  3. “OK” out of the Property dialog.
  4. Save your project, and Build. 
  5. Go look in your project folder. In our example, the project folder was in My Documents\Visual Studio 2008\Projects\Integration Services Project2\Integration Services Project2\bin\Deployment\.

In there you have a your package files named packageName.dtsx file, and one deployment manifest – an XML file named projectName.SSISDeploymentManifest.  You can open the manifest in Internet Explorer to take a look.  It tells you what it's generated by, who did it, and a list of packages. You could create this by hand easy.  As a matter of fact, Sean wrote a .Net desktop app to create a manifest file from a list of packages. “From the developers I was getting nested folders with individual files; I had it make a master manifest file.”  Sweet, sweet automation…

Now, we could go to SIS and import the .dtsx file just fine, but we'd have to do that individually for each package. When we double-click the manifest, it brings up the package installation wizard which covers all the packages.

You can deploy packages to SQL or let them sit on the file system. That's something new with SIS: In DTS your packages were in SQL; in SIS you can run them t from the file system.

Package Location

So SQL Server deployment deploys the package and stores it in MSDB.  I could instead point it to a file system; it would still import the metadata into MSDB but the package itself would live on the file system.  The advantage to this is that you have a single package. It's easier to have one package & edit it as needs be, and several different marts can call it from the same place on disk.

Back in DTS you could go into Enterprise Manager and double click on a package and edit.  However, after a package is deployed from SIS, it's compiled code.  Consider these to be like any C# application: they're exe files.  So you have two choices for editing SIS packages:

  1. go into SSIS manager, right click and export package to file system, and then edit in VS and redeploy.
  2. OR you can store the package in VSS or TFS server, some code vault, and edit/deploy that way.

-Jen McCown, http://www.MidnightDBA.com

New Videos: .NET Objects in PowerShell, Interviews, and Being out of Work

I'd like to hold a little virtual celebration here, as we've just crossed over the threshold: we have more than 24 hours of training online!  That means, if you were so inclined, you could sit down and listen to us talk about SQL, PowerShell, .NET, HTML, interviewing, Windows, and more for a full day and night…and that's assuming you didn't stop for bathroom breaks!  The prospect is just thrilling.

Let's get to the five videos that put us over the top, shall we?

  • First, we have Sean's Build .Net Objects in Powershell – I just love the topic of this video.  Here we show you how to create .NET code with powershell.  While we create an .aspx page, you can build any type of .Net object dynamically.  Use this process freely because it's just cool and easy and useful and full of awesome! 14 minutes.
  • Next, two new DBAS @ Midnight videos, Good Interview Still Don't Get the Job: Part 1 (13 minutes) and Part 2 (14 minutes) – We discuss some reasons why you may still not get the job even though you had an excellent interview.  This is NOT your stale old stuff…"be sure to bathe, bring your resume, wear a suit, don't insult the HR girl's mother".  This is our experiences, and thoughts on factors that are sometimes out of your control.  Plus we mention Star Trek.
  • And two more new DBAs @ Midnight vids, Keeping your Sanity when out of Work: Part 1 (16 min) and Part 2 (15 min) – Here we're discussing some ways of keeping your sanity and your confidence when you're out of a job, so you can ensure you get that next gig that comes along.  By the way, if you download these and play them in fast mode in Media Player (CTRL-Shift-G) it takes about half the time, and you can giggle at how funny we sound.

-Jen McCown, http://www.MidnightDBA.com

Grouping – Level Zero

“Level  Zero” doesn’t mean anything, it’s another Kung Fu Panda reference. “Iiiiii’m not an expert, let’s just start with zero…level zero.”

Note: This article is the companion to the video Grouping Basics: Level Zero.

You use a SELECT statement to get rows of data from the database. 

select ID, gameTime, thingEaten, thingTypeEaten, pointsGained from pacman order by gameTime 

One of the most basic things you can do with that data is to group it into sets of like data, in order to summarize the data. For example:

select thingEaten, thingTypeEaten, pointsGained from pacman GROUP BY thingEaten, thingTypeEaten, pointsGained  

 

thingEaten thingTypeEaten Points

bigdot

dots

50

littledot

dots

10

banana

fruit

150

cherry

fruit

150

blinky

ghost

100

clyde

ghost

100

pinky

ghost

100

 

Each row above represents one or more rows in the table that have the same thingEaten, thingTypeEaten, and points.  Right now we don’t know HOW many tables are in the database that have, say, bigDot, dots, and 50 points.  We use aggregate functions to get more information about our groups.  Like COUNT, which gets the number of rows in a group, and SUM, which totals a column in the group. For example:

select thingEaten, thingTypeEaten, pointsGained, count(*) [Count], sum(pointsgained) [Sum]from pacman GROUP BY thingEaten, thingTypeEaten, pointsGained

ORDER BY thingTypeEaten, thingeaten

 

thingEaten thingTypeEaten Points

Count

Sum

bigdot

dots

50

1

50

littledot

dots

10

8

80

banana

fruit

150

1

150

cherry

fruit

150

1

150

blinky

ghost

100

1

100

clyde

ghost

100

1

100

pinky

ghost

100

1

100

 

A couple of notes about the query we’re using: first, I’m using an ORDER BY. You don’t have to, certainly, but it makes your results more readable.  Next, it’s a good idea to use surrogate column names when you use aggregate functions.  I’m calling mine Count and Sum, just to be really creative.

Also, notice that the SELECT statement looks like the GROUP BY, plus aggregates.  This is because GROUP BY requires that each item in the SELECT statement must be included in the GROUP BY, or must be an aggregate.  To take an example of this, let’s try adding our primary key “ID” to the select statement:

select ID, thingEaten, thingTypeEaten, pointsGained, count(*) [Count], sum(pointsgained) [Sum]from pacman GROUP BY thingEaten, thingTypeEaten, pointsGained

ORDER BY thingTypeEaten, thingeaten

We receive this error:

Msg 8120, Level 16, State 1, Line 1

Column 'pacman.id' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.

We tried to return individual row information without grouping it; we essentially gave the compiler an impossible task, that is, “Go combine these rows into one summary row, using type and points as the differentiator. Oh, and while you’re at it, include each individual row ID in that group.”  It doesn’t make sense, it can’t be done (like this), so SQL rightfully blows you a raspberry.

Notice that you can control the level at which you group. Right now we’re grouping by thingEaten, thingTypeEaten, and pointsGained. If we wanted to, we could get a higher-level view of our Pac-Man game by grouping only by thingTypeEaten, and totalling our points.  Remember to remove thingEaten and pointsGained from every clause in your query: select thingTypeEaten, count(*) [Count], sum(pointsgained) [Sum]from pacman GROUP BY thingTypeEaten

ORDER BY thingTypeEaten

thingTypeEaten Count Sum

dots

9

130

fruit

2

300

ghost

3

300

 

There you go, that’s the very basics of grouping.

To review:

·         GROUP BY allows you to collapse like sets of rows into a single row

·         Use aggregate functions to get more information about each group. Examples include COUNT, SUM, AVG, MIN (minimum), MAX (maximum), and so on.

·         ORDER BY is useful for readability, but not required.

·         GROUP BY requires that each item in the SELECT statement must be included in the GROUP BY, or must be an aggregate. 

·         Get a more detailed summary by grouping by more columns; get a higher level view by grouping with fewer columns.

 

-Jen McCown, http://www.MidnightDBA.com