Category Archives: Minion by MidnightDBA

“What users are in this group?”

minion enterpriseWe solved this question.

Update: Sign up for one of our Minion Enterprise demos this coming Friday, July 3!

Minion Enterprise collects SQL Server login data, as well as Active Directory information, for an entire enterprise. The AD expansion module ties this data together to provide so much insight:

  • Find out what users are in a Windows group…especially those groups that have sysadmin privileges!
  • List all users that have SA rights on any instance in the environment.
  • Discover which SQL Server instances a specific user has access to, and via what groups.
  • Filter by environment, location, SLA, server, login type, or any combination of the data available.

These are the exact questions we’ve always needed answered, in every single shop. So, we know this will be immensely useful in your shop.

One client was recently able to reduce their SQL access on one server by two-thirds. They simply used the AD expansion module to identify the rogue group with hundreds of members, and removed that group’s rights.

Take a look at the AD expansion module demo below, and then get in touch for your own 90 day trial license of Minion Enterprise.

 

Minion Backup intro webinar June 3

minionware_logoMinion Backup 1.0 is up and available for download as of now!

Minion Backup by MidnightDBA is a stand-alone database backup module.  Once installed, Minion Backup automatically backs up all online databases on the SQL Server instance, and will incorporate databases as they are added or removed.

Join the Minion Backup webinar on Wednesday June 3

Register today for our webinar, Introducing Minion Backup on Wednesday June 1 at 12:00 PM CDT. Sean will introduce Minion Backup, walk through demos, and take questions.

We released Minion Backup

It’s awesome. It’s huge. We actually managed to get everything we planned into version 1.0. Not everything we wanted, mind you: there’s still half a ton of features we have on the docket for the next few versions. But what we have done is still massive.

One short blog post won’t cover how revolutionary (yes, we’re serious: revolutionary) Minion Backup is. One job for all schedules, yes. Availability Group aware, check. Copy, move, mirror, compress, encrypt backups. Dynamic backup tuning. Backup archival. Custom retention settings. Extensive live logging. And on and on. Since we couldn’t cover it all here, we wrote 132 pages of documentation (available in DOCXPDF, and a zipped RTF), including a favorites feature list, a quick start, how to-s, and more.

While you’re at it, take a look at our several tutorial videos on MidnightDBA.com (or at YouTube.com/MidnightDBA if you prefer).

And oh by the way, what’s with “MinionWare”?

MidnightDBA is the banner for our free training. MidnighSQL Consulting, LLC is our actual consulting business. And now, we’ve spun up MinionWare, LLC as our software company. We released our new SQL Server management solution, Minion Enterprise, under the MinionWare banner. And now, all the little Minion guys will live together on www.MinionWare.net.

Minion Reindex, Minion Backup, and other Minion modules are, and will continue to be free. Minion Enterprise is real enterprise software, and we’d love the chance to prove to you that it’s worth paying for. Get in touch at www.MinionWare.net and let’s do a demo, and get you a free 90 day trial!

Coming soon: Minion Backup, featuring table based scheduling!

minion backup
The MidnightDBA team is announcing the release of a new, free backup solution for SQL Server: Minion Backup arrives on June 1!

Minion Backup by MidnightDBA is a stand-alone database backup module.  Once installed, Minion Backup automatically backs up all online databases on the SQL Server instance, and will incorporate databases as they are added or removed.

We created Minion Backup (or MB, for short) to be the most flexible, feature-rich backup solution possible. Our goal for this initial release was to include functionality for as many possible backup scenarios as possible. We’ve included certificate backups, HA and DR awareness, restore script generation, “what if” functionality for deletes, the ability to run a batch for “missing” backups, built in manual runs, rollup and detail data in the backup logs, the ability to deactivate most settings, copy / move / stripe / mirror backup files, etc.

Table based scheduling

While there are about fifty features I’d like to talk about, I’m going to restrain myself (today) and talk about the one feature I’m most excited about (today): table based scheduling.

When Minion Backup is installed, it creates a single backup job that runs the master backup stored procedure every 30 minutes.  That master procedure checks the Minion.BackupSettingsServer table to determine what backups should be run for the current day and time.

By default, Minion Backup comes installed with the following scenario:

  • Full system backups are scheduled daily at 10:00pm.
  • Full user backups are scheduled on Saturdays at 11:00pm.
  • Differential backups for user databases are scheduled daily except Saturdays (weekdays and on Sunday) at 11:00pm.
  • Log backups for user databases run daily as often as the backup runs (every 30 minutes).

Let’s look at just a few of the columns of this default scenario in Minion.BackupSettingsServer:

ID DBType BackupType Day BeginTime EndTime MaxForTimeframe Include Exclude
1 System Full Daily 22:00:00 22:30:00 1 NULL NULL
2 User Full Saturday 23:00:00 23:30:00 1 NULL NULL
3 User Diff Weekday 23:00:00 23:30:00 1 NULL NULL
4 User Diff Sunday 23:00:00 23:30:00 1 NULL NULL
5 User Log Daily 00:00:00 23:59:00 48 NULL NULL

I’m not going to fully document this table here – I’ll be happy to send you a draft of the product documentation if you can’t wait for the release date – but you get an initial impression of how flexible this scenario can be, especially in conjunction with other settings tables. I will note that “Include” and “Exclude” allow comma delimited lists of databases (and/or LIKE operators) to include in, or exclude from, the particular backup scenario; a value of NULL means that all databases are included.

This is how MB operates by default, to allow for the most flexible backup scheduling with as few jobs as possible.

Table based scheduling presents multiple advantages:

  • A single backup job – Multiple backup jobs are, to put it simply, a pain. They’re a pain to update and slow to manage, as compared with using update and insert statements on a table.
  • Fast, repeatable configuration – Keeping your backup schedules in a table saves loads of time, because you can enable and disable schedules, change frequency and time range, etc. all with an update statements. This also makes standardization easier: write one script to alter your backup schedules, and run it across all Minion Backup instances (instead of changing dozens or hundreds of jobs).
  • Mass updates across instances – With a simple Powershell script, you can take that same script and run it across hundreds of SQL Server instances at once, standardizing your entire enterprise with ease.
  • Transparent scheduling – Multiple backup jobs tend to obscure the backup scenario, because each piece of the configuration is displayed in separate windows. Table based scheduling allows you to see all aspects of the backup schedule in one place, easily and clearly.
  • Boundless flexibility – Table based scheduling provides a stunning degree of flexibility that would be very troublesome to implement with multiple jobs. With a single backup job, you can schedule all of the following:
    • System full backups three days a week.
    • User full backups on weekend days and Wednesday.
    • DB1 log backups between 7am and 5pm on weekdays.
    • All other user log backups between 1am and 11pm on all days.
    • Differential backups for DB2 at 2am and 2pm.
    • Read only backups on the first of every month.

…and each of these can also use dynamic backup tuning, which can also be slated for different file sizes, applicable at different times and days of the week and year.

and each of these can also stripe across multiple files, to multiple locations, and/or copy to secondary locations, and/or mirror to a secondary location.

Like I said, there are a zillion and a half more things I’d like to talk about, but we’ll keep it right here for now. Reply below, email, or ping @MidnightDBA on Twitter with questions or comments. And keep an eye out on June 1!

 

Check out Minion Enterprise, our new enterprise management solution for centralized SQL Server management and alerting! 

P.S.  Anticipating a few FAQs (and I’ll add to this as things come up):

  • Yes, you can change how often the backup job runs. If, for example, you only want log backups to run hourly, set your job to run hourly.
  • Yes, absolutely, you still have the option to use the more traditional “multi job” backup scheduling. You’d just disable the single job mentioned above, and configure the new jobs with individual schedules and a parameterized master query. Easy.
  • The Include and Exclude fields aren’t the only way to include and exclude databases, but we’re not going to get into that just now.

Minion Reindex 1.1 Release (and re-release)

We released Minion Reindex version 1.1 last week. The changes are fairly small:

  • Made sure Minion Reindex handles all nonstandard naming (e.g., object names with spaces or special characters).
  • Added support for Availability Group replicas. (Basic AG support has been added by only permitting Minion Reindex to run on an AG Primary DB.)
  • Fixed formatting in Minion.Help stored procedure.
  • If you run the installation script in a database other than master, this will now automatically be reflected in the Minion Reindex jobs. (The documents still say that you have to change this manually…I need to update that…)

But, and this is important, if you downloaded Minion Reindex 1.1 before this blog post hit, you need to re-download it. Yep, Jen messed up. She posted an earlier, incomplete version of the 1.1 package, and so introduced a syntax error.  As of now, this is fixed. (Her delicious dish of crow is in the oven, almost ready to eat.)

We timed the release to coincide with Grant Fritchey’s review of Minion Reindex, which has sparked a fantastic discussion in the comments.

So, once you download the fresh, non-syntaxy version of Minion Reindex 1.1, you can chime in and let us know how you feel!

 

Video: A Better Way to Reindex

minion reindex-01Edit: Corrected the recording link.

Yestreday I taught “A Better Way to Reindex” for the PASS Performance Virtual Chapter (event link, with recording).

The recording is up, if you missed the live event, or if you want to re-watch it:  https://attendee.gotowebinar.com/recording/8483181173057914370

And by the way, the demo that fails (there’s ALWAYS one demo that fails) is due to the fragmentation routine we use, fragmenting the data way more than usual. All we had to do was to increase the reorg threshold, and it would’ve been fine. So for the record, it’s the fault of the test harness, not of Minion Reindex.

Here’s the abstract:

A BETTER WAY TO REINDEX

Let’s play a guessing game: I guess that you don’t want to spend time on index maintenance, but you know your servers need it. You want something that’s straight plug and play, dependable, and preferably free. I also guess that you don’t want to manage multiple jobs for this one maintenance task, even when you have exceptions and special settings. Am I close on this? Come and take a look at Minion Reindex. It does all those things that you want for index maintenance, plus some really innovative stuff. Watch your reindex progress live. Configure settings and exceptions for individual databases or tables. Gather fragmentation stats separately, outside your maintenance window. There’s quite a lot more so come see how you’ll make your index maintenance a lot easier.

Of course, you should go and download Minion Reindex yourself, and see the documentation, and follow along with all the fun stuff I do in the session.

Allow_Page_Locks for Reorgs

minion reindex-01There are many settings that get set one way or another in DBs and in tables. Allow_Page_Locks is one of them that you may not be able to do anything about because your vendor may require it and your situation may require it as well. Normally it’s set to true, but it does get set to false and when it does, it typically needs to stay that way.
The problem is that when you reorganize these indexes that have allow_page_locks = false, then the reorg will fail. But the problem is that if you change it, you may see increased blocking issues. So what is there to do?

Well, the answer is Minion Reindex. We allow you to define pre and post code at the table-level that you can use to switch this option on and then off again when the table is through. And better yet, we even give you the code to discover all of these issues in your database and fix them. In the Minion Reindex download folder you’ll find a Queries folder. This folder has a sql file that you can run and it will insert the table-level exceptions with the proper pre/post code. The precode sets allow_page_locks = true and the postcode sets allow_page_locks = false. This way you can still have the setting the way you need it, and perform your index maintenance too.

And of course, Minion Reindex is completely free so download it now and you won’t be sorry.

129 of 635

This stat is one of the coolest things you’ll see in Minion Reindex.
FragStats

Gathering fragmentation stats in large databases can take a long time and you have no insight into what’s going on.

With Live Insight, Minion Reindex allows you to see what’s going on every step of the way, including how many indexes you have left to gather stats on.

And we even give you the name of the table and index that’s currently having its stats gathered.  (Our reindexing operations aren’t black boxes. I use this almost every day and I honestly don’t know what we ever did without it.)

NOTE: This feature is turned on by default. Just in case you’ve accidentally turned it off, just make sure that for the current database you have LogProgress = 1 in the Minion.IndexSettingsDB table.

Live Insight…by specific table

It gets even better. You may not be interested in Live Insight for every table.

In this case you can turn off Live Insight for certain tables, or turn it off for an entire database and then turn it on only for certain tables.

This is just one of the many useful features we have in our log.

And don’t forget: you can configure at the database level specifically or use the default DB settings by configuring the MinionDefault database in the Minion.IndexSettingsDB table.

I hope this helps you configure and use Minion Reindex better.

Order by Index Usage in Minion Reindex

Last week, a user asked if he could order index maintenance operations at the table level, based off of index usage. And of course, the answer to that is definitely yes. We designed Minion Reindex to have flexibility to address custom preferences like this without a lot of hassle, and with no extra jobs.

Minion Reindex already allows table ordering: you can give individual tables a heavier ReindexGroupOrder “weight” (higher numbers are reindexed first). All you have to do is change the order based off of your criteria. We’ve written a script –  MinionReindex-OrderTablesByUsage – that updates the ordering for all tables in a database, based off of usage. You’re welcome to customize this script and change the criteria to whatever you like.

A couple notes on the script:

  • Read the introductory comments for instructions and notes. And of course, review the script well.
  • This might have been easier with a MERGE statement, but we wanted it to be compatible with all versions that Minion supports. The script does an insert or an update based on whether that table is already listed in the “Minion.IndexSettingsTable” table.
  • This script can clearly be modified to meet any number of needs, like ordering tables by row count, or even excluding tables that don’t have enough reads.
  • To use this script, you can either add a job step, or encapsulate it in a stored procedure and add it as a DBPreCode in the Minion.SettingsDB table. It’s that easy.

Feel free to use this script or alter it as you need. And if you make it do something really cool, send it back to us and we’ll add it to the community of scripts on the site. We’ve done some initial testing of this script, but nowhere near the level of testing we put the product through so if you find any bugs let us know. MinionReindex-OrderTablesByUsage

Download Minion Reindex at MidnightSQL.com/Minion

Minion Reindex arrives this Friday!

The MidnightDBA team is announcing the release of a new, free index maintenance solution for SQL Server: Minion Reindex by MidnightDBA.  We designed Minion Reindex to solve all of the reindexing headaches we’ve experienced at dozens and dozens of client sites.

The Bottom Line

Your databases need index maintenance for optimal performance, and most shops don’t have a decent solution in place.  Minion Reindex is extremely simple to implement, and deeply configurable. Junior DBAs will love the ease of use, and expert data professionals will love the extensive settings and features available.

Read more on the Minion Reindex page:

  • Check out the full features list.
  • Get the download link there starting on Friday, October 24, 2014.
  • Join us for the Minion Reindex webinar Monday, October 27.
  • See documentation and tutorials as they come in!

You can ask questions and follow along with the discussion on Twitter using the #MinionReindex hashtag.

Stay tuned for the best thing index maintenance has ever seen…