Ray Wampler’s Database Blog

Dynamic output columns from a SQL stored proc

June 23, 2009 · Leave a Comment

 Here’s an interesting problem: create a sales report that accepts a start date and an end date. The output should be a table with one row per salesperson and a column for each month showing the total sales for that person for that month, starting with the month of the start date and ending with the month of the end date.

Let’s use Microsoft’s sample Adventureworks database. If you don’t have it installed already you can download from the CodePlex samples site here:

http://www.codeplex.com/MSFTDBProdSamples/

We can get the information that we need for the report from the table AdventureWorks.Sales.SalesOrderHeader. The table has the following fields that we’ll use for the report:

OrderDate
SalesPersonID
SubTotal

We can get the sales person name from the included view Sales.vSalesPerson.

The first thought might be to use a PIVOT command. There’s even a view included with the database, vSalesPersonSalesByFiscalYears, that displays sales by fiscal year using PIVOT. However PIVOT requires fixed output columns and the requirement is that we show all the months between the start and end date, so we have a variable number of columns depending on the input dates.

The technique that I’m going to describe will output as many monthly columns as you need and will work even in Sql Server 2000, where there be no PIVOT command.

Let’s begin with the SQL that would calculate the total sales by sales person for just one month. I’ll make use of the NextMonth function that I described in a previous posting.

DECLARE @startdate datetime
DECLARE @enddate datetime

SET @startdate = ‘08/01/2003′ — just a random date for which there is data
SET @enddate = dbo.NextMonth(@startdate) -1 — Last day of start month

Select FirstName, LastName, soh.SalesPersonID,
Sum(SubTotal) TotalSales
FROM Sales.SalesOrderHeader soh
JOIN Sales.vSalesPerson sp
ON soh.SalesPersonID = sp.SalesPersonID
WHERE OrderDate BETWEEN @startdate AND @enddate
GROUP BY FirstName, LastName, soh.SalesPersonID

Now lets say that we have an arbitrary start date and end date, such as

SET @startdate = ‘08/01/2003′
SET @enddate = ‘03/01/2004′

We want our query results to sales person information, total sales for the period and the sales for each month within the period:

FirstName
LastName
SalesPersonID
TotalSales
Aug 03
Sep 03
Oct 03
Nov 03
Dec 03
Jan 04
Feb 04
Mar 04

In this example there are eight months, but there could just as easily be one or twenty. The way we can handle any number of months between start and end date is to:

1. First, combine all the sales orders into one row per sales person per month
2. Create a temporary table with the sales person info and a total
3. Loop through each month between start and end date
4. Add a new column for each month
5. Update the new column with the total for each month
6. Display the temp table at the end with all the months

The full source code for the stored proc appears at the end this blog. Most of the steps are straightforward except #4. How do we dynamically add a column for each month? First we need to generate a unique column. We can use the short name of the month plus the year for a unique name for each monthly total:

SET @ColumnName = ‘['+Left(DATENAME(mm,@currDate),3)+' '+Right(Convert(varchar(4), Year(@currDate)),2)+']‘

Now we can dynamically add a column to the table using the EXEC command which dynamically executes sql text. EXEC can be dangerous if the text you execute includes user input because it makes you vulnerable to sql injection attack, but in our case we have complete over the text we are executing. Here’s the command:

EXEC(‘ALTER TABLE ##SalesByMonth ADD ‘ + @ColumnName +’ float’)

This will add a new float column named with the month and year. To populate it with data we’ll need to execute another dynamic statement, updating the new column with the total generated in step one for each sales person for that month.

We call the first temp table #MonthlySalesData and the dynamic output table is #SalesByMonth. Our dynamic update for the newly added column looks like this:

EXEC(
‘UPDATE #SalesByMonth’ +
‘ SET ‘ + @ColumnName + ‘ = MonthlySales’ +
‘ FROM #MonthlySalesData msd’ +
‘ JOIN #SalesByMonth sbm’ +
‘ ON sbm.[SalesPersonID] = msd.[SalesPersonID]‘ +
‘ WHERE OrderMonth = ”’ + @currMonth + ””)

We put this together into a stored proc named SalesPersonSalesByMonth which you can call with a start and end date likes this:

EXEC dbo.SalesPersonSalesByMonth
@startdate = ‘07/01/2003′,
@enddate = ‘09/01/2003′

The output looks like this:

Sales Person By Month Output 

And finally here is the full source of the stored procedure which unfortunately WordPress will totally mangle, making my code unreadable by inserting hundreds of extra line feeds, many of them in the middle of a line and turning every blank line into four blank lines and messing up the font, color and every other aspect of my source.  Thank you Word Press.


IF

OBJECT_ID('dbo.SalesPersonSalesByMonth', 'P') IS NOT NULL


 

DROP PROCEDURE dbo.SalesPersonSalesByMonth


GO
CREATE

PROCEDURE dbo.SalesPersonSalesByMonth


(

@startdate

AS DECLARE @currMonth datetime  DECLARE @ColumnName varchar(10)   

-- Create a temp table for the first staff of the calculations

 

CREATE

TABLE #MonthlySalesData ( 

FirstName

LastName

SalesPersonID

OrderMonth

MonthlySales

)

INSERT

INTO #MonthlySalesData  

Select

Sum(SubTotal) MonthlySalesFROM Sales.SalesOrderHeader sohJOIN Sales.vSalesPerson spON soh.SalesPersonID = sp.SalesPersonID  WHERE OrderDate BETWEEN @startdate AND @enddate  GROUP BY FirstName, LastName, soh.SalesPersonID, DateAdd(d, -Day(OrderDate)+1, OrderDate)   

-- Create initial temp table with sales person info and overall total

CREATE

TABLE #SalesByMonth ( 

FirstName

LastName

SalesPersonID

TotalSales

)
 

INSERT

INTO #SalesByMonth  

Select

-- Set up the range for the first month

SET

@currMonth = @startdate 

-- Loop through each month from start to end

WHILE

@currMonth <= @enddate 

BEGIN

 

 

-- Name the column with month and year 

SET @ColumnName = '['+Left(DATENAME(mm,@currMonth),3)+' '+Right(Convert(varchar(4), Year(@currMonth)),2)+']'

 

-- Add the column to the output table

EXEC('ALTER TABLE #SalesByMonth ADD ' + @ColumnName +' float')

 

-- Replace all nulls with 0s so there will be no nulls in output

EXEC('UPDATE #SalesByMonth SET ' + @ColumnName + ' = 0')

 

-- Update the column with the sum for each sales person

EXEC(
'UPDATE #SalesByMonth' +
' SET ' + @ColumnName + ' = MonthlySales' +
' FROM #MonthlySalesData msd' +
' JOIN #SalesByMonth sbm' +
' ON sbm.[SalesPersonID] = msd.[SalesPersonID]' +
' WHERE OrderMonth = ''' + @currMonth + '''')

  

-- Go to the next month

SET @currMonth = dbo.NextMonth(@currMonth)

END
SELECT

DROP

DROP

GO  TABLE #SalesByMonthTABLE #MonthlySalesData* FROM #SalesByMonth

 

 

 

 

 

 

 

FirstName, LastName, SalesPersonID, Sum(MonthlySales) TotalSalesFROM #MonthlySalesDataGROUP BY FirstName, LastName, SalesPersonID   

-- Now group monthly sales data into one row per salesperson

moneyint,varchar(50),varchar(50),FirstName, LastName, soh.SalesPersonID, DateAdd(d, -Day(OrderDate)+1, OrderDate) OrderMonth, 

-- Combine all the sales order rows
-- into data by sales person by month

moneydateTime,int,varchar(50),varchar(50),datetime,

@enddate

)

datetime

→ Leave a CommentCategories: SQL Server · Stored Procedure · Transact SQL

Calculate the next month

June 22, 2009 · 3 Comments

Here’s a Sql Server user-defined function that you may find useful. Let’s say that you have date and that you need to go to the first day of the next month. This can get complicated because months have different numbers of days (30 days hath September…) and the year changes when you calculate the next month from December.

The way to do this correctly every time with the least amount of logic is to go back to the first day of the current month and then add one month. Both steps can be done using DateAdd.

To get to the first day of month (assuming no time component) subtract the day of month and add one:

DateAdd(d, -Day(@dateIn)+1, @dateIn)

If the date is 4/15/07 then the day is 15. If you subtract 15 and add 1 then the result is 4/1/07. This works whether it’s the first, the 31st, or February 29th on a leap year.

Now to get to the first day of the next month use DateAdd again to add one month. The DateAdd function figures out the correct result regardless of how many days the month has or if you’re rolling into the next year.

DATEADD(mm, 1, @dateOut)

You can now put these commands together into a convenient Sql Server user defined function:

IF EXISTS (SELECT * FROM sysobjects WHERE type = ‘Fn’ AND name = ‘NextMonth’)
BEGIN
DROP FUNCTION dbo.NextMonth
END
GO

CREATE FUNCTION dbo.NextMonth
(
@dateIn datetime
)
RETURNS datetime
AS
BEGIN
DECLARE @dateOut datetime

SET @dateOut = DateAdd(d, -Day(@dateIn)+1, @dateIn)
SET @dateOut = DATEADD(mm, 1, @dateOut)

RETURN @dateOut
END
GO

GRANT EXEC ON dbo.NextMonth TO PUBLIC
GO

→ 3 CommentsCategories: SQL Server · Transact SQL · User Defined Functions

Server Management Server Object and it’s Collections

April 20, 2009 · 2 Comments

In my previous blog I explained how to use SMO (SQL Management Objects) to create a connection to SQL Server and return a Server object. The Server object is your gateway into everything contained within the SQL Server and all of the operations that you can perform with SMO. I will spend next few posts covering the Server object in detail. This post will focus on:

SMO Server Object Collections

Databases

The most important collection that the server object exposes is Databases. This collection gives you access to all the daatabases on the server. You can access these in a number of ways. If you know the database name then you use it as a selector:

Server server = new Server(@"(local)\SQLEXPRESS");
Database dbAW = server.Databases["AdventureWorks"];

You can also access a database using an integer index:

Database dbAW = server.Databases[4];

That technique would be more useful if you wanted to iterate through the databases:

Database db;
for (int i = 0; i < server.Databases.Count; i++)
{
db = server.Databases[i];
lbDisplay.Items.Add(db.Name);
}

You could also iterate through the databases using foreach:




foreach(Database db in server.Databases)

{

lbDisplay.Items.Add(db.Name);

}

If you want to create a new database you can use the Create method of Databases collection:




private Database CreateDatabase(string databaseName)

Server server = new Server(@"(local)\SQLEXPRESS");



Database newDb = new Database(server, databaseName);

return newDb;

}

server.Databases.Add(newDb);




{


Next I will cover the other collections of the Server object:

  • Audits
  • Backup Devices
  • Credentials
  • Endpoints
  • Languages
  • LinkedServers
  • Logins
  • Properties
  • Roles
  • SystemDataTypes
  • SystemMessages
  • Triggers
  • UserDefinedMessages
 

 

→ 2 CommentsCategories: SQL Management Objects
Tagged: , , , , , , , ,

Making a connection

April 9, 2009 · 1 Comment

The first step to using SQL Management Objects is making a connect to the database server.  You do this using the ServerConnection object.  For example, if you want to connect to your SQL Express 2008 database the c# code would look like this:

ServerConnectionconn;conn = new ServerConnection(); conn.ServerInstance = "(local)\SQLEXPRESS";
conn.LoginSecure = true; conn.Connect(); 

ServerInstance takes the name of the SQL Server as a string.  If you’ve installed SQL Express locally then the default name is (local)\SQLEXPRESS.  Setting  LoginSecure = true makes the connection using Windows Authentication.  If you wanted to connect using SQL Server authentication then the code would be like this:

conn.LoginSecure = false;conn.Login = "sa";conn.Password = "PassW0rd!";

What can do once you have a connection?  You can create a Server object:

sqlServer = new Server(conn);

The server object gives you a few collections like Databases, Logins, and Settings, that you can use to access, add, delete, and modify these objects.  You use the server object to run Transact-SQL statements directly, you can view and modify all the configuration settings, and you can programmatically do just about anything that you could through the SQL Server Management Studio.

→ 1 CommentCategories: Introduction · SQL Management Objects · SQL Server
Tagged: , , , , , , , , ,

Getting started with SQL Management Objects

December 5, 2008 · 1 Comment

The first thing you need to get started with SMO is SQL Server.  If you have Visual Studio then it installs a local instance of SQL Server Express on your computer by default.  For most development activity SQL Express is just fine.

You can even download and install SQL Express by itself for free:

http://www.microsoft.com/express/sql

The DLLs for SMO get installed in the program directory for SQL Server and they appear in the list of .Net References:

Add SMO Reference

Add SMO Reference

If you want to access all the capabilities of SMO you will also need to add a references to:

  • Microsoft.SqlServer.ConnectionInfo
  • Microsoft.SqlServer.Management.Sdk.Sfc
  • Microsfot.SqlServer.SqlEnum

The top-level SMO object is Server.  To connect to a sql server is fairly easy.   First you reference the SMO in your code module (the folowing examples are in C#):

 using Microsoft.SqlServer.Management.Smo;

 To connect to SQL Server you create a new instance of the Server object, passing the name of the SQL Server instance to the constructor.  For example, if you want to connect to your local SQL Express instance the code would be:

string serverName = @”(local)\SQLEXPRESS”;
Server server = new Server(serverName);

You can then use the Server object to perform server-level tasks such as create, backup, restore, and drop databases, add logins, and enumerate processes.  It also has some useful collections:

  • Databases
  • Logins
  • Roles
  • Settings

In upcoming posts we will explore all the fabulous things you can do with these objects.

→ 1 CommentCategories: Uncategorized

Welcome to my SMO (SQL Management Objects) Blog

December 1, 2008 · 2 Comments

I’m Ray Wampler and I have decided to blog about SMO: SQL Management Objects, which is yet another Microsoft Data Access library, geared toward providing a complete administrative API for Microsoft SQL Server that you can use with any .Net language such as C# or VB.net.

I have been programming for almost 30 years and started database development back in 1984 with dBase II. (Trivia Note: there never was a dBase I.  Ashton-Tate thought it would be better marketing to start at version 2.)  When I worked at EDS (Electronic Data Systems, founded by Ross Perot) I started with Microsoft Access version 1.0 and SQL Server 4.21.  I then went to work for Pacer/CATS, a company that made ticketing and concessions software for theaters and amusement parks.  There I became the database administrator and architect, working with Access, and SQL Server.  Since then I worked with every version of SQL Server, plus Oracle and MySQL.  I still prefer Microsoft SQL Server because it offers the best performance and ease-of-use at a reasonable cost.

I began programming with Microsoft .net since version 1.0 in 2002, much of it involving databases accessed through ADO and ADO.net.  I’ve also been keeping up with Microsoft’s evolving data access technologies: LINQ and the ADO Entity Framework, as well as using various data access layer and libraries like the Microsoft Enterprise Practices Data Access Library, Nhibernate, and .Net Tiers.

Where does SMO fit into all this?  Unlike ADO and it’s predecessor ODBC, which offer generic access to any number of data sources, SMO is for SQL Server only.  Although this sounds limiting to anyone experienced with n-tier programming, by specializing in SQL Server SMO lets you easily do many things that would be difficult or impossible with a database-agnostic data access technology.

Some of the things you can do with SMO include:

·         Modify and view SQL Server settings, information, and user options.

·         View and modify SQL Server settings and configuration options.

·         Back up and restore databases.

·         Create a new database or drop and existing database.

·         Register the instance of SQL Server in the Active Directory directory service.

·         Subscribe to and handle SQL Server events.

·         Reference databases, endpoints, credentials, logins, linked servers, system messages, DDL triggers, system data types, and user-defined messages.

·         Detach and attach databases.

·         Stop processes or databases.

·         Grant, deny, or revoke permissions on the database.

·         Enumerate information about the server.

·         Read the error log.

·         View and modify SQL Server engine objects.

·         Perform DDL (data definition language) tasks on an instance of SQL Server.

·         Script databases dependencies.

·         Transfer database schemas and data.

·         Reference database objects in collections, such as tables, users, and triggers.

·         Set up database mirroring.

·         Check data, allocations, catalogs, and tables.

·         Issue a checkpoint.

·         Grant, revoke, and deny permissions to users on the database.

·         Run Transact-SQL statements.

·         Enumerate database information, such as locks or object permissions.

·         Monitor the number of transactions.

·         Set the database offline or online.

·         Change the owner of the database.

·         Update statistics.

·         Shrink the database.

·         Truncate the log.

 Hopefully the large list above answers the question: how can you have an entire blog dedicated to an obscure, specialized and little-known Microsoft technology?  As I’ve been learning SMO I’ve been Googling for help on all the tasks that I’ve needed to do, and have not found a good reference or set of tutorials.  So my each issue of my SMO blog will be dedicated to demonstrating how to do each of the above tasks and many more.

→ 2 CommentsCategories: Introduction
Tagged: , , , , , , , , , ,

Hello world!

December 1, 2008 · 1 Comment

Welcome to WordPress.com. This is your first post. Edit or delete it and start blogging!

→ 1 CommentCategories: Uncategorized