When should I use agile methods on my software project?

People have asked the “what methodology should I use” question for a long time... More recently the question has changed to should I use an agile method (which flavor is irrelevant, but sometimes the question is “should I use SCRUM?”). With every means of inquiry, a non biased question is best, should I use agile implies that there is some preference for or against it… However, as the question is cropping up more and more, it might be best to answer it directly and with as little bias as possible…

You should run a project using an agile method if the project characteristics are such that allow it and that the environment facilitates it.

The key point to take from the statement above is that a project might be ideal for an agile methodology but the context might not be right to use it… Let’s first look at the project characteristics…

There are many ways for assessing projects characteristics, some of which are summarized by Mike Griffins in his article Agile Suitability Filters. Which one you select is up to preference and the particular problem domain. For this discussion, is not important. Essentially, different aspects of a project are measured on an ordinal scale; the level on that scale determines the suitability of agile methods for that project; for example the picture below shows the Boehm and Turner Radar chart.

image

For much more see the book Balancing agility and Discipline: A guide for the Perplexed, or read the paper that stimulated the idea Observations on Balancing Discipline and Agility.

The outcome of this assessment indicates whether or not agile is suitable. An extremely important thing to take from these assessments is that every project is different. You should assess each project on its own merits and not decide to use agile methods for all projects you run.

However, there are other factors which influence the methodology choice, outside of the context of a project, for want of a better term, the environment. From experience, these factors have more of an influence on the successful use of agile methods on projects. Mike Griffins also highlights this as Non-Project factors, where he coins the phrase “methodology bias” to indicate individual’s pre-conceptions or personal preference. By far the most critical component of a successful project is the level and quality of customer involvement. To understand why, let’s put it in context by looking at the problem and how different methodologies involve the customer.

The success of a software project is down to how well the developers understand what to build, or better said:

“The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is so difficult as establishing the detailed technical requirements, including all the interfaces to people, to machines, and to other software systems. No other part of the work so cripples the resulting system if done wrong. No other part is more difficult to rectify later.”
- F P Brooks, The Mythical Man Month (page 199)

The impact is highlighted in the Standish Group Software Project Survey (see Project Management for Information Systems 3rd edition by Cadle & Yates, 2001) 16% of projects were considered a success, 53% challenged (i.e. overran or missing functionality) and 31% failed. Also, Craig Larman highlights in his book “Agile and Iterative Development – A managers Guide” that historically 45% of functionality is never used (7% always used, 13% often used, 16% sometimes used and 19% rarely used).

A waterfall methodology addresses this problem through up front effort in requirements analysis and documentation with the customer. The artifacts of the process are verified against previous artifacts to ascertain their correctness. The customer then tests the delivered software and validates its completeness. Change is managed through a defined change control process. Cost of change is high and time between requirements elicitation and validation of completeness is long, thus there is a significant risk of delivering software which is incomplete (and therefore a failed project). This approach means the customer is heavily involved at the start and at the end of the project.

Iterative Incremental models, such as the RUP, validate the requirements earlier in the process by overlaying development with requirements analysis and software design (enabling demonstration of working, but incomplete, software). The aim is to mitigate risk by validating that challenging and critical parts of the system are understood correctly. These processes still generate supporting artifacts and have verification and change control structures (similar to linear sequential models). With this approach the customer involvement is still mainly at the start and end, although the customer will be slightly more involved than on a waterfall method.

clip_image003

The essence of all agile methods is the same… They are best described by the agile manifesto and the twelve principles of agile. Agile methods raison d’être is to frequently deliver working software to a customer that adds value immediately. Change is embraced to ensure only what is most needed is delivered and this is achieved through frequent (daily) face to face customer collaboration. Agile methods demand that the customer is involved throughout; the customer is part of the team.

The reason for this discussion is to highlight the importance of customer involvement. If you cannot have access to the customer when you need them, agile fails. In agile projects there is no documentation to fall back to, the customer is the only source of information you have: without the customer an agile project will fail.

Therefore, the most important thing to do when assessing the suitability of an agile method is to determine the level of commitment the customer is willing to give to constant contribution. It makes sense for the customer to want to be deeply involved in the product you are making them, however, there are a myriad of real world reasons why it might not be possible. With any process you deploy to solve a problem customer buy in is critical… The customer must understand what is required of them and be committed to deliver on that requirement. This cannot be forced and any attempts to make a customer buy into agile will result in difficulty. Sometimes, upfront analysis might just have to do…

It is also important to consider how frequently you are actually going to deliver functionality to the customer… If the customer doesn’t receive incremental builds of working software then how do you know what you’ve developed has actually added any value? This is discussed by Alistair Cockburn in an article entitled “Are iterations hazardous to your project”. He also coins the “Agile machismo points” idea; it highlights very well the importance of the core principles of agile methods!

It is possible to run an agile project on a fixed priced or time and materials basis. However, in both cases for the project to be agile the scope must be flexible. If the project is fixed scope, there is no benefit from using agile methods. If it’s fixed scope then more than likely the requirements will form part of the contract and thus be well defined (through upfront analysis). The point of agile is to allow change such that the developers are delivering software that will be used by the customer, essentially reduce the scary waste of 45% of functionality not being used!

So in summary:

1. Assess every project suitability against a set of criteria (reuse one off the shelf, like the Radar chart above). If it’s not suitable, don’t use it. Don’t assume because it worked for one project it will work for all!

2. Do not embark on an agile project if the customer is not available throughout the project. For clarity, available means you can see them face to face at least once a day… Or at least be able to pick up the phone and talk to someone who knows the answer when you have a question.

3. Are you going to deliver evolving software to the customer to solve real business problems? Is the customer proving the suitability of what you’ve developed in the last iteration? If no, you’re probably better off iterative incremental and not agile…

4. Is the customer able to re-prioritise and add requirements as they see the software? Are you contractually allowed to without invocating a time consuming change control process? Do you have to deliver 100% of requirements? If there’s no scope for change, then don’t use agile.

Remember, there are still iterative incremental models which reduce risk and increase quality. Your chose isn’t agile or waterfall! Hopefully this blog will help answer the question; we’re all still leaning so I’m sure there will be updates!

Team work and communication is key to productivity and success

Hadi's post (The Bus number isn't agile, it's common sense ), was quite thought provoking...  Loss of knowledge, be that temporary (someone on holiday) or permanent (moving on or retirement), is a massive problem for all kinds of businesses.  This is even more critical in organisations who essentially sell their expertise, i.e. consultancy companies.  Knowledge management for large organisations is a massive priority.  However, SME's don't have the resources, expertise or drive to implement a framework for knowledge management.

The problem, which Hadi describes, is the impact short term loss of knowledge has on productivity.  In terms of delivering a project the importance of sharing knowledge is crucial. 

The long term loss of knowledge (a year after a solution is delivered) is a different problem.  The global trend (or at least the trend in the USA), as observed by Daniel Pink in Free agent nation, is that people move on in their career much quicker than before.  Loyalty to a particular company is replaced with vertical loyalty (loyalty to products, colleagues, ex colleagues, professions and family)...  Jobs, are now gigs, or as Tom Peters says WOW Projects...  You move from gig to gig, project to project.  You may do several gigs for a single company...  The key is that you are constantly developing, learning, achieving amazing things - you go to the best gig, the one with the biggest WOW factor.  People will move on - the sufficiency of the project residue is now crucial to the long term maintainability of a product.

Addressing the short term problem can also address the longer term problem.  If you subscribe to the CMMI, then your processes will ensure the repeatability of projects: a key premise is that the heroic effort of individuals to deliver a project is removed by reliable processes.  If you have a certain level of process maturity these problems are said to be alleviated.  I say, said to be, for a reason.  Long term, a new team can read the reams of documentation and start to get an understanding of what went on.  But, on a project, there is time to get up to speed...  When you're delivering the product, you don't have time to get up to speed, getting up to speed costs in terms of productivity (slippage on the plan and eventually reduced profit margins).

Short term, teaming is the answer.  In fact, effective communication is the answer.  I say effective for a reason, because communication can also be a problem.  The overhead of communication is positively correlated to team size, i.e. bigger teams more overhead (therefore lower productivity).  Brooks observed, a long time ago, small sharp teams will destroy large teams in terms of productivity and product quality.  If you have the optimum team size for your problem, then you're most likely in trouble if you work near a bus stop!  Team size is a side issue.  The issue is sharing information, or in Hadi's case, not creating knowledge silos (someone who knows everything about the GUI and nothing about the middle tier and vice versa).  The way you get the most out of your team is effectively down to task division.  In some cases task division means that you have people working solely on the GUI and others working on the back end.  For example, if you agree a contract, let's say it's your WSDL, then one team can crack on with the implementation of the capabilities encapsulated by the web methods, the other can work on bringing those capabilities to bear via the GUI.  Also, someone might want to do a certain bit of work more than someone else, they may be better at it and will do it quicker (note: just because they want to do something more than something else doesn't mean they don't also want to do the other thing...). Commercially, both these economies are important to providing good value for money and therefore cannot be ignored.  You have a problem if those teams consist of two people, one person in the GUI and the other in the service implementation team.  Now, you can still have the ability to switch between both (in the case where someone goes on holiday) without losing too much productivity.  It, however, depends on two things.  The first is the effectives of the communication; the second is level of listening of people in your team.

There are many things you can do create an environment for effective communication, and they are well documented in numerous books, in the context of software, and many other contexts.  But, specifically, I'll highlight some (and a great place to read about them in more detail is Agile Software Development):

  1. "Osmotic" communication.  Sit together.  Don't go to a meeting room to have a chat, have a chat so others can hear.  It's amazing what you hear by listening without knowing.  A great example (I can't remember where this came from, probably my project management unit on my MSc):  There was a coffee machine outside an IT lab in a university library.  The machine was just outside the office of the helpdesk staff (people employed to help stuck students through problems).  The staff complained about the noise of students congregating around the coffee machine, so it got removed.  The number of people going to see the help desk staff shot up.  The reason?  People got stuck and went for a coffee.  They met fellow students also stuck sipping their coffee pondering...  They start chatting about their problems and before they know it, they've collaboratively solved it.  On lookers, also stuck, drinking coffee heard them and were also able to try and continue.  No more coffee machine, no more environment for collaborative problem solving.  The help desk staff soon put up with the noise...
  2. Collaboration.  Kind of carries on from that example above...  Pair programming is the ultimate example of collaboration, it solves all the problems with knowledge sharing (unless they practice pair holiday taking and fly XL airways, then you might have a knowledge gap for longer than expected!)  But, just because I'm working on the GUI doesn't mean I don't have good ideas about how to solve a problem in the data access layer and vice versa.  Google, is sometimes our worse enemy.  I can Google a problem and get a solution, or look for similar implementations, design patterns etc...  However, if you solve problems collaboratively you build shared experiences which can be used when that person needs to work on what you've been working on.  And, the old adage, two brains are better than one...  Sometimes mealy talking aloud a problem you figure it out...  That, with communication osmosis is extremely powerful.  If you watch it happen, you'll also notice productivity is better...
  3. Build shared experiences.  One of the biggest barriers to effective communication is the lack of shared experiences.  You can communicate at a pace, if you share a common experience.  You always look for ways to touch into them every time you talk to someone.  Think about your recent conversations, you'll see what I mean.  So, when the customer comes in to look at the GUI, get the back end guy in there to share the experience...  If you DO collaboration, then you'll do this anyway.
  4. Create rich artifacts.  Think about the purpose of the artifact...  Is it to help Fred fix bugs in the GUI, or for Bill a year later to know everything about the product.  Is it to document a customer requirement so Jim can implement it?  Where shared experiences exist, simply create artifacts that touch into them and recreate the thoughts and experiences at the time of that conversation...  Leave markers to enable the reconstruction of ideas.  The least rich means of communication, by a massive order of magnitude, is the written word.  It is impossible to completely document an experience, so don't try.
  5. LISTEN naively!  The single biggest reason people don't like talking about their problems is because people don't listen; they assume they know the answer.  Sometimes, simply listening is all you need to do.  One of the biggest reason a product isn't a success is because people didn't listen to their focus group.  The CUSTOMER is the source of the best ideas.  Bottom up innovation is critical, not top down.  Tom Peters advocates this in "A passion for excellence".
  6. Motivation.  Don't make people collaborate; motivate them to do it...  In fact, they will automatically be motivated to collaborate, what better feeling is there than knowing you've helped someone?  Why do teachers teach?  As project managers, do not hinder communication, don’t undermine people, don't be a devil’s advocate, encourage and enable communication.  Be the facilitator; remove the interference from your team so they can effectively communicate.  Except that failures happens and encourage people to share those failures.

There's certainly a LOT more...  But it's not about software, it's about teamwork!

Now, what do I mean by levels of listening?  There are limits to how effectively individuals can communicate.  That's based on their ability.  A top class software developer can draw on their previous experience, understanding of best practice and the problem at hand to synthesise a solution which solves the problem in the best way.  When they communicate, this synthesis is happening to maximise the value they take from their conversations.  A more junior developer cannot do this yet, they'll need support.  You need to understand the makeup of your team and facilitate communication by having people on hand that can help cut through the fog.  It is important they don't just solve the problems; rather they facilitate the others in solving the problems.  If the problem is too complicated and economic reasons mean the senior guy needs to do it, make damn sure he talks about what he's doing (creates the shared experience)!  This also creates a great learning environment, which is critical to staff retention.  Hertzberg defines motivation as a function between ability and opportunity to use that ability.  The more you can do, the more you can be motivated to do.  That's the reason why training is one of the biggest motivators.  By motivating your staff, you're also increasing the composite capability of your company...  Training is win win, especially when it's free like facilitated on the job learning is...

If you do these things, then someone going on holiday means that another person in the team is MOTIVATED to pick it up and solve the problem.  It is that level of motivation that will keep your productivity at a reasonably constant level.  Also, if the project is highly collaborative, learning will be high, therefore, it'll be a good place to be...  People will stay, because the next project is the next gig they want to play at!  Teamwork wins!

One Comment Filed Under [ Software Process ]
Agility?

I was having a good discussion with a colleague about the use of agile as a project methodology.  We talked at some length about it's suitability...  Now, there are many arguments about when agile should be used and many arguments for when it shouldn't be used, however,  that's too much to go into now (maybe later)... 

However, regardless of the methodology, there is a common goal in ALL software projects.  The essence of what we do hasn't changed, and will never change as long as we do "software development".  Personally, I always think of what Alistair Cockburn says in his book Agile Software Development, I quote: 

"Software development is a (resource-limited) cooperative game of invention and
communication. The primary goal of the game is to deliver useful, working software. The secondary goal, the residue of the game, is to
set up for the next game. The next game may be to alter or replace the system or to create a neighboring system."

I also remember a quote from the Mythical Man Month along the lines of:  Conformance to user requirements is the hardest single part of developing software; the consequences if done incorrectly are the most common reason for software project failure.

Essentially, on a project, we need to find ways to get better at inventing and communicating.  If we get better at communicating, we'll get better and understanding what the customer wants and we'll get better about discussing it to invent solutions.  The better we are at inventing solutions, the higher the customers satisfaction with the product will be, i.e. they'll actually use it!  Let's face it, there is no higher measure of quality than how often something is used - quality is conformance to customer requirements...  The higher quality the residue we leave behind, the more easily a second team will be able to pick up where the first left off.  The better we collaborate (with both the customer and fellow developers), the quicker we'll get stuff done.  Whether agile is the correct mechanism for that is not really important, it just matters we are bloody good at it, because if we aren't, we fail! 

It just so happens that a lot of the stuff you do in an agile project enhances the teams ability to effectively communicate, provide a place where collaboration happens intrinsically and where innovation is encouraged.  But, that doesn't have to be just on an agile project.

Maybe, if I get some time, I'll do some follow ups on this...

LDAP Distinguished Names escaping illegal characters

I needed to escape illegal characters within an LDAP query but couldn't find anything out there to do it.  The illegal characters are defined here: http://msdn.microsoft.com/en-us/library/aa366101.aspx

Say I want to query for a group called "Test/Group" the LDAP would need to look something like LDAP://CN=Test\/Group,OU=a,DC=x,DC=y

I came up with this regular expression to clean it up.

(?<=(?:[^\\]|^)(\\\\)+|[^\\]|^)[/,+\"><;=#]|(?<=(?:[^\\]|^)(\\\\)+|[^\\]|^)\\(?!\\|[/,+\"><;=#]| $|(?<=^\\) )|^

So that translated into code a bit like "LDAP://CN=" + Regex.Replace(name, "(?<=(?:[^\\\\]|^)(\\\\\\\\)+|[^\\\\]|^)[/,+\"><;=#]|(?<=(?:[^\\\\]|^)(\\\\\\\\)+|[^\\\\]|^)\\\\(?!\\\\|[/,+\"><;=#]| $|(?<=^\\\\) )|^ ", "\\$0")

One thing I did notice was that when you ask for a users groups it is kind enough to escape most of the illegal characters for you, the only one it doesn't do is the /.  So if you're forming up an LDAP query straight from the memberOf property then all you need to do is make sure that the / is replaced with a \/...  Which makes life a bit easier and you don't need the regular expression then.  For example:

SearchResult

user = srch.FindOne();
foreach (string memberOf in user.Properties["memberOf"])
{
    DirectoryEntry
group = new DirectoryEntry("LDAP://" + memberOf.Replace("/", "\/"));
}
2 Comments Filed Under [ Active Directory ]
Moving all tables and indexes to a different file group in SQL Server 2005

 

I needed to move all the tables and related indexes to another file group but didn't want to drop and re-create the database...  I had a Google, and found some SQL on http://www.sqlmag.com/Article/ArticleID/97018/sql_server_97018.html which moved a given table to another given file group.  I updated it to move all the tables not already on that group.

Another thing I noticed when playing around with this was the behaviour in SQL Server Management Studio was that it always specifies the file group when you script off an index.  I just wanted it to use the default file group specified.  This was annoying because the script we sent to a customer now has our default file group named in it, which they don't have setup.  If you don't specify a file group then it will use the one configured as the default.  So, I also had to modify all the scripts to not specify a file group, somewhat of an annoyance...  Therefore, if you don't want to have the hassle of modifying the generated script, the script to move the tables works well! If anyone knows how to make SQL not specify a file group please let me know!

The SQL below is a modified version of the one found in the link above - go there for more information.  All I changed was fix a bug where you needed the [ ] around index / table names when calling OBJECT_ID (it doesn't like the . in the names of the objects as . separates the schema name).  Also added the loop for each table, the original script you specified a table name.

 

 

 

SET NOCOUNT ON

 

-- Note - @SourceFileGroupID and @TargetFileGroupID are the IDs of the

-- source and destination file groups. If you are not sure what

-- the IDs of your file groups are, simply run sp_helpfilegroup

-- and look at the groupid column of the result set.

 

DECLARE @SourceFileGroupID INT

DECLARE @TargetFileGroupID INT

DECLARE @TableToMove NVARCHAR(128)

DECLARE @MovePKAndAllUniqueConstraints BIT

DECLARE @MoveAllNonClusteredIndexes BIT

 

SELECT

@SourceFileGroupID = groupid

FROM

sysfilegroups

WHERE

groupname = 'PRIMARY'

 

SELECT

@TargetFileGroupID = groupid

FROM

sysfilegroups

WHERE

groupname = 'NewGroup'

 

SET @MovePKAndAllUniqueConstraints = 1

SET @MoveAllNonClusteredIndexes = 1

 

 

DECLARE @ScriptMsg NVARCHAR(512)

DECLARE @DatabaseName SYSNAME

DECLARE @ServerName SYSNAME

DECLARE @TableHasCI BIT

DECLARE @TableHasIdent BIT

DECLARE @TableHasPK BIT

DECLARE @TableHasUQ BIT

DECLARE @File1Name NVARCHAR(128)

DECLARE @File2Name NVARCHAR(128)

DECLARE @IdentColName NVARCHAR(128)

DECLARE @ColList NVARCHAR(1024)

DECLARE @indid NVARCHAR(128)

DECLARE @Type CHAR(2)

DECLARE @KeyName NVARCHAR(128)

DECLARE @AssocFKeyName NVARCHAR(128)

DECLARE @FKTableName NVARCHAR(128)

DECLARE @CIName NVARCHAR(128)

DECLARE @IsPadIndex BIT

DECLARE @i INT

DECLARE @j INT

DECLARE @SQLStr NVARCHAR(4000)

 

-- Get server and database names

SET @ServerName = CAST(ISNULL(SERVERPROPERTY('ServerName'), 'Unknown') AS SYSNAME)

SET @DatabaseName = db_name()

 

-- Some basic verifications:

-- 1. Check that file groups exist, and that the table exists.

 

SET @File1Name = FILEGROUP_NAME(@SourceFileGroupID)

 

IF @File1Name IS NULL

BEGIN

IF @SourceFileGroupID IS NULL

SET @SourceFileGroupID = 'NULL'

 

SET @ScriptMsg = N'The source file group ' + CAST(@SourceFileGroupID AS VARCHAR(64)) + N' does not exist on the database ' + @DatabaseName + N', on server ' + @ServerName + N'. Please provide a valid filegroup id.'

RAISERROR(@ScriptMsg, 16, 1)

RETURN

END

 

 

SET @File2Name = FILEGROUP_NAME(@TargetFileGroupID)

 

IF @File2Name IS NULL

BEGIN

IF @TargetFileGroupID IS NULL

SET @TargetFileGroupID = 'NULL'

 

SET @ScriptMsg = N'The target file group ' + CAST(@TargetFileGroupID AS VARCHAR(64)) + N' does not exist on the database ' + @DatabaseName + N', on server ' + @ServerName + N'. Please provide a valid filegroup id.'

RAISERROR(@ScriptMsg, 16, 1)

RETURN

END

 

IF @SourceFileGroupID = @TargetFileGroupID

BEGIN

SET @ScriptMsg = N'The file groups provided are the same. This is not allow in this script.'

RAISERROR(@ScriptMsg, 16, 1)

RETURN

END

 

-- 2. Check that the target file group is not read-only.

 

IF FILEGROUPPROPERTY(FILEGROUP_NAME(@TargetFileGroupID), 'IsReadOnly') = 1

BEGIN

SET @ScriptMsg = N'The taget file group (i.e., with file group id = ' + CAST(@TargetFileGroupID AS VARCHAR(32)) + N') is read-only. Aborting table move.'

RAISERROR(@ScriptMsg, 16, 1)

RETURN

END

 

IF OBJECT_ID('tempdb..#tablesToChange', 'U') IS NOT NULL

DROP TABLE #tablesToChange

 

SELECT

TABLE_NAME

INTO

#tablesToChange

FROM

INFORMATION_SCHEMA.TABLES

WHERE

TABLE_TYPE = 'BASE TABLE'

AND TABLE_NAME IN

(

SELECT

OBJECT_NAME(id)

FROM

sysindexes i

WHERE

groupid = @SourceFileGroupID

)

--AND TABLE_NAME = 'AssassinObject'

 

WHILE EXISTS(SELECT * FROM #tablesToChange)

BEGIN

 

BEGIN TRY

 

BEGIN TRANSACTION

 

SELECT

TOP 1

@TableToMove = TABLE_NAME

FROM

#tablesToChange

 

DELETE

#tablesToChange

WHERE

TABLE_NAME = @TableToMove

 

PRINT N'Moving table ' + @TableToMove

 

-- This temp table holds the column names of keys/constraints, and such.

IF OBJECT_ID('tempdb..#tblColTable', 'U') IS NOT NULL

DROP TABLE #tblColTable

 

CREATE TABLE #tblColTable (

Idx INT IDENTITY(1, 1),

ColName NVARCHAR(128) COLLATE DATABASE_DEFAULT,

IdxOrder CHAR(4))

 

-- This temp table is used to store the key/constraint properties

-- of the moved table.

IF OBJECT_ID('tempdb..#tblKeysTable', 'U') IS NOT NULL

DROP TABLE #tblKeysTable

 

CREATE TABLE #tblKeysTable (

Idx INT IDENTITY(1, 1),

KeyName NVARCHAR(128) COLLATE DATABASE_DEFAULT,

indid INT,

Type CHAR(2))

 

-- This temp table holds the foreign keys of the table.

-- The SQLStmt column is used to build dynamic SQL statements

-- that are related to these foreign keys.

IF OBJECT_ID('tempdb..#tblFKTable', 'U') IS NOT NULL

DROP TABLE #tblFKTable

 

CREATE TABLE #tblFKTable (

Idx INT IDENTITY(1, 1),

ForeignTableName NVARCHAR(128) COLLATE DATABASE_DEFAULT,

KeyName NVARCHAR(128) COLLATE DATABASE_DEFAULT,

SQLStmt NVARCHAR(1024) COLLATE DATABASE_DEFAULT)

 

-- This temp table holds the colunms of the foriegn key of the table.

IF OBJECT_ID('tempdb..#tblFKColTable', 'U') IS NOT NULL

DROP TABLE #tblFKColTable

 

CREATE TABLE #tblFKColTable (

Idx INT IDENTITY(1, 1),

ColName NVARCHAR(128) COLLATE DATABASE_DEFAULT,

FOrP CHAR(1))

 

 

SET @TableToMove = LTRIM(RTRIM(@TableToMove))

 

IF RIGHT(@TableToMove, 1) = ']'

SET @TableToMove = LEFT(@TableToMove, LEN(@TableToMove) - 1)

IF LEFT(@TableToMove, 1) = '['

SET @TableToMove = RIGHT(@TableToMove, LEN(@TableToMove) - 1)

 

 

-- Validate the table name and check that it exists in the system catalog.

IF @TableToMove IS NULL OR @TableToMove = ''

BEGIN

SET @ScriptMsg = N'The table name provided in the script is either null or empty, on server '

+ @ServerName + N' and database ' + @DatabaseName

+ N'. Please provide a valid table name.'

RAISERROR(@ScriptMsg, 16, 1)

RETURN

END

 

IF NOT EXISTS(SELECT * FROM INFORMATION_SCHEMA.TABLES

WHERE TABLE_NAME = @TableToMove AND TABLE_TYPE = 'BASE TABLE')

BEGIN

SET @ScriptMsg = N'The table name provided in the script is not found in database '

+ @DatabaseName + N', on server ' + @ServerName + N'. Please provide a valid table name.'

RAISERROR(@ScriptMsg, 16, 1)

RETURN

END

 

-- Check that the table is indeed defined on the source file group.

IF (SELECT TOP 1 groupid FROM sysindexes WHERE id = OBJECT_ID('[' + @TableToMove + ']') and indid IN (0, 1)) <> @SourceFileGroupID

BEGIN

SET @ScriptMsg = N'The table ' + @TableToMove + ' is not found on filegroup ' + CAST(@SourceFileGroupID AS VARCHAR(32))

+ N'. Please provide a valid table name and source file group.'

RAISERROR(@ScriptMsg, 16, 1)

RETURN

END

 

 

-- 3. If we have gotten this far, then it is ok to move the table to the

-- requested filegroup.

 

-- First thing first: Check whether the table has a clustered index.

SET @TableHasCI = OBJECTPROPERTY(OBJECT_ID('[' + @TableToMove + ']'), 'TableHasClustIndex')

 

-- If not - check whether the table has an identity column.

-- If it does - apply the CI with the new filegroup on the identity column.

-- Once done - remove the CI. If it does not - check whether the table has a primary

-- key and apply the CI there on the new file group, and then drop the CI.

-- If the table does not have an identity column, or a primary key,

-- then a new identity column is created for the table and the CI

-- is applied on it, and then the CI and the identity column are removed.

-- This whole shabang is done in order to make the CI creation as fast as possible.

-- The case where the table does not have a clustered index to begin with implies

-- bad table design, and should not be common anyhow.

 

IF @TableHasCI = 0

BEGIN

SET @TableHasIdent = OBJECTPROPERTY(OBJECT_ID('[' + @TableToMove + ']'), 'TableHasIdentity')

 

IF @TableHasIdent = 0

BEGIN

SET @TableHasPK = OBJECTPROPERTY(OBJECT_ID('[' + @TableToMove + ']'), 'TableHasPrimaryKey')

SET @TableHasUQ = OBJECTPROPERTY(OBJECT_ID('[' + @TableToMove + ']'), 'TableHasUniqueCnst')

 

-- Only if the table has no PK/UQ or clustered index, then create an identity

-- column on it. This new column will hold the CI.

IF @TableHasPK = 0 AND @TableHasUQ = 0

BEGIN

EXEC(N' ALTER TABLE [' + @TableToMove + N'] ADD

[This_Is_My_Ident_Col_Name] BIGINT IDENTITY (1, 1) ')

 

SET @IdentColName = 'This_Is_My_Ident_Col_Name'

 

-- Apply the CI on the identity column. We don't create the CI

-- as unique, since the identity column may be non-unique,

-- due to reseeding.

 

EXEC(N'CREATE CLUSTERED INDEX [This_Is_My_Clsuetered_Index_Name]

ON [' + @TableToMove + N']([' + @IdentColName + '])

ON [' + @File2Name + N']')

 

-- The table is now moved -> Remove the CI.

 

EXEC(N'DROP INDEX [' + @TableToMove + N'].[This_Is_My_Clsuetered_Index_Name]')

 

-- Finally, remove the added identity column

 

EXEC(N' ALTER TABLE [' + @TableToMove + N']

DROP COLUMN [This_Is_My_Ident_Col_Name] ')

 

END

ELSE

BEGIN

-- In this case, the table has a PK/UQ, so we might as well

-- apply the CI on the column(s) of the PK/UQ.

-- First, get the column(s) of the PK/UQ.

 

SELECT @KeyName = CONSTRAINT_NAME

FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WITH (NOLOCK)

WHERE TABLE_NAME = @TableToMove

AND CONSTRAINT_TYPE = 'PRIMARY KEY'

 

IF @@ROWCOUNT = 0

SELECT TOP 1 @KeyName = CONSTRAINT_NAME

FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WITH (NOLOCK)

WHERE TABLE_NAME = @TableToMove

AND CONSTRAINT_TYPE = 'UNIQUE'

 

-- The varialbe @KeyName now holds the name of the PK/UQ

INSERT INTO #tblColTable (ColName, IdxOrder)

SELECT COL_NAME(OBJECT_ID('[' + @TableToMove + ']'), colid),

-- append the DESC/ASC string, based on the ASC/DESC order of the PK columns

CASE WHEN INDEXKEY_PROPERTY(OBJECT_ID('[' + @TableToMove + ']'),

INDEXPROPERTY(OBJECT_ID('[' + @TableToMove + ']'),

@KeyName,

'IndexID'),

keyno,

'IsDescending') = 1

THEN 'DESC'

ELSE 'ASC'

END

FROM sysindexkeys

WHERE id = OBJECT_ID('[' + @TableToMove + ']')

AND indid = INDEXPROPERTY(OBJECT_ID('[' + @TableToMove + ']'), @KeyName, 'IndexID')

 

IF @@ROWCOUNT > 0

SET @i = 1

 

SET @ColList = N''

 

WHILE EXISTS(SELECT * FROM #tblColTable WHERE Idx = @i)

BEGIN

SELECT @ColList = @ColList + N'[' + ColName + N'] ' + IdxOrder + N' ,'

FROM #tblColTable

WHERE Idx = @i

 

SET @i = @i + 1

END

 

SET @ColList = LEFT(@ColList, LEN(@ColList) - 1)

 

-- Now, apply the CI on the primary key columns. The CI is not

-- created as a unique CI, since if the PK/UQ was added with the NOCHECK

-- option, there could be duplicate entries in the PK/UQ.

 

EXEC(N'CREATE CLUSTERED INDEX [This_Is_My_Clsuetered_Index_Name]

ON [' + @TableToMove + N'](' + @ColList + ')

ON [' + @File2Name + N']')

 

-- The last command moved the CI (and thus the table), so we

-- can now drop the CI.

 

EXEC(N'DROP INDEX [' + @TableToMove + N'].[This_Is_My_Clsuetered_Index_Name]')

END

END

ELSE

BEGIN

-- Here, the table originally had an identity. We apply the CI

-- on the identity column, and then remove it.

SELECT @IdentColName = COLUMN_NAME

FROM INFORMATION_SCHEMA.COLUMNS WITH (NOLOCK)

WHERE TABLE_NAME = @TableToMove

AND COLUMNPROPERTY(OBJECT_ID('[' + @TableToMove + ']'), COLUMN_NAME, 'IsIdentity') = 1

 

EXEC(N'CREATE CLUSTERED INDEX [This_Is_My_Clsuetered_Index_Name]

ON [' + @TableToMove + N']([' + @IdentColName + '])

ON [' + @File2Name + N']')

 

-- The table is now moved -> Remove the CI.

 

EXEC(N'DROP INDEX [' + @TableToMove + N'].[This_Is_My_Clsuetered_Index_Name]')

END

END

ELSE

BEGIN

-- Now, for the big ELSE. The ELSE applies to the case where the

-- table already has a clustered index. Here, we select the name of the

-- existing clustered index, then drop it from the table, and recreate

-- it on the other filegroup (on the same columns and order as was

-- originally defined for the table).

-- If the CI is also a PK/UQ/unique index, then we first check all foreign

-- keys for the PK/UQ/UI, drop them if they exist, drop the PK/UQ/UI

-- then recreate the PK/UQ/UI as CLUSTERED, and then reapply all the

-- foreign keys constraints. If the CI is non-unique (thus is not

-- associated with a PK/UQ/UI), we just drop and recreate it on the

-- target file group.

 

SELECT @CIName = [name]

FROM sysindexes WITH (NOLOCK)

WHERE id = OBJECT_ID('[' + @TableToMove + ']')

AND indid = 1

 

DELETE FROM #tblColTable

 

INSERT INTO #tblColTable (ColName, IdxOrder)

SELECT COL_NAME(OBJECT_ID('[' + @TableToMove + ']'), colid),

-- append the DESC/ASC string, based on the ASC/DESC order of the PK columns

CASE WHEN INDEXKEY_PROPERTY(OBJECT_ID('[' + @TableToMove + ']'),

INDEXPROPERTY(OBJECT_ID('[' + @TableToMove + ']'),

@CIName,

'IndexID'),

keyno,

'IsDescending') = 1

THEN 'DESC'

ELSE 'ASC'

END

FROM sysindexkeys WITH (NOLOCK)

WHERE id = OBJECT_ID('[' + @TableToMove + ']')

AND indid = 1

ORDER BY keyno ASC

 

SELECT @i = MIN(Idx)

FROM #tblColTable

 

SET @ColList = N''

 

WHILE EXISTS(SELECT * FROM #tblColTable WHERE Idx = @i)

BEGIN

SELECT @ColList = @ColList + N'[' + ColName + N'] ' + IdxOrder + N' ,'

FROM #tblColTable

WHERE Idx = @i

 

SET @i = @i + 1

END

 

SET @ColList = LEFT(@ColList, LEN(@ColList) - 1)

 

-- Check whether the clustered index is also the PK, or a unique constraint (UQ),

-- or a unique index (UI) that is neither a PK or a UQ.

-- If the CI is either one of the above, we first check whether any foreign keys

-- reference this PK/UQ/UI. If so - we drop the FKs, then drop the PK/UQ/UI,

-- then recreate the PK/UQ/UI on the target filegroup, and then recreate all

-- the foreign keys dropped earlier.

-- If the CI is other than the above (i.e., it is a non-unique clustered index)

-- then we simply drop it and recreate it on the target filegroup.

 

IF OBJECTPROPERTY(OBJECT_ID('[' + @CIName + ']'), 'IsPrimaryKey') = 1

OR OBJECTPROPERTY(OBJECT_ID('[' + @CIName + ']'), 'IsUniqueCnst') = 1

OR INDEXPROPERTY(OBJECT_ID('[' + @TableToMove + ']'), @CIName, 'IsUnique') = 1

BEGIN

-- This case stands for a CI which is a PK/UQ/UI.

-- First, we drop all foreign keys associated with the PK/UQ/UI.

-- These FK constraints will be reapplied on the PK later,

-- (i.e., after the PK/UQ/UI is recreated on the target filegroup).

 

DELETE FROM #tblFKTable

 

-- Get all the FK constraints associated with the PK/UQ/UI.

-- Here, we query sysreferences so we could get our hands on all the

-- foreign keys that reference the PK/UQ/UI of the table

-- that needs to be moved.

INSERT INTO #tblFKTable (ForeignTableName, KeyName)

SELECT OBJECT_NAME(fkeyid), OBJECT_NAME(constid)

FROM sysreferences WITH (NOLOCK)

WHERE rkeyid = OBJECT_ID('[' + @TableToMove + ']')

AND rkeyindid = INDEXPROPERTY(OBJECT_ID('[' + @TableToMove + ']'), @CIName, 'IndexID')

 

SELECT @AssocFKeyName = MIN(KeyName)

FROM #tblFKTable

 

WHILE @AssocFKeyName IS NOT NULL

BEGIN

-- Get the list of primary and then foreign columns

-- for the collected FK constraints. The CASCADE UPDATE,

-- CASCADE DELETE, and NOT FOR REPLICATION properties

-- of the FK are considered later.

 

SELECT @FKTableName = ForeignTableName

FROM #tblFKTable

WHERE KeyName = @AssocFKeyName

 

DELETE FROM #tblFKColTable

 

-- First, the tables of the foreign table. The select is ordered by keyno

-- so the order of columns in the FK will remain unchanged by the

-- drop/recreate operation.

INSERT INTO #tblFKColTable (ColName, FOrP)

SELECT COL_NAME(fkeyid, fkey), 'F'

FROM sysforeignkeys

WHERE constid = OBJECT_ID('[' + @AssocFKeyName + ']')

ORDER BY keyno

 

 

-- Similarly, for the primary table.

INSERT INTO #tblFKColTable (ColName, FOrP)

SELECT COL_NAME(rkeyid, rkey), 'P'

FROM sysforeignkeys

WHERE constid = OBJECT_ID('[' + @AssocFKeyName + ']')

ORDER BY keyno

 

-- We now build the FK creation statement

SELECT @j = MIN(Idx)

FROM #tblFKColTable

 

SET @SQLStr = N'ALTER TABLE [' + @FKTableName + N'] '

+ N