For most situations involving SharePoint development, I advocate some form of an Agile process. This post will go into detail about what an Agile process is, why it is usually better than a traditional approach, and how to implement your own version of Agile for SharePoint development. It is important to understand that, although this is a long post, and there are lots of links to external resources, I am just barely scratching the surface. Agile is an iterative approach to development that requires a commitment to continuous improvement of the process itself. That means that there is no end state. There is no best practice. Every organization has to adopt the form of Agile that works best for their organization, and then work to continuously adapt and improve on it as they go. I encourage you to explore the links in this article and seek out your own resources. There is no shortage of good information about Agile.
What is Agile?
I am going to borrow Wikipedia definition of Agile. This definition can be found here: Wikipedia Definition of Agile and is copied below (in part) for the convenience of discussion. It is a good starting point.
"Agile is a group of software development methods based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams. It promotes adaptive planning, evolutionary development and delivery, a time-boxed iterative approach, and encourages rapid and flexible response to change."
Below, I am going to highlight three parts to the Wikipedia definition that I think are particularly important.
"Software development methods"
Agile does not tell you what to do. In most respects, it does not tell you how to do it either. Agile provides a Manifesto and Principles. It is up to the organization to adopt a particular methodology, a hybrid of methodologies, or even invent their own methodology to come up with the how.
It is worth talking about collaboration briefly because it is such an important part of Agile. It is not just a token word in the definition, "Close, daily co-operation between business people and developers" (from the Agile Principles) is one of the key underlying principles of Agile. Because Agile is "iterative" and "incremental" Agile developers have to constantly be working with the business to understand changing requirements and an evolving set of circumstances. In fact, some of the Agile methodologies posit that a team is most effective when they are all sitting and working in the same physical location. Casual conversations, quick questions, and ad-hoc problem solving are the cornerstone of collaboration in an Agile environment.
"Iterative and Incremental"
The first underlying principle in Agile is "Customer satisfaction by rapid delivery of useful software." "Rapid delivery of useful software" is the key way to achieve customer satisfaction with Agile. Along those lines, the ideas of "welcome changing requirements, even late in development" (the second principle) and "Working software is delivered frequently" (the third principle) both underscore the importance of adapting to change and doing it quickly. Through collaboration and an iterative, incremental approach developers and businesses are able to quickly adapt to changing needs because they are collaborating and delivering software incrementally. Each increment of software delivered can be changed to accommodate new specifications.
In order to help understand why we should even consider an Agile methodology for SharePoint development, I am going to highlight a case study of Salesforce.com, point you to some research, and appeal to common sense.
Salesforce.com Case Study
In 1999, Salesforce.com started their business of developing Software as a Service (SaaS). They had a handful of developers and used a traditional Waterfall approach to development. By 2006, they had over 150 developers. Their software development life cycle had ground to a virtual halt. They were only releasing one major update per year. Their customers were not happy because Salesforce.com was not responding to customer needs in a timely fashion. Salesforce.com employees were not happy, in part, because they hardly ever got to see their work come to fruition. A lot of this is documented in a Stanford case study here: Salesforce.com: The Development Dilemma . In 2006, Salesforce.com dove headfirst into their own version of an Agile methodology called ADM. They created roughly 30 teams with 6-10 members each. Three one-month Sprints made up their first release cycle. Within the first nine months of implementing ADM, Salesforce.com delivered on 60+ "critical" features. Customers were getting features delivered in roughly half the time as before. Salesforce.com delivered the results of their Agile transformation at a conference in 2007. The slide presentation can be found here: Salesforce.com Agile Transformation . The results are hard to argue.
In his book, "Succeeding with Agile Development", Mike Cohn does a survey of the research done to illustrate the benefits of Agile over traditional approaches. The results are overwhelmingly in favor of a well-implemented Agile approach to software development. I am not going to list out all the details here. Please see Mike Cohn's book if you are interested in the details. Just know that studies have shown that Agile produces higher quality work, quickly.
If the Salesforce.com case study and the research highlighted in Mike Cohn's book isn't enough, common sense and logic should bring home the point. Software developers provide a service to their customers. They write software that provides some business benefit to their customers. In order to best service those customers and provide the most business benefit possible, developers need to listen to and understand their customers' needs and then deliver software to help them with those needs. They need to do this as quickly as possible. That is the essence of Agile. Do this repeatedly and you will have succeeded in implementing at a minimum your own form of an Agile approach to software development.
I am going to outline a simple Agile/Scrum approach that I have used in the past for SharePoint software development. Every organization is going to have their own nuances to Agile. What is important is that the essence of the Manifesto and Principles are respected.
Start with a Product Backlog. The product backlog defines, usually in the form of User Stories, what will be delivered. The Product Owner, with the help of business users, create the User Stores. Using older terminology, these are the requirements. The Product Owner, with the help of the business users and the key stakeholders then prioritize the items on the Product Backlog.
Then you define your Sprint Backlog. The Sprint Backlog defines, usually in the form of tasks, how each item on the Product Backlog will be delivered. Sprints are usually 2-4 weeks in duration. The Sprint Backlog is created by the development team. Each item on the Sprint is estimated with a number of hours. Each Sprint lasts roughly 2-4 weeks. Items from the Product Backlog are added to the Sprint Backlog and estimated in priority order until resources are at full capacity for the Sprint duration.
During the Sprint, developers meeting on a daily basis for a short meeting to discuss any roadblocks or problems they are encountering. This meeting is facilitated by the ScrumMaster, whose job it is to remove roadblocks and help problem solve. The progress on the Sprint Backlog is typically tracked using the Sprint Burndown chart. A typical Sprint Burndown chart looks something like this:
At the end of the Sprint, the new code is deployed and there is usually a Sprint Review Meeting to discuss how things could be improved.
Agile for SharePoint
Agile is rapidly gaining in popularity among software shops of all types. And for good reason. It's time it worked it's way into SharePoint development. Most companies that implement SharePoint don't realize or understand the path they are taking when they start down the path of custom SharePoint development. Assuming, however, that you are there (you either have custom SharePoint solutions to support or have undertaken a new project to implement a custom solution), you should select the software development methodology that has thus far, in my opinion, proven most successful.
Agile for SharePoint can be implemented as a comprehensive process solution for software development at companies that have extensive custom development needs and large teams of developers. Or it could be implemented by 1 developer who just wants to follow the manifesto and 12 principles when working with his SharePoint business users. The results should be the same: the consistent, rapid delivery of valuable software.
In the final blog post for this series, I will detail some of the tools and technology used to develop in SharePoint. Most of the tools and technology I focus on will be from a process perspective. That is, I am not going to show you how to build a SharePoint solution in Visual Studio. However, I will show you how to track your progress on your Sprint using Visual Studio Scrum plug ins.