Enterprise Agile: Process Adoption

Follow @scottwambler on Twitter!

NOTE: This article will soon be updated to reflect the evolution of EUP to be based on Disciplined Agile Delivery (DAD). Please stay tuned.

This article overviews the Enterprise Unified ProcessTM (EUP) and how it relates to the Rational Unified Process (RUP), suggests when you should consider adopting the EUP, and presents strategies for successfully doing so.

1. Overview

The EUP is an extension to the RUP. Figure 1 depicts the EUP lifecycle. People familiar with RUP can see that the extensions include two new phases, Production and Retirement, and several new disciplines: Operations and Support and the seven enterprise disciplines (Enterprise Business Modeling, Portfolio Management, Enterprise Architecture, Strategic Reuse, People Management, Enterprise Administration, and Software Process Improvement).

Figure 1. The Lifecycle for the Enterprise Unified Process (EUP).

Figure 2 depicts a common strategy for adopting the EUP -- first adopt the RUP, tailoring it to your environment, and then tailor the EUP extensions on top of the RUP. The RUP product from IBM is a significant resource and provides an excellent basis from which to start. For a leaner resource, the Agile Unified Process (AUP) product from Ambysoft Inc. is also an option.

Figure 2. Extending the Rational Unified Process (RUP) with the EUP.

Figure 3 depicts an augmented view of the EUP lifecycle, one that includes the phase milestones along the bottom of the diagram and an indication of how a team will cycle back to earlier in the lifecycle when they start working on a new release. In some projects you will want to rescope and rejustify the effort, hence you will want to start back in the Inception phase. Other times you may just need to rework the requirements and possibly the architecture for the new release, hence the return to the Elaboration phase. Other times you can start right in the Construction phase for your new release.

Figure 3. Augmented Lifecycle for the EUP.

2. When to Adopt the EUP

You should consider adopting the EUP when:

  1. You are successful at the RUP or other software development process. The EUP addresses the full IT lifecycle, whereas the processes such as the RUP or Agile Unified Process (AUP) address just the software development portion of the overall lifecycle. Fundamentally, if you're not successful at software development then there is little hope that you'll succeed at cross-system issues. Walk before you try to run.

  2. You have enterprise issues to deal with. The EUP adds enterprise disciplines to the RUP to address enterprise-wide concerns for organizations that have families of related products (or just several systems). The EUP can help you if you wish to start following a common architecture, improve reuse between projects, manage your IT assets as a portfolio, improve the operations and support of your system, or improve the way that you manage IT staff across projects.

  3. You need to operate and support your software after it's deployed. The final RUP phase, Transition, ends with the transition of the software to end-users. For many organizations, particularly for organizations that develop software for internal use, software must be supported after it has been deployed. The EUP adds the Production phase and Operations and Support discipline to support this.
  4. You need to retire existing applications. No application remains in production forever, you eventually need it no longer or decide to replace it – for example, many organizations are choosing to replace their home-grown financial applications with commercial off the shelf (COTS) products. The EUP adds a Retirement phase which captures the complexities of system decommissioning.
  5. You want to avoid ‘stovepipe' applications. EUP can help you migrate away from a ‘stovepipe' mentality, where applications are each developed independently and stand on their own with no thought towards interaction or reuse. Taking an enterprise view of your organization, you can build an enterprise architecture mentality and start a strategic reuse program where applications can learn to work as part of larger whole instead of as stand-alone applications.
  6. You are willing to invest in software process improvement (SPI). The benefits listed above do not come for free. There is a cost in setting up the enterprise disciplines, not to mention adopting and deploying a new software lifecycle process. Software developed for reuse must typically be reused two or three times to recoup the cost of making it reusable. You must be willing to invest in the long term to realize the benefits of the enterprise disciplines, and that implies the need for SPI.

3. Strategies for Adopting the EUP

There are several strategies which you should consider for adopting the EUP (or any process for that matter):

  1. Assess your situation

  2. Take an evolutionary approach

  3. Prove it in practice

  4. Train and mentor people

  5. Treat it like a project

  6. Gather metrics

  7. Don't stop improving

3.1 Assess Your Situation

You should start by identifying why you wish to adopt the EUP, perhaps for some of the reasons listed earlier, and then assess your organization against this criteria to define your capability gaps. This will help you to prioritize the changes you will need to make. You may wish to bring in an outsider to do provide a more objective viewpoint. Issues you should consider during your assessment include:

  1. Current "pain points". What aspects of IT are you struggling with? Having problems identifying which projects to attempt? Are project teams constantly reinventing functionality that has been built by other teams? Do you have problems hiring and retaining good staff? You need to understand your problem(s) before you can identify the solution(s).

  2. Existing RUP/AUP expertise. Does your organization have experience in the basic RUP/AUP approach? If not, then you first need to gain experience (and success) at an appropriate base software development process before considering the EUP.

  3. Acceptance of an evolutionary (iterative and incremental) approach. This may be your biggest challenge if your staff only has serial/waterfall experience. It is very easy to say that you need to develop working software each iteration, but very difficult to grasp the implications if you're used to identifying all of the requirements up front, then doing all of your design, and so on.

  4. Enterprise outlook. How well do your teams/groups/divisions/organizations work together? How well do the group(s) representing users work with the technical staff? Many project teams are used to developing systems in relative isolation, they're not used to following a common architecture, common guidelines, or even to consider the implications of what they're doing when it comes to operating and supporting their system once it's in production. Adopting the EUP will require adopting a more mature outlook than many IT professionals are used to.

  5. Acceptance of change. How well does your organization respond to change? What percentage of your staff is open minded enough to consider trying new ways of working? How many are skeptical but open minded? How are unlikely to change at all? You are likely to find the book Fearless Change to be of interest.

  6. Process culture. Is your organization used to a rigorous/prescriptive approach? An agile approach? No formal process at all? Rigorous folks will like the EUP, agile folks will demand that it be instantiated simply, anti-process folks will fight you all the way. You likely have all three groups within your organization

  7. Technology expertise. For example, if you are switching to a service-oriented architecture (SOA) approach and you haven't done it before, then you'll need to train your staff in these techniques and technologies. It is not simply a matter of sending people to Java or web services training, they'll also need architectural training and maybe even OO training, both of which take a long time to pick up.

  8. Testing expertise. Do you have the testing expertise to support your effort, or more importantly agile testing expertise? Testing plays a large part in evolutionary development because you test as you work, in fact many developers even test first. My philosophy is that if it's worth building, it's worth testing. The Full Lifecycle Object-Oriented Testing (FLOOT) method may provide some insight into the challenge that you face.

3.2 Take an Evolutionary Approach

Just as an evolutionary approach to software development is usually more effective than a serial one, so is an evolutionary approach to adopting the EUP. Start with the tailored RUP portion of Figure 2. Get that working in your particular organization first to demonstrate that you can handle developing object-oriented project iteratively and incrementally.

Then, based on your prioritized needs based on your assessment efforts, start adopting portions of the EUP. Perhaps your operations process is in dire need of repair, so adopt the appropriate portions of the Operations and Support discipline and perhaps the Enterprise Administration discipline. Once you're successful with that, an effort that could easily take a year or so, you might want to tackle Portfolio Management. Then Enterprise Business Modeling and Enterprise Architecture in parallel, quickly followed by the Strategic Reuse discipline. The point is that you can't adopt the EUP all at once, therefore you should adopt it a portion at a time, starting with your highest priority issues first.

3.3 Prove it in Practice

Adopting a process for an organization is not simply a matter of buying the RUP off the shelf or a few books and telling everyone to start following them. You shouldn't look at any process as being the answer for your particular organizational needs. With the EUP (and RUP), treat it as a process framework, not a process. They provide the framework for you to decide what portions you will implement and how you will do so.

It should be proven out by actually running a project with it. For this, you should run a pilot project. Pick a pilot project that meets the following criteria:

  1. Non-mission critical. One objective of your pilot project is to prove out the software process, not to develop and deploy an application. Pick something that will not negatively impact the business if it is late or never even deployed at all.
  2. It is still real. Nobody is going to fund a project just to test out a process. You will still be expected to deliver working software, preferably something that is reasonably complex. Otherwise, even if you succeed you run the risk of people thinking that it won't work for a "real project".
  3. Low visibility. You don't want senior management asking about your pilot project during the CTO staff meeting except in the context of your process adoption effort. The focus of the pilot project must be on proving out the software development process.
  4. Non-time critical. You want time to fix any software development process issues that arise during the pilot project without having to worry about getting the application out the door. That is not to say that you shouldn't have a timeframe in mind for the pilot project; you should. However, your objective during that timeframe is to develop and prove the software process, not the application.
  5. Small in scope. You don't need to solve a large, complex project on the initial run through of the process. Start with something manageable.
  6. Small in size. Likewise, you don't need to prove that you can handle a project team of 35 people at the first crack. A team of 7 or 8 people is likely a good start.
  7. Staffed with experienced people. You want to concentrate on the process during the pilot project, not getting people trained on use cases or Java.

3.4 Train and Mentor People

You'll most likely need several training offerings, targeted to different audiences. Everyone will probably benefit from an overview of the process that explains what the new process is in general terms. Other than that, you'll have to assess your needs. You may decide that a class discussing the iterative and incremental approach would be most beneficial. The choice is up to you – what makes sense for your organization? Your organization assessment should be the driver on what training you require.

Most likely, your people will need formal training in not just the software development process but in other skills as well, e.g., OO, Java, capturing requirements with use cases, and so on.

Much of the training will be targeted to project team members: project managers, analysts, architects, developers, etc. However, make sure that executives and senior management are properly trained, also. They need to understand the process so that they know what to expect. The people that project managers report to definitely need to be up to speed, as they will be the ones giving the PMs direction and reading PM's status reports. Senior executives need to know what to expect from their direct reports and how the new process will affect their IT planning.

Keep in mind that training is a start but it may take a while for people to develop fundamental skills. An analyst new to use cases can be taught the basics of capturing requirements using use cases but it will take a while for him/her to develop the fundamental skills on how to do it properly. Do not expect people to become experts after a round of training. In fact, do not expect them to necessary be proficient right away. Some skills take time to develop, and may only come about as the result of mentoring.

One technique to help your organization get up to speed on your process is to bring in mentors. Mentors are people skilled not only in the specific skill areas that you need (EUP, OO, etc.) but, more importantly, at knowledge transfer. Mentors objectives must be not to make sure that your projects are on time and within budget (that's the project managers job) but to get your people up to speed on specific skills as soon as possible. Use them to help speed up the learning curve for your people.

The best mentors are the people who have been there and done that. Keep that in mind when you staff your pilot project. The people on the pilot project will often end up as mentors to others in the organization. They should have the skills necessary to work with others and transfer knowledge on how to apply the process.

Your implementation of EUP will likely be different from XYZ Corporation's implementation. If you plan on bringing in outside consultants to help mentor others, bring them in early so that they can learn with you what works and what doesn't work for your organization so that they can mentor others accordingly.

You may find the essays Strategies for Effective Training and Education (T&E) in I.T. and The Longevity of IT Skills to be of interest.

3.5 Treat it Like a Project

When adopting the EUP, or any software process, it is vital to manage the effort just like any other project. Too many organizations decide that all it takes is a decree that ‘we're going to use RUP/XP/FDD/AUP/...' and that they will immediately reap untold benefits. That will not happen. Like any other complex effort, it must be managed properly. This means planning, communicating, managing of people and resources, etc.; all the things you do on a normal project.

If you are going to implement the EUP, one way to do this is to treat it as an EUP project. Follow the phases of EUP:

  1. Inception. During the adoption of EUP, you must keep in mind the needs and success criteria defined by the stakeholders. There are many stakeholders for a software process adoption project and they will vary from organization to organization, but they will probably include at least: senior management, project managers, business representatives, architects, developers, testers, quality assurance, operations and support staff and others. With your stakeholder needs and success criteria in hand, perform an assessment of your organization to determine your readiness. It's crucial to create a communication plan at this point because it is essential that you let others know: what are you doing, why are you doing it, the business case for it, success stories, etc. It is also important to start a Risks List and an Issues List early in your effort. Risks are things that may occur that could have a negative impact on your project. You try to anticipate risks and do what you can to not have the Risk materialize or, if it does, to minimize its impact. Issues are things that have already occurred.
  2. Elaboration. During an EUP adoption project, you will identify organizational changes that will need to be made and identify and acquire tools that you will need. These will provide the stable base for your work going forward. First, you will need tools to define and disseminate the process. The tools you choose to define to define and disseminate the process at your organization will vary. Many organizations use the Rational Process Workbench to tailor the RUP. Other options are leaving the RUP alone and wrapping it with your own web pages that reference the RUP material. Second, you will need software development tools which support your process. If your organization is moving from COBOL to Java-based development, you'll need a whole new suite of tools.
  3. Construction. During this phase of the EUP Adoption project, you will ‘tailor' the process and implement it via one or more pilot projects. An important decision that you need to make is whether you will tailor the RUP material to meet your needs, and thereby need to maintain it and manage updates as IBM Rational releases new versions of the RUP, or do you construct your own set of web pages which overviews your process and then provides links to the details (the AUP Product is a good example of this approach). You'll have to decide on an approach to development cases (which is a tailoring of the RUP process). Some organizations choose a single development case that covers 80% of the material the projects will use and have each project modify it as needed. Others have several organizational development cases, one for each category of project (e.g. outsourcing, FDA-compliant, research, ...), which are closer to what is needed by the teams but must still be tailored. Other important issues that you need to consider is the development/tailoring of documentation templates which meet your organizational needs and the creation of development guidance (standards and guidelines) appropriate to your situation. Finally, whatever you develop should be tested via one or more pilot projects.
  4. Transition. Once your pilot project(s) are successful, you should deploy the EUP slowly throughout your organization. Up until this point, you proved the process in one or two projects under controlled circumstances. You probably used your most experienced people on those projects. Now you must roll out the process to a much wider audience with varying skills and experience. Go slowly. Start with a handful of projects. Seed them with the people on your pilot project(s) that have experience with your newly defined process, or, if you can't do that, make those people available to mentor those projects. Build up your organization's comfort level with the process as well as your people's expertise. Remember that one of the classic anti-patterns in software process improvement is to define a new process and decree that everyone must us it at once. In doing so, you will overwhelm people and your organization with too much change at once.
  5. Production. During this phase, you will be maintaining the process. That means continuing the communications effort (perhaps via a newsletter or web site), continuing training and mentoring, and continuing adjustment of the process as you learn more and as your organization grows. You should have identified the metrics that you will be gathering by this point. You will gather them and learn from them during this phase. You should also be holding Lessons Learned sessions as projects close and adjusting appropriately.
  6. Retirement. All good things must come to an end. I believe that the EUP is the best IT process currently available, but at the same time I fully expect that something better will eventually come along. When that time comes, the EUP will need to be retired within your organization and the replacement process adopted.

3.6 Gather Metrics

You'll want to start gathering, and acting on, metrics as soon as possible. Continuous improvement is where you want to be and metrics help immensely in that regard. They can give you definitive measures on how things are proceeding. Be careful about metrics, though. Reporting the wrong information based on metrics or interpreting them incorrectly can do significantly more harm than good. Know and understand – and be able to effectively communicate – each metric you select; if you cannot do this, ask yourself "why" that metric is needed.

3.7 Don't Stop Improving

Process improvement is an on-going effort, which is why the EUP includes the Software Process Improvement discipline. Do not become complacent; look for ways to improve. Be prepared to respond to a changing business environment, where priorities may change.