Agile Software Development in the Federal Government: How to Make it Work

Despite what you hear on the news, not all IT projects undertaken by the federal government are unsuccessful. Granted, there are challenges to overcome, but a talented development team with a clear vision can build bigger and better things. In Washington, D.C., the capital of the federal contracting world, a small IT company called AIS is making a big difference at the FBI using the Agile approach to deliver an application that is being used by the FBI every day.

Agile software development, at its core, is more about focusing on interactions between team members, collaborating with customers, actively involving users, and responding to change quickly…and less about focusing on preparing comprehensive documentation or following processes. However, the AIS team at the FBI uses a hybrid approach that is heavy on constant interaction with the customer and the user community, without losing the significance of processes and documentation. For our application, AIS uses a team of requirements analysts, programmers, infrastructure engineers, software testers and helpdesk support to form a strong software development team to accomplish these tasks.

Our success is based on:

Daily Scrums

Our team uses a unique approach regarding daily scrums. There is one scrum conducted by the development team and a separate one for the analyst team. Any cross-functional team actions are discussed by the team leads on a separate call. The team leads meeting follows a Scrum of Scrums pattern where the team leads, acting as the Scrum Masters, meet weekly to coordinate the workings of the independent teams. These scrums are opportunities to discuss the team’s progress. Any stumbling blocks are briefly discussed, recorded and worked to resolution after the meeting.

Automated Builds

We recognized at an early stage of the project that the customers/end users can change their minds often about what they want, and following the traditional approach would not work. Significant investments have been made in the cloud infrastructure and automation initiatives to ensure that immediate feedback received from the customers/end users can be implemented, deployed and shown to the user the following day. This “real-time” response to emerging requirements is possible through an automated build process that is executed daily. Checked-in code is compiled, built and deployed through an automated process that is run nightly. An email is automatically sent to the development lead and the project manager reporting the status of the deployment. This automated build process is efficient and effective as it reduces the overhead involved in using the configuration managers’ time for this task. The next goal is to automatically execute the automated smoke test at the end of the automated build process which should be the last piece of the continuous integration puzzle.

Figure 1 - Automated Build and Farm Topology

Automated Testing

Automated testing is a key component in the Agile approach. Automated test scripts ensure that testing is repeatable and effectively managed. After a new environment is provisioned, the automated regression test scripts (defined in a configuration file) are executed to initialize test data. To successfully test data migration efforts, snapshots are used regularly to keep reliable backup for persistent data used for testing.

Automated load testing scripts are used to simulate a load profile for hundreds of users simultaneously accessing the UI of the web application. Detailed logging and email notifications are generated with the test results.

Figure 2 - Automated Testing

Requirements & Documentation

One common misconception of Agile methodology is that requirements documentation is not supported. The adoption of Agile principles does not mean that requirements should not be documented — it simply encourages appropriate documentation, as excessive documentation is a wasted effort. Requirements documentation is vital for developers, testers and the business stakeholders.

The size and scope of this particular application, where there are always exceptions to the exceptions, begs for detailed documentation. Our team embraces the idea of a living document where all functional and non-functional documentation is accessible and up-to-date. Business users, developers, and testers all have the requirements available at their fingertips and they all have the ability to request requirement changes. For new or more complex enhancements, the team uses mockups to depict the critical aspects of the application’s functionality, as it is the most efficient method to describe the user interface design visually. Mockups also save critical development time as they can be created quickly and provided to the customer for feedback without having to spend time developing a prototype. This keeps the customer constantly involved in the design of the application. Mockups, coupled with functional and/or non-functional requirements are created for each release.

Involvement of Stakeholders

The program’s success is partly due to its belief in active stakeholder participation. This practice is enabled by the customers’ and end users’ availability to provide requirements and their willingness to actively model together. At a minimum, the team meets weekly with the customer for a formal requirements meeting, in addition to meeting more informally as needed to hash out a topic.

Our customer, Directorate of Intelligence (DI) in the FBI, has played an active role as the product owner. They have pledged to have constant end-user participation throughout the development of the application. Active end-user participation is the most significant aspect to a system’s success, and this has only been possible because of our customer. DI knows that it is the users in the field who are most directly impacted by the application and any future development.

User conferences are held twice a year and DI invites some of their most involved agents, supervisors, coordinators, and analysts to participate. At these functions, the field personnel are invited to participate in direct talks with AIS and DI so they can express concerns about the existing program, and future enhancements they want to see. Since they are the day-to-day users, this feedback is invaluable. A demonstration of the next software release is unveiled at the conference and the representatives are encouraged to “take it for a spin” so they can see how their previous suggestions may have been implemented. In addition to user conferences, DI calls upon volunteers to participate in the User Acceptance Testing (UAT) for the next release so that any issues with the implementation can be corrected before the scheduled release.

DI also sponsors a Working Group, consisting of experienced system users, analysts, and management. This group serves two main purposes: discuss the best method of improving our development processes, and find the best method of implementing the many policy restrictions the bureau requires. Our client is obviously interested in getting the biggest bang for their buck, so DI likes to be involved in how we can streamline development costs, while still producing top-quality software. The Working Group is charged with updating priority tasks and assisting in the decision-making process of what functionality would provide the most benefit. The FBI has many, many rules and regulations it has to follow, so the Working Group also has the responsibility of staying current on the latest policy changes, and conveying those changes to DI and AIS so we can validate that there are no violations. This can include everything from making certain fields in the software mandatory, to the need for entirely new approval workflows to ensure compliance.

Weekly Demos for Customer

The investments in the cloud infrastructure where automated deployments are done daily make it possible for us to demo the application to the customer weekly, or even more frequently if needed. The customer can see any enhancements immediately after they are implemented. Feedback and discussion points are captured and change requests are added to the product backlog. The development team reviews the backlog on a daily basis and addresses each change request according to a priority assigned by the customer and analysts. In the coming weeks, program users will receive cloud accounts with access to the test environment hosting the new version of the application. The goal is for the test users to navigate through the new interface without any training and provide feedback on old functionality that has been transferred to the new system, new functionality that has been implemented thus far, and comments on the new interface design. Feedback will be consolidated and reviewed by the customer and they will determine whether each request should be added to the product backlog.

Involvement of the Helpdesk Support

An Agile philosophy applies to the helpdesk as much as it does the other components of a software development application. For the most part, the helpdesk personnel are the first unit to be made aware of a potential bug in the program. Inclusion of the helpdesk in the daily scrums allows the rest of the team to learn of these problems and begin the process of prioritization. In addition, the field often uses the helpdesk to request future enhancements or suggestions. By implementing a unified database, our team is able to track both reported bugs and the enhancement requests, and the system lets everyone track the progress of the reported item.

About Vijay Petwal

Vijay Petwal serves as a Business Analyst at AIS with over 12 years of experience in managing software development projects. Vijay acquired his MBA with a concentration in IT at GMU’s School of Management. Prior to working at AIS, Vijay worked at SRA International as a Configuration Manager and helped the company achieve CMMi Level 3 process improvement status. Vijay brings a wealth of experience in the areas of process improvement, configuration management, business analysis, requirements analysis, project management, testing, training and user support.

  • Thanks for sharing your experiences with agile. Its really important to or we can say “must to follow” agile approach in an environment where the requirements are not clear or requirements changes frequently or the change management is difficult. It also becomes clear that Government Institutions all over the globe should consider such kind of approach before automating their system as the policies of the governments are complicated and system is large.

    • Vijay Petwal

      Thank you Sir! I appreciate the comment. Yes, it is important to follow agile if the requirements are ambiguous, which is especially common on large government projects where the scope is difficult to define. Agile can be a useful tool in breaking up a large project into smaller iterations/sprints, which helps in delivering changes quickly.