Behind the Census Bureau`s Mobile SNAFUBy Jean Thilmany | Posted 05-20-2008
Behind the Census Bureau`s Mobile SNAFU
Don't worry, Americans: You will be counted as part of the 2010 U.S. census--just not as efficiently as planned, thanks to the meltdown of a major mobile computing implementation that was supposed to bring the huge data-collection project into the wireless age.
The goal was to make participating in the census as easy as signing for a FedEx package. But while your parcel-delivery service has had the handheld thing down pat for quite a while now, much of the 2010 census will still take place on paper--just as it has since 1790, when George Washington was president.
In March, the Government Accountability Office (GAO), the investigative arm of Congress, fingered the upcoming census as a high-risk operation. The designation came as a result of the Census Bureau's lagging mobile technology implementation, which was intended to reduce costs while improving data quality and collection efficiency. The bureau had estimated that outfitting census takers with mobile devices and providing associated systems would account for about $3 billion of the estimated $11.5 billion cost of the entire census.
Now those numbers, as well as the timeframe for system testing, are off the table. The bureau will need an additional $2.2 billion to $3 billion in funding over the next five years to meet its needs, bringing total costs closer to $14.5 billion.
"The American people expect and deserve a timely and accurate decennial census, and I won't rest until they have it," Commerce Secretary Carlos Gutierrez said in a statement. (The Census Bureau is part of the Commerce Department.) Much of the handheld program, on the other hand, will be given a rest until the next census rolls around in 2020.
What Went Wrong?
What Went Wrong?
In 2006, the Census Bureau contracted with wireless equipment maker Harris Corp. to get wireless handheld computers and their supporting infrastructure up and running in time for a mock census, held in May 2007. Unfortunately, the handhelds were not even close to being ready. There were a number of performance issues, such as slow and inconsistent data reporting, according to the GAO report, and the Census Bureau didn't specify how it planned to measure the device's performance. These problems persist to this day.
The lessons of the botched project revolve around poor communication between vendor and client, as well as underestimating the difficulty of rolling out a sometimes-balky technology at scale. The cost overrun and delays can be boiled down to three factors: poor contract estimate, poor program management and poor executive-level governance.
In April, Census Bureau Director Steve Murdock testified before Congress on this matter. He acknowledged that the bureau did not effectively convey the complexity of census operations to the contractor, and said problems arose in part because of ineffective communications--including information about IT requirements--between the bureau and Harris Corp.
"Once these detailed requirements were completely delineated, we had serious concerns about rising costs and our ability to complete a successful 2010 census if we continued developing the program as planned," Murdock told the House Information Policy, Census and National Archives Subcommittee of the Committee on Oversight and Government Reform.
Those detailed requirements were indeed a problem. The initial contract contained roughly 600 requirements, and the bureau later added 418 more, says David Powner, GAO director of IT management issues. What happened with the Census Bureau is a case study of why federal government IT implementations often go wrong, according to Powner, who came to government service after spending years overseeing large-scale IT implementations in the private sector. (The government spends $70 billion annually on nearly 900 IT projects.)
Step one in an escalating chain of errors that went uncorrected: The contractor presented a poorly calculated estimate. Federal entities don't require the same rigorous up-front cost estimates as their nongovernmental counterparts do.
To complicate things, that poor cost estimate was compounded by requirements creep: the tendency for clients to add more features to their wish list long after they've signed off on the original requirements. "Those of us who define requirements for systems know it isn't easy, but you need a validated set of requirements up front, and government doesn't often do that up-front work," Powner says.
Adding to the woes was the lack of oversight of the contractor. It seems that the Census Bureau didn't lean hard enough on Harris to provide continued implementation updates, but Murdock's testimony doesn't directly address such activities.
Another problem is the result of the relative newness of the mobile technology. The bureau has ventured into a new territory with technology that is not fully mature and doesn't come with an implementation road map, says Philippe Winthrop, research director for wireless and mobility at the IT analysis firm Aberdeen Group.
The Census Bureau probably didn't take market immaturity into account when signing the contract, surmises Sheldon Needle, president of CTS, which evaluates software for midsize companies. "Mobile is still a fairly raw technology so they should have had their antennas up from the beginning," he says. "If the bureau was in it two years ago, it was even rawer.
"If I were running such a job and the vendor couldn't give me a couple of referrals of large-scale sites they'd done, I wouldn't even talk to them. What happened sounds improbable, but this kind of stuff goes on all the time with huge-scale implementations."
Finally, implementations of mobile technology can be very complex. The systems encompass multiple components, carriers, devices, operating systems and applications that need to be agreed upon well before the systems are installed. CIOs also need to ensure that all parts of the system can be integrated.
A need also exists to develop business rules for each of these multiple components. "We're in new territory here," Aberdeen Group's Winthrop says.
Of course, mobile implementations can be done--and done well. Look at FedEx, United Parcel Service and similar companies. Anyone who's been asked to sign for a package knows these carriers rely on nifty electronic notepads to enter up-to-the minute information about a parcel's location.
With that in mind, Winthrop carried out an October study to isolate success factors in mobile initiatives. He found instances in which enterprise mobility has made significant inroads in organizations and is having a measurable impact on productivity.
Study findings include the following: Of the more than 580 companies queried, the best-in-class ones standardized on one platform as much as possible. Before they selected a carrier, these companies studied how individuals will actually use the devices. They also kept device content separate from device back-office hardware and security.
Winthrop admits the best-in-class practices aren't exactly a business revelation. "We're all saying those things makes sense, but my data on 580 organizations says many aren't doing it," he says. "Chances are the government didn't do as much as the best-in-class organizations did. There should be formal policies in place for choosing a carrier and implementing a system. Otherwise, it's absolutely chaotic."
Fixing Processes, Counting Heads
Fixing Processes, Counting Heads
All businesses have their share of IT disasters, but many federal projects seem engineered to go bad from the start. The GAO's Powner points to the lack of executive oversight and accountability as a fundamental difference between the federal government and the corporate sphere.
"In the private sector, there are disincentives for not performing, but they aren't any in the public sector," he says. Federal bureaucrats don't have the threat of job loss hanging over their heads, should they bungle an IT contract.
Also, corporations charge a program manager to oversee large IT investments; the federal government doesn't. "Historically, government doesn't have certified program managers," Powner says. "Qualified program managers and executives are accountable. When you have executives who are holding meetings, that can drive program management."
There has been some progress after a series of high- profile--and high-cost--federal horror stories. For example, after attempting to modernize its tax processing and database system, the Internal Revenue Service has bolstered its requirements-setting process. In an attempt to mimic executive oversight, the IRS also established a sophisticated governance structure in which managers meet monthly or quarterly to oversee business units involved in the ongoing upgrades.
Another case in point is the Department of Defense, which in an effort to more closely oversee its IT contractors, established an earned-value management program to measure the value, cost, schedule and functionality of IT deliverables. The program is meant to shine a light on a common technique used to mask scheduled problems, whereby a contractor will say it's on schedule, yet defer demonstrating functionality.
Measures like these come too late for the Census Bureau's handheld initiative. Yet the Constitution mandates a census every 10 years. So what happens now?
The bureau will do what it can with the handheld devices it has, says Commerce Secretary Gutierrez. After that, it will rely on the old standby: paper. The mobile computers will be used to initially canvas addresses--stage one of the census. Here, the device's GPS tie-in will be particularly helpful.
People who don't respond will be tracked down during a second-stage canvassing, but handhelds won't be used during this phase. "Historically, when people don't respond, you knock on doors and send them information, and that won't be done with handhelds," says the GAO's Powner.
Though bringing in manual methods this late in the game will be a hassle for the bureau, the scramble is doable. "They're going to use the same strategy they used in the past, because they know how to do it manually," Powner says. "That's how they justify this decision."