In 2009, I was director of Sunlight Labs at the Sunlight Foundation – a group of developers that worked on open government projects at the federal level. At the same time, Barack Obama had just been elected to his first term, and announced the American Reinvestment and Recovery act – a multi-billion dollar “bailout” of the American economy.
Along with the act, a website was to be created that allowed for citizens to oversee how this recovery money was being spent in order to “combat waste, fraud and abuse.” And as the leader of one of the most sophisticated transparency technology organizations in the country, I thought that this website wasn’t just something that would be smart to keep an eye on, it might be something smart for us to bid on and work on.
So I set up an open source, community bid for the project. I got a wiki running, and we went through the RFP line by line to come up with a compelling bid on how we would solve the problem. The 20 or so of us inside of Sunlight, along with dozens of volunteers crafted a bid that came to about $400,000. Taking into account that this was work that would require lots of overhead, I doubled the project price to $800,000, and sent it in.
After sending it in, I felt nervous about cranking up the price that high. It seemed like a huge amount of money to charge for what needed to be done considering how many experts we had on-staff that could craft a great website for this kind of thing almost in their spare time!
Recovery.gov ended up costing $19 Million dollars. We at Sunlight, were not even eligible to bid on the contract, so we, perhaps the nation’s leading experts on web-based transparency projects like Recovery.gov, did not even make it to the first round in the bidding process.
That was my first experience with government procurement. And it is when I started talking about it as something I thought needed changes.
Three years later, the White House called. They were setting up a “Presidential Innovation Fellowship” and wanted to work on ways to get new companies into government contracting. And they wanted me to lead a team of people to do that. It’d take six months, but the goals were clear: find ways to reduce cost, increase competition, and increase the level of talent.
My team included Adam Becker (now my partner at the Department of Better Technology) and Jed Wood, both strong developers. Like me, they had never been immersed into this space, but were frustrated by it. Government, after all, is a place where programmers can use their skills to have huge affects on society at large.
The project we shipped was called RFP-EZ, and it does three things:
Makes it easy for government to write RFPs that people can understand.
Makes it easy for people to find those RFPs, and to bid on them.
Makes it easy for government to sort through the responses to those RFPs and select the best ones.
We worked through the federal bureaucracy over six months, creating a new online platform to do those three things. We took five projects and ran them through both the normal process, and through this new, RFP-EZ process so we could compare the bids and outcomes.
The result? Bids through RFP-EZ were 30% lower than through the normal means. With just these five projects, government enlarged its small business pool by 250 new businesses. The bids came in faster, and there were more of them, and the bids on-average, tended to vary less (possibly because of well-written RFPs) than through the normal way.
The United States Federal Government isn’t the only organization with a “procurement problem”. Every large enterprise needs help figuring out how best to engage with external suppliers. And now that we are starting the Department of Better Technology, we thought we’d share the lessons we learned from RFP-EZ. The following essays are a collection of ideas from our experience: what we did, the mistakes we made, and the things we think need changing.
If you have any questions or ever need help, you can email us at .
Many organizations are interested in shifting the policy and regulations around to make the process easier and less cumbersome. Our own product, Screendoor.io, helps focus down on making simple acquisitions easy. But before you get into acquisition policy and technology, another one has to get fixed up the chain: business registration processes. Here’s what happened when Jed Wood, our colleague on Project RFP-EZ, catalogued his experience trying to register his business just to be eligible to be awarded contracts: go.dobt.co/sam-reg-process. Over 70 steps to go through!
This provides a glimpse of the process and the problem: he didn’t make it through. It’s due to poor communication: e.g. you register an “entity” not a “business” in Sam.gov, and you’re asked if arcane sections of the Federal Acquisition Regulation apply to you without links to the regulations for you to read. And it’s due to poor technology: even though Jed eventually filled out his form, the system still would not allow him to register for reasons unknown. This registration process is important – it’s not just something the federal government uses. Local governments use this central contractor registry to vet their own vendors, as do private companies: Microsoft uses this registry to vet its own suppliers.1
Ideally, this isn’t the only process a business goes through in order to just have a chance at competing for government work. They should also try to qualify for small business certifications – like being a HUBzone company or a woman-owned small business. At the same time, should the business decide to work with local government, they have to go through similar processes.
These processes take a lot of time and cost a lot of money both for businesses, who are dealing with systems that often don’t work, and for government, because most of these forms are paper and the tools on the inside (ironically because government can’t afford good, cheap technology) don’t work well and take a long time. It can often take a small business months just to get set up to be eligible to win contracts.
When the open government community talks about “Fixing Procurement” what we’re really talking about is making it so that the contracts don’t keep getting awarded to the same giant IT enterprise integrators over and over again. In general, our theory is that if you do that, then price will go down, and quality will go up.
The first step of that is increasing the pool of “eligible vendors” – because government is never going to have a completely open process where anybody can bid on anything. The complicated set of rules called “set-asides” are valuable social policies. They’re meant to ensure that when government does spend taxpayer dollars, they get the most “good” (by giving preference to service disabled veteran owned businesses, for instance) for those dollars. It’s not pragmatic or politically palpable to simply get rid of those programs, and the intent of them is good. But they could stand for improvement, and the first step of that is making it easy for people to register to contract, and to make it easy for the people who qualify for these programs to qualify for them.
That starts at the Small Business Administration at the federal level, and your locally at the business development agency. Step one in solving this problem is by streamlining the user experience via software. Turn paper forms into digital ones. Make the language look great. Every place there is a PDF file and/or a fax machine in the process, there is likely room for innovation. Once these simple barriers are removed from the process, you’ll increase the contractor pool and ensure more diversity in the procurement ecosystem.
To date, government APIs have largely been discussed as a method for releasing data to citizens. It stems from the open government’s roots in the watchdog movement – that government data ought to be free, so that it can people can police government. Though most of the time, it’s the case that for developers, (who are really the only people who can use APIs,) most government data doesn’t really require an API as much as it requires a well thought out, well formatted, and well maintained .csv file. Rarely does government data require cloud based computational capacity to add value to the raw data, and rarely does it grow so big that an API becomes more useful than managing the data locally gets unwieldy.
We need to start thinking about APIs, instead, as a method for improving the user experience of government. The registration process is a great place to start, because it’s a confusing area that’s rife with awkward user interfaces. Imagine, instead, if when you file for incorporation with LegalZoom, you’re asked to certify that you’re a woman-owned business and it instantly registers you with the SBA for an extra fee. Or that QuickBooks notices that you fall below the small business threshold for your given industry, and asks you if you’d like to self-certify for a fee?
If business registration entities like the GSA, the SBA, or local business development organizations create APIs around their registration processes they can ask the private sector to compete to see who can make it the easiest and cheapest to register these businesses for working with government at the federal state and local level.
Moreover, as long as government has provided registration free of cost via its own website, it can justify charging for API access and create a revenue opportunity for itself. It’s not charging people for something taxpayers are already paying for – it’s charging businesses to enhance their existing products. Charging a nominal amount for access to this API is probably a good API to fight fraud and prevent spam, and would likely be one that the Intuits and LegalZooms of the world would gladly pass on, and probably keep the API revenue neutral. This opens the door to ethical revenue based contracting: I’d happily build an API on top of Sam.gov’s registration process for free, in order to get $1 every time someone uses that API to register a new entity.
The one trick is to make it easy to sign up for API access. We don’t want to just shift the registration burden from citizens to developers, we also want developers to be able to register easily. The IRS’s leadership in the sector with the E-Filling API2 – the one that empowers companies like TurboTax – is a good place to start, but the barrier still likely needs to be dialed back for things less universal than paying taxes. Usually charging a small fee per-request will filter out more fraud than stringent background check requirements.
In general, government needs more process focused APIs. Everywhere there’s a government form, there’s a case for an API, and a business ecosystem to be built on top of it. Obviously some business cases (like tax filing) are more viable than others (like Race To the Top application forms). A way to prioritize this list is using a little known system called the ICR Dashboard3. The federal government has to calculate the number of “burden hours” it takes to fill out the form. The formula is simple: number of respondents times the time it takes to fill out. This system has a list of all the forms, and all the burden hour estimates.
Opportunities exist when there are high numbers of burden hours, or solely a high number of respondents. It’s unfortunate that the agency that runs this site, OIRA] doesn’t provide a list of all the government forms ranked by these numbers. They do, however, provide an XML dump of all government’s forms4. That’s a good list of where to start. And interestingly, leveraging an API first strategy may allow federal programs to avoid some Paperwork Reduction Act issues.5
The interesting thing about this opportunity is that for many of these is that if you don’t work for government, there’s still an opportunity for you. One needn’t wait for government to make an API. Often, you just need to understand what the form is trying to accomplish, who its customers are, and how to certify it and send it in to government.
We at DOBT believe there ought to be more players in this space. So we took the XML dump, and turned it into a Google Spreadsheet. Want to see which government forms take the most time and thus have the most opportunity? Now you can: go.dobt.co/oira-as-google-doc
An important lesson if you’re a programmer is to take a look at Recovery.gov, which cost $18 million dollars. But why did it cost $18 million dollars? For the answer to that, let’s take a look at the RFP6 and the somewhat-redacted winning technical proposal7 written in response. If you’re a technologist, you might come to the same conclusion I did: Government is paying a reasonable amount of money for what it’s asking. The problem is that it’s asking for the wrong thing. XML Firewalls? Data-cubing services? Seriously?
So how do you fix it? One might say: “hire a consultant to look at the RFPs, and she’ll tell you what you do and don’t need.” But government already has this – both in Technical Representative Programs (COTRs) and in open requests for comments in the procurement process. Unfortunately, each has its problems; COTRs tend to work on COTRing, not on remaining up-to-date on technology and its costs, and in an open request for information, the people who have the best input are also the people who can do the best job. This doesn’t sound like a problem, but often times you cannot bring someone in to help steer what kind of work to do, and then do the actual work themselves.
The long-term fix for these problems is to partially separate the RFP process from the procurement process. This helps on three fronts:
It helps get feedback from outside the context of a particular procurement. Instead of commenting on an RFP for “this” website, we can comment on an RFP for “a” website.
It promotes reuse. RFP content gets written over and over again, without tracking any success or results. This adds expense to the project since a), you don’t know if what you’re asking for is the right thing, and b), you’re doing work that’s repetitive.
It improves language. By soliciting feedback from a wider community, you stop the atrocious act of using phrases like “Information Distribution and Discovery Platform”. By calling them “websites”, you’ll be using the same language that’s used by folks who do the work regularly.
One way to solve these issues is by creating an Open RFP Library. It’s something we hope to work on here at DOBT, and it’s something that others are working on too.8 Imagine an open library for RFPs where a government agency can contribute their documents and share them with other agencies of all kinds, across different levels of government. And where the vendor community, the open government community, and other governments can comment on them and make them better. Imagine if people could ballpark what each RFP should cost, and imagine if you could circle back with them after the job was done to see how the work turned out.
While it would be difficult to show returns in the short term, this is a long-term play that reduces cost by making the requests better and faster. In other words: knowing what to ask for is just as important as asking the right people.
The buck stops at the contracting officer for getting the right deal. They’re the ones who select a “most appropriate” vendor, according to the terms of the acquisition, and they’re the ones who negotiate prices with that vendor. They sign the final metaphorical checks to the contractors. So why do the people in charge of spending taxpayer dollars use tools that look like this? http://go.dobt.co/co-software-1
Bad software in government begets bad software. It’s cumbersome, ancient, and a horrible user experience, which means that the people in charge of getting the taxpayer the best value for their dollar are spending their time negotiating with user interfaces, rather than negotiating with contractors. And if someone is using ugly software that’s cumbersome and that they hate using, they’re probably not going to be a good judge of what good software is supposed to look like and operate. It would be like asking a vegetarian for their thoughts on a steakhouse.
It also means that they spend less time keeping pace with technology or understanding pricing changes in the marketplace. This is how you get to a situation that I experienced inside of government, when I was pitching RFP-EZ to contracting officers as a way to make IT purchases of under $150,000. The dialogue went something like this:
Clay: RFP-EZ is a tool that will let you buy simple IT projects, like websites, really easily, as long as the total price is under $150,000.
CO: That’s impossible.
Clay: What’s impossible?
CO: You can’t make websites for less than $150,000.
Clay: Sure you can. You can actually make websites for less than $100.
CO: Well I don’t know about that. Do you have a list of companies that will make websites for less than $150,000.
Clay: Yes. All of them, except for the ones you are presently talking to.
CO: That’s crazy.
There’s a saying in large institutions that all problems revolve around three things: People, Process, and Technology. People is the hardest to solve – we have a contracting workforce that’s equipped with bad tools and compelled to do menial, low value work. How are we to expect they improve?
I think you fix this in three ways:
First, the Open RFP Library that we discussed in Section 3 helps because it makes it easier to ask for the right thing. The starting point for a purchase moves up in quality and down in price as a result.
Second, there are opportunities for policy change here, too. Raising municipal simplified acquisition thresholds for IT-related costs is necessary. Simplified acquisition thresholds are dollar amounts that, when a purchase falls below them, things tend to be a lot easier. For cities, the number hovers around $20-$40k, but it really ought to be higher for IT related purchases. The cost savings you get in both time to market (remember, this stuff depreciates faster than you can buy it), in a reduction of overhead, and in luring larger purchases under that threshold tend to be significant.
But those thresholds won’t be particularly useful unless you’ve created a cultural shift that encourages contracting officers to stay well-versed in technology.
So, third: you have to understand that Contracting Officers are career-oriented government employees; they’re not elected or appointed by anyone. They’re doing a job, and just like any other employee at any other company, they take their specialties, move around, and get promoted. This is a hard thing for people on the outside of government, or short-term appointees inside to get – they’re not susceptible to political pressure. (And for good reason.) They’re susceptible to career pressure.
What’s necessary is an inside/outside game – one where those of us on the outside work with people on the inside like the Defense Acquisition University and the Federal Acquisition Institute as well as outside professional associations like the Rising Acquisition Professionals. They need to be linked with the O’Reillys and Khan Academies of the world to create a training and certification program for getting the best out of information technology purchasing.
You want to make it so that someone who learns this stuff goes far, and someone who doesn’t gets left behind. For these folks, a badge of certification on a resume that helps them take the next leap in their career solves a lot more problems than any online petition ever will. You can’t solve career cultural problems with political solutions.
Fourth and finally: you have to create beautiful software for them. Software that works better and is cheaper than anything else on the market by a factor of 10, so there’s no excuse not to use it. That changes the expectations game. And that’s what we’re working on with Screendoor.io.
It’s easy to get distracted with the nuts and bolts of procurement. To look at the system, so broken and inefficient, and want to fix it for its own sake. But imagine a politician – say, a candidate for mayor – running for office. What do you think the chances are that our candidate will win if she runs on a platform of procurement reform?
I’m not sure we’re going to see any mayoral candidates giving speeches about increasing simple acquisition thresholds, reforming set-aside programs so that they actually work the way they’re intended to, or “creating a 21st century acquisition workforce”.9 It’s farcical.
Nobody buys a house based on the quality of its plumbing and wiring, and nobody will elect a government based on the quality of its procurement strategy. The next step in fixing procurement is understanding that fixing procurement isn’t about procurement. It’s about the things that come with it. The opportunities that get created when it does get fixed.
First and foremost, fixing procurement is about local economic development: 21st century procurement processes create jobs. The first city to implement the changes we’ve outlined in this blog is in for a massive boom. Instead of jobs going to multinational contractors, it’ll be able to work with local designers, developers and other innovators within its own community.
Those local shops will be able to create jobs, innovate further, and improve quality of life of the city’s residents. So the other reason you want to fix procurement is to fix service delivery. Constituent communications (both input and output) will work better. Imagine never having to wait in line at the DMV again. Or using your smartphone to grant yourself entry to mass transit. Or knowing what the status is of that hole in the sidewalk outside of your house. Or being able to attend a government zoning hearing via your computer or phone?
Those things aren’t just conveniences, they’re things that make people want to live there. All of these things exist in government today, but they’re incredibly hard for government to pull off because they’re expensive and there aren’t a lot of businesses that can both do the technical work to do it. Fixing procurement means that these things can happen easily and in a way that is affordable to the taxpayer.
But when you talk to government about “procurement reform” they don’t see these things. They see months of meetings with contracting officers. They see huge, intra-personnel political battles. They see committees and round tables and inertia. They see an unwinnable fight. And to local citizens? You have never seen eyes glaze over faster than when you say the word “procurement” when you’re trying to inspire people to take action.
Procurement reform is a huge opportunity, and the first government to enable the innovators in its own backyard to easily work with the city is bound to have a boom of jobs and convenience. But we, the champions of reform, must be about the jobs and convenience, not about the reform itself. It’s not about the reforms, its about the outcomes.
One of my favorite quotes is from Albert Einstein:
The significant problems we face cannot be solved at the same level of thinking we used to create them.
In the case of crafting a 21st century procurement practice, it’s worth taking that to heart. Anyone that’s ever worked at any government agency – local, state or federal – and tried to do anything “innovative” ends up frustrated with procurement and probably ends up with lots of ideas about how to fix things.
And our natural instinct is to gather a commission of those frustrated people together and think up ideas for change. Then draft these ideas into a recommendations document. Perhaps we sign some petitions and organize a bunch of people together in order to send those recommendations to policy makers, who then may implement the policy changes. And some policies change. This is often how policymaking works.
Does this sound crazy to anyone else?
It’s like waterfall software development10, but without any of the quality checks. We imagine what kind of policies might work and then implement them, never investigating whether or not those policies created the desired outcomes we were seeking.
As we move forward in figuring out what to do with procurement, I’d like to suggest that we avoid this model for crafting new policy and techniques. That is the level of thinking that created our significant procurement problems. Instead of using the waterfall technique of policymaking that got us into this mess, let’s be agile policy scientists and get us out.
That’s the battle plan for Screendoor.io:
The implications of working this way are significant because it allows for controlled failure. It makes it so you don’t need consensus of the whole acquisition community from the get-go: the seats at the table only expand when the scope of the test expands. And it means we don’t have to come up with all the answers before we affect change. We can acknowledge that we don’t know everything, and we’ll see what happens. If what happens turns out to be good, let’s implement it. If it doesn’t, we won’t.
We’re not going to solve IT procurement problems by sitting around the table drafting documents and signing petitions. We’re going to solve them by conducting tests, measuring outcomes, and implementing change based on the results.
What we’ve discussed so far are projects that involve bidding and sales – largely customizable IT integrations that come from professional services consultancies. But that’s not all that government buys, and it’s certainly not the place where the most innovation happens online.
Most of the webapps you use don’t come with salespeople. You didn’t get your twitter account by putting out an RFP for a “microblogging service” and waiting for Twitter and its competitors bid on it. The folks over at 37Signals.com don’t have an “enterprise salesforce” to compete with Microsoft Sharepoint. The past decade has ushered in a fleet of established independent software businesses whose web-based software is supposed to sell itself.
Except government can’t get to it, largely because of three reasons:
Culture. That’s “just not how you buy software”.
Purchasing requirements. It’s often difficult for government employees to sign up for these products, even if they have a purchase card. (That’s what government calls plastic cards with magnetic stripes on the back and numbers on the front.)
Terms of service. Standard terms of service for web applications usually come from a template found online, and aren’t compatible with federal law. A government employee can’t do what the rest of us do and just click through the terms of service without actually reading them; Federal guidance suggests that a federal employee cannot engage in any online service without first consulting counsel.11
That third one is the one I want to talk about today. Imagine how much money you’d spend, monthly, if you called your lawyer to review every terms of service you agreed to. Now imagine there are 4.5 million of you.
There are usually two major issues that government has with these boilerplate terms of service. The first is place of jurisdiction, and looks something like this:
You expressly agree that exclusive jurisdiction for any dispute with the Service or relating to your use of it, resides in the courts of the STATE NAME and you further agree and expressly consent to the exercise of personal jurisdiction in the courts of the STATE NAME located in CITY NAME in connection with any such dispute including any claim involving Service. You further agree that you and Service will not commence against the other a class action, class arbitration or other representative action or proceeding.
The problem is that the federal government can’t be tried in a state court, and so the federal employee who wishes to use these services cannot engage in that agreement, no matter how much authority they have.
The second problem is unlimited idemnification, which no lawyer, but especially no federal lawyer, would agree with.
So instead what happens is that the government employee goes to their agency’s legal counsel and the problem gets worse. From what I’ve seen, when they call a startup or one of these small businesses, they’re often hearing from government for the first time. And usually, the government lawyers start the conversation by using words like “you must” and “you shall”, rather than “can you” or “would you.” And sometimes the startup complies, and sometimes it doesn’t.
To their credit, GSA actually publishes guidance for social media providers for free tools12 (not commercial ones), which are quite useful in getting most of the way there. They even include a model template for a terms of service agreement.13
But this doesn’t go far enough. The right answer is for government to publish terms of service language that they would find “blanket acceptable” for free and commercial services. Startups would be able to copy and paste this language as an amendment to their existing TOS, and there would be an expedited review process within government that gives them a little “GovReady!” image, so that future federal employees know that the service they’re subscribing to is using pre-existing, pre-negotiated terms that have already been approved. This would give federal employees the ability to bypass the expensive and arduous process of having to work with their counsel’s office, and make it so that the heavy lifting doesn’t need to happen over and over again.
What will it take to make this a reality? Most of the technology businesses I know would much rather spend more time innovating, and less time working with lawyers, and those firms will have to organize and let government know that they actively want this to happen.
The cost of copying and pasting government’s amendments into their stock terms of service is drastically less than the cost of their legal team dealing with GSA’s lawyers, and government has an opportunity to save money by reducing redundant staff time and letting their employees use tools that are less expensive, and often times, work better too.
For instance, a private entity can engage in multivariate testing of a form, the federal government, ridiculously, cannot. go.dobt.co/the-law-everybody-should-hate↩
Like Beth Noveck’s WorldBank/NYU Wagner collaboration on Innovative Procurement go.dobt.co/wagner-innovative-procurement↩
Discussed in depth in this GAO Report: go.dobt.co/gao-supplier-report↩