Improving Veterans’ Digital Experience Across Presidential Administrations

Introduction

In 2013, Marina Nitze, the new Chief Technology Officer (CTO) at the U.S. Department of Veterans Affairs (VA), was determined to get input directly from Veterans. On a bench outside the VA facility in Menlo Park, California, she met a woman and her husband, a Veteran. He was going blind and his previous doctor told them he didn’t qualify for VA healthcare, but one day a different doctor encouraged him to enroll at VA. He spent eight months in a VA hospital to address mental health issues and to undergo an amputation, but fortunately the VA doctors were able to save his eyesight.

The woman raved about the great care her husband received at VA, but when Marina mentioned the wide range of benefits that VA offers, the woman asked blankly: “What other benefits?” Marina was astounded. How was it possible that a Veteran hospitalized at a VA facility for eight months didn’t know VA provided benefits besides healthcare? Marina became even more determined to make VA work better for Veterans.

A core part of making VA work better, in Marina’s vision, was improving the digital experience for Veterans. She was determined to improve the maze of websites that VA operated, including VA.gov, which was challenging to navigate and too often featured information about VA leadership or organizational structure instead of information Veterans needed about the benefits and services they had earned.

As a 28-year-old CTO who hadn’t finished college, Marina hardly fit the traditional profile of a senior technology executive in an organization the equivalent size of a Fortune 10 company. She was given no budget and no people. In many ways, she was set up to fail. But Marina was resolved to make a difference. She convinced senior VA leadership that simpler and easier-to-use digital services were important and recruited the first agency team of a new elite tech unit called U.S. Digital Service (USDS), created in the wake of the Healthcare.gov failure. By the end of 2016, Marina and the Digital Service at VA (DSVA) had built Vets.gov—a highly regarded website, separate from the main VA.gov website, that focused on VA services and information that Veterans needed the most.

With President Trump’s election in November 2016, Marina, a political appointee, left VA in January 2017. Conventional wisdom was that the incoming Trump Administration was skeptical about Obama-era accomplishments. Many in VA wondered what would happen to the DSVA. How would new leadership change VA priorities? Would the DSVA team be dismantled or slowly fade into irrelevance? Would Vets.gov be scrapped?

But, as described below, DSVA was able to adapt and thrive in the new Administration, through both support from the White House and day-to-day execution inside VA. New VA CTO Charles Worthington, who joined in May 2017, and the DVSA team continued to improve the Veteran online experience. By November 2018, they launched a brand new VA.gov (based on Vets.gov) to serve as a single “front door” to VA—winning acclaim and helping build greater Veteran trust in VA.

The success story of the digital efforts at VA surviving the uncertainties of a cross-party Presidential transition, and thriving despite subsequent substantial VA leadership turnover, presents an interesting case study. What should we take away from this story of growth and transition at VA? How did Marina Nitze, Charles Worthington, and their teams navigate a time of great uncertainty—a time when many initiatives stall or die in a new administration? And what lessons might apply in future Presidential transitions?

Background: State of VA Digital Experience in 2013

VA provides many benefits, including healthcare, pension, survivors’ benefits, rehabilitation, and employment assistance to over nine million Veterans. VA’s healthcare system has over 1,400 sites of care and represents the nation’s largest integrated health care system. For many Veterans, VA is a critical resource.

By 2013, many Veterans were looking to access VA benefits and services online, but the online experience was only available for certain types of benefits. And even when it did exist, it was inconsistent, frustrating, and often inferior to analog experiences via mail, VA medical centers, or through call centers. To access benefits online, Veterans had to engage with multiple VA websites, including VA.gov, eBenefits, MyHealthVet, Explore VA, VONAPP, AccessVA, and a VA ‘app store’.

Poor design, outdated technology, inconsistent branding, confusing navigation, and multiple logins with different security levels, usernames, and passwords made it difficult for users to find and access the correct website for their specific needs. In some cases, a Veteran would learn about a benefit on one website, fill out the application on another, and then check the status of that benefit on a third website. Other important services were only available via customized PDF forms requiring outdated versions of Adobe Acrobat and older Internet browsers to even open. Fundamentally, this confusing process created a significant barrier for Veterans to access the benefits and care they had earned.

Marina Nitze and VA

In 2013, Marina Nitze became VA’s CTO, a relatively new position to VA with no budget, employees, or any formal authority.

Nitze leveraged her background in the inaugural class of Presidential Innovation Fellows (PIFs) to find like-minded people at VA (such as Emily Tavoulareas and others at the VA Center for Innovation) and PIFs in the current cohort (Mollie Ruskin, Ben Willman, Tom Black, and her future successor Charles Worthington) who were willing to help. Armed with sticky notes and a borrowed room at the General Services Administration, Nitze and this intrepid team articulated their vision for “A 21st Century VA.” Within a few months, the team converted their post-it notes into a well-designed booklet describing their plan to improve Veterans’ experiences with VA.

Still without a budget, Nitze spent almost an entire paycheck printing out ‘glossy’ versions of this booklet. She began handing out booklets wherever she could: at meetings, in the hallway, or even on the elevator. This was not the traditional path to “getting things done” in government, but Nitze was aided by a bit of luck. On one elevator ride, she ran into Sloan Gibson, VA’s new Deputy Secretary, on his first day and handed him her booklet. Less than a week later, he came to her office and tasked her with delivering on the vision, saying, “I want this VA.”

The booklet laid out a vision to achieve a user-friendly website where Veterans could find relevant information and conduct all their transactions with VA. Most importantly, Marina wanted the chance to engage in a human-centered design process—meaning that they wanted to conduct original research with Veterans about their needs, create prototypes to test with Veterans, and continually collect end-user feedback to guide the ongoing process. This represented a divergence from how products were usually built at VA, or in government generally. To get there, Marina needed a team.

The Team and Policy Foundation

Over the next 11 months, Marina (and others, including Erie Meyer in USDS) scraped together a budget and worked to set up a structure to create a Digital Service at VA (DSVA). After proving unsuccessful at hiring the right developers, designers, and researchers through VA’s normal hiring process, Marina used a specialized hiring authority (Schedule A) to have, for one of the first times in government, engineers evaluate other engineers’ resumes and experience, instead of HR. This considerably improved the quality of the top candidates and Marina was soon able to interview and bring in the first 6 members of DSVA.

In early 2015 the first cohort of DSVA members arrived. That summer Marina and her small team began working on a new website, Vets.gov, that would be designed with Veterans, be mobile responsive, and meet the vision in “A 21st Century VA.” As the team started building Vets.gov, they found themselves rubbing up against VA restrictions while trying to use a commercial cloud provider. According to VA policy, her team could not use a commercial cloud provider for their website because the Inspector General (IG) had determined commercial cloud services were incompatible with VA rules. The reasoning Marina was told: VA could not “put the cloud in an evidence bag.” It turns out that VA forensic investigations of computers were typically done through physically impounding desktops and servers, and this practice was seen as in conflict with commercial cloud computing.

As CTO, Marina knew the agency’s future would depend on embracing commercial services like cloud computing, so she and DSVA team member Shawn Arnwine took on the challenge of changing VA’s policy in the fall of 2015. Working with the IG to update their policies, Shawn both gained approval for Vets.gov to use the commercial cloud and also laid the groundwork for many other major VA systems to move to the cloud as well. This was just one example of the many obstacles Marina and the team had to work through to begin building modern, user-centered products at VA.

Launching and Iterating on Vets.gov

While many government projects would have spent years developing a large, grand solution to unveil at the end, Marina and her team intentionally took a different approach: they created a minimum viable product (MVP) with plans to quickly improve from there. By starting small with an MVP, Vets.gov could receive vital feedback and usage data to drive enhancements and new digital products. This way Vets.gov was built with Veterans throughout the entire process and was built in a modern technology environment from the beginning.

On Veteran’s Day 2015, Vets.gov launched and the MVP included content around two common Veteran issues—disability compensation and education benefits—as well as a facility locator and a GI Bill comparison tool to determine education benefits at various schools. Then-VA Secretary Bob McDonald announced the launch with a blog post and a press release. True to their planned iterative nature, the next month Vets.gov launched the Veterans Employment Center to help Veterans find jobs more easily.

The team’s biggest early impact, however, came later in 2016 from a different product: making it easier to apply for VA health care online. During their user research, DSVA met and interviewed a Veteran named Dominic. DSVA met Dominic through a letter he wrote to President Obama describing his struggles and, ultimately, his inability to access VA health care due to problems with the outdated digital form. As researchers observed him use the old application, Dominic stumbled through the process, articulating his frustration along the way. Clunky designs with unclear information and pop-ups with legal language led to a dead-end—the application didn’t even open because he was among the more than 80% of visitors using technology that was too new for the outdated tool the VA had paid millions for. As DSVA observed Dominic using a prototype of the new application they had built, Dominic declared that he would use the Vets.gov healthcare application “over anything VA had to offer.” By July 2016, Vets.gov launched this new online health care application, designed and tested with Veterans like Dominic.

Use of the healthcare application skyrocketed, doubling overnight and highlighting pent-up demand from Veterans for better digital products across VA. This increase in volume illuminated a different problem, however, because the backend processing for these forms was error prone and often needed manual intervention. What was a manageable internal issue with a small number of daily digital form submissions was amplified beyond control once applications surged to over 500 per day. This set off a major confrontation between DSVA and the team running this backend processing, with disagreements about how best to resolve the issues. After a few hectic weeks, including Marina being told to turn off the Vets.gov application (and refusing to do so), the teams were able to resolve the major issues and get health care applications flowing more smoothly through the systems. This was a lesson for Marina and her team that became a recurring theme as they built Vets.gov; it didn’t matter how good the digital experience was if VA couldn’t process the data in back-end systems.

By the end of 2016—just over a year later—Vets.gov included 30 digital products that improved Veterans’ access to VA healthcare and benefits. Each of these products had been redesigned with Veterans in more than 500 individual Veteran research sessions. Veterans could more easily refill prescriptions, check the status of their disability claim, and apply for education benefits. More than 120,000 Veterans had applied for VA healthcare online in just the first four months (compared to annual online applications of 52,000 for all of 2015) and 50,000 Veterans had applied for education benefits, some of which previously had no online option. Availability on some digital products had improved from as low as 76% uptime prior to Vets.gov, and release cycles went from 90 days or more to daily. Using actual cost data from Vets.gov and comparing it with expected internal pricing, hosting costs were reduced by 85% as a result of developing in the commercial cloud. Marina and her team had accomplished a great deal in just two years, but where DSVA and Vets.gov would fit at VA moving forward was still unclear.

Executive Branch and Leadership Transitions

In the leadup to the 2016 U.S. Presidential election, Marina knew that DSVA was about to go through a time of transition. Marina was a political appointee and was set to leave with the outgoing administration. Organizationally, DSVA could no longer assume support from VA political appointees and the White House. They would need to prove their worth to a new set of leaders entering the fold.

Marina realized this, and months before leaving office, she worked to position her team for success through the Presidential transition. And as much as she could, she worked to position Charles Worthington as a candidate to replace her as CTO. Marina had known Charles for years, as he had been one of the founding members of USDS and rose to become its Deputy Administrator. Both Charles and Marina had entered federal service as Presidential Innovation Fellows, a program created to attract technologists and entrepreneurs to government for a term-limited tour of duty.

Marina felt that Charles shared her vision for VA and, more importantly, a complementary skillset. Marina excelled as a leader that could roll up her sleeves and get things done. Charles complemented her with strong people skills and the ability to bring disparate groups together and build a coalition. Marina thought Charles had the perfect background, mindset, and skillset, but there was not a clearly defined path for Charles to move into this new role.

Starting on Veterans Day of 2016, Marina began advocating for Charles and setting him up for success. First, Marina decided her role should be split into two: VA CTO and the Director of DSVA. Because the Director of DSVA was not a political appointee, she designated Stacey Langer, the Vets.gov product lead, to take her place. Additionally, former WebMD senior executive Clare Martorana joined the Vets.gov team, which brought experience and stability during the transition. Because Secretary Shulkin, the new Secretary of VA, was appointed only days before the incoming administration entered office, there would likely be a delay in appointing a CTO.¹ Anticipating that void in leadership, Marina knew Stacey and Clare would be critical to ensuring the team would have continued leadership upon her departure and could keep producing results during a time of transition.

After President Trump was elected in November of 2016, Marina turned her focus to winning over the new administration, which meant impressing the newly established Office of American Innovation (OAI) run by Jared Kushner. OAI’s mission was to give executive sponsorship to issue areas that were important, but not generally addressed in the daily news. According to Matt Lira, Special Assistant to the President, the new Administration wanted to “double-down on projects that were working and improve those that were being done poorly.”² Marina realized that for the DSVA team to survive, OAI’s support was critical.

Marina was able to get a meeting with OAI early in the “transition calendar” and then focused on understanding the priorities of the new administration. She learned that the Trump Administration and OAI were focused on efficiency, sound business practices, and eliminating waste. Instead of emphasizing any specific processes, she framed the work of her team as Veteran-driven and cost-effective. A real sticking point from her pitch was that “they could save a billion dollars a year in IT.”³ The combination of improving Veteran service delivery and delivering at a better price was an effective message, and quickly helped Marina gain the trust of the new Administration. As a result, Marina was permitted to attend other VA meetings with the new Administration, serving as a voice in the room for technology decisions. As Matt Lira said, “The politics were pretty irrelevant because it’s such a patriotic thing to do.”⁴

As Marina transitioned out as CTO in January of 2017, OAI indicated its support of the U.S. Digital Service, including agency teams at VA, HHS, DHS, and DoD. OAI tweeted their support of USDS, visited the DSVA offices, and invited DSVA leadership to visit the White House. In late March 2017, Jared Kushner came to a USDS staff meeting in the White House Conference Center to reassure USDS staffers that the White House supported their mission. “Those early moves are so critical,” said Matt Lira. “If we hadn’t shown the Executive sponsorship fast and early, the bureaucracy would have counterstruck.”⁵

Across the street from the White House at VA, the VA bureaucracy was doing just that, however, and DSVA was fighting for its survival. DSVA members such as Emily Tavoulareas camped out in Marina’s old office in the VA Secretary’s suite to maintain relationships and visibility as the political earth shifted under their feet. They met with members of the VA Transition Team, racing to demonstrate the obvious impact they had achieved over the past two years, before those who sought to eliminate DSVA poisoned the waters against them. And they relitigated countless issues with internal VA teams that were trying to renege on decisions made in the previous administration. At times it felt like the organization they had spent years helping was crashing down on them, but the DSVA team was able to stick together, tell their story honestly, and continue doing the work needed to improve how the VA serves Veterans.

Eventually, the White House endorsement, the internal efforts to get early engagement from the new leadership, and a demonstrated impact for Veterans proved to be effective; the DSVA team was able to maintain the portfolio of work they had worked so hard to develop and had survived a critical transition period. After four months of interviews, meetings, and process-wrangling, Charles finally entered as CTO in May of 2017 and Marcy Jacobs joined as the DSVA team lead the same month.

A New CTO

That summer, DSVA gave weekly OAI briefings, led by Marcy Jacobs and Clare Martorana, which were designed to give Vets.gov top cover from the White House. At the same time, Charles worked to rebuild VA executive support for Vets.gov as the new CTO. A lot of the top executives within VA were new and didn’t know Charles or about DSVA. Charles built relationships with executives across VA in his first months, even finding allies in people that had disagreed with Marina.

Charles then began laying the groundwork for support of a digital strategy within VA by organizing what was at first an ad hoc group of executives from across the agency—later, they would become known as the Digital Modernization Council. The first task for this group was to come up with a set of Digital Modernization principles and “north star” objectives that everyone could support.

These principles provided VA clarity in their goals and ensured VA’s leadership was on board. This elevated the conversation among executives from tactical squabbles about website brands and technology platforms, and helped different parts of the agency agree on a common vision:

“VA will deliver self-service tools on par with top private sector companies and will have the best online experience in the federal government. Every digital service will be customized to the individual using it. Interacting with VA digital services will feel like navigating TurboTax, not filling out a form.”⁶

Online Scheduling

Finally, in October 2017, Charles had an opportunity to prove what his team could do for VA executive leadership, and more importantly, Veterans. During a leadership conference in which every VA executive came together, a woman on a panel of young Veterans recounted her experience struggling to learn how to receive VA health care. She told a story of starting with a simple Google search for “make an appointment at VA,” which led her down a confusing web of links to blog posts, websites, and applications, each with conflicting information about how to make her first VA medical appointment. One VA website she found mentioned a “mobile app” for scheduling but when she searched in her phone’s app store, she couldn’t find an app. Finally, she realized the “mobile app” referenced in the article was actually a website, but then she couldn’t log in because the credential she had used in the military had an expired password that she was unable to reset. In a last-ditch effort, she called VA, but sat on hold for 15 minutes before giving up and using her private healthcare

The day after hearing this story, then-VA Secretary Shulkin told Charles that his team needed to fix this. Charles seized this opportunity and quickly sprung up a “sprint” team to improve online appointment scheduling. They worked with teams across VA to make sure all VA websites accurately described how to get an appointment and had consistent copy and scheduling information. Usage of online scheduling quickly shot up within 45 days, and Charles and his team had a quick win to show other VA executives why digital modernization was so important.

At the same time, VA’s Veteran’s Experience Office—a multi-channel customer experience office in VA that worked closely with DSVA—did research with 3,400 Veterans to better understand their online experiences with VA. The overwhelming consensus from this research was that Veterans did not understand why VA offered its digital products under multiple brands and websites. Traffic to Vets.gov had been growing, but it only represented a fraction of overall web traffic at VA. It seemed that Vets.gov, a user-centered website, was paradoxically contributing to the problem of VA’s fragmented digital experience that Marina had originally set out to solve. This led Charles to an idea: What if Vets.gov “merged” with VA.gov?

VA.gov

By early 2018, Charles and DSVA team convinced VA’s Web Governance board, a cross-administration body of VA staff that sets web policy, to reimagine the primary VA.gov website, using lessons from Vets.gov. With no extra persuasion, about a dozen VA employees volunteered to join the Web Brand Consolidation Working Group to iron out the details. This group identified the four biggest problems that had the best chance of being solved by Veterans Day 2018, which was just over six months away:

1. Veterans can’t find what they need online
2. Navigation between VA sites is a disjointed, confusing experience
3. VA.gov is organized for the administration, not its users
4. Decision-making authority for digital products is unclear

Over the next six months, the working group, the DSVA team, and three contractor teams worked tirelessly to solve these issues and relaunch an updated VA.gov on Veterans day. The team prepared a detailed launch checklist, covering everything from remaining technical fixes to who was responsible for updating the Office of the Secretary. Daily conference calls with stakeholders and offices across VA happened during the week before the launch to ensure that every item on the checklist was addressed.

With all this preparation, VA.gov launch was a low-drama event. Following its practice of making frequent improvements, the DSVA team had made the new site available publicly months earlier at preview.va.gov, giving the team confidence the new site worked. In the weeks leading up to launch, visitors to Vets.gov had been gradually directed to the new preview site, giving the team time to identify and eliminate several bugs. All VA.gov users were presented the new site at 6PM EST on November 7, 2018. A few minor bugs were quickly identified and fixed. By 9PM the new site was running so well that most of the staff focused on the launch went home.

The design of the new VA.gov—based on findings from research conducted with over 5,000 Veterans, service members, caregivers and family members—made a real difference. In the months following, the customer satisfaction score had risen over 20%, and the number of online health care application forms had jumped by 52%. By all measures, the new VA.gov was a great success, and VA leadership has incorporated the focus on human-centered design and agile development methodologies across a broader number of VA products and services.

Lessons Learned

While Marina’s “21st Century VA” didn’t lay out a perfect plan that led to a relaunched VA.gov, it did establish principles that Marina, Charles, and all of VA could follow to improve the Veteran experience. Those principles, and consistent progress toward them, enabled VA to deliver on this vision.

Too often, promising government initiatives lose momentum during shifts in leadership and formal transitions. Yet, in just four years since the publication of “A 21st Century VA”, VA has achieved many of those goals to improve Veterans’ digital experience and modernize VA benefits delivery. Challenging politics or upcoming elections shouldn’t mean that government avoids large, impactful, and multi-administration initiatives.

Here are five lessons from the VA CTO transition that apply across government:

1. Focus on mission

Marina could have been partisan and unwilling to cooperate with the new Trump Administration, as she was an outgoing Obama political appointee. But she wasn’t. She prioritized the work, and the good of Veterans, above all and understood the interests of the new Administration, adjusted her pitch to focus on where those interests and hers aligned, and helped the new Administration understand context in other meetings. She put the mission of improved Veteran service delivery first and positioned her team to continue delivering on this when she was no longer there.

Charles brought together many groups at VA by writing and getting agreement on a common vision and set of “north star” objectives, helping leaders from across VA focus on what VA needed to accomplish, rather than internal politics. He then took a risk merging Vets.gov with VA.gov. This required VA executives to come together and deliver at a scale not previously achieved at VA, and risked department-wide backlash if it failed. But it was the right choice for the mission of improving Veteran service delivery. When speaking about the VA.gov launch, Charles does not take credit for himself, instead focusing on the work the Digital Modernization Council and others did to make it happen, as well as the Veteran impact of the improved site.

2. Create momentum

Marina didn’t just write “A 21st Century VA” and claim success, but rather started delivering on the vision and stacked up early wins quickly after the document as written. She brought in talent using an amended hiring policy, changed VA’s cloud policy, and launched tools built with Veterans. She created momentum in VA, earning her credibility and giving her the opportunity to launch the new website, Vets.gov, which gave her team the opportunity to keep iterating and delivering.

Charles quickly sustained and grew this momentum. He significantly improved VA online appointment scheduling within the first year and regularly shared metrics from before and after the changes his team made to quantify the impact. This bought him credibility with VA leadership and set him up to bring together an ever-growing coalition from across VA to merge Vets.gov with VA.gov the following year.

3. Integrate digital into the end-to-end customer experience

Marina knew you couldn’t separate the online experience from the overall experience, which was illustrated above through the launch of the health care application on Vets.gov. DSVA worked through difficult internal politics to understand the backend processing issues and do its best to help resolve them. Marina knew that even if Veterans had a great digital experience, they would be frustrated if they couldn’t get the benefits they earned because of issues processing the data.

Charles recognized that Vets.gov had improved the Veteran digital experience, but also saw from internal research that Veterans were still confused about VA’s multiple websites. He created a detailed plan for how to get to a single website through a series of mergers. He wasn’t focused on any single type of benefit, but rather the overall digital experience interacting with VA.

Charles also recognized that the digital experience was part of a larger customer experience at VA that included in-person visits, mail, and phone calls. That many Veterans used several or all of these channels didn’t paralyze DSVA from making frequent digital improvements—but it did mean that the team actively worked to coordinate with other parts of VA to create an improved end-to-end Veteran experience.

4. Understand and evolve organizational culture

Marina created a new digital unit within VA, finding some people she could bring in from other parts of VA and government while also hiring from the private sector. She built an organization that valued putting Veterans’ needs at the center of their decision-making. As the first agency team of USDS, she was careful to recruit a team that embodied USDS values and fought for them to have access to modern tools like Slack and GitHub so they could communicate and work together effectively.

Charles figured out how to evolve DSVA to improve how it worked with other VA teams and leadership, without losing DSVA’s identity or focus. He didn’t unilaterally make the decision to merge Vets.gov with VA.gov, but in keeping with DSVA’s team culture, he let this strategy be dictated by internal feedback and user research with Veterans.

5. Find the right partners

Marina understood that she didn’t have any formal authority when she became VA’s CTO in 2013. Because of this, she tried to build influence through networking and helping other teams. After creating an appealing vision of a modern, digital VA, she networked inside VA to find like-minded colleagues. Then, she created Vets.gov separate from VA.gov to intentionally disconnect it from the politics and challenges of the main VA website. This ensured that her team could expand and continue to make progress despite the challenging bureaucratic environment of the larger VA.

Charles cultivated relationships with employees and executives across VA, including in the Office of Information and Technology (OIT), a larger group that initially had a contentious relationship with DSVA. This bridge-building work helped Charles gain more influence around VA. Charles’ team is now housed in OIT and able to work closely with and advise many other technology teams across VA.

While this story is told through the lens of Charles and Marina’s work, and their transition, hundreds of career government employees, people in government on a tour-of-duty, and contractors helped make all this possible. While there isn’t room in this article to include all their names, Marina and Charles were only able to succeed because of their ability to find and work with the right people in the right positions. Change at this scale doesn’t happen by two people, but through partnerships, focused on delivery, within and across organizations.