The Digital Citizen

Satisfaction with E-Gov Remains at All-Time High

by Larry Freed

The fourth quarter ACSI E-Gov Index was released today, and with good news: e-gov remains at its all-time high score of 75.2 on the ACSI's 100-point scale. There are more than 100 sites included in the Index. This quarter, in addition to releasing scores for more than 100 e-gov sites, the ACSI also released a report on offline government. You can get the e-gov report on our website and the offline report from the ACSI website.

Satisfaction took a little dip when President Obama first took office, I think in part because people were waiting to se what happened. It stayed down the second quarter. But then we saw a huge rise in the third quarter, to an all-time high e-gov score (highest since the E-Gov Index started in 2003), and it maintained that high in the fourth quarter.

I'm not saying President Obama's administration is the only factor in the unprecedented levels of high citizen satisfaction with federal websites; it isn't. The men and women running these websites (often with far fewer resources and far more bureaucracy that the private sector) have been working hard on making these sites citizen-centric long before Obama took office, and they'll be doing it long after he leaves. But what President Obama has done is promote accountability, transparency and cost efficiency through e-government while implementing a number of e-government initiatives. Those initiatives include:

• Increasing use of social media tools on government websites to engage with citizens,
• Rebuilding whitehouse.gov to include Web 2.0 features,
• Naming the country’s first Chief Technology Officer, Chief Performance Officer and Chief Information Officer.
• Launching an open government initiative that encourages agencies to take steps to increase transparency, participation, and collaboration,
• Launching Data.gov, a website that provides the public raw feeds of government information, and
• Redesigning Regulations.gov to make searching and commenting on federal rules easier.

Meanwhile, as agencies implement actions to meet the Open Government Directive’s goals, the momentum of e-government is expected to increase.

You can read all of the details of how individual agencies ranked in the report itself, and if you listen to Federal News Radio, then you'll probably hear about it today. But I did want to make two other points that I think are worth singling out:

1. In the report, we list a bunch of scores for private sector companies like Amazon, Google, Netflix, etc. It continues to blow my mind to see how many government agencies are outperforming these private sector sites noted for their innovation and customer satisfaction. For all the times that the founders of these private-sector companies have been (well-deservedly) touted for putting the customer first, there are all these federal managers doing the same thing without the recognition--or the money!! It's been proven again and again that the ACSI predicts revenues and stock prices, so the private sector companies who are excelling at customer satisfaction are probably seeing nice bonuses and/or company revenues as a result. The government folks who are scoring in the high 80's and 90's ... let's just say they aren't getting any richer by making really fantastic government websites. In fact, we can't even buy the government folks a nice dinner to congratulate them for their good performance. But they are contributing to democracy, trust, involvement...maybe not the same kind of reward as a beach house, but a reward none the less.

2. The offline reportshows that although satisfaction with offline government is a lot higher than it was a couple of years ago, it is down slightly from last year. Satisfaction with offline government is 68.9 and satisfaction with online government is 75.2. This is a sizable gap that reinforces the idea that e-government provides a tremendous opportunity to meet and exceed the needs and expectations of citizens. As more and more people have computers at home and virtually everyone that wants to has access through libraries and other public facilities, e-gov seems to be the best way to provide good information and interact with citizens. The best part? Not only do citizens prefer it, it actually saves the government time and money. Win-win.

On another note, in mid-February, we'll be releasing our inaugural Transparency Index. We've written a lot on this blog about our efforts to measure transparency, trust, participation, collaboration, and other elements of the Open Government Initiative. When that comes out, you'll get to hear me get on my soap box about how the OGI and executive directives are a great start, but that we don't see any metrics mandated by the OGI in order to evaluate success. All in good time.

Okay, so if you were going to nominate a federal manager to get royalty and riches as a result of their amazing websites, who would it be?

No Such Thing as Customer Divorce in Government

The Wall Street Journal is a favorite source of stories that address customer service issues in our economy. I found this story by Raymund Flandez in the November 10th issue of the Journal in the Small Business section to be especially timely. It discusses how some companies are dealing with customers that are habitual late payers or require extra high maintenance. Under normal economic conditions this behavior might be tolerated, but in today’s tough times, companies need to pare these customers that are not profitable. This highlights an important principal – that excellent customer service doesn’t mean that the customer is always right.

IT JUST ISN’T WORKING? SOME FILE FOR CUSTOMER DIVORCE.

"It’s the beginning of the tough-love era. Some small-business owners are ending relationships with aggressive customers who ask for more discounts and concessions or want more leeway on payment time. . . . Still, the drastic action is bringing separation anxiety for some small-business owners. For years, many have prided themselves on customer services. And some small businesses, suffering from a drop in sales, fear they can’t afford to be picky. But problematic customers need to be fired, says Valerie Zeithaml, a marketing professor at the . . . University of North Carolina at Chapel Hill.”

If you are government manager, don’t you sometimes wish you could fire some of your customers? How do you deal with the especially difficult ones?

E-Gov Knows Its Stuff

by Dave Lewan

I'll admit I was initially a little surprised to see the findings of our 2009 Healthcare Benchmark. Of the four types of healthcare categories included in the benchmark, government-run healthcare sites satisfy online browsers more than private-sector pharmaceutical, health insurance, and hospital sites. Let me say that again...people like government-run healthcare sites better than private-sector healthcare sites. I'm not going to link that finding to the larger national debate about healthcare, but still, an interesting finding given the current climate.

I know and appreciate the quality of e-gov as much as just about anybody in the world not actually running an e-gov website, but still...the assumption (fair or not) is that private sector sites have more money and less red tape when it comes to improving the online user experience. And yet:

(all scores below are satisfaction scores on a 100-point scale)

• Overall healthcare benchmark: 75

• Government healthcare sites: 79 (the e-gov health benchmark includes sites like Health and Human Services, the National Cancer Institute, the National Library of Medicine, the Office of Women's Health, the Substance Abuse and Mental Health Services Administration, the Department of Veterans Affairs and the White House Office of National Drug Policy.)

• Pharmaceutical sites: 78 (the pharma benchmark includes sites like GlaxoSmithKline and Merck)

• Hospital sites: 73 (the hospital benchmark includes sites such as Allina Hospitals and Clinics, the Cleveland Clinic, Intermountain Healthcare, Northwestern Memorial Hospital, St. John Health, and UVA Health System)

• Health insurance sites: 64 (the health insurance benchmark, which is actually part of our broader insurance benchmark and not our healthcare benchmark, includes sites like Anthem and Blue Cross/Blue Shield.

There are a lot of reasons government could be outperforming the private sector. One thing that occurred to me is that the sites in the government healthcare benchmark are mostly informational sites, not transactional sites and it's easier to satisfy people with information than transactions that actually impact their healthcare. But in our quarterly e-gov index, transactional sites actually do even better than news and information sites. Besides, a lot of the sites in the private sector healthcare are also mainly informational in nature.

I think part of the answer is that a lot of these government health care sites have been measuring and attending to online customer satisfaction for nearly a decade. Given that kind of focus, it's no surprise they come out on top.

What about you...thoughts on why government healthcare sites outperform the private sector?

Citizen Satisfaction with E-Gov Hits All-Time High

by Dave Lewan

In the quarterly American Customer Satisfaction Index (ACSI) E-Government Satisfaction Index, released today, e-gov hits an all time high this quarter, of 75.2 on the Index's 100-point scale. Folks, ForeSee Results has been measuring satisfaction with e-gov four times a year for more than six years, so this is big.

There were a couple of things that really struck me about this report:

1) In many cases, federal sites outperform private sector sites.The ACSI's threshold for excellence is generally considered to be 80. There were 29 out of 104 federal government websites that scored 80 or higher this quarter. For comparison, when we studied 100 of the top e-retail websites, only SIX scored over 80. That is really remarkable. There's lots at play here--since satisfaction is a combination of what you get and what you expect, e-gov could be benefiting from lowered expectations. But we've seen satisfaction with e-gov go up and up and up and usually expectations keep pace with, or even outpace improvements in the private sector. So while a lot of sites still have room for improvement, this is a really great story about the power of e-gov to communicate with the citizenry.

2) E-gov that satisfies citizens is still the most efficient and cost-effective channel. The federal government can save overhead costs related to call centers and localized office locations by meeting citizens’ online needs and expectations. Citizens who are highly satisfied with a federal government website are 86% more likely to use the website as a primary resource (as opposed to other, more costly channels), 79% more likely to recommend the website, and 52% more likely to return to the website.

Download the free ACSI E-Gov commentaryon our website, and feel free to leave comments here or go to Gov Loop to discuss the findings. We're using Twitter hashtags #ACSI and #egov. Larry Freed also did a post about his thoughts on why satisfaction had such a big jump.

Congrats to everyone who did so well and to those who are working hard every day to do better. An all-time high in a nationally recognized index like the ACSI should be a proud day for folks in e-gov.

ACSI Scores Out Next Tuesday

by Larry Freed

Next Tuesday, the American Customer Satisfaction Index (ACSI) will release it's quarterly E-Government Satisfaction Index.

This report is useful and interesting for a couple of reasons:

1) We can see the trend in citizen satisfaction with e-government overall. The trend over the last six years is generally up, though the recent trend has been stagnation. Tuesday's report will shed light on whether any of the efforts of the Obama administration have come to fruition in terms of improving citizen satisfaction with e-gov initiatives.

2) We can see how indivdual websites are doing compared to each other, compared to the private sector, and compared to themselves over time. There are 104 individual government websites included in the report from virtually every agency and department. Each website gets a score on a 100-point scale; the higher the better. Anything over 80 is generally considered excellent. Which websites have made the biggest improvements over the last quarter? Over the last year? Which sites are better than Amazon and Google? Which sites aren't doing so well?

3) We can quantify the role of online satisfaction on citizens' likelihood to return to the site, recommend it, even use it as a primary resource. We can do this in aggregate for the federal government overall, and for individual sites.

The data is being compiled and reported as we speak--everyone is busily getting everything together for the release on Tuesday.

Let's keep it just between you, me, and the interwebz, but it's going to be an interesting one. Can't wait to share the final findings!

The ACSI and Its Royal Connection

by Bernie Lubran

 

I first became aware of the American Customer Satisfaction Index (ACSI), the methodology that ForeSee Results utilizes in measuring citizen satisfaction with government websites, back in 1994 when I read about it in Fortune magazine.  At the time, I did some dabbling in the stock market and was always seeking out a tool or technique that would give me an edge over the rest of the “market.”  The basic premise of the ACSI, as I understood it, was pretty simple – companies that do a better job of satisfying their customers were more likely to be successful in the marketplace. 

 

I decided to test this theory and made an investment in Heinz Company which was the highest- rated company for customer satisfaction in the ACSI (I must admit that as a Pittsburgh native and proud owner of a Heinz pickle pin, there were other mitigating (i.e., non-economic) factors that affected my decision).  Low and behold, this turned out to be one of the best investments I ever made, and to this day I consider a company’s performance in the ACSI to be an important metric when making investment decisions.

 

Naturally, I was excited several years later when I was given the opportunity to meet and work with the creator of the ACSI, Dr. Claes Fornell of the University of Michigan’s Steven Ross School of Business, after the methodology had been selected to be a standard metric for federal government agencies.   I also remember well the time we both traveled to Mexico City to discuss the methodology and how it was being utilized by the US government (incidentally, the Mexican federal government subsequently implemented a similar index to track citizen satisfaction with government service delivery).   Listening to Dr. Fornell talk about how he formalized the concept of quantifying consumer behavior and measuring the impact of quality on consumers’ purchasing decisions while serving on a submarine in the Swedish navy, I developed a new appreciation for the concept of “thinking outside the box.”

 

The reason I decided to write about Dr. Fornell was a September 30 article that appeared on the Ross School of Business’ website that announced that Dr. Fornell had received an honorary doctorate degree from the Stockholm School of Economics as part of its centennial celebration.  The event took place before the King and Queen of Sweden and in the same hall that the Nobel Prizes are awarded.  Congratulations, Dr. Fornell!

The Digital Citizen Whirlwind

by Dave Lewan

We’re back, refreshed, and energized from our annual public sector user summit last week in Washington, D.C. We flew our entire public sector team out to DC for the day so the 270+ registered attendees could meet all the satisfaction research analysts (SRAs), sales people, and other staff that they work with on a day-to-day basis. The Ronald Reagan Building treated us right (except for the WiFi problems, which they tried their best to help us work around) and we loved seeing everyone there and getting some fresh ideas about how government agencies and nonprofit organizations are using the methodology of the American Customer Satisfaction Index (ACSI) online.

We started out the day with a great presentation from Aaron Smith at Pew about how the Internet is influencing civic engagement. Download Aaron’s presentation here and read more about Aaron’s research on Pew’s website. The upshot of Aaron's presentation was that there are still access issues when it comes to using the Internet to increase engagement with under-represented groups. Aaron warned that just being online wasn't enough--you have to give people a reason to engage with you online.

Next up was a presentation from ForeSee Results CEO Larry Freed about the relationship between online transparency and trust. We have a lot of agencies currently using a model that incorporates these elements, and you can read more about it in Farris Kahn's post or Larry's previous post.

Margaret Allen at the World Bank shared a fantastic case study on how the World Bank developed a road map for meeting the diverse needs of global audiences online. The World Bank has 3700 global websites, all with different content, languages, look, and feel. Few of us are struggling with audiences that disparate and varied. The World Bank has put together a strategic long-term plan with shorter-term goals to keep things moving.

John Bell at Ogilvy PR shared the five barriers standing between government agencies and effective use of social media. Heads were bobbing in agreement in the audience when John talked about management fears that social media would mean a loss of control over message and that staff were not well trained enough to represent their organization so publicly. There was some good discussion about overcoming these barriers.

Our first panel was about how several organizations are using social media on their websites. GovLoop's Steve Ressler moderated, and the participants were Amelia Wong with the U.S. Holocaust Museum, Cari Wolfson with CDC, Captain David Werner with the Navy, and Maxine Teller with DOD. These organizations are all having success communication with their consituencies through social media channels, proving that government can be as quick to adapt and evolve as the private sector when it comes to these technologies.

After lunch, Carmen Marco and Scott Morton at FAA gave a case study about how the FAA had created a "customer culture" on its website and has, in the process, increased online satisfaction from 66 to 72 while the federal government aggregate has stayed basically stable over that same time at 71. Carmen and Scott said the ACSI was their #1 ROI metric.

We heard from the ACSI's David Van Amburg about why it is so critical for a government agency to care about online satisfaction, even though there are no sales or profits in the mix like there are for the public sector. David's presentation also had us all wishing we could invest in an ACSI-based mutual fund, since an ACSI-based portfolio has beat the market so many years in a row.

Meanwhile, Christine Sonnabend from the Holocaust Museum and Joel VanHaaften from ForeSee Results led a workshop focused on some specific ways to drill down on customer segments using the ACSI, and then how to share that information internally for maximum usage and value.

Farris Khan shared some details on ForeSee's session recording tool, called CS Session Replay, which provides real-time usability by allowing agencies and nonprofits to record mouse movements in actual user sessions while maintaining total privacy for the user.

Michael Ahmadi at SAMHSA (Substance Abuse and Mental Health Services Administration) shared how his organization is using data and knowledge management to guide decisions that will improve services and access to services and information.

ForeSee Results' Usability Queen (aka Usability Services Manager) Jennifer Bailey did some live usability auditing upstairs while downstairs, Cari Wolfson with CDC talked about how the Centers for Disease Control has developed data-driven online personas, and how those have helped as the H1N1 virus made traffic spike.

Thanks to everyone who came, everyone who presented, and thanks to Cecilia Persson and the ForeSee staff that made such a wondeful event happen. As the newbie on staff, it was so exciting to see a critical mass of organizations that really value the data they get from our methodology, and to learn all the myriad ways our citizen intelligence is put to use. Government agencies and nonprofit organizations are doing some really exciting work that gets to the very core of our mission: improving the effectiveness of websites and communication by improving the user experience.

Using Satisfaction to get from Transparency to Trust

by Farris Khan

Quick Summary of this article (it's a long one!)

1. Open Government in a Nutshell
2. ForeSee Results' Predictive Model
3. Transparency and Trust: Transparency, Trust and Future Participation work within the framework of the ForeSee Results' Predictive Model.
4. Participation and Collaboration: Future likelihood of participation with the agency/department generally works within the framework of the ForeSee Results Predictive Model, though the scores may vary.Gauging future online collaboration and rating the current opportunities to participate and collaborate online are best measured as custom questions, outside of the model. This information can vary in importance depending on the mission and goals of the site.

1. Open Government in a Nutshell

Just one day into his administration (January 21, 2009), President Obama released a landmark memo entitled “Transparency and Open Government”.

The memo articulated Obama’s core belief that “Openness will strengthen our democracy and promote efficiency and effectiveness in Government.”  The memo contained both aspirational goals as well as tactical recommendations to establish what he called the Open Government Directive.

While there have been similar ideas in the past, what is unprecedented is the concept that Information Technology would be the catalyst for these changes. In the memo, Obama specifically mentioned his Chief Technology Officer (Aneesh Chopra) as a leader in success of the directive.

2.   ForeSee Results Predictive Model

For the ForeSee Results team the memo offered the opportunity to determine to what extent these concepts could be measured, and provide business intelligence to drive action on the part of our clients.

The first step for ForeSee Results was to do research with several existing clients in the government agencies/departments.  We are happy to report that this research offered immediate insights.

As Larry Freed (President and CEO,  Foresee Results) details in his blog and on this blog, highly satisfied federal website visitors (those scoring 80 and above on the ForeSee Results 100 point scale) are:
• 49% more likely to trust the agency or department overall;
• 54% more likely to participate in communication with the agency online or offline in the future;
• 73% more likely to collaborate with the agency online in the future using interactive features such as wikis or blogs;
• 54% more likely to return to the site;
• 78% more likely to recommend the site;
• 77% more likely to use the site as a primary resource for interaction with the agency overall.

While the data points to correlation between satisfaction, and trust, future participation and collaboration, our next step was to see if there was actually a causal relationship between these variables. Simply put, do positive changes in satisfaction drive positive changes trust, future participation and future collaboration scores? 

Also of interest for our team was to see if transparency on the website could be a driver of satisfaction.  If so, our clients could make changes to a variable they can control (transparency on website) to cause improvements to a variable they cannot directly control (trust in the agency/department or future participation and collaboration with agency).

If our hypothesis of a causal relationship between Transparency, Satisfaction and Trust and other variables held true, they could be incorporated directly into our core government product, CS SiteGov.

3. Transparency and Trust

Our pilot research indicates that transparency can be an important driver of satisfaction. Simply stated, on many government websites:

Transparency --> Satisfaction

In addition, it is clear that satisfaction drives trust in the agency or department:

Website Satisfaction --> Trust in Agency/Department

Taking the two conclusions together, the following relationship holds true:

Website Transparency --> Website Satisfaction --> Trust

Making a government/agency website more transparent can have a tangible impact on trust with the agency or department.

4. Participation and Collaboration

Understanding how Participation and Collaboration fit into this equation is a little trickier. 

These concepts were tested in two ways:

1.   Website ratings for existing opportunities for Participation and Collaboration, as judged by the site visitor.

2.   The visitor’s likelihood to Participate and Collaborate with the agency/department in the future. 

In our analysis, ratings for both types of questions can be established, but the number of respondents who are able to answer the questions and the ratings themselves may vary based on the type of agency and its online communication and collaboration features. 

Going back to the Obama memo, Participation and Collaboration were looked at as ideas that agencies should strive for, rather than factors by which they should be judged.

Ultimately, developing a platform that fosters citizen and private sector participation and collaboration with government will drive better perceptions of government transparency, which will lead to higher satisfaction and higher levels of trust. 

All this being said, future online participation with the agency/department did fit well as a future behavior for some entities whose goals included fostering interaction with the public.

In short…

Website Transparency --> Website Sat. --> Likely Future Participation with Agency

A battery of related P&C questions, including willingness to collaborate with the agency in the future and ratings of participation and collaboration opportunities within the website, can be asked as future behaviors or custom questions. 

Stay tuned for more as our research develops. Feel free to catch up with me on Twitter (http://twitter.com/@bobbleheadguru) or email me at farris_khan@foreseeresults.com. 

The Relationship Between Online Transparency and Trust in Government

by Larry Freed

We've seen over the years that online citizen satisfaction plays a huge role in government efficiency and effectiveness. So we decided to test out satisfaction also related to online transparency, collaboration, participation, and even trust in government, Turns out there is a huge relationship.

Online customer satisfaction directly impacts citizens’ trust in government. Not only that, online satisfaction impacts their likelihood to participate and collaborate with an agency in the future. Highly satisfied federal website visitors are:

• 49% more likely to trust the agency or department overall;
• 54% more likely to participate in communication with the agency online or offline in the future;
• 73% more likely to collaborate with the agency online in the future using interactive features such as wikis or blogs;
• 54% more likely to return to the site;
• 78% more likely to recommend the site;
• 77% more likely to use the site as a primary resource for interaction with the agency overall.

This strikes me as huge. Just by improving the website, you can impact trust, collaboration, and even cost-saving behaviors like using the site as a primary resource.

So how do you increase satisfaction? It's going to differ from site to site, depending both on existing strengths and weaknesses and on the site's mission. But in aggregate, we found that the top priorities are online transparency and online search. That means that focusing on those two things will have the greatest impact on online satisfaction.

So in short, online transparency leads to online satisfaction. Online satisfaction leads to trust, collaboration, participation, cost savings, efficiency, effectiveness, and a chicken in every pot.

We're really excited about the potential of this research. Stay tuned for more details in the coming weeks.

Digital Citizen 2009

by Dave Lewan

I didn't have much chance to tweet during today's conference (Digital Citizen '09, #fsrdc09). I'll post deeper thoughts later, but I wanted to thank everyone who came today and presented. We heard some fantastic case studies from the World Bank, The Holocaust Museum, CDC, the Navy, DOD, FAA, and SAMHSA. The federal agencies and nonprofits who were here today are all doing amazing things when it comes to keeping sites responsive to users, engaging in responsible and effective social media, and following through on strategtic plans to improve the online experience.

It was a great introduction for me to so many of our clients and partners. More to come about specific sessions after some decompression time!