DMS Quality Assurance Performance Program

DMS Quality Assurance Performance Program

DMS Quarterly Customer Satisfaction Survey December 2007 Chief Performance Officer Rob Hebeler, Q1 FY0708 1 A Few Words First The journey of continual improvement is not one for uninspired leadership or poor management. It is difficult. It is hard to always focus on how to get better on this day, and every day moving forward. Our customers expect and deserve service and product excellence. We should do nothing less to strive and meet this level of expectation. The key to success rests in our ability and desire to build relationships with each other and with our customers. Chief Performance Officer Rob Hebeler, Q1 FY0708 2 Agenda Background Executive Overview Survey Design Survey Implementation Survey Findings Closing Remarks and Suggestions for the Future Chief Performance Officer Rob Hebeler, Q1 FY0708 3 Background Survey Design Survey Implementation

Survey Findings Closing Remarks Background In January 2007, Secretary South and team embarked on a journey of continual improvement. This journey was marked with the development of an innovative Customer Quality Assurance Framework that involved the entire organization. In June 2007, the department launched a customer satisfaction survey to benchmark product and service attributes, overall satisfaction and problem resolution. The survey was distributed to a random sample of the departments 1.1 million customers. The resulting benchmarks established our starting line and launched our journey of measuring what matters. We established our continual improvement goal of a 2% increase of Extremely Satisfied for each quarter moving forward. In October 2007, the department launched its first customer satisfaction survey for the fiscal year. The Q1 FY0708 survey used an online format with a phone-based helpline. The results of the customer satisfaction surveys enables the department to Gauge customer satisfaction over time. Gauge customer product and service performance over time. Identify key areas where DMS employees excel and where there is opportunity for improvement. Chief Performance Officer Rob Hebeler, Q1 FY0708 4 Background Survey Design Survey Implementation Survey Findings Closing Remarks Our Customers 1

Who Have Needs and Expectations of Us Our Playbook 2 How We Get from Here to There Our People 3 Serving with C.L.A.S.S. 7 Our Products and Services 4 5 6 What We Provide our Customers with and How We Treat Them Process Culture Communications How We Do Things How We Behave How We Keep Everyone Informed Continual Improvement What Must Change Our Results How our Customers Keep Score

Our Improvements Opportunities Chief Performance Officer Rob Hebeler, Q1 FY0708 What Needs to be Better 5 Background Survey Design Survey Implementation Survey Findings Closing Remarks Implement best practices to create a workplace of choice that fosters recruitment, development, recognition and reward Improve our contract management capabilities Create robust strategic plans for each major program and service area Focus on the processes and procedures of our core competencies to create a springboard for world-class performance Increase the brand awareness of our programs and services to our customers Develop Human Resource practices that encourage a resilient workforce with the ability and desire to serve in an emergency or natural disaster Chief Performance Officer Rob Hebeler, Q1 FY0708

Measure what matters and continually improve the quality of services delivered to our customers 6 Background Survey Design Survey Implementation Survey Findings Closing Remarks Q1 FY0708 Online Survey Survey Question Areas Division Interaction Selection Service Product Overall Satisfaction of Experience vs. Expectation Why do you feel this way? How can we improve? Problem Resolution What types of problems? Chief Performance Officer Rob Hebeler, Q1 FY0708 7 The Moment of Truth Zone Zone of of Pain Pain Zone of Delight

Zone of Mere Satisfaction Loyalty A C D vv F Worse Worse Than Than Expected Expected Dissatisfied B Expected Merely Satisfied Better Than Expected Delighted Level of Customer Satisfaction Source: Moments of Truth by Jan Carlzon Chief Performance Officer Rob Hebeler, Q1 FY0708 8 Background Survey Design Survey Implementation Survey Findings Closing Remarks

Survey Design Used a 5-Point Scale to determine the level of satisfaction, and overall experience vs. expectation Extremely Satisfied 5 Satisfied 4 Neither Satisfied nor Dissatisfied 3 Dissatisfied Extremely Dissatisfied 2 Does Not Apply 1 vv Advocate Loyal Chief Performance Officer Rob Hebeler, Q1 FY0708 Indifferent/ Resentful Disappointed Mad

9 Background Survey Design Survey Implementation Survey Findings Closing Remarks Customer Experience = Service + Product Service Attributes build relationships and trust between the service provider and the customer. Its how we treat our customers. Friendly Listened Well Helpful Speed of Service Checked for my Satisfaction Chief Performance Officer Rob Hebeler, Q1 FY0708 10 Background Survey Design Survey Implementation Survey Findings Closing Remarks Customer Experience = Service + Product Product Attributes are those things, tangible or intangible, that the customer values (whether they are paying for it or not). Its what we give our customers. Available or Delivered as Promised Level of Quality Easy to Understand or Use Allowed me to Accomplish What I Wanted to Do Priced Fairly Chief Performance Officer Rob Hebeler, Q1 FY0708 11

Background Survey Design Survey Implementation Survey Findings Closing Remarks Survey Implementation For the Q1 FY0708 Survey, we targeted the 1.1 million DMS customers. We mailed a postcard to a random sample of 40,000 to ensure a statistically sound response. Of this group, 30,000 were given an incentive to participate (the possibility to win a $10 Florida State Scratch-off Lottery Ticket). Participants had the option of completing the survey online, or call a toll-free helpline Chief Performance Officer Rob Hebeler, Q1 FY0708 12 Background Survey Design Survey Implementation Survey Findings Closing Remarks Survey Implementation: Response Rate 1,124 surveys were collected for a response rate of 3%. We achieved a 95% Confidence Level with a 3.8% Margin of Error. The Response Rate for the

incentive group (Florida Lottery) was about twice that of the non-incentive group (1.7 times) Nearly one-third of those reporting that they have had contact within the past 3 months were from the 32% Retirement Division. 24% Human Resources People 17% State Group Insurance Chief Performance Officer Rob Hebeler, Q1 FY0708 13 Background Survey Design Survey Implementation Survey Findings Closing Remarks The Q1 FY0708 Survey Says Chief Performance Officer Rob Hebeler, Q1 FY0708 14 Background Survey Design Survey Implementation Survey Findings Closing Remarks DMS Overall Customer Satisfaction Q1 FY0708 Nearly half of the DMS customers indicated that they were extremely satisfied with the experience they have had with their DMS division. Overall how satisfied are you with how well your experience with met your expectations? TOP 2 BOX: 82%

46 0% 10% 20% 36 30% 5 - Extremely Satisfied 40% 4 50% 60% 3 2 9 70% 80% 6 90% 3 100% 1 - Extremely Dissastisfied %s are based on those who selected a division to rate. Those who said they had not had contact or left blank were filtered out of these results. However, when filtered back in the % only changes minimally, by less than 1%. Chief Performance Officer Rob Hebeler, Q1 FY0708

15 Background Survey Design Survey Implementation Survey Findings Closing Remarks DMS Overall Customer Satisfaction by Division Q1 FY0708 Division had the highest State Purchasing State Term Contracts percentage of top box scores (extremely satisfied ratings) followed by Retirement Benefits. (Divisions with less then 10 responses were not calculated.) %s are based on those who selected a division to rate. Those who said they had not had contact or left blank were filtered out of these results. However, when filtered back in the % only changes minimally, by less than 1%. Chief Performance Officer Rob Hebeler, Q1 FY0708 16 Background Survey Design Survey Implementation Survey Findings Closing Remarks DMS Overall Customer Satisfaction Q1 FY0708 Vs. June 2007 Benchmark Top Box, Extremely Satisfied, increased from 37% to 46% -- a 9% point increase over the June 2007 Benchmark Survey. The quarterly continual improvement goal of a 2% shift of Top Box (38.9%) was surpassed with a 14.2% shift of Top Box Overall how satisfied are you with how well your experience with met your expectations? Q1 FY0708 TOP BOX: 46% 46 36 9

6 4 June 2007 Benchmark Survey TOP BOX: 37% 37 0% 10% 20% 41 30% 5 - Extremely Satisfied 40% 4 9 50% 60% 3 2 70% 80% 7 90% 5 100%

1 - Extremely Dissastisfied %s are based on those who selected a division to rate. Those who said they had not had contact or left blank were filtered out of these results. However, when filtered back in the % only changes minimally, by less than 1%. Chief Performance Officer Rob Hebeler, Q1 FY0708 17 Background Survey Design Survey Implementation Survey Findings Closing Remarks DMS Customer Service Attributes Q1 FY0708 DMS maintained relatively high top box ratings on all of their service attributesparticularly, friendliness, ability to listen well and helpfulness. Percentage of Top Box Scores and Bottom 3 Box Scores by Service Attribute 50% 48% 48% 45% 42% 40% 50% 40% 35% 23% 15% 26%

25% 20% 17% 17% 30% 15% 10% 5% 0% Friendly Listened Helpful Top Box Chief Performance Officer Rob Hebeler, Q1 FY0708 Speed Bottom 3 Box Service Checked 18 Background Survey Design Survey Implementation Survey Findings Closing Remarks DMS Customer Service Attributes Q1 FY0708 Vs. June 2007 Benchmark Since the June 2007 Benchmark Survey, top box scores have risen in all service attributes. The largest increases were found in Service Checked, and Friendly. Percentage of Top Box Scores by Service Attribute and Quarter

50% 48% 43% 50% 48% 41% 43% 45% 42% 40% 40% 35% 33% 35% 30% 25% 20% 15% 10% * Friendly * Listened Helpful Q4 Top Box Chief Performance Officer Rob Hebeler, Q1 FY0708 Speed Q3 Top Box

* Service Checked 5% 0% * Denotes Statistically Significant Difference p < .05. 19 Background Survey Design Survey Implementation Survey Findings Closing Remarks Correlation Analysis of Customer Service Attributes to Overall Satisfaction Attribute Pearsons Correlation* (to overall satisfaction) Helpful .80 Speed of Service .76 Listened .72 Checked for Satisfaction at end of Call

.67 Friendliness .67 In general we see strong correlations between all of the attributes and overall satisfaction. Strong positive relations = higher overall satisfaction ratings. The level of helpfulness and speed of service is most highly correlated to the level of overall satisfaction, whereas Friendliness is the least correlated. *Pearson Correlation indicates the strength and direction of a linear relation between two factors. The value ranges between +1 (a perfect positive relation) and -1 (a perfect negative relation). Values greater than .7 indicate an extremely close relation between the factorsthat is, as one factor rises, the other factor rises in a very similar fashion. Chief Performance Officer Rob Hebeler, Q1 FY0708 20 Background Survey Design Survey Implementation Survey Findings Closing Remarks Quad Analysis of Customer Service Attributes A Quad Analysis of customer service attributes, we see that speed of service will be the primary area where the DMS can focus to impact better overall satisfaction ratings. 4.4 Friendly 4.3 Listened Average Performance Score

Helpful 4.2 4.1 Checked for Satisfaction Speed of Service 4.0 3.9 3.8 0.6 0.7 0.8 Im pact Chief Performance Officer Rob Hebeler, Q1 FY0708 0.9 In addition, better follow up / wrap up will also be an area to increase performance which in turn will help increase overall satisfaction. 21 Background Survey Design Survey Implementation Survey Findings Closing Remarks DMS Customer Product Attributes Compared to bottom three box scores, DMS maintained very high

top box ratings for nearly all of their product attributes particularly, Availability and Quality. The one exception was for Price Value, here the percentage of top box scores was nearly Percentage of Topbox Box Scores and Bottom 3 Box Scores by Product equal to bottom three scores. Attribute 49% 49% 48% 49% 50% 45% 40% 37% 35% 30% 30% 25% 16% 10% 20% 14% 15% 12% 10%

5% 0% Available Quality Easy Understand Top Box Chief Performance Officer Rob Hebeler, Q1 FY0708 Allowed to Accomplish Bottom 3 Box Price Value 22 Background Survey Design Survey Implementation Survey Findings Closing Remarks DMS Customer Product Attributes Q1 FY0708 Vs. June 2007 Benchmark Since the June 2007 Benchmark, top box scores for product attributers increased across the board. The largest increases were found in Quality, Allowed to Accomplish and Easy to Understand. The smallest increase was for Price Value. Percentage of Top Box Scores by Product Attribute and Quarter 49% 49% 41% 49% 48% 39%

50% 45% 40% 38% 40% 37% 34% 35% 30% 25% 20% 15% 10% * Available * Quality * Easy Understand Q4 Top Box Chief Performance Officer Rob Hebeler, Q1 FY0708 * Allow to Accomplish Q3 Top Box 5% 0% Price Value

* Denotes Statistically Significant Difference p < .05. 23 Background Survey Design Survey Implementation Survey Findings Closing Remarks Attribute Correlation Analysis of Customer Product Attributes to Overall Satisfaction Pearsons Correlation* (to overall satisfaction) Quality of Product .83 Allowed customer to Accomplish Goal .82 Easy to Understand .81 Availability .79 Price Value .66 Again, we see very strong correlations between all of the attributes and overall satisfaction. Strong positive relations = higher overall satisfaction ratings.

With the exception of Price Value, all products attributes correlated with overall satisfaction at or above .79. *Pearson Correlation indicates the strength and direction of a linear relation between two factors. The value ranges between +1 (a perfect positive relation) and -1 (a perfect negative relation). Values greater than .7 indicate an extremely close relation between the factorsthat is, as one factor rises, the other factor rises in a very similar fashion. Chief Performance Officer Rob Hebeler, Q1 FY0708 24 Background Survey Design Survey Implementation Survey Findings Closing Remarks Quad Analysis of Product Attributes A Quad Analysis of product attributes we see Price is a secondary Opportunity and there is no clearly defined primary opportunity. 4.5 Strengths Maintain 4.4 Available Quality Allow 4.3 Average Performance Score Easy 4.2 4.1

Price 4.0 3.9 3.8 Secondary Opportunities Primary Opportunities 3.7 0.6 0.7 0.8 Im pact Chief Performance Officer Rob Hebeler, Q1 FY0708 0.9 Since Price has the least impact on overall satisfaction and there is little to no difference between the other attributes, the best advice to give would be to focus on attributes that are the easiest to and least costly to change. 25 Background Survey Design Survey Implementation Survey Findings Closing Remarks Incidence of Problems with DMS Services

Of the customer that had contact with DMS within the past 3 months, only 12% reported that they had experienced a problem. Of those, roughly nearly half (45%) reported it to the DMS. Importantly, when asked about the satisfaction of the resolution they received, nearly half (45%) stated they were very or extremely dissatisfied. Ratings for Resolution after Experiencing a Problem and Reporting it to DMS 45% 45% 40% 35% 30% 29% 29% 26% 25% 26% 20% 16% 15% 16% 10% 5% 0% Top 2 Chief Performance Officer Rob Hebeler, Q1 FY0708 5 4 3 2

Satisfaction with Outcome 1 Bottom 2 26 Background Survey Design Survey Implementation Survey Findings Closing Remarks Customer Suggestions for Improvement of DMS The majority of customers (58%) suggested organizational changes to improve DMS (improvement of interdepartmental communications, etc.). Lagging far behind, but still ranked second, was Services (how customers are treated). Relative Percentage of Improvement Comments by Type Technology, 11.0% Categories for Improvement Organizational: Relations between departments, people, merging departments, leadership etc. Process, 8.6% Products: Tangible/Non-Tangible reference to something they receive or expect to receive. Services, 13.8% Organization, 58.3% Products, 8.1% Chief Performance Officer Rob Hebeler, Q1 FY0708 Services: How customers are treated. Process: Efficiencies in day-today operations.

Technology: increase efficiencies and means of getting product or service. 27 Background Survey Design Survey Implementation Survey Findings Closing Remarks Problems that Customers have Experienced* Password problems; information gathering problems; unable to find desired topics. Not able to get all info from one place (person) had to call another number to get info. Website is terrible. 2 months to make Drs appointment need to speed things up. People First is not user friendly requiring frequent calls to obtain help. getting hold of someone to talk to -- could be 3 days before a return call. Open enrollment through People First was much more difficult than it should have been. Web-based training videos did not function until late in the process. * Note: Unedited Customer Comments Chief Performance Officer Rob Hebeler, Q1 FY0708

28 Background Survey Design Survey Implementation Survey Findings Closing Remarks Comments that Represent Suggestions for Improvement from Customers*: Staff should be better informed; if they cant answer a question they should know the correct office to forward the person to. Keep web pages updated. Call center needs more employees.. Easier searches on data portal Have varied office hours so that you can conduct business after 5 p.m. and on Saturdays. Provide printed materials stating necessary information. Too many different agencies doing same things. Retirement and investment had to go back and forth too much. More inter agency communications. It is sometimes hard to know what each agency/division is thinking/doing. We all have personnel with strengths/weaknesses. If we

communicate better we can leverage/overcome this. * Note: Unedited Customer Comments Chief Performance Officer Rob Hebeler, Q1 FY0708 29 Background Survey Design Survey Implementation Survey Findings Closing Remarks Whats Next Chief Performance Officer Rob Hebeler, Q1 FY0708 30 Background Survey Design Survey Implementation Survey Findings Closing Remarks Rob Hebeler, Chief Performance Officer State of Florida Department of Management Services (850) 487-9887 office, (850) 491-2095 cell phone [email protected] We Serve Those Who Serve Florida DMS E-Mail Based Pulse Survey Click on the Survey Link brings you to our Web-based Customer Survey Chief Performance Officer Rob Hebeler, Q1 FY0708 31 Background

Survey Design Survey Implementation Survey Findings Closing Remarks DMS Performance Scoreboard Service, Product and Problem Resolution Performance Top Accomplishments Benchmark June 2007 Q3-07 PULSE Q3W2 Cum 60% 50% 65% 55% 45% 50% 35% 50% 65% 55% 45% 50% 50% 35% 62% 49% 65% 57% 47% 52% 34% 49% 65% 57% 47% 52% 52%

34% 62% 51% 66% 55% 48% 53% 35% 51% 66% 55% 47% 49% 49% 37% Satisfaction - Experience vs. Expectation Customer Service Friendly Listens Well Helpful Speed of Service Checks for Satisfaction Product Performance Available or Delivered as Promised Level of Quality Easy to Understand or Use Allowed me to Accomplish What I Wanted to Do Price Fairly Problem Resolution Satisfaction Department of Management Services Priorities Implement best practices to create a workplace of choice that fosters recruitment, development, recognition and reward Improve our contract management capabilities Create robust strategic plans for each major program and service area Focus on the processes and procedures of our core competencies to create a springboard for world-class performance Increase the brand awareness of our programs and services to our customers Develop Human Resource practices that encourage a resilient workforce with the ability and desire to serve in an emergency or natural disaster Measure what matters and continually improve the quality of services delivered to our customers

September-2007 Saved Implemented . Recovered Established Designed Built Communicated Our Alignment & Focus Our Motto We Serve Those Who Serve Florida Service Vision Engaged Employees; Satisfied Customers Service Mission Providing Smarter, Better, Faster Services Service Promise Communicate Concerns Immediately Listen, Learn and Grow Together Act with Integrity and Honor Strive for Greatness Serve with a Servant's Heart Key: Green = We're Getting Better; Yellow = We are "Just Okay"; Red = We are falling behind the Customer's Expectations Chief Performance Officer Rob Hebeler, Q1 FY0708 32 Customer Quarterly Survey Schedule Q2 FY0708 Launch: Mid-January; Results; Mid-February Q3 FY0708 Launch: Mid-April; Results; Mid-May Q4 FY0708 Launch: July; Results; August Chief Performance Officer Rob Hebeler, Q1 FY0708 33 Background Survey Design

Survey Implementation Survey Findings Closing Remarks Closing Remarks Overall, DMS is doing a good job on product and service aspects. Indeed, based on the improvement in scores from the benchmark to Q1 FY0708, DMS is getting better! Overall Customer Satisfaction is up 9% points over the June 2007 benchmark. Increasing gains were evidenced in nearly every service and product attribute from the June 2007 Benchmark Survey to the Q1 FY0708 Survey. Customer service attributes of helpfulness and speed of service were found to be the most strongly correlated to overall satisfaction. Simply put, customers want to be helped and helped quickly. When it comes to products, quality and allowing customer to be able to do what they set out to accomplish were the most highly correlated with overall satisfaction. With the exception of product price, all product attributes correlated with overall satisfaction at a value greater or equal to .79. Chief Performance Officer Rob Hebeler, Q1 FY0708 Only 12% of respondents reported having

had a problem with DMS. However, when customer problems did arise, problem resolution appeared to be an issue. Nearly half of those who experienced a problem and reported it, said that they were dissatisfied or extremely dissatisfied with the resolution. Quad analysis identified speed of service as a service attribute that is an primary area for changes to be made. With regard to product attributes, no single attribute emerged as a primary candidate for change; therefore, DMS might focus on those attributes that are the easiest to change. Using the information gathered from the open-ended comments about how DMS might improve, 58% of those who responded mentioned organizational changes or improvementsinterdepartmental communication, knowledge, etc.). 34 DMS Quarterly Customer Satisfaction Survey December 2007 Chief Performance Officer Rob Hebeler, Q1 FY0708 35

Recently Viewed Presentations

  • Simulation Modeling and Analysis - SHAMSUL SARIP

    Simulation Modeling and Analysis - SHAMSUL SARIP

    The Simulation Project Simulation Project Steps a.- Problem Definition b.- Statement of Objectives c.- Model Formulation and Planning d.- Model Development and Data Collection e.-
  • What is the RowanCard? The RowanCard is your

    What is the RowanCard? The RowanCard is your

    Know what your ISO card number is. The first 12 digits are printed on the back of your card. The final 4 can be found at id.rowan.edu. Each time you get a new student ID, your card number will change....
  • Monopoly (Ch.10)

    Monopoly (Ch.10)

    Gamble B: .10*5million + .89*$1million +.01*$0. The Ellsberg Paradox. Suppose you have an urn containing 30 red balls and 60 blue and yellow balls. You don't know how many blue or yellow balls there are (only that the sum of...
  • Study Abroad / Study Away Definition Academic experiences

    Study Abroad / Study Away Definition Academic experiences

    Welcome to today's Enriching Experiences icebreaker. We're going to take a few minutes to learn a little more about each other and a little more about one of the Enriching Experiences that you may have heard about as part of...
  • CGL Treatment Journey - Open Objects Software LTD

    CGL Treatment Journey - Open Objects Software LTD

    Agreed 1-2-1 interventions including, motivational interviewing, BSFT, CBT, ITEP Mapping. Weekly contact with activity coordinator including GYM, Music, Bike ASDAN. Relapse prevention & Closure . Agreed 1-2-1 interventions around relapse prevention, including triggers and high risk situations.
  • Bayesian causal phenotype network incorporating genetic ...

    Bayesian causal phenotype network incorporating genetic ...

    Bayesian causal phenotype network incorporating genetic variation and biological knowledge . Brian S Yandell, Jee Young Moon. University of Wisconsin-Madison
  • Body Paragraph Structure - please copy everything in green!!

    Body Paragraph Structure - please copy everything in green!!

    Body Paragraph Structure - please copy everything in green!! Expectations for Evidence: Minimum of three pieces of evidence per paragraph. Quotes from the novella. Events from the novella (preferably in quotes) How to quote:
  • Safety Update - National Interagency Fire Center

    Safety Update - National Interagency Fire Center

    2008 Update Safety and Health Working Team Overview Charter signed February 2008 Tactical Plan completed May 2008 Goals Subcommittees Tactical Plan Accomplishments Futuring * Safety and Health Working Team Mission Work with the appropriate State and National EMS offices and...