State Performance Case Study

Project Activities:

• Conducted a gap analysis to identify areas of state responsibility that are not addressed in the SPSS.  
• Developed interview and survey instruments to assess CMS regional office staff and state survey agency staff impressions of the SPSS.
• Interviewed and conducted surveys of CMS regional office staff and state survey agency staff to assess their perspectives of the SPSS. 
• Analyzed interview and survey data and compiled findings from the data.
• Conducted data analysis using survey data, complaint/intake data, and quality of care (data.medicare.gov) to evaluate the disconnect between survey results and quality of care.
• Conducted a pilot test of revised guidance for documentation of deficiencies among HMS subject matter experts.
• Compiled gap analysis, interview, survey, data analysis, and pilot testing results in an evaluation report of the SPSS.
• Develop a recommendations report for revision of the SPSS utilizing the evaluations report.
• Revised SPSS structure, revised old SPSS measures, and created new measures per the recommendations report and input from CMS Central Office. 
• Conducted beta testing of two SPSS measures, one that evaluated documentation of deficiencies and another that evaluated complaint intake prioritization.  
• Mined QBIC data and developed SPSS performance indicator measures. Created SQL program to create measures from mined datafiles. 
• Developed mocked-up dashboards for SPSS measures.
• Presented at the Association of Health Facility Survey Agencies and the CMS Survey Executives Training Institute.

Deliverables:

• An evaluation report of findings from all gathered data interviews and surveys.
• A recommendations report that included details on restructuring the SPSS and a revised set of SPSS performance measures.
• A SPSS database data collection tool built on HMS SharePoint to collect measure data.
• An expanded SPSS database data collection tool built on CMS SharePoint.
• State Performance Indicators Report generated for all states to evaluate various aspects of state survey agency performance.
• A mocked-up dashboard of SPSS measures.
• A revised SPSS guidance document incorporating SPSS revisions.
• IIDR review recommendation findings submitted to the government.

HMS Demonstrated Value

• Year One: Evaluation. HMS subject matter leaders met with state survey agencies and CMS offices around the U.S. to gather information and get their input on how the SPSS was or was not working. We used a combination of in-person interviews and a questionnaire to collect this feedback. HMS then conducted extensive data analysis, comparing the results of the SPSS both to stakeholders’ feedback and to other data that was indicative of states’ performance, and worked to reconcile conflicting information and perspectives.
• Year Two: Recommendations. HMS determined that the core tasks and processes involved in the SPSS were being performed using outdated tools and inefficient methods which were prone to human error. We recommended replacing Excel spreadsheets with a new system on CMS’ Sharepoint site which auto-generated information and facilitated the processes involved in SPSS’ three domains (Frequency, Quality and Coordination of Provider Non Compliance), thus saving CMS staff time.
• Year Three: Implementation. HMS implemented our recommendations. For example, we began the process of auto-generating survey-frequency information to take the burden off of CMS offices. COVID unfortunately slowed down the whole process, shifting the focus of CMS offices and states to infection control surveying. We also focused on the quality domain to ensure that complaint investigations, surveys and documentation were being properly prioritized and completed appropriately.

Agencies Involved

• Centers for Medicare and Medicaid Services (CMS): CMS combines the oversight of the Medicare program, the federal portion of the Medicaid program and State Children’s Health Insurance Program, the Health Insurance Marketplace, and related quality assurance activities.
• State Agencies (SAs): Each U.S. state and territory has an agency with responsibility for oversight of healthcare facilities, with reporting responsibilities to CMS. HMS provides technical assistance and other services to these state agencies.