Usability Testing Results - CHEFS Architecture Forum
- Shannon Farguson
Executive Summary
Our usability testing of the CHEFS form tool revealed significant user experience challenges that are impacting efficiency, user satisfaction, and adoption. Users consistently rated the system's intuitiveness between 3-5 out of 10, indicating substantial room for improvement. A 5-minute form took users an average of 18 minutes to complete—a 260% increase in expected completion time.
Testing Overview
Tool Tested: CHEFS (Common Hosted Form Service)
Similar Products: Microsoft Forms, Cognito, Typeform, Jotform, Formstack
Testing Method: User observation and feedback collection from 7 usability testing sessions
User Ratings: Intuitiveness scored 3-5 out of 10
Key Findings: Impact on Business Value
Time Inefficiency
Critical Issue: A simple 5-minute form took users an average of 18 minutes to complete
Business Impact:
Estimated 260% productivity loss across government departments
Increased frustration and reduced adoption
Higher support costs due to confusionNavigation & Wayfinding Failures
- Users couldn't locate essential functions (share link, form settings, export settings, manage form, home page)
- BC Government logo - Navigation confused users, when clicked, the Bg Gov logo took them out of the application entirely
User Quote: "The buttons never feel like they're in the most sensible place"Form Settings & Permissions Confusion
- Major confusion about access settings (Public, IDIR, Team Members)
- Uncertainty about permissions and deployment levels
Business Impact: Security risks from improper permission settingsLack of Guidance & Feedback
- No clear path through the form creation process
- Unhelpful error messages led to user abandonment
User Quote: "This is where I would give up" (after encountering unhelpful error message)
User Quotes: "I was looking for guidance, needed a natural roadmap, the flow was confusing, it wasn't linear"Technical Terminology Barriers
- Technical terms created barriers for government staff users
- Components lacked previews, forcing trial-and-error approach
Business Impact: Increased training costs and support requests
Cost of Inaction
- Continued low adoption rates across the government
- Increased support costs as users struggle with basic functions
- Time wasted on form creation (260% longer than necessary)
- Potential security issues from confusion around permission settings
- Damage to the digital transformation reputation within the government
Recommended Next Steps
- Prioritize a UX redesign focusing on navigation, information architecture, and core user flows
- Consider implementing an optional guided wizard for first-time users (directly requested by users)
- Improve error messaging to provide actionable solutions
- Simplify technical terminology throughout the interface
- Conduct follow-up testing with improved designs
Success Metrics
- Reduce form creation time by at least 50%
- Improve intuitiveness ratings from the current 3-5 range to 7+ out of 10
- Decrease support requests related to basic functionality by 40%
- Increase adoption rate across government departments
Conclusion
The form-builder in CHEFS is not the primary issue. Instead, most new users struggle with fundamental usability challenges, including navigation and wayfinding, configuring form settings, and understanding what to do with their forms after creation.
These challenges hinder productivity and adoption. By resolving them, we can drive significant ROI through time savings, lower support costs, and increased usage across the government.
