| Client | Skills | Project Dates | What is | URL |
|---|---|---|---|---|
Literacy Minnesota (Northstar Digital Literacy) | UX Research - Discovery WorkUX Research - Test PlanningUX Research - Facilitation/ModerationUX Research - Compiling Findings/DataUX Research - Presenting Findings/Report | November 1, 2017 → August 18, 2018 | What is Literacy Minnesota? Literacy Minnesota is a nonprofit dedicated to sharing the power of learning through education, community building, and advocacy. What is Northstar Digital Literacy? Northstar Digital Literacy Assessments helps individuals around the world master the digital skills needed to work, learn, and participate fully in daily life. |
Literacy Minnesota - Northstar Digital Literacy Assessments
The aim of this user research project was to gather feedback from learners to better design and future-proof Northstar Digital Literacy Assessments (a testing tool used by Literacy Minnesota and their nonprofit partners) to meet the growing needs of Twin Cities Metro communities.
Challenges
To gather, distill, and present feedback from learners, helping to improve Northstar Digital Literacy Assessments.
Design Process
Since I was leading a User Research Project, my research process followed the below process:
- Discovery (Ethnography and Contextual Inquiry)
- Qualitative (User Interviews)
- Sharing User Feedback with Literacy Minnesota
- Presenting our Project Results and Post Project Work
"We’re very excited about your project and testing. We’ve been looking forward to updating the Assessments for a long time!"
- Feedback from nonprofits in August 2018, during our Americorps Year-End Capstone Presentation
What are the Northstar Digital Literacy Assessments?
The Northstar Assessment website features various online assessments/tests covering a range of Digital Literacy topics.
These tests were created to address a growing need among community organizations to assess or improve the digital literacy of job seekers. By passing these Assessments, job seekers can earn certificates signifying their digital literacy and software knowledge to employers.
Assessing the Design and Technical Problems (Discovery)
From September 2017 to August 2018, I served in Americorps as part of a Community and Technology Empowerment Program headquartered at Saint Paul Neighborhood Network (SPNN) in St. Paul, MN.
While working at my own Americorps site (Ramsey County Library in Shoreview, MN), I quickly noticed the design and technological scaffolding of the Assessments were long overdue for a future-proof upgrade.
In recent years, the Northstar Assessments became increasingly difficult to support due to outdated technology. In late 2017, Northstar developers initiated a general redesign of 10 tests.
In addition to the needed technology updates, each test/interface would feature a newer, more modern UI and updated content.
The screenshot on the right is a snapshot of what the old Northstar Assessment test interfaces looked like in Fall 2017. They
- Had outdated graphics/buttons/user interactions
- Were made with and relied on Adobe Flash Software, which isn’t always reliable for all testing locations
- And aren’t responsive and can only be used on desktop
My Role in the Project
As part of Americorps's Community Improvement project, I advocated to lead a team of Americorps members to help Literacy Minnesota update the Assessments by gathering and analyzing feedback from people using beta versions of the Digital Literacy Assessments at the nonprofits where we served.
My role involved leading three other colleagues (each working at their own nonprofit in either Minneapolis or St. Paul) in carrying out and gathering feedback on how learners used the new test interfaces, pinpointing pain points, recording bugs and other duties as the project progressed.
I led or assisted with facilitating in-person user feedback testing once or twice a week at PPL (Project for Pride in Living in Minneapolis) and GAP (Guadalupe Accelerated Programs in South St Paul), depending on class and testing schedules. I also led Google Hangout meetings with my team twice a month to check in on goal progression.
Speaking to People Taking the Assessments (Discovery)
During our research at our nonprofit locations, we met users who had different learning needs, had different occupations and therefore wanted to take skills tests for various reasons (to complete their education, gain new skills, etc).
Below are personas I created that reflect the variety of people we talked to and gathered insights from during our research period:
Our questions in our testing sessions included:
- Does the information in the test interface match what’s being taught in the computer class you’re taking?
- Are you able follow the interface’s instructions when using keyboard keys?
- Are you experiencing any bugs while using the Interface?
Sharing Our Findings with Literacy Minnesota (Iteration)
We used Google Forms to send our user feedback data back to the development team at Literacy Minnesota (the nonprofit that hosts Northstar Digital Literacy Assessments) in order to make needed improvements to the tests’ User Interface.
To the left is a screenshot example of one of the test interfaces our users gave us feedback on. This screen asks users to fill out a shipping form for an online order.
The design of the form reset button was located too close to the form submission fields and users were confused when clicking on the Reset button when they wanted to Submit their address.
We made the suggestion to move the Reset and Submit buttons around to prioritize the Submit button under the form fields and move the Reset button off to the side as the previous location of the Reset button was too similar to a Submit or Next button on most address web forms.
Our Results: Did We Achieve Our Feedback Goal?
The results of our user testing project included
- 3-5 UI Elements Changed for an average Assessment
- 54 total user tests (meeting our goal of conducting 15-20 user tests per Assessment)
- 8 month project timeline
A Snapshot of the new Northstar Digital Literacy Assessment Interface Features:
1. An improved textual and visual hierarchy – the text and headings are sized and placed more effectively than the previous interface
2. New buttons on the top right of the interface allows users to repeat the question audio and the hamburger menu lists questions the user has answered or has yet to answer
3. The area below the interface has new features: a higher contrast progress bar, an "I Don’t Know" button which skips over a question and new Closed Caption, Mute and Fullscreen buttons
Thanks to our tester's feedback, each Test Interface had 3-5 UI elements changed.
Presenting Our User Research Project Results
In August 2018, as part of our capstone presentation, we presented our project, findings, and recommendations to Literacy Minnesota and a group of nonprofit leaders participating in Americorps.
We got rave reviews and hopeful feedback to continue the work across all Northstar Assessments!
What Happened After Our User Research Project
Our user testing gave Literacy Minnesota the feedback they needed to proceed with design and tech changes applied to existing and future Digital Literacy Assessments (with a total of 15).
To view the current Digital Literacy Assessments, you can click the link below:
Takeaways
Looking back, I’m extremely happy that I saw the opportunity to help improve the Northstar Assessments, turned the problem into a project part of the Americorps service, found a few Americorps members that were excited to work on the User Research project, and lead and helped them in gathering the feedback (semantic, not technical) we needed and very much valued.
Being flexible in leading the project and adjusting testing as needed as schedules changed also grew my leadership skills. This project helped me grow my user advocacy and team leadership skills in ways that are hard to put into words.
The most amazing takeaways from this User Research project were
- Seeing how the design updates to the Northstar Assessments resonated so much with people learning new skills.
- Inspiring one of my Americorps team members to either think about the users or people using technology and another to enroll in a Master’s program for Neurology and how technology affects the brain.
Such takeaways will continue to inspire my research work and methods for years to come!
