Marketing Funnel & Web App
How might we reinvent the Ivy League model?
When I joined the Minerva project to lead product design for the schools team, my first challenge was to build an admissions process that would admit the first full class of students.
At Ivy League Universities 50% of students come from wealthy families. We wanted to build a merit-based admissions process that was highly selective but not biased in favor of the world's most wealthy students. How might we reinvent the Ivy League model?
Why this problem?
Pain points
Many college admissions processes were biased towards wealth. Selective universities often ruled out out talented students for any number of reasons.
To many students college admissions was presented more as an achievement than a decision making process.
Tuition cost were rising and many college graduates had significant debt.
Opportunity
The university was able to offer a tuition that was many times lower than other top schools. Because the university used online teaching technology, the company cut many of the costs of a physical campus and was able to recruit high quality instructors at a reduced cost.
The company thought that many high quality students were overlooked by the most selective univesities and there was an opportunity to desiging an admission process that was both selective and inclusive.
What we already knew
Rather than rely on standardized tests, a team of university experts built a rigorous, custom admissions processes and a number of custom online assessments from scratch. The company had a reliable way of measuring learning outcomes and validating the quality of the admissions process
By offering a year of free tuition, the company was able to attract a small first class of high achieving students. But this wouldn't be suistanable as the university scaled up.
One of the founding business ideas was to build a University that could scale to be many times larger more quickly. With out the physical constraints of a campus, the goal was to triple the class size each year.
What we didn't know
Even if the team built a custom admissions process, we didn't know if students would do much more work for a rigorous and unconventional admissions processes.
An established industry of indpendent guidance counselors supported students through a predictable and standardized processes. Why should applicants do much more work for an unknown and unproven university?
Scope: 18 weeks to build an application funnel, analytics and singe page responsive web app with proctoring software and custom assessments.
12 weeks to build a native app.
Role: I led product design and project management as an individual contributor.
Team: I led product design along with 2 PMs and 12 developers.
Type: UX, task flows, story mapping, prototyping, wireframes, mockups, project management.
Tools: Sketch, HTML prototyping, front end frameworks, InVision, Zeplin.
The opportunity
I decided to approach the problem from several angles:
- Reframing the UX: In American schools a single common application can be used for most universities. Students would be reluctant to do extra work to apply to an unproven school. I wanted to design a UX that was effective at reframing the extra effort as worth it.
- Data-driven design: I would design a modular admissions process that was easy to tweak and optimize with testing. We would need many more applicants to get the same enrollment yield as established universities. The best way to do this would be to relentlessly test out new ideas & optimize.
- 10x thinking: If our notification process was 10x better than other schools maybe we could achieve the same enrollment yield. Any increase would be a huge multiplier. With special campaigns and alternative application flows aimed at high-value applicants we might be able to convert 10x as many app starts to admits.
How might we reframe the extra effort as our application process as a series of challenges?
Planning
I worked one cycle ahead of the dev team and got started by pairing with a front end developer to create a component library for the application. We built the styles and components with an atomic based css library in the first two weeks.
After that, I created task flows and wireframes of the admission process. Because of the design system and companion front end framework I could quickly prototype an MVP in HTML. For the MVP it would be good enough to build a UI around the simple metaphor of a task list, with all of the application steps. If we had extra time, we could launch an improved design later for the regular admission deadline.
Now that I had a an html prototype of an easy mvp we could fall back on, I shifted my focus to some aspirational designs that might make a big difference. To get user research started, I scheduled weekly remote interview sessions with current college students as proxies for applicants. In these interviews I reviewed and iterated on prototypes of more aspirational application flows.
Prototyping
I prototyped a redesigned application flow with a more user-centered experience that was structured around the following prompts 1. Who you are 2. How you think 3. What you have achieved.
It was hard to know from this qualitative research whether reframing the assessments as personal challenges would help with motivation. But the interviews helped me focus on a few questions that I wanted to explore further.
Would offering a practice test make applicants more likely to begin the first assessment?
If reluctant applicants took the first assessment sooner, would they be more likely to complete the application? or more likely to delay or quit?
Prototype test compilation (4 min). I reviewed and iterated on my protype ideas across weekly user testing sessions.
I started by iterating on alternate user flows that would allow students take the assessments in any order and created practice flows for each assessment so applicants could mentally prepare for the next step.
There were a number of other changes I wanted to make to the mvp relating to application choice & control that came up during user research. I decided to a/b test these ideas after launch and during the early admissions application cycle
In the final three weeks, I set up a session playback service to capture actual flows through the website and new app once we was we built it.
Some of the prototypes that explore how an applicant might navigate from one challenge to another.
Prototype for applicant-centered themes of challenge and self-discovery (left) vs the mvp design (right) which was focused on the checklist metaphor and related to our internal need to collect information.
Research
After the first of our three admission deadlines for the year, we reviewed session playback recordings and interviewed students from the founding class to create a number of story maps. We found a number of obvious problems that related to the same theme— because many of the application requirements were presented linearly in a forced flow, it was easy for students to get stuck.
In the end, we noticed some common areas where people were getting stuck. 1. Students expected to write a college essay and were confused by the accomplishment section. 2. Our two part application wasn't clear to many students who ran out of time to complete the application processes. 3. Many students weren't willing to take the online assessments unless they had an opportunity to prepare.
These findings were backed up by the admissions funnel numbers, which showed much lower conversion rates at the gap between two part application and right before taking the assessments.
As a result, we decided to explore the UX I prototyped earlier that would allow applicants to customize and control their application experience so they could become unblocked and choose the right task at the right time.
I worked with the team to ideate and iterate on new user flows so that we could begin to explore some of our ideas against our shared understanding of technical constraints and get ahead of any back-end challenges or refactoring that might be necessary.
We considered a number of changes including: a. offering practice of the online assessments without first requiring technical setup of online proctoring. b. allowing students to take the assessments in any order.
We began to explore and iterate on alternate user flows that would allow students take the assessments in any order and created practice flows for each assessment so applicants could mentally prepare for the next step.
Story map of the application process.
Flow diagram of the application.
Polish
In planning we budgeted some time for top-of-funnel experiments after the application launch and I wanted to build on some of what we learned from the application experiments.
I wanted to create an alternative application flow that started with the assessment challenges, even before asking contact information, grades, transcripts, and other application details.
For this campaign project, I led product design and UX/UI and worked with a brand designer, a visual designer, and 2 engineers. We worked closely across weekly stand-ups with stakeholders from applicant outreach and marketing teams for about 3 weeks.
To help the group better align on the problem space, I suggested a key outcome. To increase the percentage of students who take the first challenge within one week of beginning the application.
Alternative entry point to application for mobile.
An alternative entrypoint to the application.
Outcome
Of the changes that we tested, many of them were associated with a 2x-6x boost and were eventually shipped including 1. repositioning the first part of the application as an account creation step 2. asking applicants to first list a few accomplishments rather than write about one of many (which they previously mistook for an essay). 3. allowing applicants to practice any of the assessments (now called challenges) before starting.
We tested a few other ideas that turned out to be ineffective— such as full screen cards to suggest the next course of action.
For the second six week cycle, we continued to optimize the application and became confident that we were converting at a much higher rate. In the end we met our goal to increase the class size by 3x each year.
This small campaign experiment yielded a some promising results. 1. of 20,000 cards mailed nearly 6,500 visited the website and began the application flow. 2. of the 200+ complete applications in this campaign over 20 students were admitted. These numbers we're 3x-5x higher the the traditional application path. If even some of these learnings applied to the larger application group we could increases our yield 2 or 3x.
We also spent 4 weeks re-imaging the notification process in an attempt to make in 10x better. In the end, I'm not sure this was worth the time we spent on it. On the one hand we heard back from students that the notification experience was magical. On the other hand it was hard to measure the impact and the solution we came up with didn't scale well.
Since then, the university has been called The Future of College, described as What College Should Look Like, and credited with Reinventing the Ivy League Model.
Focused application entry card proposal. (4 mins). We a/b tested this but decided not to ship.
Application redesign proposal for the product team (6 mins). We decided not to build this.
More Case Studies
