Lesson Learned: How I Conducted a User Research for the First Time

Back story

The product I was doing a research for is on the earliest stage possible – an idea. I wanted to build something that will help new immigrants in Israel keep up with their career path. I believed that this product would have a strong demand as it’s a real problem for a lot of immigrants in Israel – to find job that would match their previous job in the country of origin.

You see, immigration to Israel mostly differs from immigration to other countries. Often people immigrate to other countries through a job offer, so they have a bit of stability at a new place. With Israel it’s different. But most of us who move to Israel often move to the uncertainty. The journey goes like this: we come to Israel, go to language school (ulpan) and then we’re on our own.

As a result people who have a lot of experience and developed skillset struggle to find a relevant job and start to work at low-skilled jobs.

I had some assumptions of how my product should look like and what information it should contain but I tried to set them aside and get an information from real people who face difficulties in their job search process.

Process (step + lesson)

The goal of the study

I started by establishing the goal of my research. I needed to confirm the need for such a product and find out the issues and pains people are facing along the way.

Key questions I wanted answers for:

• What are the needs and pains?
• Is the product relevant at all?
• Which criteria people use when choosing the job?

Lesson learned:

Establish overarching key questions at the start of the study. That way all of your interview questions will be on-point.

Before I just listed out the questions for respondents from my head, based on common sense and assumptions. The results I got with these interviews weren’t as useful as they could be. Having base questions made the process of writing interview questions much easier and more effective. You just take your base question and evolve it and then put all the questions you got in the logical order.

Example:

Key question

What are the needs and pains?

Evolving

What are the problems that people are facing when starting their work path in Israel after aliya? What do they need most during this period? Do they get help?

End concrete questions

1. What influenced your job choice here in Israel? Why did you choose this job?
2. What would you prefer to do for a job instead of a current one? Why?
3. Do you get any support or help regarding job search or choice?

Additional materials

Research plan (opens in Google Docs)

‍

Recruiting

I had a particular topic and a rather specific group I wanted to talk to: only people with higher education and more than 3 years of experience, who moved within a year, not more, for their view to be as fresh as possible. I had to use a screener survey to select as many relevant participants as possible.

I wrote recruiting posts in several immigrant groups for Russian and English speakers. In this post, I explained the purpose of an interview and gave a link to a screener survey.

After a couple of days, I’ve got the list of those who were willing to participate and started interviews.

Lesson learned:

This one is short. You need to collect twice more leads then you need, especially if you don’t use incentives. A lot of the people will be screened out, and also some people who expressed a desire to participate won’t be actually participating.

Example:

34 filled out the form

15 were screened out. Reasons: no job experience, moved more than 1 year ago, doesn’t have higher education.

4 didn’t reply to the follow-up invitation.

2 interviewees didn’t give any relevant information for different reasons.

Additional materials:

Screener survey (opens in Google Forms)

‍

Interviews

As it’s corona-time, I had to do online-sessions, and the simplest way for my participants was a Whatsapp call. I worked out the most effective workflow to have results of each interview right away but also not to hold up my respondents:

1. I called, turned on my speaker, turned on an audio QuickTime recorder on the laptop (I always asked the permission to record n the very beginning of the talk).
2. We talked for about 20-25 minutes. That time included some off-topic discussions that occurred sometimes. The only notes I made during the interview – were my own “AHA moments.”
3. After the end of the interview, I transcribed the audio into text.
4. Then I reread the text and took out useful chunks. I stored these chunks as a kanban board, where each column represented a separate theme.

‍

As I’m just the one person and even more-so a person without any formal experience in interviewing, it took me a while to finish the process. I’ve managed to get 6 interviews and then had to move on, but I think that the best number would be 10 or even 12 people.

By the way, while working on the interview, I was given this useful video exactly about why it’s a good idea to interview 12 people.

Lessons learned:

You need to ask all the questions, and you need to make sure that you’ve understood the reasoning behind each answer.

I guess if you’re like me and doing it for the first time, you’ll feel the need to skip some of the questions you’ve pre-written. Sometimes out of politeness, sometimes you feel you understand what this person would answer. But an interviewer is an impartial observer. You can evolve your questions in the process; you can add some questions if you feel like the timeline allows it. But you can’t skip.

The most important question is the “WHY?” question.

Always dig for the reason a person feels or experiences this. Initially, you need to remember that while composing your questions in the beginning. But also, during the interview, unplanned turns of discussion can happen, and you need to act in the moment.

Example:

Question: If now you had a choice – to contact a paid career consultant personally or use a free comprehensive website on different professions and their features – what would you choose. (question base: to establish the relevance of the problem)

Answer: Actually, the site would be useful.

And then this person started giving me some idea of what could be on that site. And after 3 minutes of this (somewhat useful) discussion, it was a bit weird for me to return and to ask, “And why did you choose the site?”

Result: I don’t understand why he would choose the site over the consultant.

Additional materials:

Notion Board Template

‍

Analysis

As I’ve already processed each interview, I had all the meaningful quotes pre-sorted. Now I had to synthesis all these results into conclusions. I worked in stages:

• I assembled all the quotes into one big kanban with the same columns, rechecking on the way if I want to move some statements to the other column, rename some columns altogether or separate some statements into the totally new column.

• I transferred those materials into Miro, so each quote would be on the separate sticky note, and the sticky notes with the column names would be a different color.

• I looked at each column and formulated a summary of all the quotes that are there. For example, in one column, I see statements regarding the fact that a community that surrounds people talks in their native language, so they can’t practice Hebrew there. My summary was – “It’s hard to practice Hebrew because some of the cities are Russian.” Those summaries replaced the column names.

• After a break for mind refreshment, I rechecked my groupings and made some changes.

• I added a new “floor” that grouped the summaries by theme: issues with language, unfamiliarity with the job market, desire for education, etc. These I also formulated as insights: so instead of “unfamiliarity with the job market,” I have “People don’t understand the Israel job market.” These were my insights., which I then moved to my Research Report.

• Finally, for the structure’s sake, I grouped them all by the initial overarching questions I posed at the start of this study: Pains, Needs, Product Relevance, Criteria to make sure that all of them were covered. I even ended up with a new group – “Existing help.”

‍

Lesson learned:

An analysis is complicated and mind-bending, and it’s not very practical to do this on your own – no news here, but I feel obliged to acknowledge this.

To help yourself keep all your notes organized right from the start. Develop your own process, use the instruments you enjoy, use the help of your teammates if possible, to get a fresh view.

Example:

I grouped my notes one way, another person would group them the other way and would maybe get some more useful insights. If you’re interested in that kind of exercise – you’re welcome to try it on your own and share the results with me. Please contact me to get the notes.

Additional materials:

My Miro board (the quotes from interviewees are in Russian, I felt no need to translate them at this point. You can contact me if you’re interested)

My Research Report (Opens in Google Slides)

‍

‍

‍