Why you shouldn’t trust bare numbers sometimes

After I conducted the research from this post, I needed to support it with quantitative data. I ran a survey to confirm my main insights and conclusions. But the results I got were confusing. My results were 50/50. I got neither confirmation nor disproof. Or so I thought.

I’ll give an example.

My respondents were people who moved to Israel 1-3 years ago. One of my assumptions made from qualitative research was that new immigrants don’t understand an Israeli job market. So one of my questions in the survey was: “Do you understand the job market in Israel?”. This was a Likert scale question.

And basically, half of the participants said yes, and another half — no (counting both “strong” or “somewhat”). Here is my 50/50. Where can we go from there? Do they understand, or don’t they understand? Meanwhile, I had five more results like this on other questions.

And now it’s time to really switch to this user-empathy-design-thinking philosophy we’ve been taught. Think of these results. Half of the sample doesn’t understand the job market in Israel. If we extrapolate — half of new immigrants don’t understand it. That means that half of them can’t efficiently look for a job. And it really doesn’t matter that the other half can. Half — is a big chunk, so yes, we have a confirmation of our initial assumption. Although the bare numbers don’t give us that right away.