Month: September 2014

Participant Interviews

For a study I’m working on, I needed to recruit 128 participants and meet with them to generate data. While I would describe the overall experience to be a positive one — interacting with people is never a bad thing, really — there are certainly some issues I faced.

First of all, the experiment protocol and script really needs to be hammered down before starting sessions. This was really critical; I thought I had the protocol down pat. Turns out the opposite was true; I found myself adding more clarifying statements to my participants than I had realized I needed before starting the interviews. Additionally, I tweaked little things that I didn’t realize beforehand that not explicitly specifying would cause issues. A simple example — I didn’t notice that I hadn’t specified the amount of mental rotations I was asking my participants to perform. The whole packet seemed unnecessary so for completeness I ended up asking for one page. Point being that the devil really is in the details — these fine-grained, granular details make a big difference.

Secondly, patience is a virtue. A major one. When dealing with human subjects, it’s important to always be “on” — to always come across as eager, natural, and understanding. This is, of course, because you’re asking for their help — Hey! Come in here and do stuff for us! — but there’s also a larger reason why you should be this way … you want to engage your participants in the work they’re doing. Acting sour, tired, or being short with your participants tends to induce a mirroring effect wherein they mimic the the personality traits of the experimenter and that will bleed into the data they’re generating. Not good! An engaged participant will generate true and honest data, the type of data that leads to real results and (hopefully) that lofty height: statistical significance. Tired participants are less inclined to do so; they will follow the path of least resistance which allows them to leave early.

Finally, you need to leave time to do the other things you have to do in life. I pursued the data for my study aggressively, seeing ~128 people in a 2 week period (10 days). And then again the following weeks after (two-session structure). This left me often exhausted seeing people from 10AM to 10PM at night with no (planned) breaks inbetween. “Hard working”, maybe, but not good for mental health. In the future, I’m resolving to limit those hours. :/

Participant Recruitment

One of the papers I’m writing currently for CHI 2014 requires collecting data from human subjects. Without saying exactly what we are trying to do, I will say that to obtain (a-priori) statistical power requires a participant pool of 128 people with 32 people to a group (so, four groups). We finalized the protocol for the study around the end of August and began subject recruitment that same week. Since the Fall semester for my institution is the first week of September this ended up being quite apropos. It would have been difficult to assemble wholly 128 people from Central NJ to come to my laboratory on Busch Campus by using the general population even if I had the whole summer to do it. The influx of students is a great boon to finishing the recruitment quickly since my institution boasts some >50,000 undergraduates alone!

Regardless, it’s evident to me that the method of posting flyers around campus is not the most effective tool to recruit participants. It strikes me as being antiquated and outdated — the idea of going around and posting flyers is a method that pre-dates the internet. If it wasn’t made for the internet, that bastion of communication, then it is typically ineffective for reaching a wide audience. Indeed, the flyer itself is asking people who read it to use the internet in the first place by sending an email.

Which led me to think about what the space of recruitment methods might be.

  1. Flyers. Tried, tested, and maybe true, this method is the most common way to recruit participants. Stick the flyer up in high traffic areas with an email address on it and various tear-out slips and then hope people get interested.
  2. Fixed installation. Grab a table, place it in a high traffic area, and try to actively recruit participants with some flashy advertorials or by haranguing them with a siren call about compensation when they pass by.
  3. Placing advertisements/flyers directly in areas that force interaction. In other words, place flyers or ads where a person would have to at least read/interact with the ad in order to resume what they were trying to do. This means placing flyers on windshields of cars or sticking them in lockers. Probably the most aggressive/invasive method of recruitment and only recommended if you need a huge amount of participants and don’t care about making people angry.
  4. Online recruitment methods. With these, you have the potential to cast a wide net but it might not land on the people who are most conveniently able to reach your location and as a researcher you don’t have a time to come and meet them.
    1. Facebook ads.
    2. Google ads.
    3. Mailing lists.
    4. Reddit ads.
    5. Craigslist
  5. Targeted advertising. Instead of posting flyers in one place, put up a large ad in a place where people will run into it. For example, I could go to lecture halls at my university and write down the study details on one of the N chalkboards available. Alternatively: campus newspapers.

Haven’t figured out anymore than those but there could be others. Generally, I’ve had the most success with flyers and online recruitment (mailing lists, Reddit). I only had the energy to put up flyers once and some of the other methods would take too much time. I’ve found really good success with mailing lists in general.

Finally, the way to really hook people beyond getting the message out is to have good incentives. Of course, this is always subject to the amount of money in the grants funding the research project. In our study, we require people to return for a second session which can make it even more difficult to get participants. Even with a small monetary amount ($10), you can still get people to show up for a single session. When dealing with more than one session, it’s difficult. Right now, we’re using $30 to bring participants in for a dual session structure and only awarding compensation at the end. ISaying you’ll have food would be a nice way to bring in participants but can distract from the task at hand, especially if you need them to be in-and-out in a tight time period.