A till

Researching chuggers: our methodology

We conducted 50 interviews with face-to-face street fundraisers in central London (TfL Zone 1), during October and November 2008.

Our method was, as a group of researchers, to locate a group of fundraisers and then allow them to approach us individually (though there were two instances in which two researchers were approached simultaneously by the same fundraiser – these were, of course, only counted once in the results.) We then conducted semi-structured conversations with them in which we observed their behaviour and asked a number of questions pertaining to the standards they should be adhering to, as laid out in the Charities Acts 1992 and 2006, the Institute of Fundraising’s (IoF’s) Face-to-Face Activity Code of Practice and the code of conduct of the Professional Fundraising Regulatory Association.  We acted naïvely throughout the interactions giving no indication that we were aware of either the rules surrounding face to face activity or the details of the charity being promoted.

More specifically we observed the following:

  1. Whether or not the fundraiser’s ID was visible (this contained their name, often names were offered.  Where they weren’t, we tried to ask, but if this seemed too awkward, we did not ask and took a description.  Thus there are two fundraisers in our results who we do not have names for.) (Required by the IoF code: 4.3.1; PFRA code: 2)
  2. Whether or not the name of the charity they were representing was visible on their person (IoF code: 3.1)
  3. Whether or not they volunteered the information of how much they were paid, how they were paid and a notifiable amount. (IoF code: 4.2; PFRA code: 1; Charities Acts 1992 & 2006)
  4. Whether or not they behaved in a way which could bring the organizations (both charity and, where applicable, agency) into disrepute (IoF code: 3.1; PFRA code: 10)
  5. Whether or not they confused, misled or obstructed us (IoF code: 3.1; PFRA code: 9)
  6. Whether or not clear information was given about the organisations being represented (IoF code: 3.1)
  7. Whether or not the interaction was terminated immediately and politely upon request (IoF code:  4.3.1; PFRA code: 6)
  8. How polite the fundraiser was (on a scale of 1-5, 5 being very polite) (IoF rules, 4.3.1)
  9. How harassing the fundraiser was (on a scale of 1-5, 5 being very harassing) (IoF code: 4.3.1; PFRA code: 8)
  10. Whether they were employed by an agency or worked directly for the charity they were representing (IoF rules: 4.2)

We also asked the following questions:

  1. Please may I see your ID (if it was not visible) (IoF code: 4.3.1; PFRA code: 2)
  2. If I were to donate, where would my money go? Can you guarantee that all my money would go to <cause>? (IoF rules: 3.1)
  3. If I were to donate, what sort of correspondence would I get? (IoF code: 4.3.1; PFRA code: 4)
  4. If they did not offer the information that they were paid, how they were paid and a notifiable amount, we asked the following: are you a volunteer for <charity>? How much are you paid? Is that on commission or hourly?  We left these questions until the last possible moment in the interaction in order to give the fundraiser every opportunity to offer the information without being asked. (IoF rules: 4.2; PFRA code: 1)

We also recorded ‘any other comments’ we had about the fundraisers. This allowed us to write up anything we noted during the interview about aggressiveness, politeness, harassment levels, pressure put on us to donate, tactics used to get us to donate and knowledge levels. 

We also had a section to comment on the group of fundraisers if we had spoken to more than one from the same charity.  This is not included in our analysis of our results however, as two Intelligent Giving researchers left before the research was complete and so we were unable to discuss all the groups of fundraisers we spoke to.

We wrote up our interactions as soon after they happened as possible (often in coffee shops).  We provided as much detail as we could and wrote down anything interesting that happened during the conversation but was not covered by our questions or observations. We recorded our results on these forms (PDF).


 

Limitations

  1. We do not have the money to sign up to Direct Debits.  Therefore, some elements of the IoF’s Code of Practice had to go untested.  For example, when a member of the public signs up, the fundraiser is required to detail all the correspondence they can expect to receive from the charity.  As we didn’t sign up, we couldn’t tell if this was being done, but we still asked about correspondence, and the majority of fundraisers were very open about it.  However, we have not included the analysis of the answers to this question in our results. 
  2. The same problem could apply to solicitation statements – perhaps the fundraiser would have given the statement just before or as we were signing up.  However, we do not think this limits our findings on this count. First, the best fundraisers gave the statement at the beginning of the interaction; second, on some instances we used our specialist knowledge of PFRA materials to know that we were being shown a statement of the notifiable amount of remuneration, but were not able to read it; and third, some fundraisers outright lied about their pay, for instance by claiming that they were volunteers.
  3. The PFRA has conducted its own mystery shopping research and its sample of fundraisers was 300.  Unfortunately, we did not have the time or the staff to match this sample size and ours stands at 50.  We still feel that the findings are significant however.
  4. The ambiguity in the Institute of Fundraisings codes (both pre- and post-October versions) made it fairly difficult to determine what fundraisers should and should not be doing.  While we have made every effort to clarify the rules with the relevant bodies, cases may still remain which require individual interpretation.  Linked to this, the fact that the Institute of Fundraising’s Code of Practice was not updated to reflect the changes in the law contained in the Charities Act 2006 (and which came into force on 1 April 2008) until October (a week after we had begun our research) meant that we were basing our research for the first fifteen fundraisers we spoke to, on the old rules.  We did also use the relevant section of the Charities Act 2006 however, and as the other changes to the code were not too relevant to the specifics of our research, this did not significantly impact upon our method or results.
  5. It emerged during research that some charities and agencies had a policy of only signing people over a certain age up (e.g. 30), in order to combat attrition as older people are apparently more likely to be financially stable and therefore less likely to cancel a direct debit.  As the oldest Intelligent Giving researcher was 26, there were times when fundraisers would not approach us.  Not only did this affect the number of fundraisers we spoke to, but some have had to be omitted from the sample as they were not attempting to sign us up, but merely tell us about the charity as we were too young.
  6. It became apparent that the question ‘how are you paid’ was very difficult to ask, and very difficult to ask consistently.  Therefore we have omitted analysis of this from our results.  Additionally, asking the question ‘if I were to donate, where would the money go?’ was intended to lead onto a conversation which would aid us in gauging how clear the information being provided was.  As such, the results of this question have not been analyzed in isolation.

 

Timeline

Our research began at the beginning of October, when we conducted a pilot to see if our intended method would work, and to identify any problems with it.  We had originally planned to undertake both interviews with and passive observation of fundraisers, but it soon became apparent that observation would not be possible.  It would have been very difficult to ensure we were always the same distance from the fundraisers and there would not always be a convenient place to sit and watch them.  It can also be hard to interpret body language correctly when not directly involved in the interaction.  We therefore abandoned the observation element of our research.

In addition, we also omitted proposed interview elements which we realized it would be difficult to implement.  For example, pre-October 2008 version of the Institute of Fundraising’s Code of Practice on Face to Face Fundraising stated that fundraisers should not accept cash donations.  It became apparent that we could not easily offer to give a cash donation and then retract the offer, so this was taken out of the method.