Wikipedia's 2010 Fundraising Appeal – Form Usability and CRO Case Study

Wikipedia recently launched their annual appeal for donations. This involves the familiar “A personal appeal from Wikipedia founder, Jimmy Wales” banner being shown some of the time at the top of the articles on the site. On clicking this, the potential donor is sent to a landing page at wikimediafoundation.org:

Form Usability Issues

A couple of nights ago I visited this page and actually considered making a donation (after reading Jimmy’s well-written appeal), but something put me off when I moved my attention to the donation form. Shortly afterwards, I wondered what this was and went back to the page for a more objective look. I picked up on several things which had contributed to the bad impression, and it struck me that the form could very easily be improved. Using the web developer toolbar for Firefox to edit the HTML of the page, I spent a few minutes making what I feel is an improved version (using the same styling) that would have made a better impression.

Below is a side by side comparison of the form, before and after modifications (scaled down to fit this blog).

Before: After:

 

 

Notes on Changes:

General Changes:

1. Labels For Checkboxes start with the active verb of the operation that the checkbox field applies to – “send” and “list”. Starting these labels with “please” or “I agree” is something that is reminiscent of the type of form where the action of the checkbox will perform better if it’s function is not immediately obvious at a glance, such as signing your e-mail up to be spammed, sharing personal data with third parties or agreeing to questionable terms and conditions. Many users would not make these connections, but regardless of this, the change promotes clearer English; with no redundant language it is much easier to see the function of the checkboxes and the overall function of the form at a glance.

2. Supplemental Information that functions as a guide or explanation to filling out a form field is visually differentiated by being rendered in a smaller font size than the label. A different text colour could also be used to accomplish this. Carrying out this change visually simplifies the form, and reduces the apparent length and complexity of field labels further.

Specific Changes:

1. Misplaced Call to Action: Should a user follow the most obvious path to conversion, of reading Jimmy Wales’ text, and then moving their attention to the form in order to make a donation, the introductory text encouraging people to “donate now” is redundant, and could conceivably cause a bad impression. Redundant calls to action (given after a typical user will already have effectively converted) bring to mind a nagging parent or spouse. For me, this immediately started the form off with a bad impression.

2. Consistency of Language: The word ‘gift’ is mentioned nowhere else on the page I landed on, in all other cases the donation being called, unsurprisingly, a donation. This inconsistency would be unlikely to actually confuse a potential donor, however things like this can contribute to give an overall bad impression of a form, particularly since ‘gift’ and ‘donation’ are not exact synonyms. This label is also written in the second person, where as every other label is written in the first.

I may have included the “tax deductible” point in the new version, however Wikipedia are working on the form at the moment, having added this text since I produced the mock-up.

3. Visual association of connected fields: The donor list/public comment options are not sensibly structured. As both fields are optional and closely (but not directly) related, this is not straightforward to present in a coherent manner. I have chosen what I feel is a good compromise in terms of how the information is presented. The original version did not make clear at a glance the connection between these fields, or that leaving a public comment is optional (some people may not wish to leave a comment – this was one of the main things that may have put me off the form). Adding an indent to the “optional comment” field visually associates it with the public donors list checkbox, further simplifying the appearance of the form.

4. Privacy Information: As it is supplemental information to the checkbox above it, this has been moved to a new line, and visually differentiated as supplemental information to the form field. It’s previous presentation made the form label for this field very long and rather clunky. I also believe that separate presentation of a privacy disclaimer could be something that surfers have come to expect to see when scanning forms.

5. Information Unrelated to the Form: The “Other ways to give” section is not part of the form and this has been identified as such by use of a visual separator. The problem with not breaking up form and non-form following text elements is that, at a very brief glance, this information  could appear to be terms and conditions or disclaimers relating to the form – presence of such things are likely to make the form appear more involved and complicated, thus making a poor impression

I restricted myself in the kind of changes made on the form to rearranging order of elements, changing text content and changing font-size and margins in CSS only. There is more that could be done with the form, for example the display of the “I would like to donate” fields could be made significantly more attractive to the eye with some simple CSS rules.

Test Possible Usability Issues before Tweaking Content and Styling.

Wikipedia have been doing A/B testing of donation pages with at the time of writing twenty variants under test. However the test appears to currently be limited to changing the Jimmy Wales text to different versions. You can see the different versions here under “A”.

This testing will reveal to Wikipedia what works best in terms of the Jimmy Wales text, as they would expect. However, what they are missing is that they may well see a greater increase in donations from fixing up (and testing) their form usability than by tweaking the spiel by Jimmy Wales. I do not believe that this is an uncommon situation – usability issues are often overlooked in favour of tweaking graphics, styling and content. When there are usability issues with conversion steps, any observed changes to conversion rates seen by changing other aspects of the page or site are likely to be smaller.

Wikipedia, to give them credit, are working on and trying to improve this form – today I’ve seen a wider version that puts the donation amount fields onto one line. However, making the whole form significantly wider to improve one aspect seems rather unnecessary. I believe some decent CSS at the level of the form fields would likely make a larger difference to presentation.

Wikipedia should be A/B or multi-variant testing these changes in the same way that they are testing different content (hopefully they are). Usability and conversion rate optimisation (CRO) can be very unintuitive, in parts about affecting subconscious reactions – I would be confident that the impr oved version of their form would perform better than the original, but times in the past I have felt like this and found that the result is the opposite when tested – this can be especially true when making several changes at once. That’s why it’s always important to test quantitatively with software such as Google Website Optimiser. This way a solid foundation of improvement is established. Unintended, negative changes are avoided, and a specific value of improvement to conversion rate for changes can be attributed with a large degree of accuracy (depending on sample size).

Read about Receptional’s usability testing services here

Share this Post: Facebook Twitter Pinterest Google Plus StumbleUpon Reddit RSS Email

26 Comments

  1. Some very good points raised here Miles, I think the main thing that stood out for me was the use of text. For a page which is making an apparant “plea” to its users to donate money, there is a very forceful undertone surrounding the text used.

    The main call to action that stands out to me is “Make a donation now” – the only thing that could make this statement worse is to add an explanation mark to the end – “Make a donation now!”.

    For something which has clearly been written with a “please support us” line to it, it is extremely surprising how a plea (aka asking) for money in the founders text quickly transforms into a demand for money once arriving at the form.

    That first line at the donate form would certainly put me off donating.

    • I think that probably is the most offensive thing, though it’s a fairly common sin and perhaps I’ve learned to mostly overlook it.

      For me, I think what had me hitting the back button was “I agree to recieve future…” instead of “send me” and the way the privacy policy information wrapped several times below it. Scanning that part of the form in some ways could give completely the wrong impression and I think that’s what happened to me.

      That’s why I believe that sort of info should be distinguished from the label visually in some way.

  2. Some very good points raised here Miles, I think the main thing that stood out for me was the use of text. For a page which is making an apparant “plea” to its users to donate money, there is a very forceful undertone surrounding the text used.

    The main call to action that stands out to me is “Make a donation now” – the only thing that could make this statement worse is to add an explanation mark to the end – “Make a donation now!”.

    For something which has clearly been written with a “please support us” line to it, it is extremely surprising how a plea (aka asking) for money in the founders text quickly transforms into a demand for money once arriving at the form.

    That first line at the donate form would certainly put me off donating.

    • I think that probably is the most offensive thing, though it’s a fairly common sin and perhaps I’ve learned to mostly overlook it.

      For me, I think what had me hitting the back button was “I agree to recieve future…” instead of “send me” and the way the privacy policy information wrapped several times below it. Scanning that part of the form in some ways could give completely the wrong impression and I think that’s what happened to me.

      That’s why I believe that sort of info should be distinguished from the label visually in some way.

  3. Hi Miles, and thank you for taking the time to write this fantastic summary.

    It is incredibly nice to see people looking at our forms and providing feedback on them. We’re taking an evidence-based approach to the fundraiser this year, and that take the form of a series of iterations, as you noticed. As you said we have been A/B testing different text and forms to try and make them as user and donation friendly as possible. We have tested elements of the form itself, not just the creative. :) I’ve passed this great summary on to our design team to inform their process.

    One important point: our fundraiser actually begins on November 8th – the banners and forms you have been seeing are the ones we’re running during our test phase. We feel very strongly that we need to launch the “official” fundraiser with a form that is as optimized as we can possibly get it.

    Some usability changes have been tested recently and more will be tested later today. Last week, for example, we found that removing the comment box from the first page greatly increased the donation rate and so have been trying variations with that. Later today we will be testing variations of the text that include your suggestion of removing the top line ask (and having one on the bottom of the letter instead). We plan to continue testing until the fundraiser begins, and continue as it runs in our attempt to optimize the experience. Hopefully the usability of these forms will increase with each step.

    Thank you, again, for your feedback. I’d encourage you (or anyone who’s interested) to send me a copy of any feedback (so that I don’t miss it) by email: philippe (at) wikimedia.org is my address.

    Best wishes,

    Philippe Beaudette
    Head of Reader Relations
    Wikimedia Foundation, Inc.

    Imagine a world in which every human being can freely share in
    the sum of all knowledge. Help us make it a reality!

    http://wikimediafoundation.org/wiki/Donate

    • Hi Phillipe, I’m really glad that you came across this article and have found it useful.

      Wikipedia is something that I use a both at home and for research purposes at work and I hoped that this case study could be constructive, especially since Wikipedia is a non-profit for everyone.

      It sounds like your testing and launch procedure for the appeal is sound (yes, I wasn’t aware that the appeal hadn’t actually launched yet), it’s just a matter of choosing the right variations to test.

      By the comment box, do you mean the banner at the top with a photo of Jimmy Wales and a quote? I noticed that wasn’t shown on some of the pages I was looking at yesterday. If so, that seems quite surprising but as I mentioned in the article, CRO can be very unintuitive.

  4. Hi Miles, and thank you for taking the time to write this fantastic summary.

    It is incredibly nice to see people looking at our forms and providing feedback on them. We’re taking an evidence-based approach to the fundraiser this year, and that take the form of a series of iterations, as you noticed. As you said we have been A/B testing different text and forms to try and make them as user and donation friendly as possible. We have tested elements of the form itself, not just the creative. :) I’ve passed this great summary on to our design team to inform their process.

    One important point: our fundraiser actually begins on November 8th – the banners and forms you have been seeing are the ones we’re running during our test phase. We feel very strongly that we need to launch the “official” fundraiser with a form that is as optimized as we can possibly get it.

    Some usability changes have been tested recently and more will be tested later today. Last week, for example, we found that removing the comment box from the first page greatly increased the donation rate and so have been trying variations with that. Later today we will be testing variations of the text that include your suggestion of removing the top line ask (and having one on the bottom of the letter instead). We plan to continue testing until the fundraiser begins, and continue as it runs in our attempt to optimize the experience. Hopefully the usability of these forms will increase with each step.

    Thank you, again, for your feedback. I’d encourage you (or anyone who’s interested) to send me a copy of any feedback (so that I don’t miss it) by email: philippe (at) wikimedia.org is my address.

    Best wishes,

    Philippe Beaudette
    Head of Reader Relations
    Wikimedia Foundation, Inc.

    Imagine a world in which every human being can freely share in
    the sum of all knowledge. Help us make it a reality!

    http://wikimediafoundation.org/wiki/Donate

    • Hi Phillipe, I’m really glad that you came across this article and have found it useful.

      Wikipedia is something that I use a both at home and for research purposes at work and I hoped that this case study could be constructive, especially since Wikipedia is a non-profit for everyone.

      It sounds like your testing and launch procedure for the appeal is sound (yes, I wasn’t aware that the appeal hadn’t actually launched yet), it’s just a matter of choosing the right variations to test.

      By the comment box, do you mean the banner at the top with a photo of Jimmy Wales and a quote? I noticed that wasn’t shown on some of the pages I was looking at yesterday. If so, that seems quite surprising but as I mentioned in the article, CRO can be very unintuitive.

  5. I would add a character count beside the comment field as there’s only 200 of them to use. 😉

  6. I would add a character count beside the comment field as there’s only 200 of them to use. 😉

  7. Hi gang – thanks again for the great comments. I continue to pass them to our design and optimization team.

    We tested a few different things on Thursday -I think we’re beginning to narrow in on it. I am ~amazed~ at the overall difference in every metric with this more optimized form. We tested variants on the ask string, and softening particular language. You can find all the details – for those of you who like to geek out with us – at http://meta.wikimedia.org/wiki/Fundraising_2010/Banner_testing.

    As always, my email box is open, and I’d love to hear any comments.

    Philippe Beaudette
    Head of Reader Relations
    Wikimedia Foundation, Inc.

    • Hi Philippe, that’s fantastic that you’re putting some of these suggestions under test and are seeing good results, and I appreciate you sharing the testing results page. There’s little or no data of a similar scale publicly available for analysis on the web at the moment and it should hopefully be a useful resource for people getting into CRO and usability, to quantitatively see some examples of changes have positive effects.

      I haven’t had chance for a proper look through yet at the data but I will do later today.

      • If you looked there for stats earlier… They weren’t there. I was running a bit behind on posting them but they are now! Along with a link to our full stats spreadsheet. We just had another test this afternoon as well that is on the spreadsheet (slowly waiting for some data) but not on that Banner Testing page yet since we don’t have everything organized.

        • Hi James, I’ve just been looking at the stats again and there’s some interesting stuff – one thing seems to stand out is how the “personal appeal” text consistently appears to give better clickthrough rates than anything else that’s been tried.

  8. Miles, you spent a good amount of time and effort trying to help the Wikimedia Foundation with its approach to donations, but I think you may miss the key point: the Wikimedia Foundation (WMF) is a terrible waste of your money. They only spend 41 cents of every donated dollar on program services, thus earning them only 1 star (out of a possible four) from Charity Navigator on the measure of organizational efficiency.

    As for Philippe Beaudette commenting here… be careful. There’s a lot about Philippe that he would probably rather the donors not know about him, such as how he and the WMF recently wired a no-compete research contract to his former employer:

    http://www.examiner.com/wiki-edits-in-national/wikimedia-foundation-director-admits-to-sweetheart-contracts

    Think before you donate, everyone! Maybe this should be the year you make a donation to your local library, which (after all) is ultimately the “reliable source” of Wikipedia’s content.

    • Gregory, thanks for your comment, but this is really not the place for a debate on these sorts of matters – this is a blog about Internet Marketing and form usability, not charity politics.

      Any further off-topic replies will be deleted.

  9. Miles, you spent a good amount of time and effort trying to help the Wikimedia Foundation with its approach to donations, but I think you may miss the key point: the Wikimedia Foundation (WMF) is a terrible waste of your money. They only spend 41 cents of every donated dollar on program services, thus earning them only 1 star (out of a possible four) from Charity Navigator on the measure of organizational efficiency.

    As for Philippe Beaudette commenting here… be careful. There’s a lot about Philippe that he would probably rather the donors not know about him, such as how he and the WMF recently wired a no-compete research contract to his former employer:

    http://www.examiner.com/wiki-edits-in-national/wikimedia-foundation-director-admits-to-sweetheart-contracts

    Think before you donate, everyone! Maybe this should be the year you make a donation to your local library, which (after all) is ultimately the “reliable source” of Wikipedia’s content.

    • Gregory, thanks for your comment, but this is really not the place for a debate on these sorts of matters – this is a blog about Internet Marketing and form usability, not charity politics.

      Any further off-topic replies will be deleted.

  10. At least you can read the text. I’m in Germany trying to use English Wikipedia, but the banner link always diverts me back to the appeal in German. My German isn’t that great and there’s no way to get to an English version. if I’m reading the site in English, why can’t I see the appeal in English?

    • That’s pretty poor if so, there must be many more people in a similar situation. You would think that, if browsing on English Wikipedia you’d see an English appeal!

      This is one of the worst things about auto redirection based on geographical location – I always think a quick prompt such as “go to the English website?” is useful and avoids the issue of visitors who might not notice a drop down to select language/location.

  11. At least you can read the text. I’m in Germany trying to use English Wikipedia, but the banner link always diverts me back to the appeal in German. My German isn’t that great and there’s no way to get to an English version. if I’m reading the site in English, why can’t I see the appeal in English?

    • That’s pretty poor if so, there must be many more people in a similar situation. You would think that, if browsing on English Wikipedia you’d see an English appeal!

      This is one of the worst things about auto redirection based on geographical location – I always think a quick prompt such as “go to the English website?” is useful and avoids the issue of visitors who might not notice a drop down to select language/location.

Leave a Comment

3 × four =