I will need to design a survey that will focus on capturing qualitative information about “stories of change”. Grantees will be asked to describe stories that illustrate a change in behaviour, opinion or attitudes as a result of project implementation. Projects are still ongoing and this information will complement quantitative data.
Any suggestions, also re the analysis, are welcome.
My organization has conducted several of this type of survey. In terms of survey design, our experience has taught us that the questions have to be phrased very precisely. If you ask loose, open-ended questions, your respondents are going to apply their own interpretations about what you want and provide you with a wide assortment of answers. In some cases we first asked a radio-button question and then asked the essay questions as a followup. Think like a lawyer and ask very tight, precise questions – leave no doubt in the respondent’s mind as to what you want.
It might also be helpful to conduct a test of your survey with a small group (7 to 10) respondents before you distribute it just to make sure you are going to get useful responses.
In terms of analysis – there are several methods that could be used. You can do a Google search on qualitative research narrative analysis and find several solutions.
My Story – When I was grad school, I was part of a team that evaluated a word-cloud like tool for text analysis. We learned that these types of tools work very well for collections of legal documents or journal articles because the various authors all used the same basic syntax and terminology. We also found that these tools do not work well with survey responses where respondents use a variety of sentence structures and terms to express themselves. Computerized text analysis tools by themselves will probably not be very useful for the analysis of your stories.