Measuring the Sense of Community in Online Communities

A strong sense of community (SOC) is considered to be desirable because it fosters pro-social behavior of community members and helps deal with external challenges/threats (cf. Sense of community in virtual communities). So every community manager should be highly interested in assessing the SOC of her/his community. But measuring the SOC and interpreting the result is by no means easy.

Sense of Community: Face-to-face communities vs. online communities

Originally, the SOC-concept was developed for face-to-face communities (geographical neighborhoods). McMillan and Chavis (1986) [1] suggest four dimensions which work together dynamically in order to create and maintain an overall sense of community:

1. Membership [MEM]: The concept includes emotional safety derived from membership, the sense of belonging and identification with the community, personal investment in the community leading to stronger bonds, and some kind of common symbol system, which unites a community. These attributes go together in a mutually self-reinforcing way.

2. Influence [INFL] means a reciprocal relationship between individuals and the community in terms of their impact on one another. Influence is a bi-directional concept, as for a group to be attractive, an individual must feel they have some control and influence over it, while, on the other hand, for a group to be cohesive it must also influence its individual members. McMillan and Chavis (1986) state that pressure of conformity from community members actually comes from the needs of individual members for consensual validation. In turn conformity serves as a force for cohesiveness.

3. Integration and Fulfillment of Needs [IFN]: Members must perceive the association to the community as rewarding for the individual (like status of membership, or the possibility to share in the success of the community, and the perceived competence of other individuals in the community who might help the member with her/his own issues).

4. Shared Emotional Connection [SEC]: The more people interact, the more likely they are to form close relationships. The more positive this interaction, the stronger the bond developed.

SOC soon became an important enabler for the success of face-to-face communities. It was said to

• increase participation and feelings of belonging, 
• hold community development efforts together,  
• lead to satisfaction with and commitment to the community, 
• enhance involvement and problem focused coping behavior, 
• have a positive impact on the sustainability of a community  
• lead to higher satisfaction, participation, and commitment within social structures (cf. [10] for the relevant literature). 

In 2002, Obst et al. [1a] published the results of their investigation about the sense of community (cf. Sense of community - Is there a fifth dimension?) They had asked 359 members of SF fandom attending Aussiecon 3, the 1999 World Science Fiction Convention how they feel about the fandom community (a - relational - community of interest with membership from all over the world) and about their neighborhood (a geographical community). McMillan and Chavis' four dimensions of SOC could be established in both communities (SF fandom and neighborhood). BUT: In-group identification was a separate 5th dimension in both community types.
All five dimensions were significant predictors of overall sense of community in both community types but acted differently with respect to the community type:

• Conscious identification with fandom emerged as the strongest predictor, while in the neighborhood setting it was the weakest predictor.
• Belonging was a strong predictor in both communities. This suggests that belonging is an important dimension of sense of community in whatever context we are examining.
• Identification, however, seems to be more important in the communities to which we choose to belong than in those communities which we may have made a less conscious decision to join.
• Influence was an important predictor in geographical communities, however not at all important in the interest community. This may again be due to the element of perceived choice. If you choose to belong to an association due to common interest the need for influence over that association may be less than the need to feel some control or influence over the area in which you live. 

Another researcher, Anita Blanchard [8], too, had found a considerable overlap between the senses of community for face-to-face and virtual communities, but there were also significant differences (cf. Sense of virtual community: Antecedents and consequences for community management):

• Influence may be less important to members in online communities than to members of face-to-face communities. (Note: Koh & Kim [8], on the other hand, stress the importance of influence in online communities, cf. Sense of community - an alternative approach).
• Additionally, members in online communities feel that they know the personalities of others and experience and observe more personal relationships than do members of face-to-face communities.

Thus, there seem to be important differences between the sense of community in virtual communities (SOVC) and the sense of community in face-to-face communities (SOC). Using the same tool for measuring the sense of community in both communities may not be rewarding.

Measuring the SOC - The Sense of Community Index (SCI)

One tool for measuring the SOC in face-to-face communities is the Sense of Community Index (SCI) which is said to be the most widely used measure of the construct. It was developed as part of the seminal New York City Block Booster action-research project. [2] The scale items are published in the appendix of Perkins et al. (1990). [3].

Unfortunately, there was never a confirmation of its intended dimensions or sub scales measuring the four aspects of SOC (MEM, INFL, IFN, SEC). Several studies tried to find out which item measured which aspect of SOC. The results are in the table. Item #10 (It is very important to me to live on this particular block.) for example "somehow" measures all four aspects, depending on the community under study.

Table: Comparison of factor structure found across studies [6]

*) Original: resident survey data from 47 street blocks in five neighborhoods in Brooklyn and Queens, New York (1985 - 1986)
**) Obst et. al: geographical community data
***) Chipuer & Pretty: adult neighborhood data
****) Obst & White: university students' community, two items removed because not applicable

Blanchard's SOVC - an adaption of the original SCI

Based on her prior findings, Anita Blanchard [7] developed a sense of virtual community (SOVC) measure, on the basis of the SCI. Given the differences between face-to-face communities, there may be items in the SCI that do not have relevance in virtual communities, and the SCI may be missing parts of the SOVC. The questionnaire had 22 items; four items from the SCI had to be eliminated because they were inappropriate. The resulting 18 items in the SOVC measure are evenly divided between the SCI and newly developed measures. The newly developed items were introduced in order to assess support and identification. The SOVC had a Likert type response set (1 = never to 4 = quite a lot), whereas the SCI had a true-false response set.
Items adapted from the original SCI

• Q1. I think this group is a good place for me to be a member.
• Q3. Other members and I want the same thing from this group.
• Q4. I can recognize the names most members in this group.
• Q5. I feel at home in this group.
• Q7. I care about what other group members think of my actions.
• Q9. If there is a problem in this group, there are members here who can solve it.
• Q10. It is very important to me to be a member of this group.
• Q12. I expect to stay in this group for a long time.

Deleted during the analysis

• Q2. Members of this group do not share the same values.
• Q6. Very few other group members know me.
• Q8. I have no influence over what this group is like.
• Q11. Members of this group generally don’t get along with each other.

Additional items related to "support"

• Q13. I anticipate how some members will react to certain questions or issues in this group.
• Q14. I get a lot out of being in this group.
• Q15. I’ve had questions that have been answered by this group.
• Q16. I’ve gotten support from this group.
• Q17. Some members of this group have friendships with each other.
• Q18. I have friends in this group.
• Q19. Some members of this group can be counted on to help others.

Additional items related to "identification"

• Q20. I feel obligated to help others in this group.
• Q21. I really like this group.
• Q22. This group means a lot to me.

Interestingly, Blanchard circumvent the problem of attributing the items to different scales measuring different aspects of the SOVC, she measured an overall factor instead. The test in listservs and Usenet newsgroups ranging from pet lovers to human resource professionals showed that the 18-items SOVC questionnaire was a more sensitive measure than the SCI.

The Sense of Community Index 2 (SCI-2) © - a revision of the SCI [9]

In 2008, Chavis et al. [9] presented a revision of the SCI, a 24 item Sense of Community Index version 2 (SCI-2). The revised version comes with a Likert-scale response format (1 = not at all, 4 = completely) and was used in a survey with 1.800 people. Unlike SCI, it was able to cover all the attributes of a sense of community described in the original theory - but in a face-to-face community context. Here are the items:
Reinforcement of Needs

• 1. I get important needs of mine met because I am part of this community.
• 2. Community members and I value the same things.
• 3. This community has been successful in getting the needs of its members met.
• 4. Being a member of this community makes me feel good.
• 5. When I have a problem, I can talk about it with members of this community.
• 6. People in this community have similar needs, priorities, and goals.

Membership

• 7. I can trust people in this community.
• 8. I can recognize most of the members of this community.
• 9. Most community members know me.
• 10. This community has symbols and expressions of membership such as clothes, signs, art, architecture, logos, landmarks, and flags that people can recognize.
• 11. I put a lot of time and effort into being part of this community.
• 12. Being a member of this community is a part of my identity.

Influence

• 13. Fitting into this community is important to me.
• 14. This community can influence other communities.
• 15. I care about what other community members think of me.
• 16. I have influence over what this community is like.
• 17. If there is a problem in this community, members can get it solved.
• 18. This community has good leaders.

Shared emotional connection

• 19. It is very important to me to be a part of this community.
• 20. I am with other community members a lot and enjoy being with them.
• 21. I expect to be a part of this community for a long time.
• 22. Members of this community have shared important events together, such as holidays, celebrations, or disasters.
• 23. I feel hopeful about the future of this community.
• 24. Members of this community care about each other.

Abfalter at al. [10] tested a German version of the SCI-2 in an online community for elderly Germans (http://feierabend.de,  approximately 163k members as of May 2011, 312 respondents). Nine items of the SCI-2 had to be eliminated because they did not measure SOVC. However, the remaining 15 measures suggested a good fit of all four dimensions of the SOC. Here are the eliminated items:
Reinforcement of Needs

• 2. Community members and I value the same things.
• 3. This community has been successful in getting the needs of its members met.
• 4. Being a member of this community makes me feel good.

Membership

• 8. I can recognize most of the members of this community.
• 10. This community has symbols and expressions of membership such as clothes, signs, art, architecture, logos, landmarks, and flags that people can recognize.
• 11. I put a lot of time and effort into being part of this community.
• 12. Being a member of this community is a part of my identity.

Influence

• 15. I care about what other community members think of me.

Shared emotional connection

• 22. Members of this community have shared important events together, such as holidays, celebrations, or disasters.

Responsible for this elimination are several particularities of the Feierabend-Community:

• Community size: In a large community, members have little knowledge about each other. How should they know whether all other members value the same things? Furthermore, in a large community the members' needs and well-being are likely to vary enormously which may explain why item #3 and item #4 had to be dropped.
• Members' motivation: In communities where getting a certain information is the members' primary concern, the individual member may not have to invest a lot of time and effort, which would explain the exclusion of item #11. But, if the questionnaire had been accidentally distributed solely among the community's core members the result might have been different.
• Anonymity: The authors assume that anonymity is the main reason why so many items measuring the membership aspect of SOVC - in this study - did not measure SOVC significantly. For example, why should I care about what other community members think of me (item #15) if I can remain anonymous?
• Community type: Some items (like item #10) are only applicable to face-to-face communities (or at least to virtual communities where members meet face-to-face on a regular basis like sci-fi fandom conventions).

Therefore, Abfalter et al. suggest to rephrase some of the eliminated items from the perspective of a member of a large online community.

And how about the practical implications?

After so many studies and empirical evidence, can we help a community manager in this situation?

I've got a product support community (X active members, Y post per day) with a SOC-score of 2.4. Is that good? Can I improve that score, given the performance of other communities in this domain? The lowest sub-score is for shared emotional connection with 1.2. Should I invest more time in improving it or is this score generally low for communities like mine?

No, unfortunately, we can't (yet).

Obviously, every approach to measure SO(V)C seems to be different - like the communities to which the measures are applied. The biggest difference is the one between face-to-face communities and online communities. Online communities, for example, use different means of communication, provide a media-rich and playful environment, and they are often anonymous. These differences raise doubt whether SOC measures for face-to-face communities can be applied to communities in a virtual setting [10].

One could use for instance the SCI-2 as it is but it must be clear that what it measures is not the "true" SOVC of the community but only something somehow related to the original SOC-concept. (Of course, remeasuring the same community with the same index from time to time may give one valuable insights into the communities evolution). But the results of community A cannot be compared with those of community B because the SCI-2 may have measured two different SOC-related concepts. Therefore, benchmarking one's community in this way is impossible.
But how about the adapted indices? Does taking one of those questionnaires and applying it to a community give one the community's "true" SOVC? The adapted versions omit several items which were irrelevant in the communities under study but which might be relevant in the community at hand. Much seems to depend therefore on the characteristics of the online community.

So, what can be done?

There is a proven SOC-concept. There are a lot of hints about how to apply it to online communities and there are items that seem to be appropriate to measure the SOVC (somehow). Suppose then, there is an index with applicable items. In a first step, one could use this same index in as many communities as possible to identify those items which measure the SOC best and form the four sub scales - controlling for such community characteristics as size, type (newsgroup, blog, support forum etc.), anonymity, members (gamers, business professionals, consumers etc.), demographics, participation level etc. At this stage, it is perhaps possible to filter out irrelevant items and variables. With many participating communities one could establish a definite set of items and variables to be controlled. The result would be a database for benchmarking purposes. Insert your community's characteristics and the database would provide you with the data from comparable communities.

---------------------------------------------------------
[1] McMillan, D., & Chavis, D. (1986). Sense of community: A definition and theory. Journal of Community Psychology, 14, 6-23. Google Scholar. See also Dr. David McMillan's website for a detailed description of SOC and the dynamics of its components and an instructive (and very entertaining) video from the CMX SUMMIT 2014.

[1a] Obst, P., Zinkiewicz, L., & Smith, S. G. (2002). Sense of community in science fiction fandom, Part 1: Understanding sense of community in an international community of interest. Journal of Community Psychology, 30(1), 87-103. Google Scholar.

Obst, P., Zinkiewicz, L., & Smith, S. G. (2002). Sense of community in science fiction fandom, Part 2: Comparing neighborhood and interest group sense of community. Journal of Community Psychology, 30(1), 105-117. Google Scholar.

Obst, P., Smith, S. G., & Zinkiewicz, L. (2002). An exploration of sense of community, Part 3: Dimensions and predictors of psychological sense of community in geographical communities. Journal of Community Psychology, 30(1), 119-133. Google Scholar.

[2] Long, D. A., & Perkins, D. (2003)."Confirmatory factor analysis of the sense of community index and development of a brief SCI." Journal of Community Psychology 31.3 (2003): 279-296. Google Scholar.

[3] Perkins, D. D., Florin, P., Rich, R. C., Wandersman, A., & Chavis, D. M. (1990). Participation and the social and physical environment of residential blocks: Crime and community context. American Journal of Community Psychology, 18(1), 83-115. Google Scholar.

[4] Obst, P., Smith, S. G., & Zinkiewicz, L. (2002). An exploration of sense of community, Part 3: Dimensions and predictors of psychological sense of community in geographical communities. Journal of Community Psychology, 30(1), 119-133. Google Scholar.

[5] Chipuer, H. & Pretty, G. (1999). A review of the sense of community index: current uses, factor structure, reliability, and further development. Journal of Community Psychology, 27, 643-658. Google Scholar.

[6] Obst, P. L., & White, K.M. (2004). Revisiting the sense of community Index: A confirmatory factor analysis. Journal of Community Psychology, 32(6), 691-705. Google Scholar.

[7] Blanchard, A. L. (2007). Developing a sense of virtual community measure. CyberPsychology & Behavior, 10(6), 827-830. Google Scholar.

[8] Blanchard, Anita L. and Markus, M. Lynne (2004). The Experienced "Sense" of a Virtual Community: Characteristics and Processes. Database for Advances in Information Systems; Winter 2004; ABI/INFORM Global p. 65; Google Scholar.

Koh, J., Kim, Y. G., & Kim, Y. G. (2003). Sense of virtual community: A conceptual framework and empirical validation. International Journal of Electronic Commerce, 8(2), 75-94. Google Scholar.

[9] Chavis, D.M., Lee, K.S., & Acosta J.D. (2008). The Sense of Community (SCI) Revised: The Reliability and Validity of the SCI-2. Paper presented at the 2nd International Community Psychology Conference, Lisboa, Portugal. Google.

There is a SCI-2 questionnaire in the section Resources of Community Geek.
For further translations of the SCI-2 into French, Spanish and Portuguese as well as for further resources see: http://www.senseofcommunity.com/show-files.php?category=11

[10] Abfalter, D., Zaglia, M. E., & Mueller, J. (2012). Sense of virtual community: A follow up on its measurement. Computers in Human Behavior, 28(2), 400-404. Google Scholar.