If you are currently developing your company’s online community strategy, and are struggling with all of the options available to you, a project to benchmark your discussion group experience is a great place to start. The members of your discussion groups will likely not only contain your most ardent evangelists (and probably most vocal critics), but will also contain the DNA to a more mature community strategy.
The intention of the benchmark is to look at the following areas:
1. Member Experience: Do members feel like they are getting what they need, in a way only your organization can deliver?
2. Community Strategy and Management: Does your organization have clear goals around your discussion groups? Is the community being managed to these goals?
3. Technology: Is your technology platform supporting member needs and community goals? Is it capable of evolving?
Community Strategy and Management could arguably be broken out into 2 separate sections, but based on several conversations I’ve had of late, the role of community management, and specifically, the community manager is evolving. It’s not just about moderation anymore. The new role of the community manager is to actually manage all dimensions of the community experience (moderation, UX, funding, metrics, etc).
The benchmarking project would be made up of several smaller sub projects and data gather exercises, specifically:
1. Benchmarking User Experience
- Member Satisfaction: Conduct a web-based survey to ask members about the quality of the user experience, feedback on the quality of message exchange, the level and appropriateness of moderation, the level of participation by members of your organization, and finally, would they recommend your discussion groups to their peers? Finally, ask about additional features or community touch-points members would like to see from you, including blogs and social networking. 50 to 100 responses to this survey would be a great baseline. For more sophisticated organizations, tying this survey into any sort of customer satisfaction, loyalty or brand-tracking research you are doing will be quite insightful. At Autodesk, we found that are Discussion GRoup members were more loyal customers than non-members.
- Usability: Gather 5-6 members from your community and have them walk you through the main interactions points they use on your discussion groups. This can be done in person, or over a web conference like WebEx or ReadyTalk.
- Find-ability: Gathering this data is very straightforward. You want to answer the following questions: Is your discussion group content showing up in google? Available from you site via RSS? How many clicks from the main flows of your corporate site?
2. Benchmarking Community Strategy and Management
- Budget: What is your total cost for hosting discussion groups? This includes staff time, moderation, license fees, hosting fees, bandwidth and any marketing you do. The other side of the coin? Who’s paying? Do you have a defined sponsor for the program, or are you asking for money quarter over quarter? Identifying additional potential sponsors helps smooth out quarterly-based funding, and also gives you a bigger checkbook for updates and platform extensions.
- Moderation: Review your moderation program. Do you have lead members assisting the moderator(s)? You should. Do you have clear and available discussion guidelines? Do your moderators have to directly intervene in the groups several times a week? A high level of moderator intervention is a big red flag that something is not working.
- Metrics & Reporting: What data are you reporting back to management? A big red flag here is “none”. That means you aren’t doing a good job of communicating value (bad), or your management team doesn’t care (even worse). What types of metrics are you reporting? Unique visitors and page views are great. Membership growth and attrition is better. Showing engagement via member participation numbers is really good. It’s also possible to do a rudimentary level of “word of mouth” reporting by highlighting key threads that net out the key issues for the period of time you are reporting against.
- Internal participation: What is the current level of participation by your organization in your groups? If it is low, you are going to hear about it loud and clear in the Member Satisfaction survey mentioned above.
- Member outreach: Do you have any sort of program in place to highlight, reward or otherwise engage your most active participants? Some call this an MVP or Lead User program.
3. Assessing Technology
Caveat: I’m not a technologist, so I would recommend getting very friendly with your web team or operations staff to help you with this part of the project 
- Performance: The 2 things you are looking for here are 1. Are the groups available 99% of the time? Significant downtime because of maintenance or database issues can wreak havoc on a communities health. 2. How fast do the pages load? Ideally you are getting sub 5 seconds (at least).
- Scalability: If your traffic and participation doubled tomorrow, could your current system handle it? Again, take your favorite systems geek out to lunch and get their opinion.
- Cost: The platform market has become VERY competitive. There are a number of vendors that have evolved their platforms beyond just discussions over the last few years. Now is an excellent time to review your existing contracts, and to re-shop your platform provider.
Once you make it to this point, you will have a massive amount of data. Because of the nature of this exercise, you will also have checked in with your membership base to guide any additional augmentations to your community, as well as the folks internally who can help fund and participate in the next generation of your community.
Now the fun starts.
You will have almost certainly uncovered opportunities to refine your existing discussion groups presence, and you likely tapped into unmet needs your members are expressing. You will almost certainly have uncovered ways in which your organization is coming up short by the amount or type of participation in your community. Lastly, you will have a good idea of current vendor capabilities with regard to their platforms. In short, you will likely have all the data you need to plan and sell a project to your management team that entails extending your current discussion group-based community experience.
The two most logical and easiest ways to extend your discussion group-base community presence are blogs and social networking.
Blogging: Corporate blogs have been in the mainstream for a good while now, but I’m still surprised by the lack of product and industry-based blogs with some of our clients. Blogs tie in nicely with discussion groups when staff that are currently participating in discussions start blogs to highlight trends in the groups, or to give members of the groups deeper insight into that persons role at the host organization, and also that persons personality and day to day life.
Social Networking: Another great way to extend a discussion group-based experience is to add social networking to the groups. This option is available in most of the latest versions of discussion software, and essentially involves creating a richer member profile, allowing members to expose their profile page, and allowing other members to browse, find and connect with them. Not only does adding social networking features add a dimension of personality to the groups, at can also support offline analogs, like in person user groups.
The takeaway: most companies could be doing a better job with their discussion groups, and could be providing and receiving more value from the current investment. Further, discussion groups provide a logical path towards engaging in more sophisticated online community building activities.
