Last year, I wrote I was joining College Confidential. Well, it seems I'm once again leaving a job I love. My last day at CC is March 19. This is the first time, however, I've left a job without knowing what I'll do next. Unfairly perhaps, things tend to work out for me. It doesn't hurt that I'm married to a talented and dedicated nurse who can support our family by taking on more shifts while I'm job hunting. In addition, I'm confident someone will be able to appreciate and afford my esoteric abilities.

When I left Stack Exchange, I worried I excelled at the job only due to deeply understanding the sort of people who are drawn to Q&A sites. (Because I was one!) How could I be sure that I'd be able to manage a forum community largely populated by people deeply interested in college admissions? To preview the end of the story, I got this encouraging reply to a post announcing new rules on College Confidential:

As frustrated as I have been by some of the problems associated with the upgrades, and I know I have done some complaining, the difference between this go round and the previous ones (pre-discord1) is that I can actually see improvements that are helpful to the users.

This is the first time since I have been on CC (5 years maybe?) that I have felt like there was genuine effort to make this a better experience for the users. Thank you.

Now a lot of other people played a role in turning around the perception that the company wasn't interested in improving the community experience. It took well over a year of focused effort from developers, designers, product managers and community managers to get this result. Business development, marketing and my own managers showed an unusual degree of sensitivity to the community. It would be false modesty, however, if I didn't admit to playing a central role in guiding toward this outcome.

My work began months before I was formally hired when an old friend asked me to evaluate the community that his organization had just inherited. In October, 2019, I created an account and posted a reply to get an idea of how the community worked. I poked around various parts of the site and made some notes. Here's a sample:

  • Lots of acronyms; no glossary?
  • I not sure what the goal of the forums are.
  • Banned users at the top of the “Most Active this Month” report.
    • Activity isn’t probably what you want to optimize for anyway.
  • Is there a way to format posts? (Guess: BBCode)
  • Curious if there are patterns to joining, becoming engaged and disengaging. Seems like students and parents might not have any reason to stay once college is over. Maybe not even once the application process has ended.

For there I listed a few recommended next steps to begin solving the many problems I observed. The list included:

  • Simplify the rules.
  • Create a theory of moderation.
  • Interview the CM, moderators and key community members to understand their needs.
  • Conduct 1-on-1 usability tests with new and existing users.
  • Set up an A/B test framework.
  • Start a community-curated FAQ.
  • Investigate moving to Discourse.

At this point I wasn't planning to leave Stack Overflow. The new owners of CC were considering bringing me on as a part-time consultant and I figured I'd keep my day job. A month later I'd decided to leave my job and the new plan was to bring me on as a Community Manager fulltime. That list of recommendations became my personal goals for the community.

At Stack Exchange I never had a clear a picture of how I was doing. For one thing, I was part of a larger community team and worked with a wide variety of product managers. For another, I could rarely focus on just one community of the dozens I was directly responsible for. Perhaps most critically, however, I was working from the base of a well-developed set of community norms. Was I getting complimented on my work because I'm an eloquent writer or because I'm successfully managing the community?2

College Confidential had been neglected and mismanaged for half a decade, as the user I quoted noted. The founders sold CC to Hobsons in 2008. The new company seemed a good steward for the community for several years. It's main contribution was building a school search that, from everything I've heard, was exceptional. A few years later, CC was transferred from one organization to another. Nobody seemed to have an idea about what to do with the community. By the time I was brought on, the only real continuity was a community manager hired by Hobsons.

My first area of concern was whether the other CM, Sorin, should stay on. Immediately after starting the job, I set up weekly 1-on-1 meetings with him. Thankfully I discovered Sorin is a skilled community manager who had been hampered by shifting priorities and poor decisions from his superiors. Hobsons trusted him to manage the community, but subsequent owners made a series of technological, marketing and even community decisions that I find baffling. Fortunately, they didn't jettison the CM who was a lifeline for the community.

Next I met the volunteer moderators. I used to say that Stack Exchange was blessed with the internet's best moderators. I still believe that, but I'm beginning to suspect that people who willingly step up to moderate online communities are exceptional people as a rule. I don't agree with every decision the CC mods make, but they care about the community and strive constantly to keep it healthy.

Once I had settled in, I asked Sorin to introduce me to the community. Not every CM works this way, but I prefer to be personally engaged:

My goal with the you, the CC community, is to listen to your frustrations with the site and address those needs as I’m able. That might not mean doing things the way you are most comfortable with. But I will do my best to make sure your viewpoint is heard.

It's a form of transparency3 that opened up the inner workings of CC's new owners to the community. Not long afterwards, my promise was put to the test when I introduced a new tagging system:

Unfortunately, we’re stuck with some of the glitches until we can work out some technical issues on the back end. (I don’t want to be cagey about it, but we’re still in the discovery phase about our plans.) When we have a better handle on how we’re going to proceed, I’d like to collect feedback on the most pressing issues.

The plans I mentioned would eventually lead to us moving the community forums over to Discourse, but that's jumping forward in the story. Online communities exist on a software platform the same way physical communities exist in a particular space. Cities inform their citizens of new construction projects and community managers discuss upcoming developer projects too. It's important because both types of communities vote with their feet.

I began identifying leaders within the community. Moderators are obvious choices, but there are often individuals who influence the community without any formal title. To get a better idea of what the community needed, I set up calls with a handful of community leaders. Mostly I listened. As I gathered more information, I started picking up on patterns:

  • Nobody cared for the recent site redesign.
  • The confidential nature of the site was important. Many users worked in the college admissions field and didn't want their affiliations publicly known.
  • Longtime members had forged close friendships with other members and preferred using private messages to interact with each other.
  • The core community cared about students, but were jaded by wave after wave of ambitious new users eager to get an advantage in applying to elite schools.

Students (and their parents) are, of course, the future of CC. Weirdly, there has long been an active category for parents but no dedicated space for students. We corrected that problem by creating the Student Lounge. The corollary to The Field of Dreams fallacy is "If you don't build it, they will go elsewhere." Cultivating the student experience without sacrificing what makes the site attractive to parents isn't an easy task. To look ahead in my story, we introduced dedicated student groups just this year which have helped grow a small core of student leaders.

It's no secret that business interests often come in conflict with the preferences of online communities. There were a lot of behind-the-scenes discussions about how to grow the business in a community-friendly way. I think all community managers know the struggle. Thankfully, the company that currently owns CC knows it depends on a healthy community in order to thrive. Still an effective CM must advocate for the community internally. Unless it is unethical or illegal, a community manger has a responsibility to effectively communicate whatever decision is made back to the community.

A decision I was eager to communicate with the community was our plan to move to a new platform. It's one of the most disruptive things that can happen to a community and so it's incredibly risky. I worked with management to explore various options such as reverting to a previous version of the site, taking over hosting from our hosting company on the same platform or moving to other forum platforms. But over and over we kept coming back to Discourse as the right answer.4

Once the decision was made last summer, I began planning with a change manager5 at the company to prepare the community. In a way we benefited from how miserable the experience was before the change. Still, you never know how people will adjust to the worst conditions.

PHP coders adjusting to painful Stack Overflow shocks in 2038.

As soon as we could invite users to try it out, we started our alpha test. We invited various community leaders to preview the site from August through September. Their feedback helped us fix many issues and gave us advocates for the change. In October, we started our beta test by inviting any member of the community who wanted to get a sneak peak to do some usability tests. I met with students and parents. Some had experienced the existing site and others had not. By the end of October, we were ready for our Charlie test. Anyone who had an account on the production site could log into the new site to try it out.

Now this didn't go perfectly smoothly. Some users worried the change was happening too quickly and without their input. We also had a frustrating glitch where a number would be randomly added to the end of a user's name. The problem stumped our developers. It turned out we had a setting that allowed our SSO service to override the username on Discourse. Somehow that got interpreted as creating a new user. But since the username was already in the system, the username would be changed to one that didn't conflict. Every time a user logged in they'd toggle the number on and off.

Finally we had a launch date. After changing to the new system, the real work began. Sorin and I worked with the rest of the team to answer dozens, maybe hundreds, of user questions. Discourse modernizes the forum experience, but it's not intuitive to everyone. Even this week there have been questions about how features of Discourse work.

There's so much I want to talk about that happened last year. I still haven't talked about the COVID megethreads or the rules that made no sense to anyone. And there are innumerable interactions that add up to a mutual understanding between a community manager and a community. As I wrap up my time at CC, I reflect on how much of myself I put into the work, how much I will treasure the past year and how I'm looking forward to whatever God puts before me.

Footnotes:

  1. He means "pre-Discourse", but I like this phrasing too. ;-) I'll explain the transition to Discourse further into this post. ↩

  2. In a very real sense, these aren't so different really. The job something like 95% communication and 5% knowing what works. ↩

  3. A frequently misunderstood concept, by the way. For more details, I recommend a pair of essays from my former collegues Meg Risdal and Shog9. ↩

  4. I have a lot to say on this topic, but I do want to publish this post sometime before my time at CC ends. ↩

  5. Confusingly, this job is also abbreviated as CM. I've also got a lot to say about the overlap of these two roles. ↩