June 15, 2010 
Will our content be found wanting?
It’s tough to be graded. In my children’s sporting leagues, the teams don’t usually keep score in the pre-k years. But, somewhere around 3rd grade, they start using the scoreboard and things get more intense.
We should probably be used to being graded by now, right? We waited breathlessly for our report cards in grade school. We stayed up late to lift up our GPA in HS and college. And even today, many of us receive numerical ratings on annual performance reviews.
But our documentation? Grade our documentation? Sacred. Off limits. Untouchable.
Not anymore
Product documentation where I work is already segmented into granular chunks. And these chunks are already on the web as discrete units. And we already track how many views each topic receives each month. And we are about to take the next step. Soon, each topic will receive a “grade” of sorts: A score systematically tabulated by usage, user feedback and a number of related factors.
What does this mean for the Technical Writing profession as a whole? It won’t be long before you’ll know which of your topics are making a positive difference, which need some help and which can simply disappear. And as one learns about topic impact, one inevitably learns about author impact as well. See, it will soon be painfully obvious which team members are writing the high-impact topics, which are writing the unintelligible topics and which are writing the irrelevant topics.
So in the coming weeks, I’ll be sharing more about how to “prepare your content for the coming judgement”. Most of us have plenty of time to get ready: This won’t be a pop quiz – More like a midterm. Late night cramming for college midterms had mixed results. And, since I’ve learned a few things since then about the wisdom of preparation, I decided to start studying early.
KnowledgeBishop 
Hi Tristan,
I am very intrigued that you are posting all your tech doc to the web and measuring the impact of each topic. This approach resonates so well with Anne Gentle’s recommendations in Conversation and Community: The Social Web for Documentation.
Does your company use a keyword glossary to help your tech writers optimize their technical documentation, in the same way that marketers optimize their content? Is there coordination between technical documentation and marketing to target the same keywords?
Keep us posted, how this approach plays out, and how it influences your writing choices.
Peg
Hi Peg, yes, I’m a big fan of Anne’s work and loved her book. We do have a keyword glossary and we do partner with other teams to achieve unity. I continually seek ways to drive toward optimization and total alignment (see my Power Ranger post, which describes the journey). Thanks for your daily influx of knowledge and encouragement! 🙂
Hi Tristan,
I found this posting really interesting.
Can you provide details about how to “prepare your content for the coming judgement.” Perhaps you’ve addressed this in a separate posting?
Robert
I think you’re leaving out a few aspects of what your company’s exercise is about. So I have a few questions.
1. What part of the topics found on the web that you’ll be tracking are also bundled in some way with the product.
2. What part of the topics are assistance topics (Help) that might be closely associated with some part of the product’s user interface?
3. Are you seeking to measure which parts of products have design issues that drive users to seek answers to basic questions about product usage?
4. Are you seeking to measure the effectiveness of the navigation architecture that connects your topics together?
5. Other than from written user feedback on a specific topic, how will you assess the effectiveness of a topic for the user?
Ok, thanks for those answers. Here are a few follow-up questions.
2.1. Based on a practice of content-reuse (especially for task topics), I would expect some percentage of the context-sensitive help topics to also be found among the web-delivered content. Do you find users look among the web-delivered topics for task topics? And if so, would you interpret that activity as indicating that the online Help is not as effective for delivering task-oriented assistance as you’d like it to be?
4.1. When you say that topics are being found via a web search, does that mean a search only within your web-delivered topics site or across the entire Internet?
4.2. Given that you say 90% of topics accessed via web delivery are found via a web search, do you interpret that activity as a clue that the navigation architecture for the web-delivered topics is not effective?
4.3 Related to Question 4.2, can you determine whether the user has first navigated among some part of your web-delivered content before he/she performed a web search?
4.4 Does your site for web-delivered assistance content provide any embedded assistance or automation for how to specify the web search?
My previous Question 4.3: nevermind. (All user navigation at your site is recorded by you, including a site-specific web search.)
Question 4.4 assumes that your web-delivered topics site offers site-specific web search.