Why? Because you have to get right down to the user level and find out what they need and the context about how they think about the information.

So, let’s think about structured data and how it affects the user’s ability to locate and use the information in the content management system; in other words the search function. The textbook approach is to define the search precision and recall ratios for the system. Search precision being defined as the ratio of relevant (as judged by the user) items retrieved to the total number of items displayed in the search results list. And recall being defined as the ratio of relevant items retrieved to the total number of relevant items in the area being searched.

I mention these terms and definitions because I know many of my traditionally IT trained professionals like to follow the textbook and have their workshops and whiteboard sessions and generate flow maps and UML diagrams. But, in projects I’ve been involved with, the terms search precision and recall are never mentioned. However, here is the typical search discussion I’ve witnessed:

• Interviewer – Ok, how do you want to search for these documents?
• Interviewee – Well I search all the time for data A, data B and data C and it sure would be nice to search on data D and data E.
• Interviewer – Ok, great, we already have identified attributes for data A, C and E, we’ll make them searchable for you. And we’ll add searchable attributes for data B and D.
• Interviewee – You know sometimes I don’t really know the data to search on but I am looking for some text inside the document, is there any way I can search on text?
• Interviewer – Oh yes this system has the ability to do text searches on any documents added to the system.
• Interviewee – Great!

One of the critical success factors of a content management system is user satisfaction with the search and retrieval. Research consistently indicates that most users won’t make more than two or three attempts at finding the information they need. (Think about your own experiences using Google.) Search is typically driven by structured data. Fail to pay attention to the structured data driving the search functionality and you’re already placed one of the critical success factors at risk.

I never use the terms search precision and recall anywhere in the design criteria. The terms are just too hard to use quantitatively and what’s relevant to one user may not be relevant to another user. But the concepts need to be firmly in mind when users are being interviewed about their search preferences and habits. Users are remarkably tolerant of search as long as they get the expected results. Unless searching using a unique ID field like document number, users expect to sift through a results list to find the document of interest. How big result listings are they willing to sift through to find the document they are looking for? 10, 50, 100 results? That’s the blend of precision and recall you’re looking for in the interviews.

Much of my work has been in the area of engineering information in the utilities and oil and gas sectors. A favorite attribute that everyone wants to search is the equipment tag number. Drawings show equipment tags; procedures describe operation, maintenance and testing of equipment; specifications and calculations provide design information for equipment; vendor information tells how to maintain and order spare parts for equipment and so on. Lots of great indexing options! Great let’s add those attributes into the data model.

Stop! This is where the dirty works kicks in.

Does anyone index that information now? No, then how will the information get into the new system? What’s the source of the equipment tags? Is it validated? Who will be responsible for the information? When will it be complete and how will it be maintained up to date?

Users will tolerate the search feature up to a point. But, when an attribute is available for searching that sets the expectation of a high search precision it should be fully functional. If there’s little indexing or controls to support the search attribute then two bad things can happen. User frustration starts to set in or worse, a business decision is made without a possibly key piece of information. It is our responsibility as the content management professionals to do the dirty work and let the client know the possible implications of adding this attribute.

So let’s say the user community is ok with a low search precision % but high recall % listing for the equipment id search example. (Or in other words a result listing of 50.) How does a good structured data design support this result?

See my next blog entry: structured data and a business classification scheme in support of search and other design features…

It’s simple.  The new Content I/O Suite.  Armedia’s Content I/O Suite allows users to collect information from all of those billions of sources, migrate it to the centralized repository in the company, and then, if needed, deliver that information to the different web portals that may need to view the information later.  In fact, it can even send the information to multiple repositories all from the same interface.  So what if Human Resources uses Documentum, and Accounting uses Laserfiche, and some departments have their own collaborative eRoom set up, but you just merged with another company who uses SharePoint and FileNet.  What if one user could capture the information coming in from the website, or through email, or through regular snail mail, and deliver it into the appropriate system, without that one user ever having to be trained in each of those systems?  Sounds too good to be true, doesn’t it?  Check it out at http://armedia.com/products/suite/.

-Kristin Evans

With this latest offering from Reveille Software and Armedia, RRT adds to the Reveille Management Console for Documentum, bringing a new level of performance and experience monitoring. With RRT in the DCTM Webtop and Taskspace application environments, you can now measure and track EACH and EVERY user’s performance and know instantly if your system is delivering your Service Level Agreements (SLAs) and if there are problems brewing with your installation. With RRT you can now see problems down to the finest of details –such as “how long is it taking EACH of my users (and ALL my users) to login in to my environment” and “what is the average time for document check-in for all my users and who are the specific users that are not seeing good performance and not meeting my SLAs.” With RRT you also get the great Reveille ability to not only diagnose a problem, such as the job queue is stalled, but proactively fix those problems.

Interested in a demo of Reveille RealTime? We will be at our booth, #122 at EMC World and we also have a Webinar for Reveille RealTime on June 2. See the website for details!