Armedia Blog

Archive for the ‘Enterprise Content Management’ Category

Armedia Case Management (ACM) vs. NIEM

July 29th, 2010 by Rahul Rana

Is there any doubt that ACM wins this battle?

But how so? Read on…

Part 1 – The challenge

NIEM is an important cog in a government case management solution. At some point in a case’s lifecycle, you may need to share the case-related information with other agencies or individuals, perhaps to get assistance in completing the case or maybe to give more information to another agency for a similar case that they’re working on. Whatever the reason, this information must be easily passed from one system to the next without being ‘lost in translation’.

NIEM (National Information Exchange Model) is a framework for sharing enterprise-wide information across all levels of government. It has a core set of elements that are standard across all the agencies and has more specific ones for the various entities and stakeholders. Agencies can now speak a common language, allowing one agency to publish data from its case management system to the XML standard, which can then be picked up and parsed by another agency that requires this information.

Now all you need is a case management system that lets you export data into the NIEM standard and then a parser to put it back in.

Pretty neat and simple, n’est-ce pas?

Almost. (Let’s face it, if it really were that simple, I’d have nothing to write about and I’d just put up some comics to keep you entertained.)

There are a few problems to watch out for:

  • Before importing or exporting, the system must check the data’s security classification. A document requiring a classification level of Top Secret cannot be sent to an agency (or individual) with only Confidential level access. Perhaps only certain parts of the document are deemed to be Top  Secret, in which case, the publishing tool should be able to extract only those parts that are at the required access level.
  • Large amounts of data may need to be continuously exported to, or imported from, an external system. Every time an update is made to case data, it may be necessary to push this information out to other agencies. So it may be better to export/import only the changed data and update the existing data. This reduces the time and cost of transfer and would help to reduce duplicated data.
  • Data may be imported that already exists in the case management system. Checks need to be in place to handle matching data during import. When duplicate data is found, is it better to replace the data or create a new version of the existing data?

Armedia Case Management comes with a tool specifically to import from and export to NIEM that tackles these problems. Stay tuned for Part Two… The Solution, which discusses how ACM implements the NIEM publishing process.

Until then, here’s a comic strip to keep you entertained (don’t we all just love Dilbert)…

Think Alfresco from Documentum perspective –Take 1

July 14th, 2010 by Balaji Sampath

Open Source ...</ins>

When you work for a while in the software you get numbed to “technologies have come and gone…” occasionally though some become commodities and others trend setters. We have seen that with many products like Apache, Tomcat, Lucene, Drupal …etc that have stabilized and matured over the past years with the help of increased development from the open source realm. Wait! Did I mention the word “Open Source” and going to talk about the enterprise content management?

So without any more ado, we have Alfresco- catering to a rapidly increasing demand of the enterprise content management solutions which is built over the open source technologies such as Spring, Hibernate, and Lucene platforms. Having done years of Documentum development and several Alfresco projects of late, I think there are some interesting overlaps and differences of approach that I feel would make the developers get adapted quicker.

With the wiki site overwhelmed with Introduction, API’s, Development, Deployment and the Forums to answer all the questions regarding the issues faced during the project phase, I am here to talk purely from the developers perspective on what’s the key areas that I happen to witness the difference from the Documentum space.

The road map for my next series of blog is going to cover each of the areas mentioned below in more detailed, code abundant and developer centric approach which will answer the questions:

  • Does this feature exist in Documentum or Alfresco or both?
  • If yes, how different is the approach?

So with that preamble, and in no particular order, I give you my list of the key areas I got hands on and learnt how different Alfresco is:

Custom data model is the core for any enterprise content management solution. The use of “Aspects” as its core is the fundamental concept for content modeling in Alfresco. Although in the form D6, aspects was introduced, how different is the use and approach in Alfresco is something I will take a deep dive in my next blog.

Alfresco Web Scripts brings together the world of content repository and the web. Being a Documentum developer earlier ways of interaction with the repository have been either using DFC API’s or DQL. In Alfresco, Web Scripts provide RESTful access to content within the repository and we can build our own interface using java script. A custom move operation is implemented using the Web Scripts and the comparison of the implementation with the Documentum would be a something to be noted.

On my last project, we had requirements for the customers to be able to permanently redact Personally Identifiable Information (PII) from existing documents stored in the repository and version the original document upon save. For various reasons, we decided to integrate the 3rd party tool Daeja ViewOne module to provide this capability. I will discuss the topic as part of this blog series.

I started this series based on my experience implementing Alfresco projects and I invite you to share any of your experiences with any part of the road map wherein you run into interesting twists and turns? Did you drive off the road to get some help? I welcome your feedback as the blog takes its shape. See you all soon in Take 2.

Software design is in crisis

July 12th, 2010 by A.J. McClary

In the 90’s, when ECM solutions were rare, we could get away with designing solely toward requirements, but there is too much at stake nowadays. A recent study sampling various IT projects reported that:

  • 62% percent of projects fail to meet their schedules
  • 49% are over budget
  • 47% have higher than expected maintenance costs
  • And get this—25% are canceled before they are ever deployed!

If you’ve been in the software industry long enough, you’ve probably seen all of these things happen. The funny thing is, it doesn’t really come down schedule, cost, or requirements—it comes from bad design. When software companies think design, they’re thinking about contractual obligations and meeting commitments with their stakeholders. Here is a typical scenario:

Company wins contract. Client provides requirements. Company builds a solution, meeting the requirements. Testers validate requirements and application is deployed to a set of users. The users hate it and the client makes new requirements, company builds to those requirements, and the users hate the next iteration…and so on.

This can be resolved by utilizing user-centered design principles in your system development life-cycle. Software is not about code, it’s about people. Instead of our clients telling us what they want, we should be telling them what they need. As engineers, we can do a much better job building solutions then they can—that’s what they hire us for. To be successful, we need to incorporate user research, information architecture, interaction design, and usability testing into our process.

“Iteration 0” – 10 Tasks to Guarantee a Slammin’ User Experience

Let’s start with “Iteration 0”. This sprint is completely devoted to user experience. The point is to get out of habit of thinking of Java Beans and database schemas and start thinking about people. Here are tasks that need to be accomplished:

  1. Gather assumptions and requirements. Take some time to get acquainted with the requirements and begin to make assumptions based on your experience with the technology.
  2. Analyze competition. Familiarize yourself with the way your competition handles things. This is not necessarily the solution you should be striving for, but is excellent to have in your back pocket to show how your solution is better or to compare solutions to initiate change.
  3. Understand goals & tasks. Comprehensive user research, interviews, card sorting exercises, and contextual inquiries help identify user needs
  4. Develop personas and scenarios. Evangelize these with everyone on your team. They can be in the form of user stories, posters, work-flow diagrams, and profiles.
  5. Build a content strategy. Find out what content you have, what needs to be developed, what needs to be expanded, and what can be cut. Also create a schedule for delivering those missing gaps.
  6. Information architecture. It’s more than figuring out what content goes where. It’s also what information is most important to your users. Identify those needs and incorporate them into your design.
  7. Prioritize features. One of the greatest advantages of user centered design is that you often find some requirements barely impact your users. Those requirements can be re-prioritized so you can focus on what’s really important.
  8. Build wireframes and interaction designs. Setting expectations on functionality, how it should look, and how it should behave reduces future UI defects.  
  9. Design a prototype. Build something you can take with you to road shows that incorporates both visual and functional design. Update the prototype during future iterations so you always have an accurate portrayal of what the product will look like at launch.
  10. Validate usability. There are hundreds of techniques, including a usability inspection, paper prototyping, and eye tracking, but put something in front of your users frequently.

Points 8, 9, and 10 should be repeated throughout all future iterations until the project’s completion. When user experience drives design, you build products people love to use.

Documentum-Composer

July 6th, 2010 by Aparna Thimmavajjala

It was year 2004, when I was first introduced to EMC Documentum. As I first fired up the DAB IDE, I felt man this IDE is Unintuitive, slow and cryptic. Apart from learning the DFC API’s in Documentum, getting accustomed to DAB itself was an excruciating experience. Finally after three years Documentum Composer came as a savior.

I liked Documentum Composer for 3 main reasons:

1. It has better interface than DAB, and its easy to use providing support for keeping multiple projects open at the same time. You need not checkout and checkin the artifacts every time you make any changes only thing you will be doing is installation of the project.

2. It comes as combined feature of both DAB and DAI where in you create the DAR and install from the same IDE into a repository.

3. Composer is built on the Eclipse platform, and the installation includes a bundled Eclipse environment. One of the benefits of the Eclipse platform is that it offers a number of paradigms that are familiar to developers thus allowing users to identify the issues at the DocApp development level instead of installation level.

Composer helps to develop applications faster and easier by reducing the learning curve through user interface consistency and the familiar standards-based Eclipse tooling framework. Composer can be run in “headless Eclipse mode”, which enables command-line scripting and automation of project installation. This feature provides self-contained, fully automated deployment of projects into a repository, without the need for a previous DFC installation or DMCL client.

One thing that I like about Composer is the ability to install into multiple repositories, assuming they share a connection broker, without restarting and on the other hand ability to work offline. To install the application or migrate the data model a user would use DAI to install the docapp archive into one or more production Docbases, where as with composer you can do both the task in one single place without the need of DAI to deploy.

It is both an artifact project, supporting artifact development, and it is also a DFS project, supporting consumer and service DFS development.

Another feature that I liked about Composer is the ability to deploy java methods as BOF module without the need for the user to copy the class files under /dba/java_methods folder in the CS location and to be able to deploy on the java method server without having to bring down the content server. Also when Business Object Framework (BOF) came out in 6.0, DAB was extended to support creation of modules which served as “container” for custom BOF code.  However, you still had to have a java environment to build JAR files that contained BOF code.  In addition, any customizations to WDK/Webtop were not supported in DAB.  The WAR build and deployment process was entirely separate from docapp installation process. Composer is an effort by EMC to come up with a single tool that help developers to Configure Documentum artifacts, Create Business Objects, Build DFS Services, Deploy Documentum applications across multiple repositories.

Once you begin using Documentum Composer the benefits are easily evident. To name a few:

  • Developing applications quickly and easily
  • Learning curve is reduced due to the consistency of user interface and familiar standards of Eclipse tooling framework

Personally I consider composer a great gift to the Documentum developer community.

Microsoft SharePoint 2010 and The Last Great Hope

June 22nd, 2010 by Jim Nasr

Anything that defines the “future of productivity” deserves a welcoming befitting a king: “the lion roars back”! Luscious superlatives take a number…

And now back to our regularly scheduled program… Yes, SharePoint 2010 has arrived (officially since mid-May). Yes, SharePoint 2010 is a good improvement over the previous incarnation (MOSS 2007). And yes, most importantly, SharePoint 2010 taps well into the new (vastly better) Office 2010. And yes, SharePoint 2010 is the silver bullet to fix all your ECM problems. Errr…ok, that’s one yes too many.

Though no silver bullet, and perhaps just a smidgen short of the future of productivity, SharePoint 2010 (and, in my opinion, even more so as a hosted solution) definitely has a place at the table. As with all things hyped and dipped in unlimited marketing resources, the trick is to figure out what is real and what is relevant to your needs. So, consider:

  • Capacity, scalability and performance have improved considerably under SharePoint 2010—no more 2000 item view list limit!! And, Microsoft has done a good job providing metrics, tools and guidelines in this area. SharePoint 2010 requires 64-bit hardware, so installation and, in particular, migration from an existing MOSS 03 or 07 platform needs to be planned carefully, even more so than before.
  • SharePoint 2010 integrates very well with MS Office 2010, including Outlook calendaring and, heretofore ignored/talented stepchild, Visio. The most exciting part of this integration is the ability to use and edit metadata across SharePoint and Office. This propels true usability and greases the skids tremendously as far as adoption is concerned. The fact that Office is the reader and editor of choice for a lot of content held in SharePoint makes this integration truly relevant.
  • SharePoint 2010 provides the option for imbedded FAST (the search engine). This is a big upgrade over MOSS 07 and brings enterprise functionality and scalability to SharePoint search.
  • As usual, Microsoft has put focus on usability and in SharePoint 2010 there are some notable application usability improvements (such as the enhanced Ribbon, rules for content storage, smarter folders, a term store to manage taxonomy, offline capability through Workspaces, and, of course, the seamless integration with Office 2010).
  • Ease of development: an oxymoron in MOSS 07—more like lots of development—at least as far as web applications were concerned. SharePoint 2010 promises to be far better with a revamped SharePoint Designer and more web services and UI and functional components out of the box. Also, the new Business Connectivity Services (BCS) promises to ease integration with third party applications (such as SAP or Microsoft CRM).
  • SharePoint 2010’s other enterprise-like features: more comprehensive administration capabilities, easier deployment over distributed networks, better backup and recovery options, auditing/compliance features. Notable improvements in breadth and depth of such features over MOSS 07.
  • Surprising improvements in records management capabilities, including the convenient, in-place record declaration. Though SharePoint has claim-to-fame largely for collaboration, and in SharePoint 2010, even more so, with a social networking slant (blogs, wikis, Cloud-tagging, activity feeds into its MySites, etc.), Microsoft has stealthily taken strides to make SharePoint a legitimate records management alternative. SharePoint 2010 is still some way short of the maturity and depth of features provided by the more established records management tools in the market, however ease of use and pull of momentum are definitely in its corner. Watch this space for more in-depth research and opinion to come…

So, some compelling reasons to drink the Kool-Aid. Maybe none more compelling though than its inevitability. Inevitable as, errr, Mr. Know It All not knowing anything at all! SharePoint is here to stay, as proven by the already-out-of-control “SharePoint Sprawl”. Also, frighteningly (CMIS implications aside), Microsoft is quite possibly the only truly one-stop-shop ECM vendor around…not that that’s really all that good: one vendor fits all or the best vendors fit together?

So, buying the hypothesis that it is inevitable, what next? Well, first, what is needed? What does the user need? No tool will solve a problem just because…in fact, in vacuum, it’s likely to compound it. What is the problem? What are the, ahem, six 5Ws (What, Why, Where, When, Who, How)? Back to basics. Yes, clarity, simplicity rule. Don’t let the expensive consultants tell you otherwise. It’s my experience that most of us spend a lot of time and induce a lot of heartburn on edge cases. Spend time on what really matters, and then de-scope and simplify that even more. If, after all that, SharePoint 2010 is still inevitable then consider these three things:

  • What does the user need?
  • What does the user need?
  • What does the user need?

Dilbert on user needs

As engineers, we will spend a huge amount of time and energy on what we would like and what works for us—can’t everything be ext-js, RESTful, Spring Observer, n-tiered, service oriented, virtualized, mirrored, horizontally scaled, run on distributed ESX on redundant 4 way blade servers?! Oh, and, please adhere to the very large and complicated Enterprise IT Policy and System Architecture document (because it’s “Enterprise” it must be the right thing to do, right?!), and to every word of every paragraph of the System Requirements Specification (SRS)—written eighteen months ago by a large committee of “stakeholders”—and be sure to offer up all of the vendor’s glorious catch-all feature-set… Surely, exaggeration to make a point…not really. Bottom line either way: no user: very bloated, very expensive, very politically damning shinny shelfware.

Getting off the user-need soapbox for a second, the next BIG thing to figure out when thinking SharePoint 2010 is migration—nine times out of ten…or maybe 97 times out of 100 it will be needed. Migration from an existing SharePoint (03 or 07) application to the 2010 toolset, migration from a legacy application to SharePoint 2010, migration from another CMS to SharePoint 2010, migration of content and metadata…pick and choose your weapon of choice. Experience tells me that migration is NEVER EASY and its scope is almost ALWAYS UNDERESTIMATED. There are a number of tools that can help, but the solution goes way beyond tool selection; there needs to be a thorough plan (actually a Plan A, a Plan B, a Plan C and maybe even a Plan D). Oh, and let’s not worry about fixing the sprawl just yet…that’s a much bigger deal!

Not to be a buzzkill, but that’s not all. There is a lot more to consider depending on your specific view of the world. One other thing for sure though is licensing. In the great (and truly annoying) tradition of most Enterprise Software vendors, the licensing is a matrix of variables and if-thens. Every family needs a resident cost-accountant to truly savor such mind-bending aesthetics…

So, SharePoint 2010 represents some interesting opportunities for users and organizations and poses some further dilemma in the ECM space. A definite improvement over MOSS 07 but not (yet?) the iPhone of its genre. Competitors beware, Microsoft is coming…is here.

Lots to look forward to!

So long, farewell, auf Wiedersehen, eRoom

June 11th, 2010 by Colin Stephenson

A simple goal – “export, transform, load” – the destination is a matter of choice.

EMC eRoom is going away.  It has been marked as End of Life (EOL) so what next?  EMC Documentum have 2 options: EMC Documentum Collaboration Services; EMC Documentum Centerstage.  Armedia’s immediate goal is to support Collaboration Services, then Centerstage but why stop there?  Why limit a client’s choice.

Armedia’s eRoom migration story is in 3 acts (and yes, I have been listening to some test pieces that I used to play in my brass banding days – check out Year of the Dragon by Philip Sparke).

Act I – The Export

Getting the content out of eRoom into an understandable format.  Of course, its not just the content, there is  a large quantity of metadata in eRoom as well.  Act I – The Export deals solely with interrogating eRoom and generating a document detailing everything about eRoom.  From communities to Files.  From eRoom Setup to databases – we mean everything.  The result: a well-formed XML document

Act II – The Transformation

As with any classic performance, after the captivating opening, Act II deals with getting to know the characters.  In this case, the transformation gets to know the XML document and gains a deep understanding of the objects held within.  The transformation is responsible for also generating a secondary XML document. This is formed to support the ingestion to a new Content Management System (CMS) and / or Collaboration System.  Currently the supported transformation is for EMC Documentum Collaboration Services.  This can easily be extended due to the flexible architecture of this utility and is simply a case of transforming XML.

Act III – The Load

The closing act is the build up to the dramatic climax which leaves the audience going “WOW!”.  eRoom Migration aims to achieve the same “WOW!”.  Now that the XML has been transformed you can sit back and let the load run automatically.  That’s it.  By using the ingestion engine of Caliente! loading all the content and metadata is simple.  Just let eRoom Migration take care of everything for you.  The only thing it does not do is say “WOW!” – we leave that to you.

Over the next few weeks I plan to talk in more detail about the approach taken and dig deeper into the 3 different pieces of the migration effort.  For those eRoom users, what do you see yourselves using in the near future?

Let there be guitar!

May 27th, 2010 by Colin Stephenson

What happens when you combine Ligero and EMC Documentum when developers have a spare moment. You get:

Rockumentum!

The question was thrown out in casual conversation – “Hey, do you think we could turn Documentum into a Jukebox?”. So what else would you do with your million dollar investment? It turns out this was surprisingly easy to do when Ligero was put in front of Documentum.

A small number of mp3 files were added to our test repository, the URL was put into the browser and in the immortal words of AC/DC – LET THERE BE ROCK!

The audio player associated with the .mp3 extension was launched and within mere milliseconds rock music filled the room. Of course this was coupled with a nice pair of LogicTech laptop speakers which provided a surprisingly good sound.

Being the nerds we are, we added a small playlist (.pls) to the repository. This was kept simple and contained purely a list of absolute references to the tunes in the repository. First we gave the playlist url to our audio player and once again we were treated to the sounds of rock music. Awesome dude.

So what next. Well, it would be simple enough to develop a few Liglets* for Ligero to build playlists for you. These could return the metadata associated with the tracks. Heck, we could even throw in the Album Art for kicks.  There are lots of things we could do.

For more fun, throw in video and and take advantage of HTML5. This could lead to .avi and possible video training, but where is the fun in that?

*Liglets: a fun and novel idea around adding scriplets to Ligero to assist with rapid development of webpages being served out of Documentum.

iPad and the pursuit of content management

March 23rd, 2010 by Jim Nasr

Too expensive. Too bulky. Too consumer oriented. Too many existing competitors. Too niche. Too proprietary. Too slow…

Oh yeah, lots of negativity. Wise words? Truly wise…if your standards are at or below Peter Griffin’s. Otherwise: NO, NOPE, COMPLETELY WRONG. Yeah, the Apple iPhone has proven otherwise. Who knew?! Even Emperor Ballmer was a wee bit off on this one “…no chance that the iPhone is going to get any significant market share. No chance.” Presumably going from 0 to 30% market share in 3 years is not significant…

Three years on and many of the “wise” words peppered on the iPhone have been regurgitated for the iPad. Jobs and his visions… Why doesn’t he leave well enough be? “A visionary, vision is scary, could start a revolution, pollutin’ the air waves, a rebel…” Ah, that explains it. Vision is scary. Thanks, Eminem.

So, are we skeptical because we’re scared? Or just because status quo is so much more palpable? Either way, maybe the better question is: are we really willing to bet against the power of the Apple iX? Let’s examine some facts:

(a) there is already an educated market out there–Apple does not need to take on the brunt of marketing the idea nor convincing people a tablet/eBook reader has a place in their world…Amazon, Sony and others already did that.

(b) people (apparently over 3 million of them) are already buying the Kindles of the world…at close to $300 per pop! Let’s see, people are buying the grayscale, relatively one-dimensional, relatively unsexy eBook readers. Hmmm, I wonder if they’d be tempted to pay $200 more to buy the multi-faceted, uber sexy iPad?! There is a chance. Though, perhaps we should ask Emperor Ballmer…

(c) those of you who’ve traveled a bit on airplanes…you’ll understand: there is a need for a legitimate alternative to the laptop for the mobile business worker. If all you really need is access to the Internet, email, a PDF reader and an occasional dabble into office applications, do you really need a laptop? Smartphones are an answer to some degree, but truly not usable beyond a simple range of activities. Can the iPad stretch that range? Does Elmer Fudd have trouble with the letter R?

(d) 10 billion and counting! Yes, that is song downloads on iTunes, but you get the point. An accepted delivery channel. A very, very large community of loyal subscribers. A very, very powerful market position to dictate terms and condition–read Walmart. A very, very large amount of existing and potential (rich) content.

All of which, brings us nicely to the germane point of our rant: the iPad opens up a whole new world of content management opportunities!

Bold? Perhaps not so when you consider: 100,000+ iTunes apps, over 2m free eBooks already available, 5 of the biggest publishers committing to distribute eBooks through Apple, web sites, images, video, audio, streaming media, et al…all available through the same well honed, well liked, well bought usable interface.

But that’s just one side of the story. There is need for digital rights management so intellectual property is protected. Source content needs to be delivered in multiple formats to meet multiple device configurations and resolutions. Content is likely linked and referenced across multiple resources. Content needs to be easily searchable. Related content needs to be easily reassembled and re-purposed. There is need for multiple renditions of rich media to support different views and use-contexts… Hmmm, this stuff sounds kinda familiar, doesn’t it?!

Is this not all consumer oriented gadgetry? Not to cite Emperor Ballmer again, but are we really going to fall into this trap again: Here ye the enterprise, thou shalt not mention consumers! The idea of doing patient healthcare, claim insurance, pipeline maintenance, criminal investigative case management, … entirely through an iPad interface tied to a back end content management system (or two!) is anything but frivolous consumer stuff. Watch this space!

The Andy Fastow Subtext…

March 4th, 2010 by Jim Nasr

File this under: “what the …”, “huh?!”, “is this ECM related…”, or “let’s bear with it for a bit, it may go somewhere…”.

So for those of you who may not have heard: The Andy Fastow Story is a far-fetched, weird and wonderful fantasy about a man who single-handedly (well almost…with a little help from a few close golfing buddies: Jeff, Ken, Michael, Rick, Ben and David, amongst others) brings down a giant company, nay, a whole industry, tailspins an entire country into recession, causes the loss of tens of thousands of job and billions of dollars of people’s savings, illegally bags tens of millions of dollars for himself and his best buddy Michael, etc… In this little tale, our hero Andy is magically transformed from an Ordinary Joe to Financial Genius with the help of his friend, and self-anointed god, Jeff. Andy then gets to make up “Andy’s World”, where everything is always wonderful for him and all bad things in the real world are shoved into a box called LJM and then a bigger box called LJM2 (and a couple of broken down pales called Chewco and Raptor). In Andy’s World, Andy and Michael get to make up all the rules. Everybody is invited to play in Andy’s World—at a fee. Everybody will win a participation prize. Everybody will lose a whole helluva lot. Errr, except for Andy and Michael of course—they always win. Until one day, when a nosy stranger starts asking questions like “Andy, how do you do it? …like, can you teach me how to stick really bad things into an ordinary box just like yours and have them just disappear and make me loads of money just like you?” Well, one thing leads to another and this nasty thing called the SEC starts poking and prodding Andy’s World, bugging Andy and asking him the same kinds of stuff; except this SEC thing is scary and has some gnarly militia working for it. The nerve of the SEC questioning whether Andy is on the up and up! Of course, just like all good fantasies, after a bout of the uglies (a little public humiliation and a few years outside the comfort of Andy’s World—chilling in an institution near Colorado), our hero rises at the end and gets to fight another day (presumably with some of those millions he bagged earlier). Unfortunately his friends, well except Michael again, didn’t do so well: poor Ken died, Ben and Rick went to jail, David became known as the world’s most infamous shredder, and Jeff was vilified around the world and got blamed for everything bad in Andy’s World. Oh, and, like millions of people’s lives’ got permanently messed up.

Pretty good story, eh?! I’ve even heard some rumors that it’s loosely based on real life

Dilbert on Enron

Of course the moral of the Andy Fastow Story is that we don’t want another Andy Fastow story. Actually, no more stories at all of that genre. Sadly, it seems to be a trite plot prone to repeat. Who can forget the Bernie Ebbers or the Dennis Kozlowski Story? Or the Lehman and AIG Show? Don’t even start with Maddof! So many copycats, it’s hard to keep track.

The answer to stop these stories, one presumes, is to put in place some controls. Tools that support the controls. And, a scary, ugly and expensive editing process affectionately called the threat of eDiscovery. Oh, and a whole bunch of government type people, lawyers, accountants, bankers and consultants. Now, surely there will never be another Andy Fastow Story. There will never be another Arthur Andersen willing to edit another Andy Fastow Story. In fact, we won’t even need to have any more authors or heroes for these kinds of stories. Our tools will auto-document happy stories where everyone wins and everyone is treated fairly. So, that’s where ECM comes in!

Unfortunately, it seems, again, that reality is just not complying with our carefully thought out plot. Dang! In spite of noise to the contrary, and HUGE vendor propaganda promoting the discovery of gold reincarnated in the all signing, all dancing eDiscovery products tied to ECM solutions, it seems there is a “fiction gap” between is and would-like to be. In fact, records management en-masse (which one would presume to be a precursor to eDiscovery products) is seemingly not much more than email archival these days. All this seems to point to the obvious. What’s the subtext here?

Perhaps the subtext is in the incentives and disincentives. eDiscovery, empirically, seems to be a dance. The goal really is not full discovery or going to court—that would be insanely expensive for most organizations—but a happy settlement, which would be less insanely expensive. Of course to get to the settlement, you still need some level of eDiscovery…to know what is insane and what is less insane! Now, what if eDiscovery were easier? ECM, records management, search and eDiscovery tools all working in unison. A nirvana to be sure. But, would that not potentially lead to greater exposure of being discovered? Would the disincentive of the lengthy, costly and archaic discovery process now be replaced with the incentive of protecting yourself through the same lengthy, costly and archaic discovery process?! Certainly the US government (through the FRCP; Federal Rules of Civil Procedure) doesn’t hope so. In fact, since the December 1, 2006 amendments to the FRCP, effectively dealing with eDiscovery is very much supposed to be on the legal and corporate roadmap of most organizations; moreover they have the burden of responsibility for adequately managing discoverable information at their own cost.

So, where does that leave us? In the spirit of Mr. Fastow and company, I suspect somewhere between pay now or pay later…which is to say, little evidence of much competence at this time. In my opinion, effective and somewhat efficient eDiscovery is directly tied to successful deployment of ECM—properly indexing, managing, storing, retrieving and archiving information, and a streamlined implementation of Records Management—identifying, classifying, retaining and disposing of formal records based on corporate and regulatory compliance policies. I’m sure the eDiscovery vendors would not entirely, or possibly at all, agree with some of these assertions. But then again they may have incentives not to…

What is a Record?

March 1st, 2010 by David Mills

This is the first in a series of blogs entries that will address major records management issues in enterprise content management.

“What is a record?”     It is the fundamental question in records management. Without a clear answer to that question, one cannot determine record titles, develop and apply retention schedules, structure file plans or manage an “item” with software. The answer also makes the difference between establishing an “item” as a record, as opposed to being just a Word document, an email, a piece of paper in a file, or a map of a city.

The answer to the question begins with a definition. The primary definition for a “record” is stated in the ISO 15489: 2001 standard. It defines a record as

“…information created, received, and maintained as evidence and information by an organization or person, in pursuance of legal obligations or in the transaction of business.”

For the US federal government, the U.S. National Archives and Records Administration (NARA) is required by the Federal Records Act to use a slightly different and more explicit definition:

““records” include[s] all books, papers, maps, photographs, machine readable materials, or other documentary materials, regardless of physical form or characteristics, made or received by an agency of the United States Government under Federal law or in connection with the transaction of public business and preserved or appropriate for preservation by that agency or its legitimate successor as evidence of the organization, functions, policies, decisions, procedures, operations, or other activities of the Government or because of the informational value of data in them.”

Although this definition is slightly different than that in the ISO standard, NARA has incorporated the ISO definition into its regulatory guidance for federal agencies.

The result of these definitions is the segregation of items into “records” and “non-records” If an item does not meet the definition, it is not considered a record and, therefore, does not require any protection or preservation efforts – it may be disposed of when it usefulness for its owner ends.

If an item does meet the definition, it must also meet certain criteria to be considered a record. The criteria are designed to establish the trustworthiness of the potential record and are inherent to the definition. They are: authenticity, reliability, integrity, and usability. These criteria are defined as:

1. Authenticity: the record is a “true” item – an accurate representation of a transaction or activity, as purported by its creator at the time of creation

2. Reliability: the record can be relied upon to be a trusted source of the information it contains

3. Integrity: the record has not been and cannot be altered. It is complete in form and data

4. Usability: the record can be repeatedly retrieved throughout its lifecycle and used as an authoritative source of the information contained with in it.

If an item meets the definition, but fails to satisfy any one or more of the criterion of trustworthiness, the value of the item is suspect and cannot be considered a record. Thereafter, it is considered a non-record and can be disposed of at any time. Conversely, an item that meets all of the criteria of trustworthiness but does not meet the definition of a record, that item, too, is not a record. An example of this is a lunch menu for a nearby restaurant that is found in an employee’s desk. Although the menu may be a trustworthy representation of the restaurant’s bill of fare, if that restaurant is not connected to the employee’s organization, the menu is a non-record. (However, it could be a record for the restaurant.)

Often confusion arises in the application of the definition and criteria to an item when determining its records/non-record status. One major causes of this confusion surrounds the item’s media type. Another source of confusion is a failure to appreciate and evaluate the content and context of an item. Media type confusion arises from outdated perceptions of what a record have been. Until recently, many considered paper to be the only items that were records. The perception may have extended even to items preserved on microfilm or microfiche, but not much beyond that. The entire realm of electronic items was ignored. However, as ISO 15489 and the Federal Records Act make clear, any item, physical or electronic, can become a record if it is evidence of an entity’s business’ conduct and transactions. Therefore, potential record items include such things as paper documents, native electronic documents, scanned images of paper documents, structured data in a database, a chart, a photograph, a recording of voicemail, an instant message, a Twitter post, an email, or even, in the case of law enforcement, a gun, a knife, or DNA material. So long as the item meets the definition and satisfies the trustworthiness criteria, it becomes a record.

The confusion involving content and context is similar to that surrounding media type, as it concerns misinterpretations of how the definition and criteria are applied. One common misunderstanding is the assumption that all emails are records and should be preserved and protected as such. However, email itself is a media type, no different from a note, a letter, or a baseball bat. The factors that elevate an e-mail to record status are the content and the context of the email. At one end of the spectrum, an email between co-workers about where to have lunch, in most instances, would not be a record. At the other end, an email from a supervisor directing one of his/his employees to undertake certain job-related tasks clearly could be a record (assuming it also trustworthy). However, when context is applied with the content, it can change the status of the item. In the example of the email regarding lunch among co-workers, if business were conducted at the lunch, the email could become a record. If the is used by the organization to prove the time and date of the lunch for the Internal Revenue Service or other governmental entity, the email could become a record. Therefore, it is not the item itself that must be compared to the definition and criteria, but rather the content and context of the item in activity or transaction.

Any sound records management program is founded on the correct definition and trustworthiness criteria of a record. If the definition is absent, or is faulty, an organization’s entire records program can cause incorrect items to be treated as records. Every organization must, therefore, ensure that its record definition and the use of that definition are clear and unambiguous.

For more detail on this subject, please visit Armedia’s Resources page
and request the whitepaper entitled “Issues in Record Definition and Declaration.”