For more information about Armedia Case Management, CLICK HERE

This mass of paper was taking up much needed physical space in HUD as well as proving to be extremely difficult and time consuming when it came to locating specific documents. To add to the constraints, HUD and its separate divisions did not have an easy way to share and collaborate on documents with each other or various other federal agencies under the existing system.

It became clear that HUD needed to digitize paper documents and implement a document management system for managing, storing and collaborating on documents. However, as a federal agency, HUD needed the system to be able to meet its specific needs to aid it in more efficiently handling FOIA requests. On this aspect, Armedia had three specific focus points for the new system:

1.      Provide a central location for managing case files
2.      Enable rich text based (i.e. full-text) and metadata (i.e. attribute) searching.
3.      Ability to share and collaborate on documents from any location on any device including tablets like iPad

Providing a Central Location

Providing a central location in which organizations manage documents and case files is a task that is at the core of Armedia’s strengths in terms of capabilities. However, one aspect was particularly important for this specific project. After scanning millions of paper documents and storing them in a database, Armedia faced the task of migrating those files to the cloud based Alfresco Enterprise Content Management system while automatically building a taxonomy based on the indexed information captured during the scanning process.

Using Armedia Caliente, a content migration product, Armedia was able to move HUD’s digitized documents and the indexed information captured in comma separated value (CSV) files into Alfresco while retaining the metadata tags (see picture below). The document’s metadata information was going to be vital to the success of the system later as this allows users at HUD to search for requested information under commonly listed categories such as name, date and location.

This centralized document management system provided HUD employees with the ability to search for documents across all case files without having to search within the different divisions. By providing the transfer of the original metadata, users can also search for documents using the same criteria they would have before.

Ability to Collaborate

The ability to collaborate and share documents with other federal agencies brought up a crucial factor in any situation dealing with government records: information security. Specifically, HUD needed to have the ability to protect individuals when documents were shared across agencies. To handle this task, Armedia integrated Daeja ViewONE Pro to allow users the ability to redact personally identifiable information before sharing across agencies or with the public. Once the redaction was burned into the document, it could be saved as a version in the Alfresco repository to keep the history of the case document.

HUD now has an extremely scalable and centralized case management system to manage its cases and the documents within them. Over three million pages of documents have been digitized and migrated into the new system. HUD Employees are able to share documents with other agencies and respond significantly faster to FOIA requests as a result of having the information at the tip of their fingers.

Click on the link for a full look at our U.S. Department of Housing and Urban Development Case Study

Daeja ViewONE Pro is a java applet designed to allow users to apply annotations to any* document.  That is just the starting point.  The applet has been developed whereby an implementer can interact with it via javascript to enhance the user experience.  The implementer can also dynamically change what controls and features are available to a user based upon the user’s role and permissions within the CMS.  I could go on but I would probability end up repeating Daeja’s website

Daeja also offers modules to extend the functionality of its offering.  Two modules that I have used are the PDF Module and the Permanent Redaction module.

The PDF module adds PDF (what else could I say?) support to Daeja ViewONE Pro.  Got a PDF which needs annotated outside of how Adobe does it?  Then this module offers that functionality.

The Permanent Redaction module came as a result of clients requiring annotations on documents but for them to be burned into the document for security purposes.  A redaction is a special annotation.  When placing a redaction onto a document it’s intended purpose is to obscure the information below.  In other words, make it unreadable, unrecognizable to anyone or anything.  This reminds me of the film Good Morning Vietnam when Adrian Cronauer if given news to read out most of the pages contain black bars.  This is redaction, and in this case permanent redaction.

This is where Daeja’s offering of the Permanent Redaction module comes in.  It can take the annotations** on the document, pass them through the Permanent Redaction Module and burn them into the document, producing either a TIFF or PDF as output.  Please do note, it is up to the client and implementer as to how and when the burning takes place and who has access to original content and who has access to burned content.

OK, enough about Daeja.  Back to the first project.  The client was not concerned about maintaining an annotations file.  They simply wanted all redactions to be burned into the document and for the document to be versioned (so as to maintain a history of redactions).  This was to support redacting PII (Personally Identifiable Information) in support of FOIA (Freedom of Information Act) requests. In this case all documents being handled were PDF.  If you remember one of the output options of Permanent Redaction was PDF.  This made it simple in that each new version was the result of burning.  This greatly simplified the model around security.

The client also wanted the burning process to be on-demand.  The burning process can be implemented in 2 ways, on-demand and background processing.

On implementing the burning process it was determined to run this in a separate Tomcat instance from Alfresco.  Not knowing how frequently documents were be worked on and knowing that the size of some of these PDF’s could in fact be over 200Mb in size offloading this to another instance seemed to be the correct way to go.  Daeja provide example code on how to implement the burning process but it is up to the implementor to customize this for the particular environment this is intended for.

This leads me nicely into Alfresco and it’s numerous API’s.  One of the many strengths of Alfresco is it’s numerous API’s to help with the flexibility of integration.  There is RESTful, Web Services, JavaScript, Java Foundation, JCR, etc. etc.  One of the common questions asked is ‘which API should I use?’.  For the Permanent Redaction integration, the implementer opted for a combo if Web Scripts and Web Services.  Web Scripts are awesome.  If the language you are using for integration can make an HTTP request then you can use Web Scripts.  Web Scripts are simple to implement, follow an MVC approach, support multiple inputs (think overloading) and support multiple outputs (need HTML, JSON or something returned?).  If the JavaScript API does not provide enough, it can be extended at the service level and exposed at the JavaScript.

Now, you may ask if they are so awesome why use Web Services as in combination with Web Scripts.  The issue for this client was the size of the PDF file.  Potentially 200Mb or more.  This is a limitation of Web Scripts in that uploading large files is a memory / performance hit.  This is the main reason Web Services was included in the combo.  As part of Web Services there is a highly optimized set of functions for streaming content into the repository.  Thankfully Alfresco also included the ability to share the authentication token between the different API’s!  To summarize for this client, Daeja ViewONE Pro is integrated into Alfresco.  When a document is opened for redaction, the applet is rendered to the browser through a custom webscript.  When the user clicks on the ‘burn’ button, the noderef of the document is sent to the Permanent Redaction instance running in the separate Tomcat instance, along with the annotation data.  The custom code running under the Permanent Redaction instance fetches the document from the repository via webscripts, applies the annotations and burns them into a new PDF.  This PDF is uploaded to the repository as a new version of the starting document.

The second project followed more of the traditional annotation model.  The client wanted the ability to annotate documents based upon content type, content format and user role and permissions.  One of the requirements was to NOT version the document.  To be honest, requirement I like.  The document itself is not changing so why constantly version it?  The way Daeja works is it maintains a separate content file with the annotations.  The initial integration  of Daeja and Alfresco was by Dr. Qu (Alfresco) and further continued by Jared Ottley (Alfresco).  This initial integration includes a basic annotation content model which associates the annotation file to the document as a child association.  Simple and effective.  The one thing that was annoying me for this client, it would be nice to version the annotations applied to the document so that there was a visual history.  Using the power of Web Scripts and Free Marker a template was created to list versions of the annotation file associated with a document, if it has one.

Some more information on Daeja is the ability to specify a server side script, cgi, etc. the call when the annotation save button is pressed.  Web Scripts are invoked via HTTP.  This means the Daeja save annotation button can invoke a custom webscript.  This webscript adds the versional aspect to the annotations file (if it is not already attached) then versions the annotation file.  This webscript also attached the Free Market template to the document to provide access to the versioned annotation files.  When clicking on a previous annotation this opens the document with the specified annotation and displays this in Daeja ViewONE Pro.  Oh, and this is in read only mode so that previous versions cannot be rewritten!

The other more interesting requirement was each annotation added to the document also needed text below this annotation to indicate who created it and when. Daeja provides this functionality as a tooltip but for compliance reasons this needed to be added to the document to be seen visually and not as a tooltip.  Unfortunately Daeja does not provide this capability (though it has now been logged as a feature request and has gotten their boffins pondering on it).  In the meantime a workaround was provided.  Again highlighting the flexibility of Daeja.  The applet provides events which can have custom coded attached to them.  In this instance when a save annotation takes place and event is triggered.  It was then possibly to have custom javascript be invoked on this event.  This is were the fun started and flash backs to maths and geometry classes happened.  It should be noted the user/date label being applied to the document was actually an annotation.  Annotations must be on the document and not fall outside the page boundaries.  This meant some calculations were needed to determine the location of the annotation and to decide if the label would be applied below or above the annotation.  The next consideration was how close to the right hand margin was the annotation.  I would like to say some complex mathematical formula was implemented but it was some simple maths that came to the rescue.  Actually this functionality is most likely cause for another blog as there are other things to consider.

This project was different to the first project but both were great learning experiences in the areas of annotation and redaction and what clients are looking for.  It has provided great information and ideas on what else could be done in the area of integration to provide even more cool integration features.

*currently Daeja supports over 300 document types!

**TIFF and PDF support a subset of annotations which can be burned into the document.

I’m not going into any deep personal thoughts on Alfresco ECM architecture today; rather I am going to share my thoughts on working with the Alfresco repository and the Alfresco Share UI from a software developer’s standpoint that had (almost) no prior experience with Alfresco, or ECM platforms in general.

I am relatively new to Armedia, and went headfirst into learning Alfresco ECM.  NOTE! It is generally a good idea to get a “crash-course” in a new product from a product expert before jumping headfirst into a new product, even if Alfresco has thorough documentation via their Wiki (http://wiki.alfresco.com/wiki/Main_Page).  My coworkers and I had the pleasure to have an “Alfresco Architecture Crash-Course” presentation shared with us from Dimy Jeannot who was working on another Armedia Alfresco project.

The high-level architecture is as seen below.  Any numerous UI technology, frameworks, etc are represented by the presentation layer (Alfresco Share, jQuery, EXTJS, etc).  The presentation layer can communicate with the Alfresco repository, or via Alfresco data web scripts (data layer).

High-Level Alfresco Architecture

The more detailed Alfresco architecture is seen below; we use the Alfresco Share UI as our example here.  Share, and any custom Share code you write, calls RESTful presentation and/or data web scripts.  These web scripts, in turn, communicate directly with Java Core Services exposed by Alfresco (i.e. document, search, node), or any custom services you may need to add for your application, to communicate with the Alfresco Repository.

Never assume that Alfresco and Share are running on the same server host.  For example, you should not hardcode localhost or any other hostname in your Share code.  Instead, use Share’s proxy URL when invoking a data web script (i.e. from a Share ftl, use Alfresco.constants.PROXY_URI when invoking your data web script -> Alfresco.constants.PROXY_URI + “<URL>”).

Where should you store your web scripts?  If you are concerned with data, store your web scripts in the data layer.  When your data web scripts are deployed, they are stored at webapps\<alfresco webapp context>\WEB-INF\classes\alfresco\templates\webscripts, create custom packages for your application.  Else you are concerned with presentation (such as from a Share dashlet), store your code in the presentation layer (it should then in turn call a data web script to get its data!).  When your presentation web scripts are deployed, they are stored at webapps\<share webapp context>\WEB-INF\classes\alfresco\site-webscripts, create custom packages for your application.

Detailed Alfresco Architecture

Looking back now and reading the Alfresco Wiki’s, the Alfresco architecture makes sense, I just needed guidance on putting the big picture together to start feeling comfortable with working with a new product.  If you ask me do I enjoy developing with Alfresco now, I would say yes <with smiles>.

When you work for a while in the software you get numbed to “technologies have come and gone…” occasionally though some become commodities and others trend setters. We have seen that with many products like Apache, Tomcat, Lucene, Drupal …etc that have stabilized and matured over the past years with the help of increased development from the open source realm. Wait! Did I mention the word “Open Source” and going to talk about the enterprise content management?

So without any more ado, we have Alfresco- catering to a rapidly increasing demand of the enterprise content management solutions which is built over the open source technologies such as Spring, Hibernate, and Lucene platforms. Having done years of Documentum development and several Alfresco projects of late, I think there are some interesting overlaps and differences of approach that I feel would make the developers get adapted quicker.

With the wiki site overwhelmed with Introduction, API’s, Development, Deployment and the Forums to answer all the questions regarding the issues faced during the project phase, I am here to talk purely from the developers perspective on what’s the key areas that I happen to witness the difference from the Documentum space.

The road map for my next series of blog is going to cover each of the areas mentioned below in more detailed, code abundant and developer centric approach which will answer the questions:

• Does this feature exist in Documentum or Alfresco or both?
• If yes, how different is the approach?

So with that preamble, and in no particular order, I give you my list of the key areas I got hands on and learnt how different Alfresco is:

• Aspects

Custom data model is the core for any enterprise content management solution. The use of “Aspects” as its core is the fundamental concept for content modeling in Alfresco. Although in the form D6, aspects was introduced, how different is the use and approach in Alfresco is something I will take a deep dive in my next blog.

• Web Scripts

Alfresco Web Scripts brings together the world of content repository and the web. Being a Documentum developer earlier ways of interaction with the repository have been either using DFC API’s or DQL. In Alfresco, Web Scripts provide RESTful access to content within the repository and we can build our own interface using java script. A custom move operation is implemented using the Web Scripts and the comparison of the implementation with the Documentum would be a something to be noted.

• Integration

On my last project, we had requirements for the customers to be able to permanently redact Personally Identifiable Information (PII) from existing documents stored in the repository and version the original document upon save. For various reasons, we decided to integrate the 3^rd party tool Daeja ViewOne module to provide this capability. I will discuss the topic as part of this blog series.

I started this series based on my experience implementing Alfresco projects and I invite you to share any of your experiences with any part of the road map wherein you run into interesting twists and turns? Did you drive off the road to get some help? I welcome your feedback as the blog takes its shape. See you all soon in Take 2.