Enterprise Content Management

Armedia Blog

Posts Tagged ‘Case Management’


Adding Full Text Search to ACM via Spring and JPA

June 24th, 2014 by David Milller

What, No Full Text Search Already?

My project Armedia Case Management (ACM) is a Spring application that integrates with Alfresco (and other ECM platforms) via CMIS – the Content Management Interoperability Standard.  ACM stores metadata in a database, and content files in the ECM platform.  Our customers so far have not needed integrated full text search; plain old database queries have sufficed. Eventually we know full text search has to be addressed.  Why not now, since ACM has been getting some love?  Plus, high quality search engines such as SOLR are free, documented in excellent books, and could provide more analytic services than just plain old search.

Goals

What do we want from SOLR Search integration?

  1. We want both quick search and advanced search capabilities.  Quick search should be fast and search only metadata (case number, task assignee, …).  Quick search is to let users find an object quickly based on the object ID or the assignee.  Advanced search should still be fast, but includes content file search and more fields.  Advanced search is to let users explore all the business objects in the application.
  2. Search results should be integrated with data access control.  Only results the user is authorized to see should appear in the search results.  This means two users with different access rights could see different results, even when searching for the same terms.
  3. The object types to be indexed, and the specific fields to be indexed for each object type, should be configurable at run time.  Each ACM installation may trace different object types, and different customers may want to index different data.  So at runtime the administrator should be able to enable and disable different object types, and control which fields are indexed.
  4. Results from ACM metadata and results from the content files (stored in the ECM platform) should be combined in a seamless fashion.  We don’t want to extend the ECM full-text search engine to index the ACM metadata, and we don’t want the ACM metadata full text index to duplicate the ECM engine’s data (we don’t want to re-index all the content files already indexed by the ECM).  So we will have two indexes: the ACM metadata index, and the ECM content file index.  But the user should never be conscious of this; the ACM search user interface and search results should maintain the illusion of a single coherent full text search index.

Both Quick Search and Advanced Search

To enable both quick search and advanced search modes, I created two separate SOLR collections.  The quick search collection includes only the metadata fields to be searched via the Quick Search user interface.  The full collection includes all indexed metadata.  Clearly these two indexes are somewhat redundant since the full collection almost certainly includes everything indexed in the quick search collection.  As soon as we have a performance test environment I’ll try to measure whether maintaining the smaller quick search collection really makes sense.  If the quick search collection is not materially faster than the equivalent search on the full index, then we can stop maintaining the quick search collection.

Integration with Data Access Control

Data access control is a touchy issue since the full text search queries must still be fast, the pagination must continue to work, and the hit counts must still be accurate.  These goals are difficult to reach if application code applies data access control to the search results after they leave the search engine.  So I plan to encode the access control lists into the search engine itself, so the access control becomes just another part of the search query.  Search Technologies has a fine series of articles about this “early binding” architecture: http://www.searchtechnologies.com/search-engine-security.html.

Configurable at Runtime

ACM has a basic pattern for runtime-configurable options.  We encode the options into a Spring XML configuration file, which we load at runtime by monitoring a Spring load folder.  This allows us to support as many search configurations as we need: one Spring full-text-search config file for each business object type.  At some future time we will add an administrator control panel with a user interface for reading and writing such configuration files.  This Spring XML profile configures the business object to be indexed.  For business objects stored in ACM tables, this configuration includes the JPA entity name, the entity properties to be indexed, the corresponding SOLR field names, and how often the database is polled for new records.  For Activiti workflow objects, the configuration includes the Activiti object type (tasks or business processes), and the properties to be indexed.

Seamless Integration of Database, Activiti, and ECM Data Sources

The user should not realize the indexed data is from multiple repositories.

Integrating database and Activiti data sources is easy: we just feed data from both sources into the same SOLR collection.

The ECM already indexes its content files.  We don’t want to duplicate the ECM index, and we especially don’t want to dig beneath the vendor’s documented search interfaces.

So in our application code, we need to make two queries: one to the ACM SOLR index (which indexes the database and the Activiti data), and another query to the ECM index.  Then we need to merge the two result sets.  As we encounter challenges with this double query and result set merging I may write more blog articles!

Closing Thoughts

SOLR is very easy to work with.  I may use it for more than straight forward full text search.  For example, the navigation panels with the lists of cases, lists of tasks, lists of complaints, and so on include only data in the SOLR quick search collection.  So in theory we should be able to query SOLR to populate those lists – versus calling JPA queries.  Again, once we have a performance test environment I can tell whether SOLR queries or JPA queries are faster in general.

Stay up-to-date on all my blogs about my Armedia Case Management projects. 

Share

Writing a Framework is Not Like Developing an Application!

May 28th, 2014 by David Milller

Armedia Case Management is both a framework and an application.  As a framework, ACM provides a scaffolding to write case management applications tailored to custom-fit a specific customer.  As an application, ACM provides pre-built web application archives (WAR files) suitable for generic customers.  For example, we provide a law enforcement WAR and an inspector general WAR.

So we’ve known for a long time ACM should be both a software development tool to write case management applications, and also a standard out-of-the-box case management app.  Like JIRA is both a software defect and issue tracking application, and also a tool to specialize your own defect and issue tracking app.  Rational ClearQuest is the same way: it is both a default out of the box application life cycle management solution, and a tool to specialize your own such solution.

So how does ACM support these twin goals?  We all know it’s an order of magnitude easier to write a purpose-built app to specific requirements than it is to write a software development tool for other people to write their own purpose-built apps.

To write a purpose-built app we can naively map requirements to the technical architecture.  Does Customer X need a case file attribute that will never be needed by any other customer?  We just add the column to the database, add a field to our model objects, update our MVC controllers to pass the field around, and update our view to show it.  When it comes time to deliver another version of the system to Customer 99, they get all Customer X’s special fields… or else we have to make a special effort to remove Customer X’s special fields, perhaps by maintaining a branch per customer.  Shortly it becomes impossible to keep straight which customers get which fields.  We don’t really have a framework at all; we have many different purpose-built apps, one per customer.

To write a framework for building case management applications, we have to add a Core Object Model.  Then we write a standard library of components, implemented in terms of this Core Object Model.  Then we write our pre-built case management WAR files using the standard library and the Core Object Model.

For ACM, our Core Object Model includes:

  • AcmBusinessObject – real world objects tracked by our customers (persons, organizations, firearms)
  • AcmContainer – case management objects that manage business objects (case files, documents)
  • AcmRelation – links (unidirectional or bidirectional) between business objects, other business objects, and containers (a vehicle contained contraband)
  • AcmExternalEvent – real world events that form the history of a container (subject was arrested; a vehicle was stopped at the border)
  • AcmAction – user actions that change the state of business objects or containers (split one document into two; consolidate two case files into one)
  • AcmBusinessRule – constraints or guidelines that can be changed at runtime by business analysts (when a document is approved, declare a record in the records management application)
  • AcmBusinessProcess – guides the life cycle of a container or business object (documents must be approved)

This object model is the set of design artifacts to write the standard component library and customer-specific applications.  The object model is the extra conceptual layer that makes the ACM framework different from just naively developing a purpose built application.  The proof is in the pudding; we no longer have to maintain one branch per customer; we can have a single ACM framework source tree, with customer-specific projects that import the framework artifacts.

Share

Slides From Today’s Webinar

March 4th, 2014 by Allison Cotney

In case you missed today’s webinar in which we walked through our Armedia Case Management solution framework with Alfresco Records Management, here are the slides from the presentation!!

 

Armedia Case Management is a web-based, workflow-driven case management software solution designed to help organizations capture, investigate, and manage cases.

Powered by Alfresco, Armedia Case Management provides a complete case management infrastructure to create and manage case information through its lifecycle. From initiation, assignment, investigation, evaluation and disposition Armedia Case Management tracks the case and the relationships to all of the collected information. Armedia Case Management provides pre-built law enforcement and inspector general workflows and data types, and can also support other organization types.

In this webinar, we walked through

  • How to declare a record in Alfresco Records Management within Armedia Case Management
  • How Armedia Case Management delivers easy compliance with Presidential Memorandum on Managing Federal Government Records as well as paperless initiatives

Stay tuned for a recording of the webinar!

Share

Alfresco Content.gov 2014 – Armedia Case Management Presentation

January 29th, 2014 by Allison Cotney

Thank you to everyone who turned out for our presentation this morning at Content.gov!!

In case you missed the presentation, here are our slides from the event. For more information about Armedia Case Management, CLICK HERE to learn about our upcoming webinar!

Share

Armedia Case Management: Getting Some Love

January 8th, 2014 by David Milller

It’s been a long time (OK, a few years) since I last wrote about Armedia Case Management.  What’s been happening since then?

We wrote the Armedia Case Management  framework using Spring and Spring MVC.  It was successful: we supported a few customers.  One customer was very large and required many customizations.  For this customer we extensively modified the framework until this implementation and that customer entwined together into a close-knit whole.  Untangling customer-specific code from framework code became very difficult.

We also learned some lessons.  Today Armedia Case Management requires extensive custom coding.  This is good for small customers that don’t require much customizing anyway.  And it’s good for large customers that can afford extensive bespoke development.  But it’s not good for mid-size potential customers; they could not afford the amount of customizing necessary for Armedia Case Management to make sense in their environment.

So we are ready to move Armedia Case Management to the next level, giving it the care and loving attention it so richly deserves.

We will support custom workflows via the Activiti workflow engine, not by custom code.  Activiti allows us to model the lifecycle for each Armedia Case Management business entity (case files, tasks, documents).  More importantly Activiti reduces the time and the custom code required to support each new customer.  We will want customers to have just the exact workflow they need; we don’t want to force customers to use a pre-defined workflow.  Activiti allows us to customize the workflow cost- and time-effectively.

We will implement user actions using the Mule ESB, not custom Spring- based code.  Each user action is embodied in a Mule ESB message flow.  These Mule message flows will interact with the Activiti workflows.  At the most granular level, Armedia Case Management itself will be a large collection of Mule flows.  When we configure Armedia Case Management to support a customer, we assemble the desired Mule flows into a specific Armedia Case Management application for that customer.  Again, this will reduce the amount of custom code and the time required to deploy an Armedia Case Management solution.

This component-assembly approach also allows us to easily build special-purpose modules such as Records Management support.  The first next-generation Armedia Case Management module declares records in the Alfresco RMA.

From a management perspective, we will have a defined engagement model.  This engagement model specifies a basic pricing structure and defines how Armedia can host Armedia Case Management implementations on behalf of our customers.  Again, this reduces the time and effort required to stand up Armedia Case Management for a customer.

These are exciting times for Armedia Case Management.  I hope to report more soon…

 

Share
Copyright © 2002–2011, Armedia. All Rights Reserved.