For more information about Armedia Case Management, CLICK HERE

Most projects include building security, network security, and various other types of security policies which are generally inherited from the environment where the task is occurring and are fairly defined and stable.  Controlling access to the building, access to the network, access to files and folders, and general access to computing resources are all well established.  However, when we sit down and try to implement some form of security into the software we develop, it gets left out or deferred and needs to be added in at the last moment.  Why is it deferred? Well, how many requirements are really created around security?

Let’s throw a few examples out there:

• The system will only permit access to authorized users
• The authorized users will only be allowed to perform tasks they have been granted
• The authorized users will only be able to see the data they are allowed to work on
• The system will provide role and group based access controls
• The system will log all unauthorized access to data (is this security or logging –you decide)
• The system will log access to all data (Again, security or logging? Well, both)

These are very similar from project to project and pretty high level and very few, usually a dozen or so, but, without them in place, the system is essentially, non-functional.  Well, with the other functional requirements, what the user wants to see and do, which usually number in the hundreds, it’s easy to see why the security component gets little attention.  Except for the initial log-in, the users don’t really experience what is happening in the background, except for an odd warning about doing something they aren’t allowed to do.  A correctly implemented security model won’t even let the user know that features exist beyond what they are permitted to see.  What prevents them from seeing hidden features by glancing over their co-workers shoulder – well, that’s another issue.

So what happens when one of the most critical components of an application is not designed in from the onset? Besides chaos, a lot of redevelopment which is required to implement the missing feature, which leads to missing project deadlines and at worst not meeting implementation times. This is especially true for Fixed Firm Price (FFP) contracts where this can be disastrous to the bottom line, not to mention project failure – after all who will accept a system that cannot meet their basic requirements. Things are changing for the better, as I have seen projects now that have distinct security implementation team which interacts with the other development teams rather than being a side-attraction.

Naturally, for small scale web projects which are usually secured with 2-factor authentication (userid and password), the implementation is fairly simple and can be supported by the OS or a very simple database table.   If, however, you also need to control what the user can do (functionally) once authenticated, it starts becoming complex as you now have to maintain Access Control information for the user as well.  If you now further need to control what data they user can see, use, or modify, the security implementation gets even more complex.   Finally, if you need to implement CAC card integration, IP based authentication, CPU based authentication, or N-factor authentication (more than just a User ID and PWD, such as secure tokens, bio-metric, key-card …) , then additional complexities arise which need to be implemented in code or at the enterprise IT level.  Even now, we still haven’t addressed any encryption and data security requirements that could exist (ie. The system shall encrypt all personal data – does that just mean when it’s stored or also when transmitted??).  It is for this reason that security requirements must get high priority in any project where application, data, and system security are paramount, with the design engaged from the onset of the project.

The actual size of the security team will vary depending on the size of a project, but at least a single resource, with knowledge on all aspects of security (yes all, not just software), should be assigned as the single point of contact.  This person/team, throughout the project, must be involved intimately at the architectural level and will have the final say in any security testing scenarios.  By being involved at the architectural level, no design will be allowed to move forward until the security implementation has been vetted and approved.  Regardless of the security implementation, the primary goal is to confirm that all teams are compliant, which in essence translates to a compliant and secure application.  Whatever tests are needed to validate compliance should be defined during the design process as there is no better way to validate a system then to penetrate it by knowing its internals.  If you know how the security model is implemented and are still unable to compromise it, and then be default, the system is secure.  Open source systems fall into this category since all their code is accessible; everyone knows how to try to gain access, however, due to a well-implemented security model, they cannot.

At this point, we can keep going on about how imperative it is to make security design a high priority in any application design, or for that matter, any system design.  How would you like it if the car you bought didn’t have a lock or key to get it going?  Instead, I’ll just finish up with stressing that security is not a feature that can be put on the back-burner.  Everyone talks about it – you hear about cyber-security everywhere these days, but when it comes to the actual implementation, it gets compromised and inadequately implemented.  Considering that the raw power of computers is such that most systems can be compromised by simple attacks (such as brute force attacks), effort must be put into not just installing measures at the front gate, but beyond that as well to control losses if a breach were to occur.  It is also essential that management personnel understand this when planning a project to account for the level of effort it entails and to ensure that it is effectively budgeted for the entire project duration.

Now that we have addressed the issue of security in software system (and other systems), where do we go from there?  The best security is achieved by informing and teaching the user base, from the users up through the developers and managers.  I have covered a lot of items in this quick prelude to discussing security in information systems, but each should be followed with further details and training.

The situation, the project was with one of the leading product safety regulators in the world.  The international arm of the organization was leading an initiative to create a global information pool for product safety recall information in an effort to make product safety efforts across the world more coordinated and effective.  This would be a one stop, one shop view to “any recall – any where – any time” product safety recall information.

Armedia was asked to perform an initial assessment of what it would take a build a global recall information pool, build a roadmap for the process and structure the approach for the global consortium.  This was a pure play Information Technology (IT) strategy project where the focus was on creating an IT approach to meet a complex business situation.

What I loved about the project was the challenge of a highly unstructured business situation where the project sponsors understood the pain and knew the desirable outcome, but they had little understanding of what needed to be done.

Building an approach for an IT system in the area of product safety was a challenge, as I soon discovered.   Several factors contributed to the challenge, they included,  (a) plethora of legacy IT deployments, (b) changing data structures and data definitions over time, (c) different data structures and definitions across countries, and (d) project execution headwinds (Note: Our team ran into a situation of competing for attention amongst different priorities both within the organization and with 3^rd parties outside, in this case the consortium of product safety organizations from other countries).

Sound familiar?

So we walk into the situation not knowing all the pieces of the puzzle and the Armedia team immediately began an assessment of the situation.  Quickly the team structured the analysis into the following topical areas:

1. Strategic
2. Governance and Oversight
3. Functional Scope Definition
4. Usability, Data Search, and Discovery
5. Architecture
6. Deployment and Execution
7. Operations

Next came a deep dive into each area to develop specific recommendations and a roadmap at a granular level. Data categorizations, normalization and data definitions were all important areas of focus given the complexity of handling these items across different jurisdictions. I believe that our team’s recommendations around this area were excellent and well received by the international community, especially because our approach required minimal impact to current product safety operations, which was a key requirement by the sponsors.

We brought value in terms of providing structure and simplification for execution to what seemed at the outset to be a complex problem.  The Armedia report on “Considerations for Pursuing Global IT Interoperability for Publicly Available Product Recalls ” was published by the OECD (Organization of Economic Development) and has been accepted as the strategic guidance document by the international product safety working party set-up for the purpose of creating the global recall pool. The execution work on report recommendations has started and is currently underway.

This complex business issue was in need of a technology execution plan that was intentionally kept simple, and yet met the unique requirements of the customer. My team and myself guided our efforts in order to meet these criteria, and in the end the Armedia team successfully developed recommendations that provided both strategic and efficient solutions to this international IT challenge.

I’ve been in the software development business for 20 years.  Over the course of those years I have been involved in a lot of proposals, project plans, project re-plans, and estimating exercises.  I have used a variety of approaches and processes from very rigid, overly-documented approaches, to looser, Agile approaches to develop estimates.  One thing I have seen, regardless of the approach, is that project plans rarely reflect reality.  I have observed three frequent oversights when  developing estimates that always seem to bite project teams once reality meets the plan.

• Developers will spend 5% of their time every week on project management activities.  Time and budget are not usually allocated to developers to attend status meetings or company staff meetings, write input for the project manager’s weekly report, mentoring, or help with business development efforts, but it happens.  So, figure that 2 hours of every week will be consumed by activities of this type.
• Developers (especially those on the customer site) will spend another 5% of their time attending impromptu meetings,
  attending  ad hoc JAD design meetings, giving demos, supporting the project manager or customer, or dealing with customer drive-by meetings.
  Again, these are not activities easily anticipated or scheduled, but they occur all the time, and can negatively affect project performance.  Figure that 2 hours every week will be consumed by activities of this type.
• Developers new to a project need time to get up to speed on the project, the requirements, the environment, the tools, the processes and practices, the rhythm of development, and the project culture.  Don’t expect developers — even senior, seasoned developers — to be 100% productive from day 1.  Expect that it will take at least 40 hours for a developer to settle into the project  before he will be as productive as the rest of the team.  Schedule tasking accordingly.

I know it doesn’t seem like much, but over the course of a 5 week sprint, a team of 5 could lose 100 hours to these otherwise “invisible tasks”!  And, adding team member #6 will not immediately improve performance.

I mention these things because when I (and I suspect most project planners) sit down to estimate development times, we tend to estimate in a vacuum.  What I mean is, when asked how long it will take to develop Widget X or complete Requirement Y, we tend to give estimates that assume a perfect environment, the steady state of the universe, and that developers never get distracted by other project activities.  Or, that developers can be dropped into running projects and be 100% productive from the word “go”.

Hedging your task estimates by 10% and easing new developers into running projects can give your project plans a little buffer and more accurately reflect reality.  This advice will certainly not be popular with project managers or business development folk who continually strive for lower costs.  However, it may be to everyone’s advantage — yours and your customer’s — to produce a realistic project plan that can be met, as opposed to apologizing half way through the project that you have run out of resources for no ascertainable reasons.

I have a project that I wanted to add a couple buttons that would allow the user to navigate away from a “Drop-Off” library to the libraries where files are directed by my content organizer rules. So, without thought to SP Designer I opened my VS 2010 and commenced to coding. The great thing about building your buttons in VS is the flexibility you have – but being so flexible means you have to build a lot even if you only want a little!

In this instance I only wanted my buttons in one library. Using VS2010 you start with an elements.xml file. Unfortunately the elements file can only point to a “type,” not a specific library or list. So, even though I only wanted to see my buttons in my “Drop Off” document library, all the document libraries would also display the buttons. You can certainly get around that by omitting the type and buttons and code them in or setting visibility, but I haven’t done that myself so I was less interested in getting the coding right and figuring out where to insert the coding.

Enter SharePoint Designer 2010. It provides an easy nterface for creating custom actions that aren’t complex. You can create buttons that will navigate to a form, initiate a workflow, or navigate to a URL. Even better, it only applies to the document library or list where you are adding the custom action – exactly what I wanted!

You can certainly add an image to your button – this was something I overlooked initially and was driving me crazy that I “couldn’t” add a button image! Well, of course you can add an image. Just move that scrollbar down on the side of the “Create Custom Action” screen to get to the “Advanced custome action options” – duh!

So, a couple limitations right off:

• You can’t create tabs, groups, or context groups with SP Designer so far as I can see
• SP Designer placed my buttons in the “Manage” group of the “Documents” tab. There does not appear to be anyway to direct your buttons to a specific ribbon group or tab that you might prefer.

In this case the SP Designer provided all the functionality I needed and saved me time I would have spent coding, testing, and deploying!

Turns out there are a few tricks.

We use Apache Chemistry as our CMIS client.  Chemistry’s checkin method allows you to provide a file content stream, such that the act of checking the file in also updates the file contents:

checkin(boolean major, Map<String,?> properties, ContentStream contentStream, String checkinComment)

However, sending a non-null contentStream to this method makes it fail!  You will get the famous “object reference not set to an instance of an object” message.  You must send null for this parameter.  So how do you update the file contents?  Right before the checkin, you can call the setContentStream method, as illustrated in the below code snippet.

Also, in the properties parameter, you must provide a property for the object name.  This is also illustrated in the below snippet.  You may provide other properties if you want, but if you don’t provide the object name, the checkin will fail.

The JUnit test case below illustrates how to connect to SP2010 via WSDL, find an object by path, check out the object, update its content stream, and check it back in.

Why? Because you have to get right down to the user level and find out what they need and the context about how they think about the information.

So, let’s think about structured data and how it affects the user’s ability to locate and use the information in the content management system; in other words the search function. The textbook approach is to define the search precision and recall ratios for the system. Search precision being defined as the ratio of relevant (as judged by the user) items retrieved to the total number of items displayed in the search results list. And recall being defined as the ratio of relevant items retrieved to the total number of relevant items in the area being searched.

I mention these terms and definitions because I know many of my traditionally IT trained professionals like to follow the textbook and have their workshops and whiteboard sessions and generate flow maps and UML diagrams. But, in projects I’ve been involved with, the terms search precision and recall are never mentioned. However, here is the typical search discussion I’ve witnessed:

• Interviewer – Ok, how do you want to search for these documents?
• Interviewee – Well I search all the time for data A, data B and data C and it sure would be nice to search on data D and data E.
• Interviewer – Ok, great, we already have identified attributes for data A, C and E, we’ll make them searchable for you. And we’ll add searchable attributes for data B and D.
• Interviewee – You know sometimes I don’t really know the data to search on but I am looking for some text inside the document, is there any way I can search on text?
• Interviewer – Oh yes this system has the ability to do text searches on any documents added to the system.
• Interviewee – Great!

One of the critical success factors of a content management system is user satisfaction with the search and retrieval. Research consistently indicates that most users won’t make more than two or three attempts at finding the information they need. (Think about your own experiences using Google.) Search is typically driven by structured data. Fail to pay attention to the structured data driving the search functionality and you’re already placed one of the critical success factors at risk.

I never use the terms search precision and recall anywhere in the design criteria. The terms are just too hard to use quantitatively and what’s relevant to one user may not be relevant to another user. But the concepts need to be firmly in mind when users are being interviewed about their search preferences and habits. Users are remarkably tolerant of search as long as they get the expected results. Unless searching using a unique ID field like document number, users expect to sift through a results list to find the document of interest. How big result listings are they willing to sift through to find the document they are looking for? 10, 50, 100 results? That’s the blend of precision and recall you’re looking for in the interviews.

Much of my work has been in the area of engineering information in the utilities and oil and gas sectors. A favorite attribute that everyone wants to search is the equipment tag number. Drawings show equipment tags; procedures describe operation, maintenance and testing of equipment; specifications and calculations provide design information for equipment; vendor information tells how to maintain and order spare parts for equipment and so on. Lots of great indexing options! Great let’s add those attributes into the data model.

Stop! This is where the dirty works kicks in.

Does anyone index that information now? No, then how will the information get into the new system? What’s the source of the equipment tags? Is it validated? Who will be responsible for the information? When will it be complete and how will it be maintained up to date?

Users will tolerate the search feature up to a point. But, when an attribute is available for searching that sets the expectation of a high search precision it should be fully functional. If there’s little indexing or controls to support the search attribute then two bad things can happen. User frustration starts to set in or worse, a business decision is made without a possibly key piece of information. It is our responsibility as the content management professionals to do the dirty work and let the client know the possible implications of adding this attribute.

So let’s say the user community is ok with a low search precision % but high recall % listing for the equipment id search example. (Or in other words a result listing of 50.) How does a good structured data design support this result?

See my next blog entry: structured data and a business classification scheme in support of search and other design features…

Here’s how it works:

1. Sign up for an Amazon Mechanical Turk account
2. Embed tracking code into your web pages, I recommend Crazy Egg—so you can collect more precise data and present it with overlying heat maps
3. Instruct several “users” to go through a series of scenarios and watch the data come in

This technique is a nice way to analyze how user’s react to certain calls-to-action, navigation items, and placement of graphics/text. I’ve been using this as a mechanism to determine placement of components via A/B testing, but it could just as easily be used to also test  how long it takes for them to get to their destination—if they can figure it out at all. Just rotate two different designs across all of your participants and see which one performs better.

Did I mention this is basically free? It costs around $0.05 cents per scenario, per user. Testing a 100 users could cost you around $5 bucks. Compare that to a traditional test, you can save thousands of dollars just by implementing this one technique into your lifecycle.

While I’d love to say that “the numbers don’t lie”, they can be very deceiving. Even though you can use this technique to track trends in human behavior, bad data doesn’t necessarily mean bad design. While this can be very effective, I want to stress that is not a replacement for traditional user testing and should only be used to verify your previously conducted research.

There are two reasons why you should conduct this test along side many other styles of user testing. The first reason is because it lacks qualitative feedback. When you collect only quantitative data, you are missing out on some very important pieces of information, like “wow” factors, user quotes, and moments of frustration. The second reason is because you’re not actually testing authentic users, so this data is probably not going to be relevant to your website audience. It does, however, do a great job at measuring user behavior.

I think it’s a very big mistake to neglect usability testing. This is a great way to incorporate real data into your development efforts.

And now back to our regularly scheduled program… Yes, SharePoint 2010 has arrived (officially since mid-May). Yes, SharePoint 2010 is a good improvement over the previous incarnation (MOSS 2007). And yes, most importantly, SharePoint 2010 taps well into the new (vastly better) Office 2010. And yes, SharePoint 2010 is the silver bullet to fix all your ECM problems. Errr…ok, that’s one yes too many.

Though no silver bullet, and perhaps just a smidgen short of the future of productivity, SharePoint 2010 (and, in my opinion, even more so as a hosted solution) definitely has a place at the table. As with all things hyped and dipped in unlimited marketing resources, the trick is to figure out what is real and what is relevant to your needs. So, consider:

• Capacity, scalability and performance have improved considerably under SharePoint 2010—no more 2000 item view list limit!! And, Microsoft has done a good job providing metrics, tools and guidelines in this area. SharePoint 2010 requires 64-bit hardware, so installation and, in particular, migration from an existing MOSS 03 or 07 platform needs to be planned carefully, even more so than before.

• SharePoint 2010 integrates very well with MS Office 2010, including Outlook calendaring and, heretofore ignored/talented stepchild, Visio. The most exciting part of this integration is the ability to use and edit metadata across SharePoint and Office. This propels true usability and greases the skids tremendously as far as adoption is concerned. The fact that Office is the reader and editor of choice for a lot of content held in SharePoint makes this integration truly relevant.

• SharePoint 2010 provides the option for imbedded FAST (the search engine). This is a big upgrade over MOSS 07 and brings enterprise functionality and scalability to SharePoint search.

• As usual, Microsoft has put focus on usability and in SharePoint 2010 there are some notable application usability improvements (such as the enhanced Ribbon, rules for content storage, smarter folders, a term store to manage taxonomy, offline capability through Workspaces, and, of course, the seamless integration with Office 2010).

• Ease of development: an oxymoron in MOSS 07—more like lots of development—at least as far as web applications were concerned. SharePoint 2010 promises to be far better with a revamped SharePoint Designer and more web services and UI and functional components out of the box. Also, the new Business Connectivity Services (BCS) promises to ease integration with third party applications (such as SAP or Microsoft CRM).

• SharePoint 2010’s other enterprise-like features: more comprehensive administration capabilities, easier deployment over distributed networks, better backup and recovery options, auditing/compliance features. Notable improvements in breadth and depth of such features over MOSS 07.

• Surprising improvements in records management capabilities, including the convenient, in-place record declaration. Though SharePoint has claim-to-fame largely for collaboration, and in SharePoint 2010, even more so, with a social networking slant (blogs, wikis, Cloud-tagging, activity feeds into its MySites, etc.), Microsoft has stealthily taken strides to make SharePoint a legitimate records management alternative. SharePoint 2010 is still some way short of the maturity and depth of features provided by the more established records management tools in the market, however ease of use and pull of momentum are definitely in its corner. Watch this space for more in-depth research and opinion to come…

So, some compelling reasons to drink the Kool-Aid. Maybe none more compelling though than its inevitability. Inevitable as, errr, Mr. Know It All not knowing anything at all! SharePoint is here to stay, as proven by the already-out-of-control “SharePoint Sprawl”. Also, frighteningly (CMIS implications aside), Microsoft is quite possibly the only truly one-stop-shop ECM vendor around…not that that’s really all that good: one vendor fits all or the best vendors fit together?

So, buying the hypothesis that it is inevitable, what next? Well, first, what is needed? What does the user need? No tool will solve a problem just because…in fact, in vacuum, it’s likely to compound it. What is the problem? What are the, ahem, six 5Ws (What, Why, Where, When, Who, How)? Back to basics. Yes, clarity, simplicity rule. Don’t let the expensive consultants tell you otherwise. It’s my experience that most of us spend a lot of time and induce a lot of heartburn on edge cases. Spend time on what really matters, and then de-scope and simplify that even more. If, after all that, SharePoint 2010 is still inevitable then consider these three things:

• What does the user need?

• What does the user need?

• What does the user need?

As engineers, we will spend a huge amount of time and energy on what we would like and what works for us—can’t everything be ext-js, RESTful, Spring Observer, n-tiered, service oriented, virtualized, mirrored, horizontally scaled, run on distributed ESX on redundant 4 way blade servers?! Oh, and, please adhere to the very large and complicated Enterprise IT Policy and System Architecture document (because it’s “Enterprise” it must be the right thing to do, right?!), and to every word of every paragraph of the System Requirements Specification (SRS)—written eighteen months ago by a large committee of “stakeholders”—and be sure to offer up all of the vendor’s glorious catch-all feature-set… Surely, exaggeration to make a point…not really. Bottom line either way: no user: very bloated, very expensive, very politically damning shinny shelfware.

Getting off the user-need soapbox for a second, the next BIG thing to figure out when thinking SharePoint 2010 is migration—nine times out of ten…or maybe 97 times out of 100 it will be needed. Migration from an existing SharePoint (03 or 07) application to the 2010 toolset, migration from a legacy application to SharePoint 2010, migration from another CMS to SharePoint 2010, migration of content and metadata…pick and choose your weapon of choice. Experience tells me that migration is NEVER EASY and its scope is almost ALWAYS UNDERESTIMATED. There are a number of tools that can help, but the solution goes way beyond tool selection; there needs to be a thorough plan (actually a Plan A, a Plan B, a Plan C and maybe even a Plan D). Oh, and let’s not worry about fixing the sprawl just yet…that’s a much bigger deal!

Not to be a buzzkill, but that’s not all. There is a lot more to consider depending on your specific view of the world. One other thing for sure though is licensing. In the great (and truly annoying) tradition of most Enterprise Software vendors, the licensing is a matrix of variables and if-thens. Every family needs a resident cost-accountant to truly savor such mind-bending aesthetics…

So, SharePoint 2010 represents some interesting opportunities for users and organizations and poses some further dilemma in the ECM space. A definite improvement over MOSS 07 but not (yet?) the iPhone of its genre. Competitors beware, Microsoft is coming…is here.

Lots to look forward to!