SOA - Integration Industry Pulse

Tracking SOA Transactions

2009-04-08T02:23:01Z

This week AmberPoint introduced its Business Transaction Safety Net™. This non-invasive solution monitors, records, provides search capabilities and can isolate root cause of a failure for end-to-end business transactions that cross applications and platforms. This release doesn't just track transactions - it allows real time visibility into the overall distributed application and can provide alerts or other actions in the case of failures.

Finding root cause of failures in distributed heterogeneous environments, where a transaction may involve multiple applications and platforms is a highly complex issue. While some tools enable you to pre-define criteria, Amberpoint now provides an automated way to search the log using technical or business criteria to discover the cause of the problem. The search capability enables operations to quickly figure out what went wrong. It supports policy based logging of all transaction history. You can define what to keep, what to encrypt, and even what to do with the data.

Tracking the cause of a failed transaction across distributed processes and platforms is a complex and time consuming process. Therefore, it's probably fairly easy to figure out an ROI for this solution. If you have the problem, and automated solution can save lots of time and money.

The key is having the right metrics to make that decision. That's also part of governance. While SOA governance is typically discussed in terms of design time and run time repositories, in a larger sense it's about clearly defining the roles, responsibilities, processes, policies, and metrics, and then monitoring and managing them, throughout the entire lifecycle, taking corrective action when needed. It requires aligning bonuses to overall success (something the financial industry especially needs to work on).

Hard times tend to make people and organizations marshal their resources more carefully, and spend on only what is necessary. When it comes to IT, how do you know what is absolutely necessary? What are the requirements, guidelines and metrics used? Who defines them? Who ensures they are followed? Once you get the roles, responsibilities, processes, policies and metrics right, there's a lot of great technology out there than can automate management for you. Success demands that you know how to use the technology effectively. How do you do all that in hard times? Pick your battles carefully. Only do what is necessary, but do ALL that is necessary. Effective governance is an absolute necessity.

Can ITIL help fix SOA?

2009-03-24T21:25:31Z

Tony Baer presented his research on the connection between SOA and ITIL to the SOA Consortium. The inspiration for Tony's research was a concern over the growing numbers of governance silos that were becoming apparent. After being briefed by a vendor who announced an SOA life cycle management platform Tony immediately recognized the overlap across numerous other tools. Tony concluded that there was significant similarities between an SOA Service life cycle and the ITIL Service Management Lifecycle and considered how exchange of information between different groups at specific points in the life cycle could help integrate the silos.

Tony focused on run-time SOA governance, because he said that is where you see all the issues. He gave an example for the interaction between development and IT operations where the development group monitors and manages SOA at runtime and IT operations monitors and manages the underlying infrastructure. When an SLA issue arises with the SOA service, development analyzes the problem as the monitoring system automatically submits a trouble ticket to the service desk. IT operations then applies incident, problem, and change management processes (all defined in ITIL) if the problem is attributed to infrastructure.

Todd Biske was on the call, and talked about the role of a Service Manager. Tony enthusiastically embraced the concept as a way of assigning clear responsibility for life cycle management of each service.

I was very interested to hear what Tony and others on the call would have to say about ITIL and SOA because I believe that governance is key to SOA success. One of the most oft touted benefits of SOA is that it creates business alignment. However, there is nothing in the SOA concepts and precepts, including loose coupling, layers of abstraction through composite services, reuse, that actually delivers IT and business alignment. Governance does that.

While Tony focused on run time governance, I feel that the governance process must start at the very beginning. In fact, ITIL has some excellent processes for determine which IT projects to invest in, and track the value of IT investments to the business. ITIL also includes the definition of roles and responsibilities. IMHO this is crucial to SOA governance. A single service may be used by different parts of the organization. Policies governing that service may be defined by different people in the organization. While it all hits the fan at run time, defining this much earlier in the process using ITIL processes and best practices, will make run time much more manageable. Once you know who is responsible for what (which I'm afraid is a far more complex issue than just having a service manager for each service) then the technology is available to send alerts to the right person at the right level, or to trigger an automated process to remediate the problem.

Another big issue, not currently part of ITIL but which can be part of the Enterprise Architecture framework, is a data architecture for defining the policies and procedures for repository integration. It is clear that multiple repositories will remain. There is the LDAP repository for security information, the CMDB defined in ITIL for configuration management, the SOA registry/repository, and the runtime repository, and probably other home grown metadata repositories. Each of these repositories is the authoritative source for some information, and probably also duplicates information in other repositories. The problem with redundant data is when it is out of synch. Integration technologies and techniques make it possible to automate the synchronization of information across repositories. But unfortunately different parts of the organization own the different repositories and do not usually work together to ensure consistency.

IMHO what is needed is not a handoff of information during runtime, but an overall methodology, including processes, roles and responsibilities, best practices, metrics, and governors, to ensure the business maximizes the value of IT investments, and IT can manage the software and infrastructure in a cost effective and agile manner. ITIL also has frameworks for defining who gets value of out of services, which can help identify how service development is paid for.

I view ITIL as a good overall framework for finding best practices on managing IT assets to maximize business value. This is what will create IT and business alignment. Organizations generally choose a subset of ITIL that will work within their organization. These ITIL processes should be full integrated with enterprise architecture processes. SOA should be considered a subset of the overall enterprise architecture and not a separate new effort. In other words, rather than an interaction between ITIL processes and SOA processes, I am suggesting a full integration of portfolio management, enterprise architecture and SOA within the organization, through an overall governance framework. In fact,I would go as far as to say that governance silos are a recipe for failure, and think this is what we are seeing with the current state of SOA.

Sun Releases Enterprise Open Source Platform

2009-02-10T19:51:46Z

Today Sun announced the release of its open source GlassFish platform. Sun has embraced open source and has released the integrated platform as a more palatable alternative for the enterprise. The issue with open source is the time and cost to integrate different components, and support. According to Sun, the GlassFish platform offers price/performance savings of 90% over proprietary alternatives. The integrated platform makes it easy to install and manage, and Sun offers enterprise support. The platform is extensible and fully integrated with MySQL.

The fact that the platform is pre-integrated goes a long way to lowering the bar for organizations adopting open source. One of the hidden costs of open source is the skill sets, time and resources for implementing and supporting open source. Furthermore, an enterprise class open source platform includes a stack of technologies. Integrating and configuring the stack represents another cost. Each development team may configure the stack differently, increasing overall enterprise maintenance and support cost.

The GlassFish platform provides a standardized and integrated stack, helping to simplify management and decrease costs. The platform includes, a complete LAMP stack, a portal for web site development and collaborative work spaces, a full Enterprise Service Bus (ESB), and enterprise scale management and monitoring for SNMP. Technologies include Sun Web Server, Apache HTTPd, GlassFish, lighttpd, Memcached, Mod_jk,perl, ruby, PHP, Ruby, Python, Squid , and Tomcat). While Sun is not expecting customers to dump their WebSphere and WebLogic platforms, they are betting that the downturn will give organizations a fresh reason to look at open source for its price/performance advantages.

An integrated and supported open source platform lowers the barriers and decreases the risk to large organizations who may very well be taking a conservative strategy to the economy. An easy to use cheaper platform should be well worth at least a serious look by organizations looking to lower infrastructure costs.

Sun is betting many organizations will opt for the Enterprise supported edition. For organizations already doing open source, there may be significant cost savings through standardizing LAMP configurations across the enterprise. The pricing model for support is based on the MySQL Enterprise pricing.

MySQL Enterprise
(per server per year in USD)
•Basic • $599
•Silver • $1,999
•Gold • $2,999
•Platinum • $4,999

GlassFish Portfolio
(per server per year in USD)
•Basic • $ 999
•Silver • $2,999
•Gold • $5,999
•Platinum • $8,999

Organizations wishing to standardize on the portfolio will be interested in the GlassFish Portfolio Unlimited which starts at $120,000. For the price of a single CPU of Oracle Enterprise Edition, companies with up to 1000 employees can deploy an unlimited number of GlassFish Enterprise Server, Web Stack, Web Space, or ESB servers with full 24x7 production support. Includes 24x7 access to GlassFish Manager.

Open Group Releases TOGAF 9 Today

2009-02-02T21:10:48Z

Today the Open Group announced the release of TOGAF 9. Perhaps not earth shattering news to many, based on the whole scuttlebutt about SOA being dead, which to me says that the discipline of architecture is dead. However, as the design points of this release were making TOGAF easier to use and easier to plug into external frameworks including ITIL, those who are struggling with architecture within the enterprise may wish to take a second look at TOGAF 9.

The Open Group has been around for 25 years, and claims over 7,800 participants from 350 member enterprises. They are completely vendor neutral, and work with other frameworks and standards. Some other interesting statistics include over 90,000 downloads of TOGAF, 8,491 certified practitioners, and 529% growth since October 2006. When it comes to enterprise architecture, I don't think there is a competing alternative that comes close.

TOGAF 9 encompasses the entire enterprise architecture life cycle, which is important as architecture is a never ending journey, always changing and evolving. The figure below depicts the TOGAF Architecture Development Method (ADM) which covers the entire architecture life cycle.

The Open Group members were looking for an evolutionary step for version 9, not a revolution. Design goals for the release were greater usability, closer alignment with the business, and more guidance on SOA. To deliver on this, the Open group has expanded detail and clarification of existing proof points, focused on holistic enterprise change, created clear links between business and IT objects, including a metamodel for an enterprise architecture repository, and has provided more consistency of output deliverables. TOGAF 9 was developed around a web based view, and has a more modular structure than the previous version. Below is a depiction of the components.

There is also more guidance on how to use TOGAF in a consistent way within the enterprise. While there is no single architecture that fits all needs in all enterprises, there is a body of knowledge and capabilities to help organizations more successful in their enterprise architecture endeavors. This is what TOGAF 9 attempts to do - to show organizations how to apply the architecture development method (ADM) . And version 9 includes hooks to ITIL, Zachman, and other frameworks. This means that organizations looking to get a leg up on their SOA implementation by purchasing, for example, one of IBM's industry solutions, will then have a way to incorporate the framework into TOGAF and bring it under a consistent life cycle management methodology. This will help organizations develop architectural competency and consistency - both essential to long term architectural agility and ROI.

For this release the Open Group has also created a metamodel for the TOGAF framework. In another post I plan to compare this framework to the OMG SOA Consortium framework. In an effort to increase usability, the Open Group has also published a metamodel on how the different components of TOGAF 9 relate to each other.

I was looking forward to discussing this release with Alan Brown, President of The Open Group and Judith Jones, President and founder of Architecting the Enterprise which does TOGAF certification, particularly in the light of the recent pronouncement by some analysts that SOA is dead. SOA, is after all, a specific style of distributed architecture. If SOA is dead, then it is in large part because organizations are not particularly adept at the discipline of enterprise architecture. Bringing TOGAF into an organization and going through TOGAF certification represents a big commitment to architecture.
Were organizations making that commitment during the down turn?
According to Judith Jones, the answer is yes. When I asked her what reason the organizations who have recently become members cited for joining. She answered that these organizations view the downturn as both an opportunity and a necessity to take a very strategic look at their business operations and architecture, determine what is holding them back, and deciding not to keep making the same mistakes. According to Jones, these organizations are viewing architecture discipline as the basis for moving forward with agility. They regard competency in enterprise architecture a critical need. Interestingly, she noted that new members include financial services and retail organizations - both which have been hit hard by the recession.

Caga Tió and Other Improbable Beliefs

2008-12-24T20:39:42Z

I recently took part in an old Catalonian Christmas eve tradition of hitting the "Caga Tió" which means, and I promise that I am not making this up, "shit log." Instead of Santa, Catalonian children have a shit log. Before Christmas they feed the log, cover it with a blanket, and keep it warm near the fire (but not too near). Then on Christmas Eve Caga Tió is covered with a blanket and the children gather round beating it with a stick chanting a song:

Caga Tió,
Caga Turró,
avellanes i mató,
si no cagues bé
et daré un cop de bastó.
¡Caga tió!"

English translation:
Shit, log,
shit torrons,
hazelnuts and cheese,
if you don't shit well
I'll give you a blow with a stick.
Shit, log!

Here's the chant we used:
Shit well my log,
Shit for the Lord (this is after all, a religious holiday)
If you don't shit well
I'll beat you to hell
¡Caga tió!

Then you pick up the back of the blanket and there is a delicious chocolate dessert. No ummm kidding. Catalonian children love their shit log for the treats and gifts it brings. (For more on this fascinating holiday tradition you can check out this video)

So what does this have to do with IT? Well, it occurred to me that some IT professionals believe in some magic bullets no less strange than thinking a log will excrete dessert and gifts. For IT, there is always the next magic bullet. I've been around long enough to witness a few - CASE, 4GLs, Client/Server, EAI. In 2008 the magic bullet was SOA. In 2009 cloud computing is being set up as the next magic bullet.

To be clear, I am not equating SOA with a Caga Tió. It just got me thinking about magical thinking. For example, the belief that you can use the same skills, methods, and organizational structures of the past to create new agile service oriented enterprises. Frankly, you will probably have better success throwing a blanket over your IT installation and hitting it with a stick. Maybe it will excrete reusable Web services.

The real hard truth is that logs don't shit candy and gifts and you don't achieve SOA success by implementing a few Web services or wrapping some legacy code in a WSDL interface. Transforming IT means transforming roles, responsibilities, processes, and methods. Anne Thomas Manes of the Burton Group has reported that most companies she has interviewed are not reporting success with SOA. The problem, according to Anne, is that the "techies have not been able to explain to the business units why they should adopt a better attitude about sharing and collaboration--which is the fundamental cultural shift required for SOA to succeed. The pervasive attitude is "What's in it for me?" As one of my interviewees said, "Altruism is not an enterprise strategy". The only success story she reported was of an organization which re-organized itself around functional capabilities and established strong incentives for reuse.

The bottom line is that if organizations wish to become more agile during the downturn they are going to have to give up magical thinking processes and belief in magic IT bullets, hunker down and do some real work around organizing for success. This means investing in retraining, and reinventing processes and methods so they can move past the obstacles that exist today. IMHO one of the most important things organizations need to get right is governance. Setting policies and procedures, enabling flexibility through federation while maintain control over what is core to the organization. Business and IT governance need to go hand in hand, and need to become central to any effort to bring greater agility to the organization. It is even important for successfully moving forward while investing less. It's about making your investments count, and tracking and measuring success.

And to those who say you can do proper governance in a spreadsheet, I would suggest that you may have a better chance of success by throwing a blanket over the spreadsheet and hitting it with a stick. But it's not all about having a registry either. It's about what you do with it. You need governors for proper governance. You need policies, procedures, and you need to track them throughout the life cycle. You need reward structures that guide people to desired behavior. You need the right organizational structure to enable governance of the policies, and ensure that business gains value from its investments.

In hard times, business and IT need to give up their magical belief that technology will fix the business, and start focusing on how to optimize the way business is done. Technology enables the story, but business should use technology to its advantage, and give up believing it will magically solve all the business problems.

Save your magical thinking for Santa and Caga Tió.

Wishing you all very happy holidays.

SaaS Development Platform

2008-12-24T01:53:14Z

One of the major trends for 2009 will surely be cloud computing, also called SaaS (software as a service). In bad economic times, organizations have less money to spend, and even more pressing needed to do more with less. Usually technology helps enable increased productivity. But paying for a software license, hardware, installation, training and support adds up, sometimes to more than the budget of a single project.

Integrating technologies to enable new types of Web 2.0 applications may be out of reach for organizations facing funding freezes as deep as the snow in the northeast this week. The answer could very well be cloud computing or SaaS. You get the advantage of all the latest technologies, but at a fraction of the cast because it's a shared service.

Recently SpringCM announced it will be licensing its SaaS platform for application development. SpringCM has integrated 22 different content technologies, including imaging, document capture, search, version control, business process automation, fax, e-mail, records management, and a managers' workflow console, into a single SaaS application. The platform has enterprise class security, and each individual has a separate login in with permissions.

The SpringCM 5.0 application development can be used for content rich applications including (but not limited to) accounts payable, logistics, contract management, proposal management, and compliance applications. Solution partners have built reusable templates on the Spring CM platform which can be configured for clients including customized menus and toolbars, folder and metadata models, workflow templates, custom reports and eForms. The SpringCM 5.0 platform also enables integrations with third-party systems such as Enterprise Resource Planning (ERP) and Customer Relationship Management (CRM).

The SpringCM platform can be used by Independent Software Vendors (ISVs), consulting organizations, document services organizations such as Business Process Outsourcers (BPOs) and scanning bureaus, to create on-demand versions of their offerings, at a fraction of the cost and time it would take to build their SaaS platform.

According to Dan Carmel, CEO, applications built on the SaaS platform take approximately 3-4 weeks to deploy. There is a $1500 setup fee that includes 12 hours professional services. Carmel stated that most applications are up and running in the first 12 hours. There is also a $50 per seat charge for a specific application.

Partners that are offering solutions built on the SpringCM platform include:

• Clickability, which is integrating with SpringCM's existing marketing asset management capabilities to create the industry's first "Word to Web" SaaS solution linking marketing content creation, review and approval to comprehensive Web publishing.

• Pitney Bowes Inc. has developed an application to leverage the SpringCM 5.0 platform to streamline Freedom of Information Act (FOIA) compliance for government agencies and case management professionals.

• Ricoh Netherlands, Belgium, and Luxemberg, world leaders in document-based technology solutions, are exploring market opportunities and several content applications on the SpringCM platform to expand Ricoh's presence in the total enterprise solutions market.

• Conversion Technologies International, Inc. has developed the Healthcare and Patient Records Management solution to speed the capture, management, and workflow automation of patient health records, resulting in significant cost savings and efficiency improvements.

• Harding Group Document Management Solutions has developed the AkuTrust eRecords Management (ERM) solution for the streamlined management, retention and review of student records in institutions ranging from K-12 through higher education saving to improve records management while saving money.

• MACH 2 Solutions has developed the Digitized Patient Information Management solution for capturing clinical data at the point of creation, turning hospital clinical healthcare data into valuable and actionable information and simplifying the management of patient health information as well as HIPAA compliance.

• Security Compliance Advisors has developed the ComplianceCare for FINRA Document Compliance solution that enables SEC registered brokers to automate and standardize over 52 specific compliance events required for mandatory compliance and associated audits.

• US Archive & Imaging Services, Inc. has developed the Digital Mail Toolbox, a mailroom solution for non-profit service providers to enable the digitizing, management and distribution of incoming mail, and all related documents, in order to improve accountability, efficiency and cash flow.

• Waypoint, LLC has developed Integrated Business Capture for Government Contractors, a solution for managing pre-award business services to create revenue growth for clients selling into federal, state & local government as well as commercial markets where new business is won through structured, competitive procurement processes.

Predictive Analytics - A Good Investment for a Down Economy

2008-11-25T20:07:18Z

I know things are tough out there. Belt tightening is a fact of life when the stock market is tanking, and the countries financial institutions and major industries are going down the toilet. But the fact remains, that to get through it, companies are going to have to invest something just to stay afloat.

My pick for an IT investment that has the potential to deliver ROI in the downturn and real competitive advantage once things start looking up is predictive analytics. Predictive analytics is a capability based on complex event processing (CEP). CEP takes disparate events from across different processes, and correlates them to patterns. For example, a set of events might indicate that a service level agreement or a compliance regulation is about to be violated. It alerts managers to problems BEFORE they impact the business. While this can have real competitive advantage in enabling managers to recognize and act upon opportunities, it also has great money saving potential in a down economy.

In the economic downturn, as companies need to keep their eye on the ball and do more with less, they cannot afford to spend money is non-value add activities, such as tracking down and remediating system failures which impact the business. In this climate it is even more important to invest in technologies to reduce risk of failure. Predicting problems before they happen, and then automating the solution as much as possible, reducing the time and number of people necessary to fix problems is even more important now.

While CEP is being talked about quite a bit in the press, I prefer to focus on the predictive analytics. In my view, CEP is the means to this ends. Predictive analytics will help companies keep their eye on the ball. Once a pattern is recognized, the system can initiate automated processes to remediate the problem, or alert the stakeholder who can step in and avert the problem before it ends up costing the company money. This capability is useful to all levels of management, from systems management up to business management. Richard Nikula, Senior Corporate Solution Architect of Nastel, defines predictive analytics as "leveraging data to provide context sensitive decisions, or more simply put, applying available information to understand current and future behaviors."

I'm a true believer in predictive analytics. If you can have the system find problems, alert the right person, and even automate some solutions, it will save time, money, avert potential disasters as well as put you in a good position to take advantage of competitive opportunities. And in this economy, it's going to take all our focus and tight management to even survive. We can't afford to make mistakes.

If you want to learn more about predictive analytics, tune into the December 4th webcast: An Ounce of Prevention vs. a Pound of Cure: Predicting and Preventing SOA Problems Before They Happen.

SOA Adoption for Dummies

2008-11-14T20:13:33Z

There's a new SOA Dummies book out there, not to be confused with Service Oriented Architecture for Dummies by Judith Hurwitz, which I hear is coming out with a new edition. SOA Adoption for Dummies is a small thin book - under 100 pages - much like the Service Oriented Architecture: Getting it Right book (reviewed in a previous blog). The book was written by Miko Matsumura, VP and deputy CTO, Bjoern Brauel, VP and Deputy CTO, and Jignesh Shah, VP of SOA Product Management and Marketing, all from Software AG. Click here for another ebizQ review of this book.

You can read this book on a plane ride, which is a good thing. Even better, you can learn some valuable best practices for SOA adoption. What I liked most about it was the continual advice that SOA success requires attention to the human element. Boy is that ever true! The book presents both a SOA architecture blueprint and an organizational blueprint, talks about tribal warfare and how it can impact the success of projects.

The book also introduces a very high level methodology called SOA Rocket Science. I asked Miko Matsumura what format the methodology existed in and his answer was "We do have a methodology called GEAR, but since rocket science is so new, it's not yet baked into GEAR. GEAR is pretty close though." However, I couldn't find anything about GEAR on the Software AG website. My guess is that SOA Rocket Science was a cutesy way to talk about SOA adoption. The three SOA rocket science principles are:

1. Keep the pointy end of the rocket up. Meaning you should measure continuously and make course corrections along the way
2. Keep moving up. Meaning that you need to engage funders, executives, other stakeholders, and show business value.
3. Don't stop till you are weightless. Meaning you should automate policies and processes to make governance effortless.

In all fairness the Dummies title pretty much demands cutesy, although I must say that Service Oriented Architecture for Dummies actually manages to be clever rather than just camp. But the two books have different purposes. While the Hurwitz book is meant to be a full SOA primer, this book is focused on adoption, and the importance of aligning organizational polices, processes, roles and responsibilities to enable business agility and SOA success. Instead of being a full primer, it is meant to be read in an hour or two. You can download it on the Software AG site , or you can attend SOA in Action on Wed., Nov. 19th, for an opportunity to win one of 5 copies.

Keep the pointy end up and don't stop until you're weightless. Do you think that could work as an enterprise methodology?

SOA: Getting it Right

2008-11-11T16:49:23Z

SOA: Getting it Right is a mashup book, edited by Jim Green, CEO of Composite Software, with chapters contributed by Jim Green, David Besemer, CTO of Composite Software, Luc Clément, Co-chair of OASIS UDDI Specification Technical Committee, Hub Vandervoort, CTO of Progress Software, Paul Butterworth, CTO of Amberpoint, Hemant Ramachandra, Managing Director of Business Systems Integration for BearingPoint, Jeff Schneider, CEO of Momentum SI.

In the first chapter, Green admits that some of the authors have different contradictory views of certain subjects. In the opening chapter, Green states "Hopefully, this makes the book richer, and doesn't introduce confusion." IMHO, what it mostly does is make the book uneven, and sometimes confusing. But there is still much to be admired here.

One of the helpful design mechanisms of the book is that each chapter begins with a summary of key recommendations. They key recommendations are hard to argue with, and I think add value to the book. The chapters include: Designing Services, Registries and Repositories, Enterprise Service Buses, Runtime Management, Organizing for Success, Capability Development, and a final chapter on pulling it all together including where to start, how to define scope and measure success, and a summary of the key recommendations from each chapter in the book.

My favorite chapter was Hub Vandervoort's explanation on ESBs. I recommend you pick up the book for this chapter alone. I've been doing research on ESBs since the term was coined, and there was much I learned from this chapter. Vandervoort discusses different types of ESBs, and the different applications they can be used for. He includes a chart on usage patterns and scenarios for different industries. Whereas most of the other chapters were at a pretty high level, there is much in the ESB chapter to sink your teeth into.

My big Ah Ha moment came while pondering the difference between David Besemer's depiction of service levels(p. 13) and Hemant Ramachandra's service layer model (p. 80). Besemer defines Physical Services which lie just above the data source and transform data into a form utilized by higher level services, Business Services which embody the bulk of transformation logic that converts data from physical form into required business form, and Application Services which leverage business services to provide data to consuming applications. Ramachandra's layers include business capability, business service level, technical component level, technical service layer. Now, granted that Besemer focuses just on data services, it would be nice to see how these relate to other breakdowns of service layers. My Ah Ha came when I realized that while we talk as IF people knew how to design services this is not the case at all. There is no one agreed upon reference model for service layers. Most of the organizations I have spoken with have low level data access services (read database calls - JDBC, ODBC calls). While we may be starting to tire of talking and writing about SOA, we still have a long way to go before realize the full benefits.

Bottom line, I think this book deserves a place on your bookshelf. We are going to be giving away 5 copies at SOA in Action. Join the action on Wed., November 19, for a chance to win.

Innovation World

2008-11-04T18:55:35Z

I'm down in Miami at Software AG's Innovation World. The first innovation was that they changed the name of the annual customer conference from Integration World to Innovation World to better describe the expansion of the stack and the reason customers implement it.

One of the most interesting sessions I attended was an analyst round table. This was a far ranging discussion on how organizations are adopting SOA, why some are failing, and which technologies are likely to be most important in the future, and which are most important in the economic downturn.

One of the topics we discussed was how do you succeed with SOA, and is SOA even relevant going forward. While most of the analysts there felt that SOA was going to be the dominant architecture pattern going forward, it will take the better part of the next decade to get there. Dave Linthicum , always the provocateur, put forth the vision that angry masses were waiting at the gates with to torches, because it's too difficult to implement. Personally, I think a big problem is that the skill sets are not yet there, and that's why we're seeing the failures. Also, the press will soon, or is already starting to experience SOA acronym fatigue. But even if we stop using the acronym, few in the room envisioned a retreat from SOA, because it's really the only architecture that makes sense. However, organizations will need to understand how to incorporate other architectures, including cloud computing and EDA, and how to manage the disparate parts as a whole.

This is an area where I think Software AG may have an advantage moving forward. They are focused on reducing complexity by making things easier to install, and easier to use with integrated Eclipse based tooling. The whole platform is integrated, which Software AG says lowers the total cost of ownership.

Additionally, and very importantly, Software AG is enabling federated ESBs and repositories. The next version of CentraSite Active SOA will have some very strong support for federated SOA. It will even support development life cycle policies, which means companies can ensuring that development processes and policies are monitored and managed. Federated SOA is most likely to be the way most large organizations deploy SOA. It is complex, so any help here is greatly appreciated.

There were some discussions about how this might move to cloud computing, but Software AG is not announcing any plans to support Cloud Computing - at least not until their customers start demanding it.

Another interesting discussion was whether the underlying applications are the backbone of SOA, with BPM on top to integrate processes across the underlying systems, or whether the ESB is the backbone. The problem is that many organizations who have ESBs the center of their SOA universe have not been successful. There was a venture capitalist in the room who reminded us all that money is the center of the universe, not IT infrastructure. This was a useful point of view. In fact it seems that Software AG, a fiscally conservative organization, is indeed following the money. According to CEO Karl-Heinz Streibich, Software AG's software has the lowest TCO in the industry.

One of my favorite Albert Einstein quotes is "Make things as simple as possible, but no simpler." All distributed architectures are complex. There are many underlying platforms. The role of middleware was supposed to smooth out the differences to make it easier, but middleware itself has become complex. Abstracting complexity is the key to SOA success.

Microsoft's Azure Cloud Computing Platform

2008-10-30T19:54:37Z

This week at its developers' conference Microsoft announced, Azure, its new cloud computing platform, and gave analysts a peek during a conference call. The industry trends of cheap computing, cheap storage, high bandwidth, and a proliferation of devices with internet connections, are creating a demand for cloud based computing, according to Microsoft. Microsoft is calling this the "third tier of computing" in which the first two tiers are functionally rich PCs, back end application servers, and then the Web tier that supports both personal and business processes. Never mind this fact that this is a gross simplification of distributed tiered computing, the general idea is that the Web will become an application tier.
Microsoft is describing this tier as software plus services. These services include comprehensive development services, consistent programming models (read .NET only here), security and privacy, and control and customization. Below is how Microsoft is depicting the capabilities of the platform.

Microsoft will provide Azure as a SaaS offering which will only be hosted in Microsoft data centers. Initially the services will be accessible from anywhere, but will be hosted in the US. Eventally Microsoft is planning data centers around the world. There are no plans to ship versions which can be deployed within the enterprise. Microsoft is going to provide an SDK so developers can design, test and run application on a local machine before deploying to the cloud. While Azure supports REST, SOAP and XML, only .NET services can be deployed on the platform.

Azure includes advanced tracing and logging to allow development personnel to understand what is happening with the application. It provides rules based access control. The ESB will provide integration with other applications. While Microsoft stated that the problems of traditional integration can be resolved through mediation through messaging, and cloud computing offers a way to connect with partners and suppliers, to me there was a disconnect in this message. SaaS platforms offer cost effect solutions when partners and suppliers can simply connect with one platform

Microsoft stated they have been providing SQL Services in the cloud since the Spring of this year, and the service has proved popular with developers seeing an expensive way to scale out storage. However, the question remains as to how organizations will use cloud computing in the future. Neil McAllister of Infoworld makes some very good points about cloud computing creating lock in. He also sites issues with version control, and keeping up with new versions of the cloud platform and how that will impact existing applications, or more precisely, force you to change them. All important to keep in mind when evaluating development and deployment platforms.

My major concern is manageability. My consistent mantra for the past 20 years of designing distributed systems has been "Don't build what you can't manage". I asked Microsoft about testing these cloud apps. Answer - not there yet, it's a good opportunity for partners to provide a solution.

What about managing the services in the cloud? Microsoft's repository is Oslo. However there is no specific plug in to integrate the management of policies and SLAs for the cloud based services. Microsoft stated that this may be coming sometime next year. In fairness to Microsoft, I think this is a general issue in the industry. There are too many different repositories for different kinds of policies and no integration or federation between them. Bottom line - how do you manage the policies end to end? This is a general issue that organizations should be a bit concerned about.

How about security, a very big concern with cloud computing. Microsoft realizes this. Their answer today "they will be providing additional information as they go forward with the platform."

Azure is not yet GA and Microsoft has not yet announced a release date. We are still in the very early days of cloud computing. What should organizations be doing now? I think they should be evaluating what makes sense to put in the cloud. Data services and on-demand as needed, off-site storage makes a lot of sense to me. How many of you now are backing up your laptops with an online service? Collaborative applications also make a lot of sense. These types of applications are easier and cheaper to deploy in the cloud. Another use of Azure would be to make Microsoft applications, such as Office, available as a SaaS offering. This would make a lot of sense given the rise of Google apps.

So while cloud computing is certainly in our future, my advice to organizations is to move cautiously, see what makes economic and business sense, and DON'T BUILD OR DEPLOY WHAT YOU CAN'T MANAGE!

For more information on Azure, check out David Chappell's white paper, available on the Microsoft site.

The Rise of Network Services

2008-10-17T05:11:13Z

This week I had an opportunity to speak with Lynn Lucas, Director, Enterprise Architecture Marketing and Chris Wiborg, Enterprise Architecture Marketing Manager, who are both part of Cisco's enterprise architecture marketing team. Their goal is to define the relevance of network services to Enterprise Architects and CIOs.

With the acquisitions of Reactivity and Securitant, Cisco has expanded its network services and created an architecture they call SONA - Service-Oriented Network Architecture.

The services offered include:

Of interest here is what Cisco separates into Exposed Services and Transparent Services. The transparent services are in light grey and indicate services which do not require application developers to specifically write API calls to. The services are automatically provided by the network. A big plus of this architecture is the virtualization services offered by the network. The network seems the right place for these services, making them pervasive and manageable. Exposed services have callable interfaces which application developers can use.

The management services are focused on traditional network management, monitoring packet loads. However, SONA also has the ability to do deep inspection on message traffic and correlate with data for upstream application level visibility and management.

While SONA has capabilities which overlap ESBs, according to Cisco, it does not compete with an ESB. The suggestion is that the network will provide better performance for some types of services, but Cisco did not provide specific guidelines, or ROI examples of which services perform better, are less expensive to deploy and manage, or increase business agility when they are deployed in the network. Also, Cisco did not share a implementation reference model of how these services are deployed. For example, Quality of Service is deployed in different Cisco offerings. Then the question arises of how you meet a business level service agreement which you have to correlate policies and run time management across many different layers in the architecture.

The real challenge is that SONA is a conceptual architecture, not a product or packaged solution. It is meant to give an overview of all the types of services they offer through the network. However, my hunch is that enterprise architects are going to need a little more help in determining where the real value of SONA lies for their organization. Anyone out there currently using network services? Are their use and benefits obvious to you?

This month Cisco announced an integrated offering with Microsoft of a "headless Windows Server."This is an implementation of a Microsoft Windows Server 2008 on Cisco's Wide Area Application Services (WAAS) network appliance. The solution improves the remote-office user experience of centralized applications through Cisco WAN optimization technology and helps customers lower IT costs by reducing the number of Windows-based physical servers required in remote offices. The Cisco WAAS wide-area network (WAN) optimization appliance offers virtualization capabilities for hosting Windows services in branch office environments. This type of packaged solution, which is an example of using SONA, is far easier to explain in terms of business use and value. Cisco also offers industry solutions that incorporate SONA concepts.

The last thing that puzzles me is why Cisco is targeting just CIOs and enterprise architects. If anyone would understand the value of having certain services in the network I would think it would be the network administrators. In the early days of designing distributed systems, I used to go in and talk to the networking guys, help them understand the new network demands of distributed applications, and encourage them to get involved early on in projects, BEFORE the application was deployed and suddenly brought the network to a crawl. I think it would be helpful for network administrators to get some tutoring on SOA, to understand the new types of applications being deployed across the networks, and work hand-in-hand with enterprise architects to develop the full set of infrastructure services needed to ensure the scalability, reliability, security and performance of new business solutions.

IBM Focuses on Business Outcomes

2008-10-15T15:54:15Z

Last week IBM gathered the analyst community together to preview a series of SOA announcements which, to my ear, contained more marketing approaches and services than product innovations. Senior IBM managers participated in the announcement, including Robert LeBlanc, GM, Software Sales, Tom Rosamilia, GM, WebSphere Software, Manoj Saxena, GBS, VP, Global Solutions & Asset Management, Mike McCarthy, VP, GTS Middleware Services, and Sandy Carter, VP, SOA & WebSphere Marketing, Strategy, and Channels.

IBM is touting its position as a leader in SOA implementations based on experience with 7022 customers and 7420 business partners. Now they are announcing a "First-of-a-kind, customized go-to-market approach" that includes a 100 city tour, with customers speaking in each city. The agenda and discussion will be localized and customized for each city. Over 1000 customers are participating in these presentations. One third of the cities are in emerging and growth markets.

IBM's approach is to focus on business outcomes and provide new products and services to enable a successful business outcome. The outcomes include driving agility into the organization, taming chaos by process, creating a smart SOA foundation, and meeting new customer needs.

Drive Agility into the Organization. Includes industry specific solutions and key agility metrics, and of course SOA middleware to implement all. Manoj Saxena introduced SmartBusiness INSight, which provides past and future process insights. Saxena also announced a set of industry specific frameworks with business trends and processes, market opportunities and threats. The full offering includes business design services, approaches for linking core business processes to business strategies, processes, and IT execution, and for prioritizing SOA/IT initiatives based on business priorities and pain points, as well as the IBM CBMSOMA Methodology which integrates development across CBM/BPM/SOMA.

For me, the most interesting announcement Saxena made was about the key agility indicators which IBM built from over 16000 engagements from global services. The announcement was for over 300 Key Agility Indicators over different industry processes. This KAIs are also being embedded into WebSphere Monitor and Modeler. These industry specific key agility indicators are potentially very valuable IP if they provide managers with insights that would not have otherwise been available. Saxena also announced the release of Chemical and Petroleum product development frameworks. So far, there have been nine frameworks released. These include a framework to monitor the profitability of products, and one to monitor call center performance vis-a-vis average in the industry and best in class. There is a framework for retail inventory turnover and sales per store and one for Insurance to determine combined ratio of policies written. There are frameworks for monitoring losses and expenses ratios, and other key things that drive business outcomes.

Taming Chaos by Process. This set of announcements, discussed by Tom Rosamilia, focuses on BPM enabled by SOA. According to Rosamilia, in this release IBM focused on making BPM easier to install, easier for business people to use, and easier to buy with more attractive pricing.

IBM is making BPM more accessible through Lotus mash-up technology, providing easy to use, unified, integrated interfaces for different user roles such as business leader, business user, business analyst, etc. For example, a business analyst would need an interface to modify rules and change policies. A process owner or analyst will have an interface for monitoring the end-to-end performance of the process. Now a common user interface crosses several products including WebShpere Monitor, Process Server, and Modeler. The BPM Suite is going to include more content management integration with the Active Content Edition. The second edition will be available in December.

According to Rosamilia, the world of events is "exploding" and with the next release discussed WebSphere Business Events & eXtreme Scale of IBM was focusing on the expansion of the billions of transactions processed by CICS and other platforms each day. IBM WebSphere Business Events, released in May, is the product purchased from Aptsoft. The next release will feature tighter integration with the rest of the portfolio, including integration with the ESB, integration flows, and into monitor to make policy updates. WebSphere eXtreme Scale provides in-memory management working with thousands of JBVs, and up to 1 million events per sec, to tackle big problems. The CICS Transaction Server Support Pac will be available in December, and will convert transactions into events and integrate with Tivoli to enable bilateral sharing with business events.

BPM Healthcheck Services, originally announced at IMPACT back in the spring, apparently was popular among customers and helps them identify current strengths and shortcomings in different dimensions and articulates a BPM Strategic Roadmap. Rosamilia also announced an expansion of this offering.

Build a Smart SOA Foundation. This business outcome includes a collection of products and services focused on the underpinning of SOA strategy, including Portfolio Strategy Analysis & Planning, Information Agenda, Entry Points, Connectivity, Governance, and security. The full list included in the SOA foundation offering includes:
• Business Value Assessment
• Online Value Analyzer
• WebSphere Portal
• QuickStart / Design & Implementation Services for WebSphere Portal
• InfoSphere Information Server
• IBM Mashup Center
• WebSphere Process Server
• WebSphere MQ File Transfer Edition
• QuickStart services for DataPower
• WebSphere Application Server
• Rational Asset Analyzer
• WebSphere DataPower LLM & B2B
• WebSphere ESB 6.2
• Detailed Business & IT Best Practices By Maturity
• Telelogic System Architect
• Rational Requirements Composer
• SOA business and infrastructure consulting services
• Tivoli Security Policy Manager
• Rational AppScan
• Professional Services for security

This is obviously the IBM laundry list, and it's a long one - not necessarily a good thing when trying to figure out how what is needed to implement a specific solution. During the call, IBM focused on a few of the areas. One message was that WebSphere Application Server (affectionately known as WAS) is a key underpinning of the SOA strategy and expected to be a key SOA development platform moving forward. A recent survey by ebizQ showed that the number criteria for choosing a development platform is existing skillsets, and the #1 skill set in organizations today is Java. From the beginning application servers have evolved from TP monitors, and kept adding services. This trend will surely continue as WAS has added support for Web Services, ESBs, and other services to enable it to remain a viable development platform for new SOA solutions.

Sandy Carter spoke about the role Rational Asset Analyzer in developing new types of solutions. The Rational Asset analyzer is being touted as the way to develop an inventory of services based on existing assets.

As part of this announcement IBM talked about connectivity enhancements, using Data power to enhance security at the perimeter. DataPower offers low latency messaging and will work with financial services and telco offerings. IBM also announced enhancements to XML parsing, and added B2B support for the ESB, including trading partner management and governance in DMZ for transactions.

Meet new customer needs. IBM put Web 2.0 and green computing under this business outcome. This includes easy customer assembly of mash-ups, easy access to enterprise SOA and web data, new widgets that support multiple platforms, and enhanced XML. WebSphere Portal now has a richer more responsive Web 2.0 experience for new customers, new AJAX user interface, new Rest interfaces, consumption of feeds, and new BPM integration. The full list of products, services, and education includes:

• IBM Mashup Center
• WebSphere Portal
• WebSphere sMash
• Rational Application Developer
• Insight on Using SOA for Green
• Smart SOATM Sandbox through Cloud Services
• Smart SOA Social Network
• SOA Certifications
• Innov8 Video game
• Carbon management dashboards through SOA
• IBM Academic Initiative
• SOA courses online and in-classroom
• IBM Innovation Centers
• Industry Framework Program

The areas focused on in the call were Rational Application Developer, which is being touted for SOA development, and enhanced Web 2.0 development tools, including visual development focused on the combination of Web 2.0 and SOA.

Sandy Carter also highlighted two new green offerings, including pre-packaged scoreboards to monitor green initiatives, including green focused KPIs such as water and energy usage. Additionally, IBM has published 7 insight papers around green SOA highlighted best practices around the world.

IBM is also offering what can be called a try before you buy option through the SOA Sandbox offered through cloud services. This allows customers to experience SOA solutions free for 30 days. The Sandbox includes business patterns, scenarios and use cases. IBM has also set up innovation centers around the world which support solution partners.

Some of the products and services announced during the call are available now, and some will become generally available in Q4. Parsing through the range of offerings announced, it seems to me that IBM's products, services, solutions, and educational offerings are aimed at closing the skills gap for customers to help ensure their success with SOA. At first my reaction was that we were being called in to applaud a marketing strategy. But the thing is, it really is a good marketing strategy, and one which capitalizes on IBM's unique strengths, so despite my initial reservations, I do in fact applaud it the general approach.

One question I hear often these days is "does SOA really work? We're hearing more about failures." The unfortunate truth is that while SOA really and truly is our best hope for delivering IT and business agility, it is not simple. It is not one product, not one solution. It is a new way to think about how to develop systems, and skills are in short supply and high demand. IBM stated that 61% of customers are in foundation or basic stage of SOA. They need help. The long, long list of IBM products for each business is an indication of the complexity of the problem. Focusing on industry specific solutions is a good way to get around that problem.

If you attend any of the sessions on the 100 city tour let us know what you think!

Systinet 3.00 Provides Enterprise SOA Governance

2008-10-14T18:28:30Z

Last week HP announced the release of HP SOA Systinet 3.00, which is aimed at making helping customers successfully move from a single SOA project to enterprise scale SOA. In actuality Systinet 3 is already in use by some customers. The date of the GA announcement coincided with the release of an ROI case study.

The three primary groups of users for Systinet are (1) Architects/Architecture Groups, and Centers of Excellence; (2) service providers; and (3) service producers. Architects, architecture teams, and centers of excellence can use Systinet 3 to monitor, manage, and elevate the consistency of business and governance process.

Services providers, the person or group responsible for the business service and overall evolution of the service, can use Systinet to understand and track how services are being used, and how they evolve to meet continuing business needs.

Service Consumers use services to create business solutions. Consumers need to know what services are available, levels of service supported, policies and processes. The repository is the central resource for all service information and usage.

The growth strategy for the enterprise class Systinet 3 is to expand across IT teams and Architects, then proceed into change management teams, and then upstream into portfolio management teams. Having a single platform across all helps support Business and IT alignment.

New features in HP SOA Systinet 3.00 include:
• Best-practices-driven automation. These are pre-built life cycles and templates that help bring non-experts up to speed quickly, and allow users to customize service life cycles using wizard-driven, graphical and application programming interfaces. Kelly Emo, HP Software's SOA Product Marketing Manager, stated that when new group needs to participate they can have something to use on day one. They just need to attach policies to pre-built lift cycles.
• Role-based visual dashboards that provide relevant information in a format that relates to each user's responsibilities.
• Integration across management tools integrates end-to-end life cycle management. Systinet can trigger business policies based on service quality through integration with HP Service Test Management or prompt actions to manage rogue services in production through integration with the HP Universal Configuration Management Database (CMDB).
• Governance of business processes. Organizations can increase productivity in their business process management initiatives with business processes that are easier to discover and reuse.
• Enterprise scalability. Organizations can automate repetitive tasks across a large number of services with new support for bulk operations and life cycle cloning. This simplifies the implementation of major changes.
• Expanded support for standards. Systinet now supports BPEL. It registers BPEL process artifacts, and associates them with owners, stages, policies, and governance processes. This integrates the management of BPEL processes. Systinet also now supports Open CSA (previously SCA) and WSDL 2.0.
• Solution Offerings. Best practice solution templates, methodology, global SOA competency centers, and EDS services.

A demo of the software and the IDC report on "Reducing Operations Costs and Improving Customer Experience with HP SOA Management" are available.

Enterprise 2.0

2008-07-30T19:32:02Z

Last week ebizQ held its Enterprise 2.0 virtual conference. The event had some great presentations by Gartner analyst David Mitchell Smith and Forrester's Rob Klopowitz. David Smith stated that the future of Web 2.0 is NOT Web 3.0. While organizations are adopting social networking and cloud computing, just two of the technologies and uses attached to the term Web 2.0, as they move forward they will focus on separate initiatives. Rob Klopowitz talked about the innate tension that occurs when you introduce innovative and somewhat disruptive technologies into IT organizations that typically try to control all computer and network use.

The conference included a live panel with Dan Woods, CTO and Editor, Evolved Media, Nathaniel Palmer, President, Transformation+Innovation; Executive Director, Workflow Management Coalition and Puneet Gupta, CEO and Founder, Connectbeam. While David Smith reported that Enterprise 2.0 initiatives have often been led by IT, in an audience poll done during the panel the attendees incidated that the majority of their initiatives are being led by business users.

By coincidence, the same day I got my new MIT Technology Review and the issue was focused on the "Future of Web 2.0". While the analysts pointed out the ways the new capabilities of Web 2.0 are changing the ways organizations work and collaborate, the MIT articles pointed out that no one has actually been able to figure out how to make money from social networking. However, it is growing very rapidly and most pundits feel it is only a matter of time before they can monetize all the eyeballs on the social networking sites.

So what does this mean to the organization? First of all, we found that while the terms Web 2.0 and Enterprise 2.0 are useful to describe the new capabilities, vendors are not lining up behind the term, and the analysts indicated the terms will soon be obsolete. And while the vendors creating the new technologies have not yet figured out how to make money, and enterprises have not yet defined an ROI, social networking is starting to seem like an inevitability, which may replace email in many instances.

In fact, the virtual conference environment itself is an example of Web 2.0. It includes a networking lounge and public chats in the lounge and in booths, and person to person chats so visitors can contact other visitors. Allison of Dorsey Consulting contacted me through the chat and said "Great experience! I will be back. Your team did a good facilitation job today, and I enjoyed it. Poked in booths, chatted with people, made a couple contacts I will follow up in. Almost like a trade show . . . .Onward to virtual experimentation!".

As we experiment and experience more in the virtual world the Web 2.0 nomenclature might go away, but it seems the technologies are here to stay - and change the way we work and interact. Is your organization on the bandwagon yet?