Granularity and services? There are two schools of thought here: Coarse grained and fine grained. Lately there has been much confusion around this issue especially from those who consider fine grained services always a bad thing. Allow me to clear things up.

The coarse and fine grain trade off is really a matter of latency and usability. At the end of the day, you should move to a good SOA, with the right services exposed that do the right things, and not be as concern about granularity. Thus, web services that implement fine grained interfaces, and are meant for local invocation, will work well. Moreover, web services that implement fine grained interfaces and are meant for distributed invocation, and is on a low-latency network, will also work well.

Indeed, this is really an issue around service design, and we should really get off of this fine grained/coarse grained thing. It’s just causing unnecessary confusion. So, let’s talk service design, and in the process, how granular services should be.

Posted by davel in | Permalink | Comments (0)

Although many formats exist within most application integration and SOA problem domains, we will confine our attention, for the purposes of this manifesto, to the following:

• Alphanumeric
• Binary integers
• Floating point values
• Bit fields
• IBM mainframe floating points
• COBOL and PL/I picture data
• BLOBs

In addition to these formats, there are a number of formatting issues to address, including the ability to convert logical operators (bits) between systems and the ability to handle data types that are not supported in the target system. These issues often require significant customization in order to facilitate successful communication between systems.

In data conversion, values are managed in two ways: carrying over the value from the source to the target system without change, or, modifying the data value dynamically. Either an algorithm or a look-up table can be used to modify the data value. One or more of the source application attributes may use an algorithm to change the data or create new data.

Algorithms of this type are nothing more than the type of data conversions we have done for years when populating data warehouses and data marts. Now, in addition to using these simple algorithms, it is possible to aggregate, combine, and summarize the data to meet the specific requirements of the target application.
When using the look-up table scenario, it might be necessary to convert to an arbitrary value. “ARA” in the source system might refer to a value in the accounts receivable system. However this value might be determined, it must be checked against the look-up table. Integration servers may use a currency conversion table to convert dollars to yen, which may be embedded in a simple procedure or, more likely, in a database connected to the integration server. The integration server may also invoke a remote application server function to convert the amount.

The application integration and SOA architect or developer may encounter special circumstances that have to be finessed. The length of a message attribute may be unknown, or the value may be in an unknown order. In such situations, it is necessary to use the rules-processing capability of the integration server to convert the problem values into the proper representation for the target system.

Posted by davel in | Permalink | Comments (0)

Accounting for the differences in application semantics is the process of changing the structure of a message, and thus remapping the structure and data types so that it is acceptable to the target system. Although it is not difficult, application integration and SOA architects need to understand that this process must occur dynamically within the integration server.

This process can be defined within the rules-processing layer of the integration server by creating a rule to dynamically translate data, depending on its content and schema. Moving information from one system to another demands that the schema/format of the message be altered as the information is transferred from one system to the next.

Although most integration servers can map any schema to any other schema, it is prudent to try to anticipate extraordinary circumstances. For example, when converting information extracted from an object-oriented database and placing it in a relational database, the integration server must convert the object schema into a relational representation before it can convert the data within the message. The same holds true when moving information from a relational database to an object-oriented database. Most integration servers break the message moving into their environment into a common format and then translate it into the appropriate message format for the target system.

Related to the concept of accounting for the differences in application semantics, accounting for content changes is another important aspect of transformation. In short, it’s the reformatting of information so that it appears native when sent to a target system. The information needs to appear native, requiring that changes be made to source or target systems. More later...

Posted by davel in | Permalink | Comments (0)

SOA Expert Podcast Show 35...Great Debate, SOA vs. Traditional Enterprise Architecture

Listen to the latest SOA Expert Podcast

Posted by davel in | Permalink | Comments (0)

I spent some time over the weekend updating my SOA levels.

Level 0 SOAs are SOAs that simply send SOAP messages from system to system. There is little notion of true services, but instead, they leverage Web services as an information integration mechanism. Hardly SOA, but certainly a first step.

Level 1 SOAs are SOAs that also leverage everything in Level 0 but add the notion of a messaging/queuing system. Most ESBs are level 1 SOAs, leveraging a messaging environment that uses service interfaces, but really does not deal with true services (behavior), and instead moves information between entities as messages through queues.

Level 2 SOAs are SOAs that leverage everything in Level 1, and add the element of transformation and routing. This means that the SOA can move information from source and target systems, leveraging service interfaces, as well as transform the data/schemas to account for the differences in application semantics. Moreover, by adding the element of intelligent routing, you're able to route the information based on elements such as source, content, and logical operators in the SOA.

Level 3 SOAs are SOAs that leverage everything in Level 2, adding a common directory service. The directory provides a point of discovery of processes, services, schemas, and such, allowing all those who leverage the SOA to easily locate and leverage assets such as services. Without directories, the notion of service reuse--the real reason for building SOAs--won?t work. Directories are typically standards-based, including UDDI, LDAP, and sometimes more proprietary directories such as Active Directory.

Level 4 SOAs are SOAs that leverage everything in Level 3, adding the notion of brokering and managing true services. Here is where the brokering of application behavior comes into play. In other words, at this level we are not only about managing information movement, but the discovery and leveraging of true services.

Finally, Level 5 SOAs are SOAs that leverage everything in Level 4, adding the notion of orchestration. Orchestration is key, providing the architect with the ability to leverage exposed services and information flows, creating, in essence, a "meta-application" above the existing processes and services to solve business problems.

Posted by davel in | Permalink | Comments (1)

SOA Expert Podcast Show 34...SOA Press Conference at Business Integration 2006 in Frankfurt

Listen to the latest SOA Expert Podcast

Posted by davel in | Permalink | Comments (0)

Heading back from the Frankfurt conference soon, and I did spend a lot of time with the attendees trying to get a feel for SOA need. There were a few surprising patterns that began to emerge from my many meetings.

First, SOA is much more hype than reality. No brainier there, the marketing guys are well ahead of the technology, and those purchasing the technology know that.

Second, no clear direction and guidance. There is direction and guidance; you just need to know where to look, and which guidance to take.

Finally, and most surprising, the big software vendors don’t seem to understand the needs of their own user base and don’t seem to understand the value of SOA. I heard this many times this week, including the following comments:

“Our vendor keeps shoving their older technology at us and calling it SOA, when clearly it’s not. This is disturbing, and confusing, and the vendors are loosing creditability.”

“We are getting technology pushed at us instead of solutions. We’ve been down this path before, it’s not a good thing.”

“Nobody is defining the value of SOA for us during the sales cycle, and it’s difficult to justify the costs before somebody does that. Our SOA implementation will go into the millions of dollars….”

Just seems to be an emerging pattern. My suggestion is that we all learn to define the value first, than the requirements, than the solution, and only then the technology. The big guys need to understand this else risk losing customers and credibility. I’m not sure which is worse.

Posted by davel in | Permalink | Comments (1)

Linthicum SOA Presentation Picture here
I had a great time this morning at the conference doing a talk on implementing SOA, see photo. Good audience and the message was well received….in short:

1. SOA is complex, hard, and takes a lot of work.
2. You need to do your homework up front, no exceptions.
3. No quick fixes, and technology is selected after you understand your requirements.
4. You need to make sure you have a semantice, services, and process level understanding of your domain.
5. You can’t skimp on testing.

Leaving tomorrow. I’ll miss the beer, not the food.

Posted by davel in | Permalink | Comments (0)

I’m in Frankfurt this week doing a keynote at the Business Integration 2006 conference. I’m actually speaking on Tuesday, and then doing a short press conference afterwards.

Europe seems to be catching on to SOA more so than traditional integration in the past. In fact, there are a few good SOA companies that are beginning to emerge who are based in Europe. Also, you need to remember that many of the SOA companies out of Boston are actually funded and supported by European investors and developers. It’s clearly a global economy now, and software companies are scattered all over the place. Perhaps that’s a good thing.

Things I’m going to look for this week include:

• Progress being made by European companies with their initial SOA implementations. What’s working and where, and can I blog about it?
• Good companies to watch, and work with. Who’s got something new and exciting?
• Good beer. You can’t just work ya know.

Not sure how dependable my Internet connection is going to be, and blogging from my Blackberry is the definition of frustration. However, I will give you the scoop at some point.

Posted by davel in | Permalink | Comments (0)