We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

Business Ecology Initiative & Service-Oriented Solution

Michael Poulin

Agile Architecture Revolution: a new buzz, a new dream, or a new reality? Part 2

user-pic
Vote 0 Votes

(see Part 1)
I would argue that architectural solution above a project level should cover business tasks end-to-end, not for an individual iterative agile time-box. The iterations in architecture exist but in between the levels of architectural solutions: the details of architecture appear in more numbers when the process moves from the enterprise level to the programme and cross-project level. At the project level it stops and is handed over to the development team. We have to find a regular mechanism that could help us to define an "enough level of details" to be defined outside of agile development. Here is one proposal for this matter.

Assume we have a strategic task in a company whose development prefer working in Agile Methodology. A Solution Architect (SA) has to deal with many aspects including compliance, security, strategic directions, finance, relationship with key business stakeholders and alike before the solution reaches the level of architectural design and, then, development. An architectural design is a document where the SA articulates a few alternative solutions/designs, compares the (in SWOT) and enumerates architectural decisions. In general, it is recommended that the design addressed details not lower than the level of components or combinations of components. This relates to both applications and integration.

The level of detail in the architectural design is based on so-called estimate tolerance. In many organisations where I worked, business asked SA to estimate proposed architectural solutions from the resource, duration and funding perspectives (in addition to architectural qualities). The estimate exercise was usually supposed to be quick and performed as a prerequisite for funding the project, i.e. it has to be relatively cheap. The problem, however, appeared when an SA asked how accurate, i.e. detailed, the estimate should be. The more details are considered, the more accurate the estimate becomes.

Besides historical statistics, an accuracy of an estimate may be "measured" only via a tacit confidence of the estimator but this results in the willingness of the stakeholder to accept the fact that the real cost of the solution may be much more or not that much more than the estimate shows. This willingness is expressed as a tolerance of the stakeholder to the under-estimated cost.

For example, if an estimate tolerance is set to 75%, this means that the SA is allowed making mistakes that may lead to the increase of the cost of the solution up to 1.75 times and that the stakeholders will be OK with this. On the other hand, constructing such approximate solution does not take a long time, a collection of many details of business requirements, and, thus, relatively cheap. Also, this means that the significant part of the design should be performed by the development team (in agile manner). I have illustrated this relationship in the Figure below.

EstimateToleranceVSAgile.JPG

Figure

In practice, an estimate tolerance is set to 25%, 50% or 75%. The decision of which one value to be chosen belongs to the stakeholders. One of the criteria in this section is a confidence in the ability of the development team to perform at the required level of quality. For example, if a development team is unfamiliar with the corporate policies, customs, processes and influential power or if the team consists mostly of junior developers, or if a solution seems to require a long period of delivery time while the development team turnover is high, it would be better to increase the estimate tolerance to compensate risks in delivery by more detailed architectural design.

In any case, the more accurate the estimate should be, the more costly it will be. And all these happen before the project is funded, i.e. somebody has to pay for it.

In other words, an estimate tolerance defines how granular an architectural design should be and how much of the work should be delegated to the agile team. Not more, not less.

So, if here is such thing as an Agile Architecture, you decide. I'll watch for an Agile Architecture Revolution.

WebSiteAdvert.jpg


ActualBookCover.jpg

7 Comments

| Leave a comment

Let me declare I am a business person as such some of what you said I really did not get! I also run a Business Software Technology company that with original R&D now puts down the claim to be the start of commoditisation of business software - the ultimate re use of code. It has been a business driven development and as such I am comfortable contributing to the "BPM Forum" but I thought I would see if I can get you to "get it" in the context of 2 issues you raised "agile" and "estimates"
Agile is only a methodology good but only treats the symptom of poor software technology which requires coding business logic which actually has not changed since commerce started. People create all source information, undisputed, and what we did was identify generic tasks human and system that address all required business logic including workflow rules and user forms. We made them objects inside the Oracle RDMS, built a “process engine” inside the database and then built a graphical model where the build takes place this will give you a flavour http://bit.ly/qlSUvM
Estimates as the world’s first “Agile Software” build is very quick as core code never changes so iterative build takes place once the agreed outcome of what is required from the new process application. A fairly good estimate can be given by counting the number of user forms that becomes the number of man days to build the first working cut of the application working with users for instant feed back even change with them. Complex processes such as configurators or means testing may require more time but fairly easy to estimate once the business rules are established. However legacy exists so we can use as required which can add time but we discourage changing legacy”! Over time legacy system may become redundant but by then you have protected users from any “consequences” as they are closed down and archived.
There is little doubting legacy needs to undergo “modernisation” and what you describe will be part of that. However I believe such an “outside in” people supporting technology will aid that cause but as you can see it is very business driven – but will IT “get it”……? As for revolutions “commoditisation” of software is long over due. See interesting research here http://www.igi-global.com/publish/call-for-papers/call-details/733#.T7AdC7qrXUU about "Model-Driven Software Engineering". The big question is given choices which “isation” will business decision makers support with investment?

David, I am glad that this post has caught an attention of a business person such as yourself. I would be very interested in learning what you particularly "did not get"; I have to improve my arguments.

Returning to your comments, let me say that I agree with only the first part of “Agile is only a methodology good but only treats the symptom of poor software technology which requires coding business logic” – yes, Agile Methodology treats the symptoms but the illness is in business, not in IT, in this case. In particular, this methodology was created to address massive ad-hock requirements from business users for the functionality of User Interfaces, which these users cannot formulate accurately because, in many cases, did not know what whey needed or wanted. There is a huge layer of IT work that has nothing to do with UI and with such type of agility.

I do not have doubts that coding business logic into applications and, especially, into persistence storage (database) is the example of bad practice because a cost of changes of such solution is too high and requires too much time while correct (agile, if you want) approach must be flexible, i.e. change adoption should be quick and cheap.

What you described about business objects in the Oracle RDMS is only one of possible solutions; an alternative that works equally well is using Rules Engines (that also persisted but the focus is not on database here).

“A fairly good estimate can be given by counting the number of user forms that becomes the number of man days to build the first working cut of the application” – this is good for User Interface but how this works for ETL or message routing? Sorry, I meant interaction beteen databases and communication between applications via an intermediary (you can imagine a Postal Hub where letters from senders directed to intermediary Hubs before getting distributed to the addresees). Nonetheless, my post is not about this level of estimate and agility.

I was talking about architectural estimates (typical for large organisations) that take place weeks before first user form or a code are created. This is the time of making business decisions whether to run a project, how much it might cost and how long it might take. At this level, architects evaluate possible combinations of existing applications vs. products in the market; how both of them fit into governing policies, how they may be audited and proofed compliant with regulations, an so on.

Service orientation does not recommend but require separation between business logic implementation and supporting utilities. This is how service become flexible, composable and reusable.

I have not looked into the article you refer but from my personal experience, developers and PM hate "Model-Driven Software Engineering". It disallow any short-cuts and quick (and dirty) fixes; you have to modify the model first, every time you mking a change, plus, it makes quite visible that the role of developers becomes a support role only.

I am sure that IT, the part of IT that works with business users and User Interfaces, whould be happy having a tool/mechanism that can generate proper form/screen sets and flows (User Journey) automatically. However, I doubt thay this example would cascade to development of other SW coponents easily.

I think the opening words went right over me!
"Architectural solution above a project level should cover business tasks end-to-end, not for an individual iterative agile time-box. The iterations in architecture exist but in between the levels of architectural solutions: the details of architecture appear in more numbers when the process moves from the enterprise level to the programme and cross-project level..... ? A classic example of the business IT divide! Is architecture not about secure reliable delivery of business information it is not about the business application?

I think you are right business people have not historically known what they wanted but partly because IT has failed to both listen and deliver and that's where agile helps to bridge the gap. However agile software actually closes the gap. But people at work do actually have ideas and suggestions but need confidence IT will deliver. Here is a recent quote from a user as we handled change to an application. “It captures all my weird and wonderful ideas and all done without telling me that I am expecting too much!”

“There is a huge layer of IT work that has nothing to do with UI and with such type of agility.” Yes and all taken care of in Agile Software see the Q&A in the link I gave UI is just one task type. This takes you direct to the Q&A http://bit.ly/rjLx38

“I do not have doubts that coding business logic into applications and, especially, into persistence storage (database) is the example of bad practice because a cost of changes of such solution is too high and requires too much time while correct (agile, if you want) approach must be flexible, i.e. change adoption should be quick and cheap.” “And you have to modify the model first, every time you making a change, plus, it makes quite visible that the role of developers becomes a support role only.”
Wrong and demonstrates you have not yet “got it”. Change is very quick. The model is the application. There is no code generation no compiling it uses a declarative technique. Take a change say a regulator now requires all transactions over £100,000 must be approved by third party. So you go the model take a copy off line then go to the tasks involved. In the link joining them open it up and put in a new rule if over £100,000 it goes to new link to take information to the new approval authority. They will be allocated a role and if required URL. A UI form would be created to dynamically populate with required information and set up for that user to accept or reject and make comment whatever. It then returns to the original process. Very simple form and this whole exercise may take an hour. You then click to activate and it immediately becomes live using version control and no disruption to the business. This is all handled by the business analyst skill set not a programmer there is no need for code change. You are right in your language a “developer” has no role in this! That is what happens in commoditisation of software and why MDE and the emphasis on “Engineering” to get to the end result is now gaining prominence.

“..this is good for User Interface but how this works for interaction between databases and communication between applications via an intermediary”
I doubt that this example would cascade to development of other SW components easily.” Handles it all you can create documents in the browser set up templates that can be changed, even send for approval whatever the process requires send emails and readily communicate in variety of ways with external databases /systems internal or external. As for what it can build well the generic nature of the tasks it will build any business process. It even has intelligent process capability! Take HR you would not reinvent payroll processing but you can quickly build all other requirement custom the way you want your organisation to run. All simple process hiring and firing holiday request, compassionate leave, expenses, annual reviews, training schedules whatever it is all about people and process. Your imagination is the limitation but also recognising good existing applications should be used within a process another task type is to call a program.

This debate is needed as it both highlights the divide between “IT” and business and the urgent need for the step change to “commoditisation”. I think you have a hugely difficult role trying to sort out the legacy mess with the resultant complex infrastructure. But at the end of the day business want operational efficiency, better real time reporting and that elusive ability to change their applications quickly. All aspect of ICT need to work together focusing on their core competency and business application logic belongs to and is responsibility of the business.

user-pic

David,
In your first paragraph, you have got into the trap architecture semantic: architectural solution is more about architecture implementation rather than about architecture. Architecture is usually expressed in functions and their relationships, everything else is a set of levels of architectural solutions. Business Architecture differs from Technical Architecture but both of them contribute in to Business Service that realises particular business functionality. Architecture is not about applications at all but business information is a part of Business Architecture. But this is a separate topic.

I’d say that the opinion of your user should be regular, normal for a delivery process of IT. I do not see any special “agile” quality here.

Agile Methodology in development has its niche and preconditions. If the preconditions are not met, Agile Methodology should not be used. For example, it is impossible to create a service in an agile manner because if offered functionality at any moment does not meet consumer needs (and there are other services in the market that offer this functionality but difere in cost, quality, supported policies, etc.), no consumer will choose your service. If your service is not chosen, you are getting out of funds and, eventually, out of business. In this example, I am talking about Business Service that comprises manual/human and automated (IT) implementation parts. [I read the PDF but have not found an answer to what did you mean]
David, as professional and many times certified architect with PM certification as well, I can officially state that no application can be a model and vice verse. If a change to an application is very quick this has nothing to do with the model but with the right componentisation of the application implementation. A model is an abstraction (see Model Driven Architecture - MDA).

When you externalise business logic from the application into a decision making component like Rules Engine, you do not change the code but only a script or set of rules or configuration. But this is not MDA. However, you have said “A UI form would be created to dynamically populate with required information” – this is THE code generation, isn’t it? So, we talk about either MDA/MDE/MDD or automated generation of UI based on the Rules Engine. As I said, this may be good when you need a process worker to start using an automated means in the business process.

But a business process is only one of several elements of a Business Service. There are such IT elements as exchange of data between applications beneath the UI and without it generated UI is dead. You cannot build an inter-system integration in an agile manner; it either works or does not (a case of pregnancy of 15%).

As I said already, I believe that the tool you describe is very useful for certain IT tasks.
However, “business driven” solution, IMO, is not only about UI, it is about end-to-end, and in some areas of this journey no people should take place. From another hand, business efficiency is a tricky thing: I know a company that has its core systems not integrated via data exchange channels and they are very efficient… because a battalion of Typers in Malasia re-type all needed data overnight from one system to another, and this is much cheaper than to build an automated integration.

Though these debates deviate from the topic of my post, it was interesting to explore around agility between Business and Technology.

Michael
I think the point on IT architecture is that business people do not get it they just want reliable secure delivery and rely on technicians to do their best. They do not even get to the semantics!

There is nothing new in “agile” as a project management philosophy. But in being applied to IT has been over hyped as trying to persuade business that there is no “gap” by giving quicker feed back but build is by programmers writing in a different language? Yet as I have said business logic just does not change so why are we coding this over and over? This brings me to your implementation of business services question by agile software.

The generic task types covering human and system requirements addressing all business logic are pre-built “objects” and stored in the RDBMS (something even IBM thought not possible and Microsoft tried to patent 10 years after us!). The forms are designed as required for each task and stored ready to be configured. Included are powerful links that join up the tasks in required sequence and where just one place rules are readily handled as is one to many and many to one. The in built process engine acts as the automated orchestrator. As soon as a recognised user signs on with its pre-programmed capability the form is dynamically populated with associated data for that specific task. There is no code generation. One might argue it is a sort of runtime compiling – the other analogy made by others is a bit like SOA but in a contained “box” the database pulling people, their roles, the task type and data all together.

I mentioned that only after we had engineered this capability we decided to make it easier to build an application by placing a graphical interface with simple click drag and drop over the “engine”. I can absolutely assure you that the resultant graphical representation is the actual application in as such it is the build environment just as are programmers’ lines of code. If you want to change it is all done through this graphical interface and once built/changed and checked your business logic (including rules capability) just click a button from design to run and the process engine automatically set it all up to run – no code generation. Again I assure you that it handles the whole “journey” supporting users end to end in any application including any system driven requirements for example manipulation of data.

Handling legacy is a big issue for all and there are a number of ways including a SSMQ and extensive tag library to connect to bring in data to either be used by the engine or populate the user form building say a mash up. Whist this new application integrates into the business system we discourage “integration” into legacy systems. As you point out the legacy side can be time consuming and very expensive. The reason this new MDE approach can do this is because the inherent architecture is handling front and back office as a process hub not a traditional layer. As such it supports high levels of automation and thus efficiency with quick ROI. It can be part on a legacy modernisation exercise allowing orderly archiving of old legacy to eliminate duplication of data - like removing re entry of information by a “battalion of typers”…….hope it is not a regulated environment!

Yes this is rare discussion as business person engaging an IT professional and neither of us walking away shaking heads……yet.

David,

I tend to agree with you on “that business people do not get it they just want reliable secure delivery and rely on technicians to do their best “. It is IT has to learn the business, not the other way around. However, I have one condition: business has to treat IT as another business unit, not as a servant (in any industries, automations enables business that cannot move a finger w/o IT). This is why one of my proposals (to look back and find the lost value) is to split business and related IT into groups performing/realising certain business functions at a low level (Business Services). Everything else in the business area should be just different compositions of these functions, i.e. teams of business and technical people. Certainly, there will be small central/shared units in both business and IT but above all there should be a cross-functional cross-departmental management team that is responsible for thos compositions and creation of new Business Services (and related capabilities).

I never doubted the tool you are describing. My only remark is that your tool generates the only specific type of applications, that than UI and forms. These last two do not constitute a “business application”, IMO, because it is about business logic realised by the application. In many other cases, the applications realising this logic do not have (need) human interfaces.
However, I would not say that automation of the kind you describe eliminates architectural estimate and related “game” of tolerance and cost/time. This game is usualy plaid at the departmental/LOB/enterprise level and across several very different applications, many of which are COTS. This is what my post about.

As of legacy applications, I think that I less care about the than about ability to interact with them. In general, about application integration-ability. IN your description, I have not found any pre-allocated hooks for any integration and this is scary. Assume, you have to meet a FSA requirement about Sanctions Checking for all end-users who deal with finance (in any way). There are sepcial (soficticated) systems that effectively work with Sanctionsl Lists all over the world. What is required is to intercept a communication session of your end-user with your application and check where this end-user is not in the Sanctions List; if he or she is, the application might need to be locked.

This is only one example of unpredicted needs for cross-application integration. Can your solution handle this global/cross enterprise task?

I agree there is a need to split “IT” handling different functions – the separation of business logic from delivery technologies puts the analysts skill set driving application build and subsequent change but they do need to work with the other aspects of IT to see delivery of the solution. This “commoditisation” of business software in hands of the CIO team will see “IT” quickly become as you describe a valued business unit?
Again agree that there are many non human tasks what we call system tasks see the list of both human and system all part of requirement for that dream of cutting out coding. These exclude the extensive capability in the tag library in the presentation layer to quickly connect data and forms

User tasks
A normal task
A form task
A report task
A web report/form
A program task
A pending task

System tasks
A calculation task
A VB script task
An import/export task
An event task
A sub process task to a process (where libraries of frequently used function processes can be built up for re-use)
A Server Side Message Queue task
A finish task

13 in total but some are client server orientated and not now used. Then the “link tasks” true and false with rules capability and that is it; this builds business applications core code need not change. The user only sees the UI but a lot happens behind the scenes even if it is a machine sending a signal/message it is catered for. Calling a COTS handled as well as extracting data (as long someone knows how to get into the legacy!) No doubt a good architecture helps but if required just focus on point to point.
As I have mentioned we do have “pre-allocated hooks” that set up connectivity to legacy. We have some custom APIs but these are readily custom code built as required; no magic bullet for that given the huge variation in legacy! But remember we do not need to integrate to build working dynamic solutions. As part of a modernisation program the more inadequate systems that can be made redundant the better for all?
Interesting thought on dealing with Sanction Lists and with domain knowledge no reason why this cannot be tackled. Global/cross enterprise readily handles as long as knowledge exists where to “hook in” and that’s where business rely on the system architect. Whatever the result will an application that can readily change as regulations change. If you are in UK lets see what can be done?
And so I think we conclude on quite a bit of agreement despite we come from different spectrums of the “IT” interpretation problems. Persistence does pay off…..!

Leave a comment

In this blog, Michael Poulin writes about business and technology ideas, concepts, methodologies and solutions leading to service-oriented enterprise, the primary instrument for obtaining business objectives in fast-changing environments.

Michael Poulin

Michael Poulin is an enterprise-level solution architect working in the financial industry in the U.K. and the United States.

He specializes in building bridges between business needs and technology capabilities with emphasis on business and technical efficiency, scalability, robustness and manageability. He writes about service orientation, application security and use of modern technologies for solving business problems. He contributes to OASIS SOA standards as an independent member and is listed in the the international "Who's Who of Information Technology" for 2001. View more

Subscribe

 Subscribe in a reader

Recently Commented On

Categories

Monthly Archives

Blogs

ADVERTISEMENT