Recently in Industry Trends Category

I've been spending the year assessing where enterprise architecture -- and SOA in particular -- are headed as we begin to leave the first decade of the 21st century. The good news: It's clear that EA and SOA are delivering real value to most businesses today. However as we'll see, there is also considerable room for performance improvement. Trying to map out the direction of these disciplines has resulted in quite a snapshot with many interesting details and nuances that I'll share here throughout the rest of the year.

More immediately of interest we can see some trends beginning to stand out clearly right now:

1) Modern software architecture, with SOA as the top-level organizing principle, seems to have more and more trouble keeping up with the rate of change in most organizations. Not only have the timelines and schedules of IT and architecture groups never aligned well with business activities, it's now becoming an endemic problem as I've shared experiences with architects around the world recently. There's a growing sense that the business is "pulling away" and despite enterprise architecture and SOA being intentionally proactive, the default stance is one of short-term reactive response and trying to "clean up afterward", an activity which few businesses want to pay for. In short, our traditional services delivery models seem to move too slowly these days to embrace business opportunities as they occur.

2. Consumption of SOA and service-based IT is still too low. Depressingly low in some cases. The early "field of dreams" approach to services that some organizations practiced (build them and they will come) combined with the proliferation of JBOWS (just a bunch of Web services) finally morphed at last in many organizations into more mature services landscapes with some actual governance. But now we see that the subsequent complexity, rarefied skills and tools, and additional constraints ended up stamping out a lot of SOA consumption at the edge. These days the evidence is beginning to mount (presented below) that a certain style of services -- combined with mechanisms that actively drive consumption and outfitted with real business models -- could address this. There are other emerging solutions to drive consumption of SOA as well. Without consumption and uptake, SOA cannot access the "return" in ROI. Systematic removal of the obstacles to participation in SOA is the goal in this view.


3. The focus on SOA still tends to be on the [over]engineering of seams and processes instead of removing constraints on the business and increasing ready access to value. We are still not providing good enough reach to our enterprise data, IT systems, or people. Despite heavy investments in IT for 30 years and the openness and interoperability intents of SOA, the majority of our enterprise data remains submerged and inaccessible to most business users, silos are still prevalent, and there still lacks the human dimension, the latter which can represent the use of more effective architectures of participation or the involvement of more than IT in the strategic development of SOA.

While important cross-cutting concerns -- like security efforts or the imposed siloed architectures of enterprise software suites (ERP, CRM, HRM, etc) or even turf boundaries (see Conway's Law) -- are also responsible for many of these issues, at the core seems to be a project-oriented focus instead of a product-oriented focus with SOA. This is not to say that good engineering is not important, but that we are too often aiming at the wrong part of the SOA tolerance continuum.

Looking to successful models of SOA for answers

As an industry it's also apparent that we have too few examples of good SOA to point to and use as reference models. Most success stories are behind the walls of other organizations that are just as inclined to keep the secrets of success to themselves. Best practices and effective approaches therefore are not always likely to spread when they should. In fact, it's fairly easy to argue that the evolution of SOA has often moved far faster in standards bodies than by validation of effective new ideas in the field that are only then adopted and standardized when they work well.

Interestingly, this means the openness and visibility of today's modern Web seems to provide living examples of what's effective at driving forward business results with SOA. By this I mean when it addresses the key issues around improving the response to opportunities and directly enabling self-service, all while remaining highly scalable and protecting users and data. While I've been writing about Web-Oriented Architecture (WOA) for some time now, it's in truth a parallel "track" for SOA that's evolved organically in the wilds of the online world to meet many of the same challenges that we have in our organizations today.

Related: Fixing Enterprise Architecture: Balancing the Forces of Change in the Modern Organization

Key to the story of WOA is that at least one critical feedback loop is present in online businesses that is much less evident in most SOA efforts today: Their business will fundamentally thrive or die based on the adoption of their services. Most new online businesses offer an API (the Web version of SOA) at the very outset these days and it's now common for the services themselves -- instead of the visual Web pages -- to be the dominant point of usage early on. Online firms offer APIs in easy to use formats and then market and evangelize them extensively precisely because there is an urgent need and these are the best ways to drive adoption. There is also more direct and immediate business value in this model since carefully measured use and consumption is the yardstick by which online firms are measured. It's also what they monetize, an entire area that's seriously underdeveloped in enterprise SOA for both internal and external customers. This is a yardstick that would change a lot of what we focus on in SOA.

Imagine how we might behave and prioritize our efforts if our success was largely based on the number of points and overall level of consumption of SOA services, internally as well as partners and customers. Unfortunately, as Joe McKendrick noted yesterday in "SOAs too big to fail", we may be increasingly isolated from that pressure as the business becomes more and more dependent on SOA. Ironically, this seems to be encouraging an organizational structure (which often is the IT department itself) that is isolated from market pressures.

How WOA can help SOA reach the next level of performance

While there are certainly many aspects of WOA that need to have an enterprise context added to them to work for traditional businesses, you can see from the visual above that a successful and capable service-based approach has emerged naturally side-by-side with classical SOA. At their roots, they both have great commonalities (HTTP, interoperability, and recombinant software), and some key differences as well. Intriguingly however, these often line up with many of the issues we're now seeing holding back the next level in business performance of SOA:

  • Responding to change faster. Reducing execution chokepoints by making SOA more self-service (running your SOA like a business) by broader audiences in the organization.
  • Increasing consumption. Using new service delivery models that make SOA the easiest, cheapest, and fastest way to solve problems.
  • Actively enabling access to value. Opening up broad access to data and people using models such as deeply linked REST-based data webs and open supply chains. Adopt browser-based approaches to consumption such as enterprise mashups tools and user distributable widgets that project the SOA across the organization.

We certainly don't need to throw away everything that we've done with SOA to achieve these performance improvements, far from it. However, we must change much of the focus on what we do and how we do it. There are also some key additions to technical approaches that will be required such as adopting enterprise REST, making IT systems more mashup friendly, and updating existing delivery models such as portals and intranets to be more malleable, self-service, and connected to SOA data and resources.

The bottom line: Since many of the very best examples of SOA that works are the models we seeing being proven out in large scale on the Web, we must learn as much as we can from them (and the price is right, even the most compelling lessons here are free.) While enterprise SOA will never be identical to Internet-based WOA, we can borrow the best ideas from the success stories that have emerged on the Global SOA (which is what I call the service-enabled side of the Internet).

We are also starting to see evidence that a more mature melding of Web-oriented service models with the enterprise is actually starting to happen. For example, an InformationWeek survey this year showed that on the technical side of the SOA/WOA stack, SOAP adoption appears to be dropping in enterprises (54% a year ago to a projected 42% in the next 18 months) while REST is on the rise (14% to 24% over the same time frame). The recent announcement of the opening up of EMML is also showing that WOA concepts around lightweight composite applications (mashups) have moved into SOA territory.

While emerging initiatives such as JBOSS's REST-* may or may not help with this transition, as more of the foundation of WOA is available in organizations, richer and more effective outcomes on the business side of the the WOA equation are possible as well. No doubt this will take years, but this evidence and others shows that we are already beginning to incorporate the lessons of WOA into enterprise SOA.

In an upcoming post, I'll look at emerging new SOA practices that are coming from other sources as well.

Note: I'll be holding a discussion on The Future of SOA and enterprise REST at ebizQ's SOA in Action on October 28th. Please register to participate today.

Are Web-based approaches providing insight to your SOA effort? Or are you getting effective new ideas from elsewhere?

There's been a lot of discussion in IT circles and online lately about the future of enterprise architecture. As the highest level and most strategic technical function in most organizations, good enterprise architecture is vital to the vision, capability, and healthy operation of today's modern organization. As any IT architect will happily tell you, in today's 21st century enterprise, technology is used pervasively to enable, automate, streamline, and transform business processes of every description. At its very best, enterprise architecture provides the bright lines that articulate the full range of possibilities for a business, even describing how to go about getting there.

But there's long been something fairly unsatisfying about the relationship that enterprise architecture has had with the business side of most organizations. Recently there's a growing realization that traditional enterprise architecture as its often practiced today might be broken in some important way. What might be wrong and how to fix it are the questions du jour.

Examining Today's Common Enterprise Architecture Pitfalls

This morning here on ebizQ Brenda Michelson examined the latest list of enterprise architecture pitfalls as part of the larger discussion that's taking place these days about EA. With organizations such as Gartner proclaiming the advent of newer approaches such as emergent architecture and reporting out the aforementioned pitfalls, there's a lot of change in the air at the moment. A recent piece in ComputerWorld interviewing the deputy CIO of NASA immediately noted that enterprise architecture is "far more difficult to properly implement than most IT professionals realize". Richard Veryard contributed to the discussion today as well, noting that the trouble with these diagnoses is that while essentially correct (perhaps even obvious) they aren't by themselves actionable information and therefore can't help lead to better outcomes.

In other words, problems aren't solutions and EA definitely seems to have some problems to solve. In particular I'd argue that there are two primary reasons for most of the reported woes. One is that EA is apparently quite difficult to do well and the second is its overly centralized nature. In this view, which I'll for now call a "next generation" perspective of enterprise architecture, there must be more agents responsible for enterprise architecture that are in a position to do it properly, carry it out effectively throughout an organization, and are connected deeply with the business, both locally and widely. And because EA is increasingly seen as a core business activity (even if it's all too often not) as well as a technical discipline, the current situation of relatively enforced -- though unfortunately quite often naturally occurring -- separation has caused many of the problems that we're seeing today reported as pitfalls (having the wrong architects, not engaging business people, etc.)


Looked at this way, enterprise architecture is thus the responsibility of almost everyone in an organization that makes longer-term decisions (business or technical), especially when they are captured and represented in a IT system somewhere. But EA is too difficult for this to be responsibly done by anyone but capable enterprise architects, right? This then is perhaps crucial to understanding the problem.

Reexamining The Fundamentals

One key way in which civilization (and technology) advances is when things that were formerly difficult become easier to do. Concepts like abstraction, encapsulation, componentization, dependency management, etc. made it possible for software architecture to evolve to a point where it could be intellectually mastered and directed at a systems of systems (i.e. enterprise-wide) level. Key to this whole discussion seems to be the truism that software architectures are most successful when certain constraints are left open. These can be well-defined interfaces or the use of open standards, or anything in which a distinct and important point of flexibility is explicitly left open for use and change down the road when more information is available or when the situation changes.

Identifying these points where choice might be needed and making it possible is one of the most important and central activities in software architecture. This is even more true of its cosmopolitan big brother, enterprise architecture. In fact, I've argued in the past that the simplest possible definition of the job of a technical architect is "to put off key design constraints until the last responsible moment." The magic, of course, lies in knowing what should be locked down and what should be less constrained.

We are perhaps learning to what degree that enterprises and the resources within them have properties and abilities that can't be declarative prescribed or controlled in a straightforward manner. From my personal experience, I'd say the more constraints and controls that are imposed architecturally, in general, the poorer the outcomes. From my research in the past on how SOA and Web 2.0 are actually two sides of the same coin, it's apparent that we have a lot to learn from very large yet loosely cooperating architectures that seem to work so successfully on systems of systems like the Internet. This is particularly true when the technical architecture is the business, a convergence that's only going to increase in our organizations, not decrease. The key point here is the excessive architectural constraints diminishes available options, limits flexibility, and ultimately reduces outcomes. The more control you assert over a system, the less it will be able to do. The converse is true as well.

The fact is that enterprise architects are often looked at these days as the police, invoking "governance" as a mandate to do things that sound right on the face of them: introduce standardization, reduce duplication and waste, and improve the levels of integration and connectivity between systems. If if these outcomes are desirable (and they can be), several important things are missing. One is that in any large system, local actors will often (but not always) have a more concrete and clearer understand of local situations and how to take advantage of them. Top-down enterprise architecture often fails to incorporate this or tries -- rightly or wrongly, and too often the latter -- to adapt the local conditions to a more unified, global environment. Second is enterprise architecture's generally poor role as a direct enabler of business opportunities and improvements. Again, at its best, good EA certainly does this. But average EA is about imposing familiar structure and patterns, not inspired and insightful business solutions.

So unfortunately, in this way, far too often enterprise architectural involvement actually creates a bottleneck for business activities and becomes an active dampener on what is possible. It also tends to be carried out on a schedule and with requirements that aren't aligned well with the local actors. Consequently, it's rare to see a business unit run to IT and ask for more enterprise architects to help them with their challenges.

A New Vision for Enterprise Architecture

There are two ways to go about solving these problems. One is to push even more control and oversight up into the purview of enterprise architecture to "get it right". In this way we can give EA efforts more resources so they can be more responsive and spend more time in the trenches. However, I think most people would agree that's not going to be the right answer. The other is to make enterprise architecture far more autonomous, distributed, and loosely coupled than it is today. That is, spread the wealth, greatly increase architectural capacity (via decentralization), loosen up the constraints, and try to guide the outcomes taking place at the edge of the organization.

A large number of good things can come from this relatively simple (though far from consequence free) act. (Like any good two-edged object challenges will also result.) In practical terms, this means moving to a practice of enterprise architecture focused more on delivering choice and options (a consumption-oriented view, as in people wanting to consume what you have or can do because it will help them) and one that is free of as many essential constraints as possible (practically, only truly essential ones like security, legal requirements, and the most critical business needs.)

The Implications of "Next-Generation" Enterprise Architecture

What does all this mean in practical terms? How does it change the picture of enterprise architecture in businesses today? Well, there are a number of significant and far ranging implications, some that are surprising but that I mostly find encouraging. From what is apparent at this point, they are:

  • Highly pliable software platforms that are more customizable, on-demand, and self-service. We're already witnessing this with the rise of Software-as-a-Service (SaaS), cloud computing, enterprise mashups, collaborative intranets that can be maintained almost entirely by workers (Enterprise 2.0), and the growing use of smaller, more granular microapplications such as widgets, gadgets, and visual feeds to combine and connect things together across corporate networks. Applications these days are also more open and federation-friendly, frequently offering Web services and APIs with a high degree of customizable features that make them easier to tailor for specific situations. While there is still a long way to go before we have fully mashable enterprises, today's newest IT solutions are already simpler and sleeker yet more powerful and open than ever before, providing more on-the-ground control, adaptability, integration-readiness, and customizabiity. This is the often worker-driven canvas upon which next generation enterprise architecture must function and thrive.
  • Business units and their workers will become more architecturally-aware and assume more local responsibility for EA. Enterprise architects had to learn the business, now it's the business' turn. The whole story here is about strategic advantage through the application of technology and everyone needs to get on board. Workers need literacy in basic EA principles and knowledge of what their enterprise tools are capable of (as well as when they should get help.) With enabling guidance from professional, full-time enterprise architects (probably still from IT or the CTO/CIO's office) the deeper technical issues can be addressed and the classical traps avoided. In this model, enterprise architects realize they can no longer do it all the themselves, must enable local efforts with the tools and ideas to succeed, and formally deputize deeper down into their organizations while still providing moral support and central guidance, though only when it's essential. How far should they go in doing this? That remains an open question but it's probably surprisingly far down into the modern organization. The key point: Business architecture today is enterprise architecture.

    Am I advocating making everyone an EA expert? Not necessarily, I'm advocating spreading EA responsibility much farther across the organization into capable hands and enabling far more foot soldiers who have a stake and real determinism in what they do.

    Intriguing sidenote: I recently had the privilege of having a conversation with Paul Preiss, the founder of the world's largest formal body of software architects, the International Association of Software Architects (IASA) and he told me that they have learned through many years of instructional development that they can teach almost anyone to be an effective enterprise architect, even if they are non-technical.
  • Competition across the enterprise. While duplicate software and data was previously considered wasteful, it's still surprisingly common in most organizations. The number of duplicate and unnecessary customer databases alone that most businesses maintain even today stands testimony to both the challenges of overcoming Conway's Law as well as a nod to the powerful tribal and organizational boundaries that exist in all organizations. However, overlap in emerging IT solutions on the edge of our organizations for a time actually isn't a bad thing. We just don't have good enough competitive mechanisms for IT solutions inside most organizations; the best perceived solutions to business problems are usually picked up front and then rarely changed, with only occasional drivers for dramatic improvement as systems become outmoded.

    The implication here is that new distributed solutions can crop up, and if successful can rise and become a better and more useful solution. Or they will die away as their faults and failures are understood and a new crop of solutions emerge. The role of traditional enterprise architects here will be to help manage such competition so that it's least disruptive and help manage change.
  • Traditional enterprise architects become mentors to the business units. As business units assume more, though again, not all, control of the architecture of their business (and some of the technology underlying it), they will need serious help. Often lots of it, especially at first. While mentoring is often 2nd nature to architects, who frequently have to work through the hands of others, the increasing distribution of architectural responsibility is going to require considerable bridge building, education, hand-holding, advice, and problem solving. The best part, done right and with a EA help desk (perhaps with an internal community development effort), the phones will ring in IT once again.
  • Better business outcomes. Because more architectural capacity lies across the organization and can help itself within a larger number of situations, better results often occur. This means quicker response to business situations, faster and better application of technology to business problems, and less rework.
  • Now the bad news: Making sure chaos doesn't ensue. The proliferation of new self-service technologies and increasing use of outsourced IT solutions is already giving IT a headache by giving it more to manage, integrate, and connect with. Portfolio management of highly decentralized enterprise architecture is going to require new tools and techniques, though these were needed anyway. Is a 10x increase in maintenance and management effort going to be required? It's hard to say but it's likely. This is where centralized enterprise architects can do enormous service to the organization by getting out ahead and working out ways to enable choice by working with local groups to create common technology platforms and some simple ground rules to keep the new EA manageable. Again, the concept here is think globally, act locally.

I should be clear that some organizations, and probably ones that are having unusual success with their enterprise architectural efforts, are likely doing some of this to some degree or another already. What this discussion is about is what can organizations do today if they want to improve their enterprise architectures in a practical ways using new perspectives, many of which are taken from the hard-won lessons learned of recent years.

Finally, of course, one of the big problems with this whole discussion is that it smacks of architectural astronautics. However, this is the very heart of the problem itself; a lack of close connection to ground reality with only an impartial, external stakeholder's view into the business. Looking at the pitfalls list circulating this week it's clear that opening up and increasing ownership and responsibility of enterprise architecture (something that's going to require a big step up for many managers in most businesses) can address many of the problems being discussed right now. In fact, I'd wager it's one of the few things likely to work.

Given the trend for business units to try to solve more and more IT problems themselves, all of this is not just theory, it's rapidly becoming practice and traditional EA is at risk on many fronts as a result. This means as businesses increasingly mind meld with their technologies and begin to attain truly integrated strategic control over their destiny, the 20th century view of IT and business will almost certainly undergo a transformation like no other it's encountered before.

Where do you see enterprise architecture going? Is EA about to undergo a major transformation or are the current challenges just temporary?

While reports vary these days on the exact level of demand for cloud computing by businesses, it's clear that there is fairly broad general interest if the features and cost are right and the risks are reasonable. In the short term, the organizations looking for effective new business strategies in the downturn are finding that cost is a compelling argument for cloud computing, especially for transitioning non-critical IT operations or to inexpensively harness cloud computing for resource intensive but low-risk tasks like software testing.

The real question for many IT departments is whether the cost of transition to an external compute cloud will be low enough to benefit from any medium-term savings. That process might be costly enough in itself to delay any significant ROI by outsourcing to the cloud. However, as we'll see below, there's now some indication that leading cloud computing vendors like Amazon are realizing that the current cost incentive just might not be enough, particularly for enterprises ready to commit for longer terms. As Microsoft gets ready to launch Azure and Google App Engine gets serious about the enterprise, expect pricing models to get even more competitive.

Compute Cloud Pricing for Amazon, Azure, and Google

Earlier this week Amazon lowered the costs of their Elastic Compute Cloud (EC2) reserved instances for organizations prepared for sign up for 1 to 3 year terms of usage (and who presumably understand their ceiling on steady state compute capacity.) This brings the cost of a continuously available cloud computing instance down by 30% to as little as 4.3 cents an hour (or a 3.0 cents/hour simple rate):

Cloud Computing Cost Reduction: Amazon's Web Services EC2 Reserved Instances Pricing

* - The Effective Hourly Rate is computed based on full-time (24x7) usage.

Note that this is just for computing power and some scratch storage. Dedicated long-term storage requires a separate cloud computing service such as S3 and other features as well, so this is just a comparison of raw core compute power. Back of the envelope calculations says that a cloud data center of 100 servers could be operated around the clock on EC2 for well under $5 an hour, or just under $38,000 a year all in (servers, power, cooling, facilities, basic operations), an impressive number. Note that Amazon's EC2 regular pricing for on-demand instances remains at $0.10/hour for Unix/Linux and $0.125 for Windows instances. It's worth noting that Amazon does not offer reserved instances for Windows.

For those planning their cloud usage, you can use Amazon's handy cloud services calculator to run your own scenarios.

Compare this with the pricing recently announced for Microsoft Azure:

Cloud Computing Pricing: Microsoft Azure Services Pricing

Note that this makes Microsoft a hair cheaper (by $0.005/hour) than Amazon for Windows instances for the time being. When Azure launches, Microsoft says it will offer subscriptions that "provide payment predictability and price discounts that reflect levels of usage commitment." At that point, Microsoft will likely be much cheaper than Amazon for Windows instances, the latter whose commitment to Windows in the cloud seems fairly uncertain at the moment. On the other hand, you can't run Unix/Linux on Microsoft's cloud at all and that's not likely to change. Interestingly, both Amazon and Microsoft offer 3 and a half nines of service level availability (99.95%), making comparing the service prices even easier.

The last of the big three clouds is Google App Engine (GAE) and it has the exact same pricing as Amazon's EC2 for on-demand instances or $0.10/CPU hour. GAE is fairly restrictive in terms of what it will run in its cloud, supporting only Java and Python applications at this time, but it's also free for production use to a ceiling of 6.5 CPU hours per day and one daily gigabyte of transfer.

Cloud Computing Pricing: Microsoft Azure Services Pricing

Conclusion: 5 Lessons From Today's Cloud Computing Value Propositions

Taking a look at all this, I've come away with five conclusions about the top providers of cloud computing today given their current pricing and feature sets:

  1. Amazon is currently the lowest cost cloud computing option overall. At least for production applications that need more than 6.5 hours of CPU/day, otherwise GAE is technically cheaper because it's free until this usage level. Amazon's current pricing advantage is entirely due to its reserved instances model. It's also the provider with the most experience right now and this makes it the one to beat with low prices + maturity. However, expect subscriptions from Azure to give it a run for its money when Microsoft's cloud platform formally launches in a few months (probably November).
  2. Windows costs at least 20% more to run in the cloud. Both Microsoft and Amazon offer almost identical pricing for Windows instances while Google App Engine is not even a player in Windows compute clouds. There are undoubtedly cheaper offerings from smaller clouds but they are less likely to be suitable for enterprise use, though certainly there are exceptions.
  3. Subscriptions will be one of the lock-in models for cloud computing. Pre-pay for your cloud to get the most value and you'll get great prices. But you'll be committed to providers for years potentially without a way to leave without stranded investments.
  4. Better elasticity does not confer major price advantages. GAE is one of the most granular of the cloud computing services, only requiring for you to pay for what you actually use (for example, you have to commit to at least an hour of compute time at a time from Amazon) but does not provide a major cost advantage for large applications.
  5. You can't pay more for better uptime and existing SLAs are not sufficient for important business systems. It's unclear why, given open questions about cloud reliability, why no vendors will offer differentiated service where enterprises can pay more for a better SLA. The best you can get right now is also the worst, or 99.95% uptime. This is about 4 hours of expected but unscheduled downtime a year. For business critical applications, this is still too much. This will end up being an opportunity for other vendors entering the space though I expect the Big 3 listed here will improve their SLAs over time as they mature.

I'm hoping to cover additional cloud computing providers and their pricing/value propositions more here this fall to round out our understanding of the competitive advantages that their cost models can provide businesses. In the meantime, expect few changes from the big players until after Azure launches and limited ones from the smaller players, which often use services such as EC2 to provide their own offerings. As Andrew McAfee recently pointed out, the transition to new economic models is long but usually inevitable and cloud computing is no exception.

What are your experiences with cloud compute pricing? How did it affect your decision to switch (or not) from internal infrastructure to the cloud? Please leave your comments below.

Even through the downturn the pace of change in the technology landscape in the last several years has moved at a breakneck pace. Transformations in the marketplace itself as well as major new trends in culture and society are putting concerted pressure on businesses and IT departments to find new ways to adapt. Powerful new technologies such as cloud computing and open supply chains have appeared on the stage that are dramatically changing what is possible as well as how much it costs.

Generational shifts are also contributing to this "future shock" with millenial workers expecting modern workplaces that are up-to-date with the latest capabilities including social software, mobile applications, and self-service IT. Many organizations I speak with, particularly in government and financial services, report that they just can't attract the best talent if their workplace is clearly behind the times.

All of this is a lot to manage to and plan for, but today's leaner, recession-impacted organizations often have to do more with less while changing the way they have to think about what they do. Thus, to help get a strategic map of what's lying at the intersection of IT and business this fall, I thought I'd kick off my inaugural post here on ebizQ with a look at the top 18 emerging topics that will likely to be significant to most organizations in 2009.

As you can see in the visual below, I've tried to map whether these changes are business or technology ones, or as is often the case, both. Each one is classified as to whether it's a brand-new area, is in early adoption, or if it's an approach that is being revisited. The latter classification is usually due to new ways of using it or if there has been a broad rethinking of the subject that's likely to spur a new wave of adoption, such as is probable with SOA. Finally, using color coding, I've tried to identify whether the trend will affect mission critical systems, central functions, or just edge areas of the organizations. There is a legend for these at the bottom.

Emerging Technology Trends at the Intersection Of Business and IT for 2009

Below is a breakdown of each of the topics above, many of which I'll be exploring here in depth in coming months. Some of these will initiate far reaching changes to an organization like Social CRM, Open APIs, and SOA + WOA, while others will have notable impact but be far more localized initially like microblogging and Mobile Office. Note that these are in alphabetical order and not in order of importance, which will vary for each organization.

18 Emerging Business/IT Topics for 2009

  1. Automated Risk Management. The proliferation of interconnected systems and data in most organizations increasingly spells trouble as complexity mounts in today's IT landscape. This is particularly the case as application silos get opened up and SOA achieves deeper interconnections between far-flung parts of organizations. Ensuring that new and modified business applications continue to comply with industry regulations and corporate policy in a world where enterprise mashups and lightweight agile processes are becoming more common is essential. Software support for this space is growing but still limited.
  2. BPM. Process optimization and automation has been a popular topic since the advent of the first workflow systems. However, the recent rise of truly effective BPM tools and standards has ushered in a new area of business process design and automation for the average organization. BPM was previously common mostly in highly process-oriented and/or strongly procedural organizations. In particular, tools that make BPM accessible and usable by line staff are becoming a practical reality. Tools such as the open source ProcessMaker and Intalio are good examples of what's leading the pack from a technology perspective but the advent of emergent architecture (see below) is combining with BPM to create a close connection with the business.
  3. Cloud Computing. By offering economies of scale, access to innovative new application models, agility gains, and reduction in the costs of management and support, cloud computing has a bright future in most organizations despite ongoing reservations about security, control, and performance. A recent survey found that 22% of organizations were already using cloud computing for business critical applications. While most organizations are still kicking the tires, either by creating private clouds or using public ones for testing purposes, cloud computing offers compelling cost and time-to-market opportunities for most organizations this year.
  4. Crowdsourcing. In a seminal new piece, the Wall Street Journal recently explored how technology is allowing us to tap into new wells of innovation that were previously too expensive or difficult to reach. Organizations now have access to virtually everyone on the planet over the Web and can invite them to participate in co-creating new products, improving existing ones, or otherwise transforming their ideation and decision making processes. Taken from the model of open source software, crowdsourcing has become an increasingly popular technique with numerous success stories such as the Netflix prize or Threadless, a community-designed clothing store. The software, however, to enable this business model for most organizations is just in its infancy and most solutions are homegrown, though LG famously used Crowdspring recently to design one of their phones.
  5. Content Discovery. Enterprise search has largely been a disappointment in most organizations, until recently. The ongoing improvements to classical ECM and DMS systems combined with social computing models such as Enterprise 2.0 and Web-Oriented Architecture are finally surfacing (and creating) large amounts of content, making it relevant and reachable via internal search engines, often just because newer social media tools encourage the forging of links. The insistent demands for E-Discovery and content exploitation combined with renewed attempts in reaching submerged information in databases, documents, file systems, and applications is beginning to reap rewards in many organizations. This is often due to overhauls to enterprise architecture as well as the deployment of more modern and increasingly open versions of business applications, even if enterprise search engines are not improving much in and of themselves.
  6. Emergent Architecture. The discipline of enterprise architecture is evolving and becoming more collaborative and inclusive, yet more autonomous as well. Arising from resource constraints as well as software solutions that can be more directly guided by the business, emergent architecture is getting a considerable attention this year as an approach to IT and business architecture that can successfully combine grassroots, bottom-up initiative with the needs of the enterprise in an efficient, agile, robust, and adaptive way.
  7. Enterprise 2.0. Coined by Harvard Business School professor Andrew Mcafee, Enterprise 2.0 refers to the application of social tools to freeform collaboration in the workplace. Blogs, wikis, and social networking software are typical examples of such tools and with about half of all organizations globally deploying Enterprise 2.0 applications this year. It's likely that early adoption status won't last long for this category of productivity and knowledge retention software.
  8. Enterprise Mashups. The mashup approach to software development is now so common on the Web that it's hardly remarked upon, but it's still fairly rare in the enterprise. Organizations such as JackBe, IBM, and others are changing this and bringing the ability to create applications extremely rapidly (typically in hours or days) by building on top of existing business services and accessing stores of enterprise data. Business intelligence is one of the most sought after types of application at a strategic level in most organizations and are one of the most likely places you'll see enterprise mashups first, in the form of dashboards and management consoles.
  9. Microblogging. While many Web 2.0 tools are flourishing in the workplace, blogs have lagged behind, often because of the time and writing skills required to fully exploit them. Microblogging, with the canonical example of the popular service Twitter, can solve this by constraining what is shared socially to small chunks of information, which take less time to produce and consume. Microblogging can create valuable information streams within organizations and broad situational awareness, often to the point that RSS feeds are not required to keep track of key information coming in from across the organization and the Web. While enterprise-class versions of microblogging tools have emerged, most notably SocialText Signals and Yammer, adoption will be uneven though quite strong in some organizations this year.
  10. Mobile Office. The arrival of Apple's iPhone ensured that palmtop business computing wouldn't be far behind. The desktop-class Web browser and addition of an App Store to the iPhone proved that usable business computing devices could be much smaller than the laptop. This has made the smartphone market one of the brightest places in the technology world this year and opens up a new front for software developers -- both commercial and enterprise -- as users demand access to their business data and functionality from their phones. Expect to see a growing focus on business software for mobile devices increase throughout the fall.
  11. Non-relational databases. While the debate about relational vs. non-relational databases continues, there are just some applications that greatly benefit from new forms of storing and accessing very large amounts of data or work better using a document-centric model. While these type of apps are often coming from the Web world, non-relational databases such as CouchDB, Amazon SimpleDB, Mongo, Scalaris are creating compelling new models for application storage, often in a very Web-oriented way.
  12. Open APIs. Taking your private SOA, opening it up to partners, and attaching a business model to it is still something that mostly Web companies do, but an growing number of traditional businesses are opening up their data to partners now in a form of decentralized, self-service business development. The federal government is also following the lead and has been opening up data sets to the world over the Web with initiatives such as As Amazon has longed proved, there is major revenue in cloud services if you only have compelling capabilities or data to offer the marketplace, which most businesses do. The challenge is that APIs require a different set of business skills and mindset, so it's only early adopters at this point, but major success stories have already emerged. View John Musser's great State of the API Market for details.
  13. Productivity-Oriented Programming Platforms (POPPs). Developing Web applications, which is an increasingly large segment of the business application development market, has been getting consistently easier and faster over the last few years using the latest productivity-oriented languages and frameworks such as Ruby combined with Rails, Java and Grails, or PHP and CakePHP. There are often as much as 5-10x productivity improvements with these platforms that greatly reduce the amount of development time yet use the latest best practices and approaches to software development. While Java and .NET remain the dominant platforms for enterprises today, the growth in new software development is increasingly going to the POPPs. Not insignificantly, developers also tend to prefer working with them though actual job openings still dominate with older languages and platforms.
  14. Real-Time Security. With social computing tools and mashups potentially creating data spills and exposed services with vital business functions, security tools that can identify a myried of problems in the enterprise architecture stack in very short amounts of time are of increasing interest. The window in which "the horse can leave the barn" before the door is shut is growing smaller. Security tools must be able to almost instantly detect worms, trojans, security holes, XSS attacks, data leaks and other problems. These are caused not only by flaws in traditional applications but in Web 2.0 software in particular, which tend to be much more open and change more frequently. There are a growing number of vendors which supply such solutions which include Ajax and social media filters and other capabilities.
  15. SOA + WOA. Service-oriented architecture remains the top level organizing principle for enterprise IT and business systems today. However, the lackluster returns that were often achieved frequently resulted in low ROI and subsequent disillusionment. While the whole history of SOA is too detailed to go into here, new simpler and lighterweight models, including Web-Oriented Architecture, are pointing the likely way forward. In short, while SOA has famously struggled of late, it remains the best model for how to architect our enterprises and pragmatic techniques are coming from the Web to help make it more effective on the ground. In short, SOA will get a second shot in most organizations.
  16. Social CRM. The world of customer relationship management is getting a big Web 2.0 makeover this year. Companies such as Helpstream, Lithium, and others are remaking the staid corporate CRM into one in which much more value is mutually exchanged with the marketplace.
  17. Social Computing. While Enterprise 2.0 and Social CRM are also on this list (and are subsets of Social Computing), there is a broader transformation taking place in many organizations right now that includes the strategic application of social software to real business problems. These include social media marketing, vertical industry-specific social networks, and social business models. While Social Computing adoption has been taking place for years, 2009 is proving to be a major year for many organizations.
  18. Unified Communication. The world of UC is getting much harder now that so many new communication channels are proliferating. While online services such as Friendfeed have figured out how to centralize social activity effectively, unified communication is just starting to catch up. Even organizations that have effectively deployed UC in the past will have to do it again to account for the social computing and mobile waves that are hitting them this year.

Please enclose your own thoughts on what's going to be big this fall in IT and business below in comments.