We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

Open for Business

Noam Tamarkin

Interface large amount of data between systems

Vote 0 Votes


In the system that I am responsible on, we use SAP ERP as our back office for payments and charge.

Since we have produce a monthly batch of about 1,000,000 lines we need to pass a large amount of data in one batch via interface.

There are two main challenges:

1. Moving large amount of data between different technologies

2. Control of the success/fail status on the line level

3. Manage a complex scenario that involves updates from both systems in the process

My solution was to use elements to cover all challenges.

The first element was a database table as a staging place for the data and recording line level status. The table was designed to hold the basic line data and two additional fields for status from each system.

The other element was a middleware system to trigger actions on the systems on both sides of the interface. The other task of the middleware was to midiate the technologies (.NET and SAP).

Just to to complete the picture I want to mention other alternatives that we discussed:

1. Transfer files (it's so 90's)

2. Line-by-line call and process (good idea in general but not with large data)

3. Web service (again, problematic with large data)

I invite you to comment on your own experience.

Yours, Noam

Enhanced by Zemanta

Leave a comment

In this blog, Noam Tamarkin provides ideas for improving and better integrating your applications.

Noam Tamarkin

Senior software architect and CTO. Experience in solution design and implementation. Holds the ability to understand complex business processes and translate them to technology. Expert in Enterprise applications, integration, SOA, SaaS. Experienced in project management, technical infrastructure, procurement and manufacturing.


Recently Commented On

Monthly Archives