Native Apps At The Client & Cloud

Srinivasan Sundara Rajan

Subscribe to Srinivasan Sundara Rajan: eMailAlertsEmail Alerts
Get Srinivasan Sundara Rajan: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn

Related Topics: Cloud Computing, Cloud Data Analytics, Cloud Development Tools


Increase Business Agility in Cloud Data Integration

CDC with Informatica Cloud Platform

In my previous article on Cloud Integration Platforms, I considered most of the real-time queue-based integration tools, which uses event-based message queues to integrate the application on premise to an application hosted on Cloud. However, real-time message-based integration is applicable to most situations, but there are some variants.

Batch Integration
There are requirements  within an enterprise that a source system and target system are integrated at regular intervals.  This is generally accomplished through ETL Tools (Extraction, Transformation and Loading)  while leveraging the  Scheduling options of respective tools.

These are some of the situations of a batch interface between a source and target .

  • Customer Orders on a E-Commerce Portal may be stored in a data store and the orders will be fulfilled by the provisioning system over a nightly batch job
  • A job portal may accept the applications during the day and perform the matching of jobs to profiles over the nightly batch job

There are many such situations where a source system can feed its data to a target system over scheduled intervals, typically at night time.

Change Data Capture  & Cloud Integration
The business agility of an enterprise is best realized if the integration between a source and target system is done in the least possible time so that customer satisfaction is  improved. The batch  windows for this integration can be completed quickly only if the data  that is changed since the previous run is processed, but not all the data. This is known as CDC (Change Data Capture).

Typically the CDC Option recognizes business events - such as customer creation or order shipment data - by capturing the database inserts, updates, and deletes underlying these events as soon as they occur. This captured stream of database activity can be delivered to multiple targets in real time or batch.

Integration between On Premise ERP applications and with Cloud-based SaaS applications can benefit from this CDC (Change Data Capture) option and this pattern is valuable for business agility.

Informatica Cloud Services & Cloud Platform
Informatica,  which has long been a player in the ETL and Data Integration space, has established a stronger  presence in the Cloud Integration too. We find several options for Cloud Integration from their product site.

Informatica provides Cloud Integration as multiple options.

  • SaaS model in the form on Cloud Services
  • PaaS model in the form of Cloud Platform

Informatica Cloud Offering & CDC
We find some  good implementation of the CDC  (Change Data Capture)  Pattern  on the above mentioned Cloud Integration Offerings.

Some of the features of the CDC Option  are listed below.

  • Capture updates from relational and pre-relational databases on mainframe, midrange, and commodity systems
  • Access changes once and route them to multiple targets without using staging tables or intermediate queues
  • Extract all updates, or view only the most recent version of selected data
  • View data only from committed transactions
  • Deliver data while preserving business-level transactional semantics
  • Capture changes without having to rely on specialized programming skills
  • See only the latest change to a data element so you can deliver the latest version of that element-even when hundreds or even thousands of changes are made to the element over the course of a business day
  • Connector services including a secure agent that reaches through the firewall to deliver powerful cloud-to-on-premise data integration, such as Salesforce CRM to Oracle, or Salesforce CRM to JD Edwards, as well as cloud-to-cloud data integration.

Informatica Cloud Services delivers powerful bi-directional data loading and extraction capabilities and goes much further. With Informatica Cloud Services you can easily set up advanced scheduling in order to automate everything from simple to complex multistep integrations between various cloud and on-premise systems.

The below diagram from Vendor Site Courtesy, explains the  Informatica Cloud Integration solutions and the CDC Option can be part of the same.

Utilizing the best of SaaS applications on Cloud while continuing to use the touch points on premise is best realized if we can process only the changed data so that the processing is most efficient, and hence the CDC  pattern of Cloud Data Integration is most relevant in this situation.

More Stories By Srinivasan Sundara Rajan

Highly passionate about utilizing Digital Technologies to enable next generation enterprise. Believes in enterprise transformation through the Natives (Cloud Native & Mobile Native).