Data Integration is one of the most important elements of enterprise information management (EIM). It’s also one that is often least understood or appreciated.

The end goal for any EIM environment or strategy is to transform data to insights that drive actions. In order to complete the data-insight-action journey, companies need to effectively clean, normalize, combine and enrich distinct sources of data within their central data warehouse or data lake to capture a 360-degree view of the patient, the healthcare professional (HCP), and the environment in which they operate.

As disparate data sources flowing into an organization expand, data integration becomes increasingly fragmented and complex. Today’s healthcare data environment includes structured and unstructured data designed for a variety of applications. The same healthcare data lake will include data from internal systems, third party vendors, healthcare organizations, public research, and social media (to name a few). And each set may represent a different population, region or country – each of which adheres to a unique set of rules, languages, and local preferences.

In its raw form, this data cannot be simply merged and analyzed. Companies need sophisticated platforms as well as industry domain knowledge of therapeutic areas, ontologies and methodologies for defining data attributes to bring all this data together, clean it, format it, and link it to create that 360 view. Only then can they begin to analyze these assets and find answers that drive new business insights.

This requires a number of different tools, capabilities and the expertise of a skilled data manager.

Integration strategy: The data integration journey begins with a strategy that includes both the technical skills required to build and support the EIM environment, along with a data framework that defines what data assets the company wants to integrate, what types of data users need, and any format or structural requirements that must be applied to the data. They also need to understand the structure, delivery, and flow of that data. In some cases, data assets like electronic medical records and insurance claims will be updated monthly or weekly; while others, like Google Analytics, or wearable device readings from clinical study patents, will update in near real time. When companies choose data integration platforms and tools, they should look for a technology layer that possesses capabilities to handle the technical and business process flows across the spectrum of disparate data sources and that can seamlessly adapt to changes in these process flows.

Establishing this foundation will help companies choose the best tools for data integration and analytics, and ensure they deliver measurable value to the organization.

Reference data: Reference data is the backbone of how companies manage data around customers, products, organizations, peers and patients. Most data assets will have the same entities within them, but they will look and feel different due to variations in data collection strategies. For example, the same healthcare provider might be referenced in a specialty pharmacy database, medical claims records, and your own CRM system.

MDM: For companies to harmonize entities across different sources they need to be able to sort which entities are the same person, and which are different people with similar names, addresses, etc. Companies need Master Data Management (MDM) and reference data management solutions provide this clarity, delivering a concise and accurate view of the entity. The MDM system enables users to connect the dots, tying all of the transactional data flowing into the organization via every channel. That includes results of sales calls, medical information requests, media and events, and other customer and clinical touchpoints.

Warehouse platforms: With today’s volume, velocity and variation of data, companies need adaptable warehousing platforms that can easily handle processing large data volumes. We encourage clients to seek out cloud warehouses, and cloud platform capabilities. For example, Spark can help manage, transform and translate data, and Snowflake or Azure Synapse provides a data storage layer, which can bring all this data into a single data lake to link assets with the MDM system.

Business rule management systems. Companies then need to formulate business rules to generate insights from this data. This includes a territory alignment engine (for commercial use cases), the use of conversation algorithms and natural language processing, and deidentification systems to transform raw data into a meaningful data that can be utilized for analytics while still maintaining data privacy. It is important for these rules to be defined by data consumers since they know how the data will need to be used for clinical and commercial applications.

We have seen many implementations where the business rule management layer has been encapsulated within software codes making it difficult to access outside of the software development team. This limits flexibility in the system and prevents users from scaling those rules across your globe. When the platform allows for self-serve use, it becomes easy for the rule and platform to evolve with the needs of the organization.

Spread the word

With all these layers in place, companies can propagate the data to their data science and commercial teams to generate insights that are relevant to each group. For clinical teams, that can mean defining the size and location of patient populations, gathering data about comorbidities and adverse events or identifying relevant subpopulations of trial patients, whereas commercial teams can use it for things like defining brand potential, and identifying next best actions for sales outreach.

There are a lot of tools in the market to support data integration, so it’s important to be judicious. Selecting tools that fit the skills of the data management team, the needs of end users, and the types of data flowing into the organization will ensure your EIM system meets the needs of all stakeholders now and in the future.

IQVIA provides the gold standard for pharmaceutical market data as well as clinical research. IQVIA employs thousands of data scientists, healthcare professionals, technology experts, and Data Integration professionals who oversee processing of more 100 billion healthcare records annually. To learn more about IQVIA’s data services, contact us here.