Breaching the Data Dam

Part One of our two-part look at the data technology required to support modern customer experience programmes.

Data Dam 3.png

Why marketing teams need their own data platform

by damian williams | june 2018

Marketing technology is a hot discussion topic at a senior level in most enterprises, particularly between CMOs and CIOs. A common question that comes up from the IT side in these discussions is “Why do I need yet another data platform?”

In reality, that’s the wrong question because:

IT does not need another data platform, Marketing needs a dedicated one.

Many enterprises have spent years and millions of dollars upon realising Enterprise Data Strategies that revolve around consolidation and centralisation. Key drivers of consolidation include cost reduction and simplification of the technology stack. Centralisation of data provides benefits focussed around improved systemic control and compliance.

Although these strategic approaches undoubtedly deliver multiple benefits across the organisation, there is usually one exception – the marketing department. 

The problem with most Enterprise Data Strategies from a marketing perspective is that they are essentially driven by compromise. In many enterprises, there is an overriding need to create and maintain an immutable System of Record (SoR) to satisfy external reporting and regulatory requirements. This need results in compromises because every department in the organisation has different requirements, and it is impossible to meet them all. 

In a digital landscape that is increasingly fragmented, with consumers using ever-increasing numbers of platforms and devices, marketers are becoming hamstrung by traditional Enterprise Data Strategies. Common enterprise data environments fail to provide marketing departments with what they need to drive the types of experiences that their customers and stakeholders are demanding. 

Key signs that your Enterprise Data Strategy is not supporting your marketing programme

  • Your data lake is more like a dam. Modern marketing requires quick access to data in order to drive timely and relevant experiences. However, most enterprise data warehouses and data lakes require skilled analysts to access and manipulate the data that they hold - skills that marketing teams usually don’t possess. Consequently, marketing data requests often join a significant backlog for prioritisation and execution. This means that, in an increasingly real-time and reactive world, the opportunity to send a campaign can be lost.  

  • The cost to add data is prohibitive. Systems of Record used for business critical and regulatory reporting typically cannot be changed without going through multiple layers of approval. This usually adds layers of cost and time to the process of adding data to the environment. Marketing departments struggle to justify the cost of sourcing data for campaigns that run for relatively short amounts of time, and the IT change timescales involved are often too long to make it worthwhile.

  • Your data environment is not designed to handle scale. Enterprise data warehouses and data lakes are often built to store and report on large core transactional data sets. However, even in the largest of organisations, the transactional data set may be dwarfed by the amount of digital data that can be generated by web analytics and marketing platforms. To handle this scale and preserve capacity for critical SoR data, digital marketing data can be heavily filtered or summarised, limiting its usefulness for driving campaigns. Speed is also an issue – most centralised data infrastructure is not designed to handle the real-time data feeds and decisioning required by modern customer experience programmes.

Not all marketing data needs to be centralised

Although there will always be a need to feed some marketing data back into the centralised enterprise data store (e.g. opt outs, business process and reporting data), there is a significant amount of operational marketing data that will never need to be centralised.

  • A lot of marketing data is only used by marketers, for example: email opens, complete web analytics feeds and paid media impressions.

  • Marketing data can be transient, in that it may only be required for a one-off campaign or for a short period of time (e.g. – seasonal, promotional or partner data).

  • Some data is platform-specific, for example: specific data tables that are required by a marketing platform for its operation but are derived from a larger data set.

Holding marketing data in a dedicated platform makes a lot of sense for marketers who need to run time-sensitive, reactive campaigns. However, it also makes sense from an IT and data governance point of view because:

  • It preserves the integrity of SoR data sets so that business critical reporting and processes are not affected by relatively unstructured and transient marketing data.

  • Marketing teams can have SoR data access limited to only that required by their marketing platforms. This improves auditability as it becomes easier to track what was shared with the marketing department and when.

  • IT capacity planning becomes easier without the ‘lumpy’ demands for data from marketing teams when large campaigns are being run.

You cannot drive a modern digital customer experience programme from the same data infrastructure that is acting as your System of Record.

The only real solution to the different needs of IT, Compliance and Marketing is to have a dedicated data platform for marketing. This platform needs to be fed from a variety of sources, including Systems of Record, to generate a single view of marketing data. The single customer view can then be distributed in relevant subsets (segments) to downstream marketing decisioning and delivery platforms, and then augmented with marketing campaign response and interaction information.

Only by implementing a dedicated Customer Data Platform (CDP) can enterprise marketers deliver the customer experiences that are increasingly demanded of them. 

But what of the other elements of the modern marketing technology stack? A common question that is asked by CMOs is whether any of their existing marketing technology platforms can potentially be co-opted to act as a Customer Data Platform. In part two of this article, we’ll look at the other types of data platform in the marketing technology stack.

TAGS | systems of record, marketing technology, customer data platform

about fulcrum hub

Fulcrum Hub is a technology business that enables enterprises to rapidly automate personal experiences for each customer, at scale. We provide a next generation Customer Data Platform built to solve real enterprise problems around data integration, campaign targeting, personalisation and performance attribution.

Strip blue NEW.png



Fulcrum Hub "Kea" Release

In June we will be deploying the Fulcrum Hub “Kea” release, our second major release of the year. This release brings the Hub to life with major usability and performance improvements and extends our levels of configurability. 

KEA release trimmed.png

New features and enhancements contained in this release

  • Concurrent Processing - With the implementation of a queuing system, we are now able to run multiple jobs concurrently, which will increase the Hub’s already fast processing time and allow for both horizontal and vertical scaling to further increase its performance.

  • Hub API v1.0 - In order for our customers and our own applications to interact more efficiently with the Hub we have released the first version of the Hub API.  This version of our API provides the ability to view and modify a limited set of the Hub’s functionality and to monitor its processing.

  • Hub Web UI v1.0 - To enable our customers to be able to better monitor and operate their Hub instances we have released the first version of our Web UI.  This release is focused on supporting the functionality of the first version of the API.  

  • Hub Data Visualiser v1.0 - The power of leading open-source analytics and data visualisation tool Grafana has been integrated into the Hub’s Web UI to give its users the ability to create a wide range of engaging graphs and reporting dashboards from the data the Hub holds.

  • Configurable channels - new Hub channels can now be created and modified through the Hub Web UI instead of them having to be created by Fulcrum. This enhancement has major productivity benefits as it enables Hub customers to add, enable, disable and modify Hub data sources and sinks (channels) without any need to contact Fulcrum.

New integrations to the Hub

  • Google Analytics integration - The Hub can now pull data from Google Analytics and ingest it for decisioning, reporting and export. This adds to the Hub’s existing Adobe Analytics support and allows it to use either or both of the 2 main web analytics platforms as data sources.

  • Microsoft Dynamics 365 integration - The Hub can now pull and push data to and from the Microsoft Dynamics 365 CRM.  This allows the Hub to enrich its own customer records with CRM data and to enrich the CRM records with the Hub’s data. 

  • Delacon Call Tracking Service integration - The Hub can now pull data from the Delacon Call Tracking service to provide visibility into inbound calls generated from paid digital media campaigns.

  • Shoretel Unified Communications Server integration - The Hub can now ingest call logs from the Shoretel Unified Communications Server to allow it to attribute in and outbound calls with leads and customers to further enrich the customer records that it holds.  

Bug fixes contained in this release

  • Vacuum command bug where it could make DB slower

  • SFTP command timeout which could lead to the Hub blocking indefinitely

  • SFTP command to download only files

  • Logging bug where a job could fail on some environments



Blue hexagon banner.png


Modern marketing decision-making

Why a centralised approach to decisioning is not necessarily the best for marketing. Read on to learn more about:

  • The three key types of decision in digital marketing

  • The pitfalls of centralised decisioning

  • Common tensions between Marketing and IT

  • The advantages of taking a decentralised approach

Decisioning head NEW.png

Making marketing decisions at enterprise scale

by damian williams | May 2018

Decisions, and the process of making them, is an important part of the digital marketing customer lifecycle. Marketing decisioning, put simply, is the process to determine which message a customer should receive through a given channel or medium.

Many organisations take a relatively simplistic approach to decisioning and attempt to use a single platform to make decisions. This platform then pushes data to a delivery tool (or tools) for message generation and presentation.  

This works well for small organisations that have a limited set of relatively simple campaigns to decision for, and a simple set of rules for deciding which messages an individual should actually receive. This rule set is commonly known as a contact framework.  

In these simple cases the decisioning platform can also be the delivery platform. However, in most large enterprises, the decisioning process and contact framework is far from simple. 

The single-platform decisioning approach does not scale to larger organisations that communicate through multiple channels, with a large set of campaigns and a complex contact framework.  

In large enterprises, there are typically three different types of decisioning that influence marketing campaign delivery:

  • Campaign Decisioning (also known as customer targeting and personalisation) determines which contacts should be targeted by a given campaign and generates personalised data to populate that campaign.

  • Message Decisioning: this is where the actual message or offer that a customer should receive is determined. This is particularly important for multi-step campaigns, where a contact may receive several different messages based upon custom logic and data.

  • Contact Decisioning: when a contact is a candidate to receive multiple messages or offers in the same time window, this is the process that determines how many of these messages the contact will actually be presented, based upon a prioritisation and/or suppression framework. 

Although each of these types of decision is distinct and relevant to a different stage of the campaign journey, many large enterprises do not split out decisioning into distinct stages. A key reason for this is that large organisations tend to centralise processing - because it generates a sense of control and makes it easier to trace what processing has occurred.

An outcome of adopting a centralised decisioning approach is the provisioning of large platforms and associated infrastructure to perform marketing data processing. Also, under this approach, delivery and presentation platforms perform no decisioning and are only responsible for message rendering and/or delivery.  

Unfortunately for enterprise marketers, a centralised decisioning approach has a number of key issues that result in poor marketing outcomes:

  • Long campaign turnaround times due to complex campaign setup and the significant operator expertise required. 

  • Under-utilised capability of marketing delivery platforms, resulting in campaign experience being less ‘rich’ and engaging than what is potentially possible. In a centralised decisioning approach, the maximum level of experience that can be delivered is limited by what the decisioning platform can do, not by what the delivery platforms can achieve.

  • High cost of change as all contact rules and decisioning logic are centralised in a single enterprise-wide platform. 

  • Confusion and complexity as decisioning is merged into a single process without distinct stages and separation of concerns. Complex decisioning configuration in a single platform leads to a lack of clarity within the enterprise as to what the contact rules are and the impacts that they have.  

Of course, the main reason for the friction between a typical enterprise IT-led approach and an ideal marketing approach is the appetite for change. Marketers need to be inherently flexible and reactive to match changing consumer needs, while IT managers are driven by needs to centralise operations, control costs, and reduce the need for change.

The ideal enterprise approach to decisioning

To achieve a winning balance between the needs of marketers and IT managers, enterprises need to focus upon developing a decisioning capability that can provide all of the following:

  • Fast campaign turnaround times to meet changing customer needs and mitigate against competitor activity.

  • Reduction in the need to engage costly IT resources to deploy marketing campaigns.

  • An environment where test and learn approach can be adopted, enabling marketers to optimise campaigns based upon performance.

In this approach, decisioning is not centralised in a single platform – rather it is divided between platforms that have capabilities best suited to perform each type of decisioning and for the type of operator that will be driving it.  

  • Campaign decisioning in a Customer Data Platform or dedicated decisioning platform that delivers a set of campaign independent records per contact with targeting flags and personalisation.

  • Message decisioning in a marketing platform that includes journey building capability. This delivers fully personalised messages for delivery to contacts for the given time window.

  • Contact decisioning in a marketing platform with contact framework capability that allows for the delivery of prioritised and personalised massages to contacts.

A key advantage of this approach is that it does not require any specialist IT or development resource to design, decision and deploy campaigns, instead using existing marketing and data analyst resources more effectively.

This approach also allows marketers to utilise the full capabilities of modern multi-channel marketing platforms to build out complex and rich customer experiences. Having the contact framework closer to the point of delivery also allows alternative models to be generated and evaluated as all of the potential messages are available for review.

Adopting a decentralised decisioning approach for marketing does not mean increasing complexity or adding additional cost. In reality, done well, it is a ‘fit for purpose’ approach that utilises existing resources and technology better without impacting the integrity of the enterprise data framework.

Fulcrum Hub enables marketers to automate the decisioning process across the three key types of decisioning, and can be deployed quickly with minimal impacts upon existing IT infrastructure. For further information on the capabilities that we can bring to your marketing technology stack, please contact us.

tags | marketing automation, digital transformation, data decisioning, customer journeys




Unlocking customer data

How to drive customer experience transformation using data from legacy systems

Unlocking customer data transparent graphic only.png

How to transform your customer experience without breaking your systems

By Rory Watt | May 2018

Digital-led customer experience transformation is one of the hottest topics for CMOs in today’s market. According to IDC, over two thirds of global enterprises are looking to make the strategic shift from traditional to digitally-led strategy, with over one third expecting to have fully adopted digital transformation by the end of this financial year.

As with many major strategic shifts, digital transformation is largely consumer-led at its core. Consumers are now demanding to deal with brands through their own channels of choice, and new market entrants are primed to disrupt industries that are slow to react to the pace of digital change. Senior marketers have been well aware of this trend for some time and have now started to lead transformation initiatives at the C-level.

According to recent research by Gartner, CMOs are now outspending CIOs, making significant purchases of marketing-related technology from their own capital and expense budgets as they look to engage customers through a rapidly-expanding number of channels. However, the purchase of modern marketing technology systems is only one part of driving successful customer experience transformation. 

CMOs now have access to a range of powerful solutions to deliver great customer experiences across multiple digital and offline channels simultaneously. However, the performance (and ultimately payback) of these systems can be seriously impacted by an inability to access customer data. 

In any large organisation, customer data is usually held in a range of legacy systems and databases, and in formats that end delivery platforms struggle to use effectively. Liberation of this data – and creation of the near-mythical Single Customer View – is often pushed to IT as a systems architecture challenge. As IT teams serve the entire organisation, marketing then has to compete with a range of other areas for resource, and data integration gets wrapped up into broader change programmes that can take years.

The challenge for CMOs is that their experience transformation programmes can’t wait years for access to customer data. This leads to executives purchasing a range of point solutions to meet the specific requirements of new and emerging channels, and ultimately, fragmented multi-vendor marketing technology stacks that don’t work together seamlessly. In fact, even single-vendor stacks have integration issues as they are often the result of a series of acquisitions by the lead vendor attempting to gap-fill capability and pull together an end-to-end solution.

Mixed vendor environments are, and will remain, a reality for CMOs in the digital age. The challenge isn’t really about technology consolidation, it’s more about working out how a range of technologies can work best together, using and enriching a common set of marketing data. The answer to this challenge is in an emerging sector of marketing technology, the Customer Data Platform.

Customer Data Platforms act to unify marketing data from a wide range of sources, transforming it into usable, targeted information. Essentially, they are the ‘marketing glue’ that acts to stick together a disparate range of legacy databases, third party data sources, CRM systems, and marketing delivery platforms, making the whole stack work together to deliver the end experience.

Of course, data integration solutions have existed in the market for years. However, many of these solutions are complex, requiring specialised IT support to maintain, and expensive data science resource to operate. The advantage of the Customer Data Platform is that it only looks at data relevant for marketing and is ‘owned and operated’ by the marketing department. 

Customer Data Platforms optimise data specifically for marketing outcomes. Marketing-specific data integration isn't just a matter of getting two systems to talk to each other - the data also has to be optimised to make campaigns easy to implement. There is little point sending complex data models to delivery platforms that will then require in-depth work by marketing delivery teams to get campaigns live. Customer Data Platforms make the campaign deployment task easier by optimising data before it reaches the end delivery platform, reducing complexity and increasing the number of campaigns that can be deployed within a given timeframe. 

Modern Customer Data Platforms, like the Fulcrum Hub, are also relatively inexpensive and quick to implement. And as they are designed with easy integration in mind, they typically require minimal IT change – meaning they can be up and running, delivering value within weeks, not years.

tags | Data integration, customer data platform, customer experience, digital transformation


Blue hexagon banner.png