Common Reasons for Incorrect Data in Web Analytics

Incorrect data is an acute problem for online businesses. While most businesses have started to realize the importance of data-mining and performance-based marketing, it seems that some of the effort has been going in the wrong direction. The point is not about having as much data as possible, but rather having the data you can actually work with – structured, complete and correct.

Below I will outline some common sources of erroneous or incomplete data and give some advice on how to avoid the mistakes.

Unclear or imprecise definitions of KPIs.

Even if your company is not very large, make sure that everyone dealing with data understands precisely what is included in statistics and how it is calculated. It even makes sense to create a written document defining the KPIs, no matter how simple or self-explanatory they might seem.

For example, if you want to calculate CTR (click through rate) for banner advertising, which is, in essence, the number of clicks on the banner divided by the number of banner impressions, you can ask yourself a range of questions. Do you want to apply unique banner impressions and unique clicks (that is, per user)? At what intervals do you want to measure the CTR (per hour, per day, per week)? How do you go about natural fluctuations in CTR (e.g. during the day vs at night), do you want to make them part of your statistics or just ignore them and take the average? How do you group the data: by region, by traffic source, by banner type, etc.? If you run banner tests, do you want to exclude the test data from the overall statistics?

As you can see, the answers to these questions might greatly influence the outcome, i.e. the CTR your  data analyst will produce at the end of the day.

Technical issues

The more complex a system is, the more likely it is to malfunction. Always check for technical issues if dealing with data inconsistency. For this, it is best to work with statistical and technical benchmarks, based on the normal system behavior in the past.

Technical problems influencing the data consistency may, firstly, be caused by the under-performance of the system itself (for example, the banners are not served for a period of time, or are not displayed correctly in some browsers). Secondly, even if the system functions well, there might be problems in capturing or storing the data (e.g. not enough RAM to perform operations causes the database server to crash). And thirdly, if you do not query the database directly but let the data flow through a business intelligence system, there might be all kinds of compatibility problems between the systems.

Incorrect calculations

This is the least predictable source of data inaccuracy. It starts with how the data is collected and aggregated in the system. Even if one has enough clarity on how the KPIs are constructed, there is always a chance that the setup of the analytics system will divert from the desired parameters. Furthermore, if data processing and analysis are largely done manually, the probability of a mistake rises with every step. Also, merging the data from several sources can impact the data consistency in a negative way, especially if some parameters have to be converted or rounded up before the merging.

Thus, human factor should not be underestimated, and the only way to minimize the number of errors is double checking the calculations, or better still, automating as much of data processing as possible.

In conclusion, if you find that the data does not “seem right”, even after you have excluded every possibility of a mistake, you might need to look for the reason outside of your analytics system. One of my previous blog posts looks at this issue in more detail.

Share this post

Mobile Marketing KPIs: What to Consider

This post gives an overview of essential mobile marketing KPIs that would be considered for a performance dashboard. It also outlines a general framework of how these KPIs relate to each other.

mobile_kpis
Mobile marketing KPIs structure

Conversion funnel is an important part of any performance dashboard. The funnel shows how the user travels through different stages and interacts with the ad and then with the product.  A simple conversion funnel for a mobile marketing display campaign advertising for an app would be:  Impressions-Clicks-Installs-First Interaction, with percentage share between them.

Return KPIs can be measured as monetary values: revenue per user, lifetime user value, etc. However, non-monetary KPIs (number of installs, daily active users, daily active paying users, average session length, etc.) are also important, especially in the cases where monetization is detached from the download (free apps or image campaigns).

Cost KPIs will largely depend on your payment model for the advertising. Possible ways are:

Cost per mille (per thousand impressions)

Cost per click

Cost per install (also possible: cost per lead, cost per download)

Cost per engagement

Cost per revenue or revenue share models

ROI is calculated as a percentage share of return on what has been invested in advertisement, for a product that does not monetize immediately, the ROI is calculated per time period (daily, weekly) in user cohorts.

The framework at the bottom of the picture shows how the KIPs can be adapted to achieve a significant level of preciseness and data consistency.

Timeframe includes both how the KPIs are aggregated (daily, weekly, monthly) and what time periods they cover (past, current or future/trend data).

Segmentation is essential for getting the working data. Making the segments too broad will impact the data consistency negatively and working with micro-segments is mostly too difficult to implement and does not allow to make solid conclusions based the data sample.  As an example, consider segmenting the data by country or region, by traffic channel or by device type.

Sample statistics includes the parameters that analyze the quality of data and how the sample behaves in general. Instead of just aggregating and averaging the data, consider such parameters as median, min. and max. amounts and standard deviation. Besides, look at the size of the segments you work with: which of those have the strongest impact and why? Also, learn to recognize patterns in your timeframe, and how the data fluctuates on a daily, weekly or seasonal basis.

After you have decided on the KPI data you will include for your dashboard and reporting, consider making the data more visual and more structured for the intended users. In one of my next posts, I will cover the issue of data visualization in more detail.

Share this post

Do You Know Your Personal KPIs?

The notion of Key Performance Indicators, or KPIs is by no means new in the business management world. With the growth of online businesses and the rise of real-time data collection the importance of structuring, sorting and filtering the increasing amount of internal data has increased as well.

However, most business do not really have insights into what KPIs are relevant for them and work with a standard set of metrics without adjusting it to the nature of their business. Here are some simple steps on how to build your personal system of KPIs.

Look at your business more closely. Two simple ways of doing it are designing a sales funnel for your business (that is the size of the target market, the number of leads at each stage of the buying process and how they are managed by your business system); or looking at the customer journey (at the stages a potential customer would go through in your system: from the lead to the buyer, including post-buying behaviour). The major difference between two approaches is that the fist one is more internally driven whereas the second one is externally driven or customer based.

By looking at how your marketing and sales funnel narrows through different stages you can work out relevant KPIs that will help to monitor the numbers in the funnel. Some examples are in the table below.

Total size of the target market Size of the segment you hope to reach with marketing campaigns
Number of customers who would visit the shop Number of customers who would start the buying process
Number of customers who would make the purchase Number of customers who would rebuy

Using the second, customer oriented approach you would analyse where your customers are coming from, when they mostly visit the page, and how they navigate through your shop and make their purchase decisions. This approach allows you to develop a series of “soft” or qualitative indicators that will likely be helpful in adjusting and analyzing the data you would derive using the first approach.

Hard facts about the customer: age, gender, location Soft facts about the customer: interests, hobbies
Buying situation: home, work, time of the day Buying behavior: time spent on site, number of revisits
Shopping basket: size, contents After purchase: feedback, cross-selling, rebuys

After you have a list of metrics that measure your business, decide on the most relevant indicators or KPIs that will be looked upon in the first place.

Design a dashboard for your KPIs that shows a snapshot of data you look at in the first place, as well as the development of indicators over time. In designing the dashboard you should consider the past data (e.g. average over the past month/year), the present data (think of how often you will update it: hourly, daily, weekly, etc), and the future data, i.e try to work out a system of variables in your system and incorporate this data into plotting a trend for KPIs.

Develop the data collection and management system that would meet your needs. Think of the tools you use (you could acquire a ready-made solution for data management, or design and program your own unique system, which will often be costly but will allow for the necessary flexibility and help to prevent data leakages). You should have basically two systems in place – one for collecting operational, daily data, and the other one for evaluating marketing or sales campaigns that only take place at certain intervals. Do not let both data flows melt into one in your system, otherwise you will have difficulty explaining the changes in KPIs over time.

Set the targets in your KPIs based on company business goals, historical values, industry average or benchmarking. Evaluate the incoming numbers at regular intervals, check them against the goals and predicted values you had set in your system, and if necessary, adjust those goals after more data has flown into the system.

Also, do not collect data just for the sake of collecting data. Your KPIs should actually be involved into your decisions, whether on operational or strategic level or at least give you valuable insights into your business and how it functions over time or under different conditions.

Share this post