Through the Looking Glass: Website Reporting Approaches

In this article I will describe some common approaches to reporting and analytics.

If you already have analytics system in place, it is time to think about reporting on your website. In this article I will describe some common approaches to reporting and analytics.

Ad-Hoc Reporting: This type of reports is done only once, usually to answer a research question or after a major update. For example, if you want to find out where the site visitors geographically come from but suspect this will not change significantly month on month. Regular reporting  can be done monthly, weekly or daily to track performance over time.

Reporting by segment summarizes information using the segments you have (see my post on segmentation). Detailed reporting examines the data at page level (this way you can find out how different pages perform and what key user journeys are). Holistic reporting  contains website data in general (for example, you want to display KPIs of the whole site). Most likely you will combine different levels of data granularity in your report.

The reports you produce will differ depending on the stakeholder group you create them for. For example, reports for senior management will demonstrate trends and bird’s eye view of data. Reports for Marketing department may contain numbers on campaign performance, click-through rates and page/product rankings.

Analytics reports can also produce different types of analysis from descriptive to casual. At the same time, the goal of reporting may include increasing conversion, analyzing user journey, leading users to certain pages, increasing time on site, decreasing bounce rates, optimizing campaign performance, understanding SEO or referral traffic, etc.

You will most likely need to decide how to benchmark or evaluate the data. One way is to compare the data to the previous reporting period (e.g. June data to May data). This, however, does not take into account seasonal or weekday fluctuations, which may have a significant impact on performance. Another way is to compare the reporting period to the same period in the past, e.g. June 2017 to June 2016. This will provision for fluctuations, however, will not consider how different the conditions could have been in the past or what overall traffic growth was realized. A third way to approach benchmarking is compare your performance to that of your industry segment or your main competitor (this data will seldom be available). And finally, if a business goal is set, the KPIs will be evaluated against this goal.

I hope this article has given you some ideas what can be included into your reporting. Apart from setting the framework, it is important to provide human and technical resources for successful reporting.

Picture source: Unsplash

Share this post

Divide and Rule: Types of Segmentation in Web Anaytics

In order to analyze traffic to your website you need to segment it. A lot of times, possibilities for segmentation will depend on the analytics system you use and the data you track. In this article, I will outline how you can create segments using the data commonly provided by analytics software. I will also evaluate, to what extent you can make use of the created segments.

First group of segments: user-based

Geo and tech characteristics of users belong to this group. If you have a way to find out the gender or the age of your users (registration, dialog windows), you can segment visitors even more granularly.

User-based segments: by device type.

Geo-segmentation includes segmenting by cities, regions, countries as well as browser languages.

Geo-data is usually collected based on user IP. Although this information seems to be interesting (often presented in form of maps), it is only relevant if you offer a location-based product/service (e.g. only available in a few cities) or if your website is localized for several countries and you want to make sure that international users are routed to the correct country site.

Tech segments are based, for example, on device type or browser name and version, as well on as the user domain. From my experience, these parameters are not going to be used extremely often either. If you get a significant number of visitors from a certain device or a certain browser,  you should better make sure that your site works on them. Other than that, it useful to group sessions where an HTTP or a JavaScript error occurred (if an error is recurring and you can reproduce it, a fix is needed).  Additionally, IP address or user domain name are used to exclude internal traffic (i.e. the traffic from your company).

Second group of segments: content-based

As can be guessed from the name, such segments relate to the content of the site. In other words, by applying this segmentation, you can see what happens on certain parts of your website or on groups of pages. If set correctly, this gives a structured and precise insight into the website performance.

Some ways to apply this segmentation are:

  • By product (if you have a few products, e.g. 1-5)
  • By brand or by product type (if your website showcases a lot of products)
  • By department/division (if this is how your site is organized)
  • By content for different customer groups (gender, age, type of customer: business or private…)
  • By site component (forms, product description, FAQ…)
  • By country site (if the data is not split in the database)

This is by no means an exhaustive list. By looking at your site, you may come up with a more suitable (for you) way to segment content. Then you can use the grouping, for instance, to compare how different products perform or to create a product ranking by popularity/engagement.

Third group of segments: traffic-based

This type of segments will contain general break-up of traffic, without precise description of particular users. The most common segments will include: by traffic source, by referring domain, by time of visit (day of the week, hour).

New vs repeat/returning visitors segment is based on the cookie set on a visitor’s device (however, as a growing number of visitors do not accept cookies or delete them regularly, the data from this segmentation should be treated as an approximation).

Such segments are extremely useful in order to analyze how the performance changes depending on the type traffic flowing to the site. What is more,  traffic type will most probably have a larger impact on performance rather than user-based characteristics,

Fourth group of segments: interaction based

This is a large group of segment that describes how visitors interact with the site.

You can create segments by number of page views per visit (page depth), by bounce rate, by time on site, etc. For the majority of these parameters, you will need to come up data ranges, e.g. number of page views per visit may be split into the following segments: 1 view, 2-3 views, 4-6 views, 7 or more views. Besides, you can include events – such as forms submission,

This type of segmentation will enable you to start the analysis “from the other end”, e.g. what are the traffic sources or pages in best performing segments.

As you can see, applying correct segmentation to site traffic will provide you not only with more data but also with more insights into what influences the amount of traffic and its performance over time. But what if you feel that besides segmenting you need to group and generalize traffic somehow? This would be a clear case for cohort analysis, which I may talk about in one of the next posts.

Share this post

360 Degrees: Types of Data Analysis

According to J. Leek, there are six data analysis questions we can ask. Let us look at these questions in detail. As an example, I will use a website of a marketing agency. The conversion goal of this website is the contact form that users fill out and send. We want to ask questions about the conversion.

Descriptive analysis: What is..?

Here the goal is to describe a set of data, without inferring anything from it or making a prediction. It can be performed by itself or be a starting point of a more in-depth analysis. An example in case of an agency would be the number of forms sent or the conversion rate (e.g. the number of submissions divided by the number of page visits).

Explorative analysis: Where is..?

This analysis looks at the data more deeply, discovering the connections between different variables. However, this cannot be used for prediction or does not necessarily imply causation. In the example above, exploratory analysis can be used to look for connection between form submissions and day of the week or where the form was placed on a page.

Inferential analysis: Who is..?

In this type, we infer about a larger group of users from a small group. E.g. we can set up a user survey or conduct focus group research with some users to find out how they interact with the site and the contact form in particular.

Predictive analysis: What will be..?

This analysis makes prediction about future occurrences based on some known variables. For example, we could predict the fluctuation in the number of form submissions according to the day of the week. Prediction should not be confused with causality, e.g. if there is a peak of form submissions on Monday, it does not mean that Monday causes form submissions.

Causal analysis: Why is…?

This is used to define one-to-one relationships between different variables. Causality will almost always look at the average cases, so some outliers or different behaviors should not be excluded. As an example, placing the form at the top of the page will usually increase the conversion, as opposed to placing it below-the-fold. Here, a better form placement actually causes more visibility and increases the chance of a submission.

Mechanistic analysis: How is..?

This a rare and difficult type of analysis that allows for understanding how exactly variables influence each other in individual objects. This would often bring the analysis of data down to the set of equations.

All in all,  you will need to perform different types of analysis of the raw data to get a clear picture and come up with a set of recommendations.

Share this post

Common Reasons for Incorrect Data in Web Analytics

Incorrect data is an acute problem for online businesses. While most businesses have started to realize the importance of data-mining and performance-based marketing, it seems that some of the effort has been going in the wrong direction. The point is not about having as much data as possible, but rather having the data you can actually work with – structured, complete and correct.

Below I will outline some common sources of erroneous or incomplete data and give some advice on how to avoid the mistakes.

Unclear or imprecise definitions of KPIs.

Even if your company is not very large, make sure that everyone dealing with data understands precisely what is included in statistics and how it is calculated. It even makes sense to create a written document defining the KPIs, no matter how simple or self-explanatory they might seem.

For example, if you want to calculate CTR (click through rate) for banner advertising, which is, in essence, the number of clicks on the banner divided by the number of banner impressions, you can ask yourself a range of questions. Do you want to apply unique banner impressions and unique clicks (that is, per user)? At what intervals do you want to measure the CTR (per hour, per day, per week)? How do you go about natural fluctuations in CTR (e.g. during the day vs at night), do you want to make them part of your statistics or just ignore them and take the average? How do you group the data: by region, by traffic source, by banner type, etc.? If you run banner tests, do you want to exclude the test data from the overall statistics?

As you can see, the answers to these questions might greatly influence the outcome, i.e. the CTR your  data analyst will produce at the end of the day.

Technical issues

The more complex a system is, the more likely it is to malfunction. Always check for technical issues if dealing with data inconsistency. For this, it is best to work with statistical and technical benchmarks, based on the normal system behavior in the past.

Technical problems influencing the data consistency may, firstly, be caused by the under-performance of the system itself (for example, the banners are not served for a period of time, or are not displayed correctly in some browsers). Secondly, even if the system functions well, there might be problems in capturing or storing the data (e.g. not enough RAM to perform operations causes the database server to crash). And thirdly, if you do not query the database directly but let the data flow through a business intelligence system, there might be all kinds of compatibility problems between the systems.

Incorrect calculations

This is the least predictable source of data inaccuracy. It starts with how the data is collected and aggregated in the system. Even if one has enough clarity on how the KPIs are constructed, there is always a chance that the setup of the analytics system will divert from the desired parameters. Furthermore, if data processing and analysis are largely done manually, the probability of a mistake rises with every step. Also, merging the data from several sources can impact the data consistency in a negative way, especially if some parameters have to be converted or rounded up before the merging.

Thus, human factor should not be underestimated, and the only way to minimize the number of errors is double checking the calculations, or better still, automating as much of data processing as possible.

In conclusion, if you find that the data does not “seem right”, even after you have excluded every possibility of a mistake, you might need to look for the reason outside of your analytics system. One of my previous blog posts looks at this issue in more detail.

Share this post

Do You Know Your Personal KPIs?

The notion of Key Performance Indicators, or KPIs is by no means new in the business management world. With the growth of online businesses and the rise of real-time data collection the importance of structuring, sorting and filtering the increasing amount of internal data has increased as well.

However, most business do not really have insights into what KPIs are relevant for them and work with a standard set of metrics without adjusting it to the nature of their business. Here are some simple steps on how to build your personal system of KPIs.

Look at your business more closely. Two simple ways of doing it are designing a sales funnel for your business (that is the size of the target market, the number of leads at each stage of the buying process and how they are managed by your business system); or looking at the customer journey (at the stages a potential customer would go through in your system: from the lead to the buyer, including post-buying behaviour). The major difference between two approaches is that the fist one is more internally driven whereas the second one is externally driven or customer based.

By looking at how your marketing and sales funnel narrows through different stages you can work out relevant KPIs that will help to monitor the numbers in the funnel. Some examples are in the table below.

Total size of the target market Size of the segment you hope to reach with marketing campaigns
Number of customers who would visit the shop Number of customers who would start the buying process
Number of customers who would make the purchase Number of customers who would rebuy

Using the second, customer oriented approach you would analyse where your customers are coming from, when they mostly visit the page, and how they navigate through your shop and make their purchase decisions. This approach allows you to develop a series of “soft” or qualitative indicators that will likely be helpful in adjusting and analyzing the data you would derive using the first approach.

Hard facts about the customer: age, gender, location Soft facts about the customer: interests, hobbies
Buying situation: home, work, time of the day Buying behavior: time spent on site, number of revisits
Shopping basket: size, contents After purchase: feedback, cross-selling, rebuys

After you have a list of metrics that measure your business, decide on the most relevant indicators or KPIs that will be looked upon in the first place.

Design a dashboard for your KPIs that shows a snapshot of data you look at in the first place, as well as the development of indicators over time. In designing the dashboard you should consider the past data (e.g. average over the past month/year), the present data (think of how often you will update it: hourly, daily, weekly, etc), and the future data, i.e try to work out a system of variables in your system and incorporate this data into plotting a trend for KPIs.

Develop the data collection and management system that would meet your needs. Think of the tools you use (you could acquire a ready-made solution for data management, or design and program your own unique system, which will often be costly but will allow for the necessary flexibility and help to prevent data leakages). You should have basically two systems in place – one for collecting operational, daily data, and the other one for evaluating marketing or sales campaigns that only take place at certain intervals. Do not let both data flows melt into one in your system, otherwise you will have difficulty explaining the changes in KPIs over time.

Set the targets in your KPIs based on company business goals, historical values, industry average or benchmarking. Evaluate the incoming numbers at regular intervals, check them against the goals and predicted values you had set in your system, and if necessary, adjust those goals after more data has flown into the system.

Also, do not collect data just for the sake of collecting data. Your KPIs should actually be involved into your decisions, whether on operational or strategic level or at least give you valuable insights into your business and how it functions over time or under different conditions.

Share this post