Vanity Metrics

Consider (aggregate) metrics like the number of website visits, post likes, comments, bounce rate, etc. Absent any business context, these are usually referred to as vanity metrics.

Such vanity metrics tend to look great in reports (if they’re improving), but they cannot easily be linked to quantitative business objectives, such as Return-on-Investment or Customer-Lifetime-Value. Neither do they come with any explanation as to why they have their specific value.

It’s nice that the number of monthly website visits (vanity metric) has increased by 7% last month, but how do we know whether this can be attributed to our new marketing campaign?

Here’s another example:

In order to reduce friction during registration, we’ve decided to remove the Captcha from our sign-up page at the beginning of last month (in addition to some other improvements to our landing page). By the end of last month, we’ve gained 1408 new users.

What would be the conclusion of this experiment? More signups (vanity metric) is great, but there aren’t any numbers to back up the hypothesis that the Captcha removal has, in fact, reduced sign-up friction for new users. Based on the data, there’s no conclusion to be drawn here.

In order to run this test properly, you’d have to perform an experiment where two fresh and independent batches of visitors are presented with the two different versions of the sign-up page (with and without Captcha), all other parts of the website being equal.

Only if the sign-up rates for visitor batches A and B differ significantly (depending on sample size), you’ll be able to conclude that the Captcha has any impact on sign-up friction.

As a final note, there’s clearly a negative connotation to the word “vanity metric”, but that’s only because they’re often unsuitable for making quantitave business decisions. The number of likes on a company’s Facebook posts would be useful for gaining qualitative audience insights and performing (basic) sentiment analysis, for example.

Cohort Analysis

Simply put, cohort analysis refers to the technique of analysing the behaviour (think: website visits, registrations, purchases) of independent batches of users over time. This timespan can either be a day or two (when selling cheap consumer trinkets via a webshop) or several months (for enterprise products with long sales cycles).

For example, if you’d like to know whether the presence or absence of testimonials on your landing page has any impact on purchase rates for your product, you should consider running the following 2-week cohort analysis experiment.

• Prerequisite 1) Set up a funnel via Funnelbranch for tracking Landing-age-to-Purchase conversions
• Prerequisite 2) Make sure you have a steady stream of independent users visiting your website, either organically or through a marketing campaign (Google ads, FB ads, etc.)

As for the actual experiment:

• Week 1) Set up your website such that every visitor gets to see your landing page without testimonials
• Week 2) Set up your website such that every visitor gets to see your landing page with testimonials

Then, by comparing the conversion rate between weeks 1 and 2, you’ll be able to determine the impact of landing page testimonials on purchase rates.

When setting up a funnel via Funnelbranch, there’s an important parameter called session expiry. If you think your users convert from first-time visitors to buyers within a day, you should set your funnel’s session expiry to 1 day via the dashboard.

By default, Funnelbranch allows you to track cohort sessions for up to 3 days.

If you’re looking to track cohorts over a period of weeks or months (to figure out whether customers tend to return to your e-commerce store within 6 months after signing up via one of your marketing campaigns, for example): please contact us.

Project

We recommend setting up different projects for different websites or applications. Multiple funnels can be created for each project.

Funnel

A funnel represents a sequence of steps (i.e.: stages) to be completed in the correct order by users and visitors of your website or application.

Stage

Each stage represents a step in the funnel sequence. Stages can be triggered by URL visits or custom actions.

Session

A session is defined as a single user or visitor matching against one of your funnels, either partially or entirely.

Funnel Setup: Session Expiry

An important parameter to configure when setting up a funnel is the so-called session expiry, which is set to 24 hours, by default.

For example, if a new user visits your homepage, Funnelbranch immediately starts recording a new session, and only stops recording after 24 hours. This means it doesn’t matter for your conversion metrics whether this user immediately registers, or whether they leave your website and return later to register (within 24 hours).

By default, the funnel session expiry time is capped at 3 days. If you’re looking to have this maximum increased for your project, please contact us.

Funnel Setup: Optional Stages

When setting up a funnel (via the dashboard), it’s important to exclude any optional stages from the funnel definition. If a visitor matches the 1st and the 3rd step of a funnel, but hasn’t matched the 2nd step, then they’ll end up being recorded as having only matched the 1st step.

Note that you can always define a second funnel with slightly different stages, though.

Funnel Setup: Stage Triggers

Funnel stages are triggered when visitors interact with your website. They can either be triggered by URL visits or by custom actions submitted via your (client-side) Funnelbranch integration.

Below's an overview of possible trigger conditions for each of these two trigger types.

JavaScript

Refer to the README file of the funnelbranch.js GitHub repository for installation and API documentation.

JavaScript - Frameworks

Refer to the README file of the Funnelbranch Script Wrapper GitHub repository for installation and API documentation.

Other Clients

If you’re looking to integrate from a mobile application or a backend server, please contact us to discuss a direct integration with our API.