Flow vs Other L1s — A cohort analysis

# Intro

When I think of measuring user-retention, the best tool that comes to mind is “Cohort Analysis”. But, first we need to define what is Cohort analysis. Here is the textbook definition:

Cohort analysis is an analytical technique that categorizes and divides data into groups with common characteristics prior to analysis. This technique is typically used to make it easier for organizations to isolate, analyze, and detect patterns in the lifecycle of a user, to improve user retention, and to better understand user behavior in a particular cohort.

That’s hard to ingest and understand, so here’s a good example from Bill Su:

Let’s say that your customer Bob entered your online store four months ago in response to a 50% discount, looked through your wares, and bought a tiny trial set of your avocado cosmetics.

As a business owner, it is natural to ask yourself — are people like Bob coming back to my store as result of that trial set purchase?

So you ask your store attendant (their name is “Cookie”) to keep track of behavior of people like Bob, and see if they return and make a purchase, and how frequent those returns are.

In this case, “people like Bob” i.e. people who were attracted by the 50% discount are a cohort and you’re trying to analyze whether these people come back month after month.

## Cohort Analysis Example

Here is a real world example of using Cohort analysis to measure users who have launched an app first time and revisited the app in the next 10 days.

Source

From the above retention table — Triangular chart, we can infer the following

• 1358 users launched an app on Jan 26. Day 1 retention was 31.1%, day 7 retention was 12.9%, and day 9 retention was 11.3%. So on the 7th day after using the app, 1 in 8 users who launched an app on Jan 26 were still active users on the app.
• Out of all of the new users during this time range (13,487 users), 27% users are retained on day 1, 12.5% on day 7, and 12.1% on day 10.

# Applying Cohort Analysis to blockchains

In the blockchain world, there is no shortage of blockchains and dApps. With all the EVM compatible L1s these days, you can find most of the same dApps deployed everywhere. Therefore, for an L1 to grow it needs to not only attract users but retain them month over month.

## Methodology

What is a cohort? — In this analysis, I’m considering the users who joined in a particular month (i.e. the month in which the first transaction was done) as part of the same cohort. Then, I calculate how many of these users are transacting again in the next months.

What does the cohort size show? — The cohort size for each month shows how many new users joined that L1 in that month.

How to read the tables? — To read the tables, start with the month name in the “Cohort” column and then add the month number in the “Month 1, 2, 3 …” columns. E.g. Cohort of April ‘22 will have June ’22 as Month 2.

## Flow

The data for Flow only goes back to April in Flipside’s tables so we’re seeing a really small table here.

• We can see that 92% of the April cohort was retained in May. However, beyond May, we see the number of users retained drops to 6%!
• Similarly for May cohort, we’re only seeing 4% users retained in the next month.

## Ethereum

Ethereum tables go much farther back, so we have a larger time-period to explore here.

• We can see that the January cohort was the largest with 37M wallets. Since January, the cohort has been decreasing and we’re down to 4.5M in the June cohort.
• We can see that Ethereum is showing much better retention with the 33% of the January cohort retained for the first 2 months. Even at Month 6, we’ve retained nearly 14% of the users.
• For the Feb-June cohorts, we’re down to around 10–20% retention in the first months and then high single digits in the subsequent months.

## Solana

Solana’s data is a lot slower to query so I could only go as far back as February ’22. We notice a few key things immediately:

• The monthly cohort sizes are a lot smaller than Ethereum
• On the other hand, the user-retention is much higher in almost every month.
• For e.g. Month 1 retention for May ’22 is 45% compared to Ethereum’s 11%. Month 2 retention for Feb-May is nearly 2x that of Ethereum.

## Polygon

Even Polygon shows a higher user-retention than Ethereum. The higher retention than Ethereum is likely due to low gas-fees.

However, compared to Solana, Polygon’s user-retention is lower in most cohorts.

## Osmosis

Surprisingly, Solana is not the best performer in this L1 comparison. That crown goes to Osmosis.

• For the Jan ’22 cohort, Osmosis shows 69% user-retention in the first-two months, 56% at month 4 and 33% at Month 6!
• For reference, 45% was the Month 1 user-retention for Solana’s May ’22 cohort but it dropped to 23% in Month 2.
• Even in subsequent months and as we get deeper into the bear-market, we can see two digit user-retention percentage across the board.

## Algorand

• Algorand along-with Flow is the weakest performer in this comparison with only one month of double digit user-retention for Month 1!
• For most other months, the user-retention is low single digits.
• Things did improve a bit in April but the numbers are still lacking compared to other L1s.

# Conclusion

In this analysis, we applied the Cohort Analysis tool to measuring user-retention on various L1s. Here are the major findings:

1. Solana and Polygon both exhibit better user-retention than Ethereum which is likely due to lower gas-fees.
2. Flow and Algorand are both the weakest performers with many cohorts exhibiting single-digit user-retention.
3. Osmosis represented the best user-retention in this analysis with the January cohort exhibiting 56% user-retention at Month 4 and 33% at Month 6.