## Is your feature request related to a problem?

Currently knowledge on query optimization in clickhouse is pretty centralized. This means that when rewriting queries a lot of manual work needs to be made by a select few to ensure we don't lose performance in our wide product.

## Describe the solution you'd like

Have a benchmarking suite.

## Describe alternatives you've considered

Winging it

## Additional context

Some requirements:
- Run benchmark against master + labeled branches, indicate when performance changes
- Use database time over runtime for more accurate measurements
- Use dedicated server (over CI) for running the clickhouse node
- Have a good and representative dataset that we can filter in various ways as we improve performance. Generated datasets require a lot of maintanence and will invalidate previous measurements.

Initial proposal:
- Use [Airspeed velocity](https://asv.readthedocs.io/en/stable/using.html) library to do the benchmarks
- Set up a clickhouse installation with a copy of some of our teams data (or anonymized data) to run against
- Set up tests. Use fixed time-frames in benchmarks when possible.
- Integrate it with PRs, e.g. as with https://github.com/sympy/sympy/pull/21757/files, also run against master. Set concurrency=1
- Publish results somewhere

#### *Thank you* for your feature request – we love each and every one!

ee/clickhouse/materialized_columns/util.py
from datetime import timedelta
from functools import wraps
from typing import no_type_check
from django.utils.timezone import now
from posthog.settings import TEST
def cache_for(cache_time: timedelta):
    def wrapper(fn):
        @wraps(fn)
        @no_type_check
        def memoized_fn(*args, use_cache=not TEST):
            if not use_cache:
                return fn(*args)
            current_time = now()
            if args not in memoized_fn.__cache or current_time - memoized_fn.__cache[args][0] > cache_time:
                memoized_fn.__cache[args] = (current_time, fn(*args))
            return memoized_fn.__cache[args][1]
        memoized_fn.__cache = {}
        return memoized_fn
    return wrapper
def instance_memoize(callback):
    name = f"_{callback.__name__}_memo"
    def _inner(self, *args):
        if not hasattr(self, name):
            setattr(self, name, {})
        memo = getattr(self, name)
        if args not in memo:
            memo[args] = callback(self, *args)
        return memo[args]
    return _inner