Skip to content

[FR]: Support custom clickhouse health check timeout #5397

@ew0s

Description

@ew0s

Proposal summary

It'll be great to have ability to change duration of clickhouse (and other dependencies) healthcheck timeout through env vars

source

@Override
    protected Result check() {
        try {
            return template.nonTransaction(connection -> Mono.from(connection.createStatement("SELECT 1").execute())
                    .flatMap(result -> Mono.from(result.map((row, rowMetadata) -> row.get(0))))
                    .map(o -> Result.healthy()))
                    .block(Duration.ofSeconds({$ANALYTIC_DB_HEALTHCHECK_TIMEOUT})); // better to have ability to change this parameter through envs
        } catch (Exception ex) {
            return Result.unhealthy(ex);
        }
    }

Motivation

We have a restriction from db team to use only https connections to clickhouse, so that there are often cases when round trip in out infrastructure (network + ch healthcheck query) overlaps current timeout and k8s tend to restart pods in such cases.

Currently our backend pods restarted more than 200 times in 4 days.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions