您现在的位置是：首页 > Javascript

当前栏目

ClickHouse集群搭建部署

集群部署

2023-04-18 16:24:11 时间

文章目录

一、单机版安装和启停
二、集群部署搭建
- 1、手动搭建clickhouse集群
- 2、使用docker-compose快速搭建clickhouse集群
三、集群扩容
- 如何保证扩容后的数据均匀分布？
四、集群缩容
docker-compose 中的一些文件

一、单机版安装和启停

单机版的安装相对简单，官网介绍了好几种办法：

https://clickhouse.com/docs/zh/getting-started/install/

这里提一下tgz安装包的方式，目前官网的文档有点问题(不知道将来会不会更新)。从21.2.xx之后的版本的相关tgz包已经移到 https://repo.clickhouse.com/tgz/stable/ 下面了，官方文档的curl拉的路径全部都是https://repo.clickhouse.com/tgz。另外，如果通过官网的命令获取最新版本，甚至在 https://repo.clickhouse.com/tgz/stable/ 下可能都找不到对应的tgz包，遇到这种情况，我们可以自行找一个合适的版本下载

#获取最新版本的clickhouse
export LATEST_VERSION=`curl https://api.github.com/repos/ClickHouse/ClickHouse/tags 2>/dev/null | grep -Eo '[0-9]+.[0-9]+.[0-9]+.[0-9]+' | head -n 1`
#如果最新版本下载不到，可以获取指定版本的clickhouse
export LATEST_VERSION=21.12.3.32
#21.2.xx 之后的tgz包路径变成https://repo.clickhouse.com/tgz/stable下
#21.2.xx 之前的还在https://repo.clickhouse.com/tgz下
curl -O https://repo.clickhouse.com/tgz/stable/clickhouse-common-static-$LATEST_VERSION.tgz
curl -O https://repo.clickhouse.com/tgz/stable/clickhouse-common-static-dbg-$LATEST_VERSION.tgz
curl -O https://repo.clickhouse.com/tgz/stable/clickhouse-server-$LATEST_VERSION.tgz
curl -O https://repo.clickhouse.com/tgz/stable/clickhouse-client-$LATEST_VERSION.tgz
 
tar -xzvf clickhouse-common-static-$LATEST_VERSION.tgz
sudo clickhouse-common-static-$LATEST_VERSION/install/doinst.sh
 
tar -xzvf clickhouse-common-static-dbg-$LATEST_VERSION.tgz
sudo clickhouse-common-static-dbg-$LATEST_VERSION/install/doinst.sh
 
tar -xzvf clickhouse-server-$LATEST_VERSION.tgz
sudo clickhouse-server-$LATEST_VERSION/install/doinst.sh
sudo /etc/init.d/clickhouse-server start
 
tar -xzvf clickhouse-client-$LATEST_VERSION.tgz
sudo clickhouse-client-$LATEST_VERSION/install/doinst.sh

启动/关闭clickhouse服务：

clickhouse start
clickhouse stop

clickhouse安装完之后会注册linux服务，因此也可以通过linux系统服务来启停clickhouse：

#注意！如果使用linux系统服务启动clickhouse，后面不能用clickhouse stop来关闭，不然系统会重新拉起clickhouse
/etc/init.d/clickhouse-server start
/etc/init.d/clickhouse-server stop

安装完测试连通性：

clickhouse client --port 9000

二、集群部署搭建

1、手动搭建clickhouse集群

clickhouse 集群是非主从结构，各个节点是相互独立的。因此，和hdfs、yarn的集群不同，我们可以根据配置，灵活的配置集群，甚至可以将一个节点同时分配给多个集群。

clickhouse集群的概念主要就是用于分布式表和表的副本

在这里插入图片描述

上面这张图有3个节点，这3个节点组成了2个集群。

想要配置集群，需要在 /etc/clickhouse-server/config.xml的 <remote_servers> 标签下添加相关集群信息。或者在/etc/metrika.xml中进行配置。

如果要在 /etc/metrika.xml 中配置，需要确保metrika.xml已经被config.xml包含进去了：

<remote_servers incl=“clickhouse_remote_servers”> remote_servers记得加incl属性

<include_from>/etc/metrika.xml</include_from> config.xml 将metrika.xml包含进来

要实现上图的集群架构，ck1、ck2、ck3的/etc/metrika.xml配置分别如下：

ck1配置：

<yandex>
    <clickhouse_remote_servers>
        <!--自定义集群名称-->
        <test_cluster1>
            <!--定义集群的分片数量，2个shard标签说明有2个节点-->
            <shard>
                <!--定义分片的副本数量，这里副本只有1个-->
                <replica>
                    <host>ck1</host>
                    <port>9000</port>
                </replica>
            </shard>
            <shard>
                <replica>
                    <host>ck2</host>
                    <port>9000</port>
                </replica>
            </shard>
        </test_cluster1>
    </clickhouse_remote_servers>
 
    <zookeeper-servers>
        <node index="1">
            <host>zk1</host>
            <port>2181</port>
        </node>
    </zookeeper-servers>
</yandex>

ck2配置：

<yandex>
  <clickhouse_remote_servers>
    <test_cluster1>
      <shard>
        <replica>
          <host>ck1</host>
          <port>9000</port>
        </replica>
      </shard>
      <shard>
        <replica>
          <host>ck2</host>
          <port>9000</port>
        </replica>
      </shard>
    </test_cluster1>
    <test_cluster2>
      <shard>
        <replica>
          <host>ck2</host>
          <port>9000</port>
        </replica>
      </shard>
      <shard>
        <replica>
          <host>ck3</host>
          <port>9000</port>
        </replica>
      </shard>
    </test_cluster2>
  </clickhouse_remote_servers>
 
  <zookeeper-servers>
    <node index="1">
      <host>zk1</host>
      <port>2181</port>
    </node>
  </zookeeper-servers>
 
</yandex>

ck3配置：

<yandex>
  <clickhouse_remote_servers>
    <test_cluster2>
      <shard>
        <replica>
          <host>ck2</host>
          <port>9000</port>
        </replica>
      </shard>
      <shard>
        <replica>
          <host>ck3</host>
          <port>9000</port>
        </replica>
      </shard>
    </test_cluster2>
  </clickhouse_remote_servers>
 
  <zookeeper-servers>
    <node index="1">
      <host>zk1</host>
      <port>2181</port>
    </node>
  </zookeeper-servers>
 
</yandex>

配置完之后，无需重启clickhouse服务，clickhouse会热加载这些配置。我们可以分别登陆这3台clickhouse，通过 select * from system.clusters; 查看当前节点所属集群的相关信息：

在这里插入图片描述

配置好集群之后，我们就可以基于配置好的集群创建分布式表了：

--在集群test_cluster1上的各个节点创建test表（也就是ck1、ck2）
create table default.test on cluster test_cluster1(id Int8,name String) engine = MergeTree order by id;
--基于test_cluster1创建分布式表
create table test_all as test engine =Distributed(test_cluster1,default,replicaTest,rand());

2、使用docker-compose快速搭建clickhouse集群

当集群机器数量众多，一台一台操作会非常麻烦。另外，如果我们手上没有服务器，又想深入研究clickhouse集群的一些特性时，就可以通过docker快速的搭建起clickhouse集群。

这里简单介绍一下docker-compose，docker-compose会根据定义好的配置文件帮我们启动多个docker container，省去我们一个个容器的操作工作。

下面的docker-compose.yaml是我经常用来快速搭建一个clickhouse集群的docker-compose配置文件：

version: "3.7"
 
services:
  ck1:
    image: yandex/clickhouse-server
    ulimits:
      nofile:
        soft: 300001
        hard: 300002
    ports:
      - 9001:9000
    volumes:
      - ./conf/config.xml:/etc/clickhouse-server/config.xml
      - ./conf/users.xml:/etc/clickhouse-server/users.xml
      - ./conf/metrika1.xml:/etc/metrika.xml
    links:
      - "zk1"
    depends_on:
      - zk1
 
  ck2:
    image: yandex/clickhouse-server
    ulimits:
      nofile:
        soft: 300001
        hard: 300002
    volumes:
      - ./conf/metrika2.xml:/etc/metrika.xml
      - ./conf/config.xml:/etc/clickhouse-server/config.xml
      - ./conf/users.xml:/etc/clickhouse-server/users.xml
    ports:
      - 9002:9000
    depends_on:
      - zk1
 
  ck3:
    image: yandex/clickhouse-server
    ulimits:
      nofile:
        soft: 300001
        hard: 300002
    volumes:
      - ./conf/metrika3.xml:/etc/metrika.xml
      - ./conf/config.xml:/etc/clickhouse-server/config.xml
      - ./conf/users.xml:/etc/clickhouse-server/users.xml
    ports:
      - 9003:9000
    depends_on:
      - zk1
 
  zk1:
    image: zookeeper
    restart: always
    hostname: zk1
    expose:
      - "2181"
    ports:
      - 2181:2181

上面的配置文件定义了4个容器，其中3个容器分别运行clickhouse服务，1个容器运行zookeeper服务。

配置后docker-compose.yaml后，进入该配置文件的目录，执行 docker-compose up -d 就会一起启动这些容器，clickhouse集群也就快速搭建好了。通过docker-compose down可以卸载集群。

上面docker-compose.yaml中引入的./conf/users.xml、./conf/config.xml、./conf/metrika.xml 等内容会在本博客的最后贴出。

在这里插入图片描述

三、集群扩容

假设当前有个集群test_cluster，有两个节点，该集群下面有张test表。集群配置如下：

<test_cluster>
      <shard>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck1</host>
          <port>9000</port>
        </replica>
      </shard>
      <shard>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck2</host>
          <port>9000</port>
        </replica>
      </shard>
</test_cluster>

test表的相关创建语句：

--在集群各个节点创建test表
create table default.test on cluster test_cluster(id Int8,name String) engine = MergeTree order by id;
--在某个节点创建分布式表
create table test_all as test engine =Distributed(test_cluster,default,replicaTest,rand());
--写入若干数据
insert into test_all values(1,'zhang'),(2,'li'),(3,'zhao'),(4,'qian'),(5,'sun'),(6,'wang'),(7,'tian'),(8,'he'),(9,'zheng'),(10,'dong');

之后我们想往这个test_cluster新增一个节点。扩容的步骤大概如下：

在这里插入图片描述

1、在新节点安装clickhouse，进行配置（加上原有集群的相关配置）
编辑新节点的/etc/metrika.xml (ck3为新节点)：

<yandex>
  <clickhouse_remote_servers>
    <test_cluster>
      <shard>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck1</host>
          <port>9000</port>
        </replica>
      </shard>
      <shard>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck2</host>
          <port>9000</port>
        </replica>
      </shard>
      <shard>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck3</host>
          <port>9000</port>
        </replica>
      </shard>
    </test_cluster>
  </clickhouse_remote_servers>
 
  <zookeeper-servers>
    <node index="1">
      <host>zk1</host>
      <port>2181</port>
    </node>
  </zookeeper-servers>
 
</yandex>

2、在新节点新建该集群的相关本地表
因为test_cluster集群下面只有一张test表，因此我们只要在新节点下创建test表即可：

--创建test表，结果就是ck1、ck2、ck3都有test表
create table if not exists default.test on cluster test_cluster(id Int8,name String) engine = MergeTree order by id;

3、修改集群旧节点的config.xml配置，加上新节点
在ck1、ck2的/etc/metrika.xml中全部加上ck3的配置：

<yandex>
  <clickhouse_remote_servers>
    <test_cluster>
      <shard>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck1</host>
          <port>9000</port>
        </replica>
      </shard>
      <shard>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck2</host>
          <port>9000</port>
        </replica>
      </shard>
      <shard>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck3</host>
          <port>9000</port>
        </replica>
      </shard>
    </test_cluster>
  </clickhouse_remote_servers>
 
  <zookeeper-servers>
    <node index="1">
      <host>zk1</host>
      <port>2181</port>
    </node>
  </zookeeper-servers>
 
</yandex>

修改完配置文件后，因为clickhouse会自动感知到config文件变化，因此我们修改的内容会马上生效。

后面可以通过下面的方式验证是否扩容成功：

--在ck1、ck2查询system.clusters表，看ck3是否已经加进来了
select * from system.clusters
--往之前创建的分布式表test_all表中再插入若干数据
insert into test_all values(1,'zhang'),(2,'li'),(3,'zhao'),(4,'qian'),(5,'sun'),(6,'wang'),(7,'tian'),(8,'he'),(9,'zheng'),(10,'dong');
--去ck3查看是否有数据写入
select * from test

4、通知客户端更新节点列表

如何保证扩容后的数据均匀分布？

根据写入的场景我们可以分开分析：

1、数据是通过分布式表来写入

这种情况，我们可以通过设置集群的权重，让后面的数据优先写入新节点，比如：

<yandex>
  <clickhouse_remote_servers>
    <test_cluster>
      <shard>
        <weight>1</weight>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck1</host>
          <port>9000</port>
        </replica>
      </shard>
      <shard>
        <weight>1</weight>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck2</host>
          <port>9000</port>
        </replica>
      </shard>
      <shard>
        <weight>99</weight>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck3</host>
          <port>9000</port>
        </replica>
      </shard>
    </test_cluster>
  </clickhouse_remote_servers>
 
  <zookeeper-servers>
    <node index="1">
      <host>zk1</host>
      <port>2181</port>
    </node>
  </zookeeper-servers>
 
</yandex>

将ck1、ck2的权重都设置为1，ck3的权重设置为99，这样后面写入的数据大部分都会写入到ck3中。等到ck3的数据差不多和ck1、ck2持平了，再将权重改成一样

2、数据是在客户端层面直接往各个节点的本地表写入

这种情况就需要稍微改造下客户端的程序，让客户端可以优先选择新节点的本地表进行数据写入，直到各个节点的数据平衡

四、集群缩容

假设当前有个集群test_cluster，有三个节点，该集群下面有张test表。集群配置如下

<yandex>
  <clickhouse_remote_servers>
    <test_cluster>
      <shard>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck1</host>
          <port>9000</port>
        </replica>
      </shard>
      <shard>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck2</host>
          <port>9000</port>
        </replica>
      </shard>
      <shard>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck3</host>
          <port>9000</port>
        </replica>
      </shard>
    </test_cluster>
  </clickhouse_remote_servers>
 
  <zookeeper-servers>
    <node index="1">
      <host>zk1</host>
      <port>2181</port>
    </node>
  </zookeeper-servers>
</yandex>

现在我们需要下掉一个节点（ck3），大概需要进行以下步骤：

在这里插入图片描述

1、对外停止服务

防止操作过程中客户端读取到的数据不完整

2、转移要下线节点的数据

这里需要将要下线节点的数据转移到其他的节点去，数据迁移可以使用以下方式：

--在ck1执行下面的sql，将ck3的部分数据写到ck1的本地表中
insert into replicaTest select * from remote('ck3:9000','default','replicaTest','default') where id % 2 = 0;
--在ck2执行下面的sql，将ck3的部分数据写到ck2的本地表中
insert into replicaTest select * from remote('ck3:9000','default','replicaTest','default') where id % 2 = 1;

执行完上面的sql后，ck3的数据就迁移到ck1、ck2中去了

3、修改剩余节点的集群配置

在ck1、ck2的config.xml配置文件中去除ck3的配置：

<yandex>
  <clickhouse_remote_servers>
    <test_cluster>
      <shard>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck1</host>
          <port>9000</port>
        </replica>
      </shard>
      <shard>
        <internal_replication>true</internal_replication>
        <replica>
          <host>ck2</host>
          <port>9000</port>
        </replica>
      </shard>
    </test_cluster>
  </clickhouse_remote_servers>
 
  <zookeeper-servers>
    <node index="1">
      <host>zk1</host>
      <port>2181</port>
    </node>
  </zookeeper-servers>
</yandex>

4、通知客户端更新节点列表

docker-compose 中的一些文件

users.xml :

<?xml version="1.0"?>
<clickhouse>
    <!-- Profiles of settings. -->
    <profiles>
        <!-- Default settings. -->
        <default>
            <!-- Maximum memory usage for processing single query, in bytes. -->
            <max_memory_usage>10000000000</max_memory_usage>


            <!-- How to choose between replicas during distributed query processing.
                 random - choose random replica from set of replicas with minimum number of errors
                 nearest_hostname - from set of replicas with minimum number of errors, choose replica
                  with minimum number of different symbols between replica's hostname and local hostname
                  (Hamming distance).
                 in_order - first live replica is chosen in specified order.
                 first_or_random - if first replica one has higher number of errors, pick a random one from replicas with minimum number of errors.
            -->
            <load_balancing>random</load_balancing>
            <allow_ddl>1</allow_ddl>
            <readonly>0</readonly>
        </default>

        <!-- Profile that allows only read queries. -->
        <readonly>
            <readonly>1</readonly>
        </readonly>
    </profiles>

    <!-- Users and ACL. -->
    <users>
        <!-- If user name was not specified, 'default' user is used. -->
        <default>
		<access_management>1</access_management>
            <password></password>

            <networks>
                <ip>::/0</ip>
            </networks>

            <!-- Settings profile for user. -->
            <profile>default</profile>

            <!-- Quota for user. -->
            <quota>default</quota>

            <!-- User can create other users and grant rights to them. -->
            <!-- <access_management>1</access_management> -->
        </default>
	<test>
	    <password></password>
	    <quota>default</quota>
            <profile>default</profile>
	    <allow_databases>
	        <database>default</database>
                <database>test_dictionaries</database></allow_databases>
            <allow_dictionaries>
	        <dictionary>replicaTest_all</dictionary>
            </allow_dictionaries>
	</test>
    </users>

    <!-- Quotas. -->
    <quotas>
        <!-- Name of quota. -->
        <default>
            <!-- Limits for time interval. You could specify many intervals with different limits. -->
            <interval>
                <!-- Length of interval. -->
                <duration>3600</duration>

                <!-- No limits. Just calculate resource usage for time interval. -->
                <queries>0</queries>
                <errors>0</errors>
                <result_rows>0</result_rows>
                <read_rows>0</read_rows>
                <execution_time>0</execution_time>
            </interval>
        </default>
    </quotas>
</clickhouse>

config.xml:

<?xml version="1.0"?>
<!--
  NOTE: User and query level settings are set up in "users.xml" file.
  If you have accidentally specified user-level settings here, server won't start.
  You can either move the settings to the right place inside "users.xml" file
   or add <skip_check_for_incorrect_settings>1</skip_check_for_incorrect_settings> here.
-->
<clickhouse>
    <logger>
        <level>trace</level>
        <log>/var/log/clickhouse-server/clickhouse-server.log</log>
        <errorlog>/var/log/clickhouse-server/clickhouse-server.err.log</errorlog>
        <size>1000M</size>
        <count>10</count>
    </logger>

    <http_port>8123</http_port>
    <tcp_port>9000</tcp_port>
    <mysql_port>9004</mysql_port>
    <postgresql_port>9005</postgresql_port>
    <interserver_http_port>9009</interserver_http_port>
    <max_connections>4096</max_connections>
    <keep_alive_timeout>3</keep_alive_timeout>
    <grpc>
        <enable_ssl>false</enable_ssl>
        <ssl_cert_file>/path/to/ssl_cert_file</ssl_cert_file>
        <ssl_key_file>/path/to/ssl_key_file</ssl_key_file>
        <ssl_require_client_auth>false</ssl_require_client_auth>
        <ssl_ca_cert_file>/path/to/ssl_ca_cert_file</ssl_ca_cert_file>
        <compression>deflate</compression>
        <compression_level>medium</compression_level>
        <max_send_message_size>-1</max_send_message_size>
        <max_receive_message_size>-1</max_receive_message_size>
        <verbose_logs>false</verbose_logs>
    </grpc>
    <openSSL>
        <server>
            <certificateFile>/etc/clickhouse-server/server.crt</certificateFile>
            <privateKeyFile>/etc/clickhouse-server/server.key</privateKeyFile>
            <dhParamsFile>/etc/clickhouse-server/dhparam.pem</dhParamsFile>
            <verificationMode>none</verificationMode>
            <loadDefaultCAFile>true</loadDefaultCAFile>
            <cacheSessions>true</cacheSessions>
            <disableProtocols>sslv2,sslv3</disableProtocols>
            <preferServerCiphers>true</preferServerCiphers>
        </server>

        <client> 
            <loadDefaultCAFile>true</loadDefaultCAFile>
            <cacheSessions>true</cacheSessions>
            <disableProtocols>sslv2,sslv3</disableProtocols>
            <preferServerCiphers>true</preferServerCiphers>
            <invalidCertificateHandler>
                <name>RejectCertificateHandler</name>
            </invalidCertificateHandler>
        </client>
    </openSSL>
    <max_concurrent_queries>100</max_concurrent_queries>
    <max_server_memory_usage>0</max_server_memory_usage>
    <max_thread_pool_size>10000</max_thread_pool_size>
    <max_server_memory_usage_to_ram_ratio>0.9</max_server_memory_usage_to_ram_ratio>
    <total_memory_profiler_step>4194304</total_memory_profiler_step>

    <total_memory_tracker_sample_probability>0</total_memory_tracker_sample_probability>
    <uncompressed_cache_size>8589934592</uncompressed_cache_size>
    <mark_cache_size>5368709120</mark_cache_size>
    <mmap_cache_size>1000</mmap_cache_size>
    <compiled_expression_cache_size>134217728</compiled_expression_cache_size>
    <compiled_expression_cache_elements_size>10000</compiled_expression_cache_elements_size>
    <path>/var/lib/clickhouse/</path>
    <tmp_path>/var/lib/clickhouse/tmp/</tmp_path>
    <user_files_path>/var/lib/clickhouse/user_files/</user_files_path>
    <ldap_servers>
    </ldap_servers>
    <user_directories>
        <users_xml>
            <path>users.xml</path>
        </users_xml>
        <local_directory>
            <path>/var/lib/clickhouse/access/</path>
        </local_directory>
    </user_directories>
    <default_profile>default</default_profile>
    <custom_settings_prefixes></custom_settings_prefixes>
    <default_database>default</default_database>
    <mlock_executable>true</mlock_executable>
    <remap_executable>false</remap_executable>

    <![CDATA[
         Uncomment below in order to use JDBC table engine and function.

         To install and run JDBC bridge in background:
         * [Debian/Ubuntu]
           export MVN_URL=https://repo1.maven.org/maven2/ru/yandex/clickhouse/clickhouse-jdbc-bridge
           export PKG_VER=$(curl -sL $MVN_URL/maven-metadata.xml | grep '<release>' | sed -e 's|.*>(.*)<.*|1|')
           wget https://github.com/ClickHouse/clickhouse-jdbc-bridge/releases/download/v$PKG_VER/clickhouse-jdbc-bridge_$PKG_VER-1_all.deb
           apt install --no-install-recommends -f ./clickhouse-jdbc-bridge_$PKG_VER-1_all.deb
           clickhouse-jdbc-bridge &

         * [CentOS/RHEL]
           export MVN_URL=https://repo1.maven.org/maven2/ru/yandex/clickhouse/clickhouse-jdbc-bridge
           export PKG_VER=$(curl -sL $MVN_URL/maven-metadata.xml | grep '<release>' | sed -e 's|.*>(.*)<.*|1|')
           wget https://github.com/ClickHouse/clickhouse-jdbc-bridge/releases/download/v$PKG_VER/clickhouse-jdbc-bridge-$PKG_VER-1.noarch.rpm
           yum localinstall -y clickhouse-jdbc-bridge-$PKG_VER-1.noarch.rpm
           clickhouse-jdbc-bridge &

         Please refer to https://github.com/ClickHouse/clickhouse-jdbc-bridge#usage for more information.
    ]]>
    <remote_servers incl="clickhouse_remote_servers">
        <test_unavailable_shard>
            <shard>
                <replica>
                    <host>localhost</host>
                    <port>9000</port>
                </replica>
            </shard>
            <shard>
                <replica>
                    <host>localhost</host>
                    <port>1</port>
                </replica>
            </shard>
        </test_unavailable_shard>
    </remote_servers>

    <zookeeper incl="zookeeper-servers">
    </zookeeper>
    <builtin_dictionaries_reload_interval>3600</builtin_dictionaries_reload_interval>
    <max_session_timeout>3600</max_session_timeout>
    <default_session_timeout>60</default_session_timeout>
    <query_log>
        <database>system</database>
        <table>query_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
    </query_log>
    <trace_log>
        <database>system</database>
        <table>trace_log</table>

        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
    </trace_log>
    <query_thread_log>
        <database>system</database>
        <table>query_thread_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
    </query_thread_log>
    <query_views_log>
        <database>system</database>
        <table>query_views_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
    </query_views_log>
    <part_log>
        <database>system</database>
        <table>part_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
    </part_log>
    <metric_log>
        <database>system</database>
        <table>metric_log</table>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
        <collect_interval_milliseconds>1000</collect_interval_milliseconds>
    </metric_log>
    <asynchronous_metric_log>
        <database>system</database>
        <table>asynchronous_metric_log</table>
        <flush_interval_milliseconds>7000</flush_interval_milliseconds>
    </asynchronous_metric_log>
    <opentelemetry_span_log>
        <engine>
            engine MergeTree
            partition by toYYYYMM(finish_date)
            order by (finish_date, finish_time_us, trace_id)
        </engine>
        <database>system</database>
        <table>opentelemetry_span_log</table>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
    </opentelemetry_span_log>
    <crash_log>
        <database>system</database>
        <table>crash_log</table>

        <partition_by />
        <flush_interval_milliseconds>1000</flush_interval_milliseconds>
    </crash_log>

    <session_log>
        <database>system</database>
        <table>session_log</table>

        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
    </session_log>
    <top_level_domains_lists>
    </top_level_domains_lists>
    <dictionaries_config>*_dictionary.xml</dictionaries_config>
<user_defined_executable_functions_config>*_function.xml</user_defined_executable_functions_config>
    <encryption_codecs>
       
    </encryption_codecs>

    <distributed_ddl>
        <path>/clickhouse/task_queue/ddl</path>
    </distributed_ddl>
    <graphite_rollup_example>
        <pattern>
            <regexp>click_cost</regexp>
            <function>any</function>
            <retention>
                <age>0</age>
                <precision>3600</precision>
            </retention>
            <retention>
                <age>86400</age>
                <precision>60</precision>
            </retention>
        </pattern>
        <default>
            <function>max</function>
            <retention>
                <age>0</age>
                <precision>60</precision>
            </retention>
            <retention>
                <age>3600</age>
                <precision>300</precision>
            </retention>
            <retention>
                <age>86400</age>
                <precision>3600</precision>
            </retention>
        </default>
    </graphite_rollup_example>

    <format_schema_path>/var/lib/clickhouse/format_schemas/</format_schema_path>
    <query_masking_rules>
        <rule>
            <name>hide encrypt/decrypt arguments</name>
            <regexp>((?:aes_)?(?:encrypt|decrypt)(?:_mysql)?)s*(s*(?:'(?:\'|.)+'|.*?)s*)</regexp>
            <replace>1(???)</replace>
        </rule>
    </query_masking_rules>



    <send_crash_reports>
        <enabled>false</enabled>
        <anonymize>false</anonymize>
        <endpoint>https://6f33034cfe684dd7a3ab9875e57b1c8d@o388870.ingest.sentry.io/5226277</endpoint>
    </send_crash_reports>
    <include_from>/etc/metrika.xml</include_from>
</clickhouse>

metrika1.xml、metrika2.xml、metrika3.xml 见第二章ck1、ck2、ck3的配置文件。

猜你喜欢

深入理解JavaScript的事件冒泡与事件捕获
16位汇编通用寄存器
（19）目标检测算法之模型预测并保存结果到xml中，自动生成标签文件/自动标注
C++课程设计房产中介管理系统（代码全）
Golang每日一练(leetDay0004)
python读取bam文件&&OSError: no BGZF EOF marker； file may be truncated
贪心算法（几种常规样例）
树莓派4B-Python-控制DS18B20（温度传感器）
今天就要让你看懂 “最高效的单调队列“
python 基于tkinter商品销售系统
FPGA和IC设计怎么选？哪个发展更好？
python字符串，特定字符的删除问题
【C语言】3天速刷C语言（指针初识）
优化库——ceres（三）实战案例
Python数据结构-----栈1.0（栈的介绍与操作）
python-获取UUID
IP核简介及PLL_IP核的调用
C语言实现不使用第三个变量交换两个数的内容
MySQL数据库基础
Springboot扩展点之ApplicationContextInitializer

zl程序教程

当前栏目

ClickHouse集群搭建部署

文章目录

一、单机版安装和启停

二、集群部署搭建

1、手动搭建clickhouse集群

2、使用docker-compose快速搭建clickhouse集群

三、集群扩容

如何保证扩容后的数据均匀分布？

四、集群缩容

docker-compose 中的一些文件

相关文章