整理自 StarRocks 官方中文文档《自动化集群快照》,维护方:StarRocks 文档团队。本文固定核对 branch-3.5 文档;集群快照从 v3.4.2 起支持,在该版本文档中仍标记为 Beta,且只适用于存算分离集群。全文已于 2026-10-05 核对,下面的 SQL、Shell 和 YAML 只做静态审查,没有连接数据库或执行恢复。
要恢复一个存算分离集群,仅仅留住表数据还不够。CN 管理的数据已经放在对象存储中,但 FE 保存的目录、用户、权限以及导入任务等元数据仍在本地。自动化集群快照的作用,就是把恢复所需的元数据也保存到存储卷,使完整的集群状态有机会在故障后重新组装起来。


先明确快照能保存什么
集群快照记录某一时刻的集群状态,覆盖 Catalog、数据库、表、用户及权限、导入任务等对象。外部依赖并不自动包含在内,例如 external catalog 使用的配置文件和放在本地的 UDF JAR。因此,演练材料除了快照,还应有这些依赖的独立副本、适配当前版本的部署文件,以及能够访问存储卷的授权。
在这里,“自动化”有一个很具体的含义:系统持续维护紧随集群最新状态的一份快照。新快照生成后,历史快照会被删除,最终只保留一个。3.5 分支文档明确说明,快照任务只由系统触发,不支持手动创建。它不能替代能够回到多个历史时刻的备份链,也不能因为名称中有时间戳就被理解为任意时间点恢复。
灾难恢复的前提是数据和元数据本身仍然完整。云存储的备份和跨区域复制可以参与远程或跨区域灾备,但本文不会把“存储服务有这些功能”等同于“已经获得一致且可恢复的副本”。实际演练必须同时检查副本、访问权限与外部依赖。
启用自动化快照并理解保存位置
自动化集群快照默认关闭。官方给出的启用语法如下,方括号表示可选部分,尖括号表示需要替换的值,不能原样执行:
ADMIN SET AUTOMATED CLUSTER SNAPSHOT ON
[STORAGE VOLUME <storage_volume_name>]
storage_volume_name 指定快照使用的存储卷;不填写时使用默认存储卷。创建存储卷的方法见原站 CREATE STORAGE VOLUME 文档。恢复演练开始前,应确认该卷的地址和访问权限与准备保全的数据一致。
每次 FE 完成元数据检查点并生成新的元数据镜像后,系统会自动创建快照。名称格式是 automated_cluster_snapshot_{timestamp}。元数据快照保存在以下目录,而数据快照仍位于原始数据所在的位置:
/{storage_volume_locations}/{service_id}/meta/image/automated_cluster_snapshot_timestamp
FE 配置 automated_cluster_snapshot_interval_seconds 控制自动化周期,默认 600 秒,也就是 10 分钟。这个默认值描述触发周期,不能被写成灾难发生后必然只损失 10 分钟数据的保证;还需要观察任务是否完成,以及对象存储中的数据是否齐备。
分别检查快照和快照任务
快照清单和任务执行状态应分别查询。前者可看到最新快照,以及尚未清除的快照记录:
SELECT * FROM information_schema.cluster_snapshots;
| 字段 | 含义 |
|---|---|
| snapshot_name | 快照名称 |
| snapshot_type | 快照类型;本篇锁定的 3.5 文档仅支持 automated |
| created_time | 快照创建时间 |
| fe_journal_id | FE 日志 ID |
| starmgr_journal_id | StarManager 日志 ID |
| properties | 用于尚未可用的功能 |
| storage_volume | 保存快照的存储卷 |
| storage_path | 快照存储路径 |
任务视图用于判断自动化过程走到哪一步,以及失败时的具体信息:
SELECT * FROM information_schema.cluster_snapshot_jobs;
| 字段 | 含义 |
|---|---|
| snapshot_name | 对应快照名称 |
| job_id | 任务 ID |
| created_time | 任务创建时间 |
| finished_time | 任务结束时间 |
| state | INITIALIZING、SNAPSHOTING、FINISHED、EXPIRED、DELETED 或 ERROR |
| detail_info | 当前阶段的具体进度 |
| error_message | 错误信息(如果有) |
注意状态值 SNAPSHOTING 的拼写应按接口原样使用。看到任务创建并不等于快照可恢复;应保存对应任务的完成状态、存储路径与日志 ID,再以恢复演练验证使用它的结果。本文没有任何真实查询输出,表格列出的是接口字段。
关闭功能会清理历史快照
ADMIN SET AUTOMATED CLUSTER SNAPSHOT OFF
以上语句并非无损暂停按钮。原文明确说明,禁用自动化集群快照会自动清除历史快照。不要把它安排成“先停止写入快照,再研究恢复”的通用准备步骤。确有关闭需求时,先明确将失去哪些恢复材料,并按自己的备份方案保全所需副本;本次写稿没有执行此语句。
在独立环境恢复 Leader、其他 FE 与 CN
以下保留原文的恢复顺序,并补充演练保护说明。应先准备同版本的恢复节点、完整的快照和数据副本、存储权限及外部配置。恢复环境的地址、目录和存储路径必须经过人工核对,避免把示例中的目录清理动作误用到仍承担业务的原集群。
-
必要时调整存储卷与恢复配置。如果保存快照的存储位置已经改变,必须把原存储路径下的所有文件复制到新路径,并修改 Leader FE 的
fe/conf/cluster_snapshot.yaml。仅移动元数据镜像而遗漏原始数据文件,不满足原文的恢复前提。模板见下一节。 -
启动 Leader FE。使用集群快照模式:
./fe/bin/start_fe.sh --cluster_snapshot --daemon -
处理其他 FE。原文要求先清理其
meta目录,再使用 Leader 的地址和 edit-log 端口启动。本文不提供可直接粘贴的删除命令;先停止待恢复节点、保存原目录的可恢复副本,并再次确认它属于本次独立演练节点。确认完成后才进入原文的启动步骤:./fe/bin/start_fe.sh --helper <leader_ip>:<leader_edit_log_port> --daemon按照角色把节点加入集群:
-- 添加 Follower 节点: ALTER SYSTEM ADD FOLLOWER "<follower_host>:<follower_edit_log_port>"; -- 添加 Observer 节点: ALTER SYSTEM ADD OBSERVER "<observer_host>:<observer_edit_log_port>"; -
处理 CN。原文要求先清理 CN 的
storage_root_path目录,再启动 CN。这里同样必须确认目标是独立恢复节点,先保全原有文件,不能把路径清理套用到任意运行中的目录。./be/bin/start_cn.sh --daemon随后将 CN 加入集群:
ALTER SYSTEM ADD COMPUTE NODE "<cn_host>:<cn_heartbeat_service_port>";
如果第一步修改了 cluster_snapshot.yaml,新集群会按其中的信息重新配置节点和存储卷。完成启动后,应核对 Leader、Follower、Observer 与 CN 的实际地址及角色,确认它们属于计划中的恢复拓扑,并检查目录、权限、任务及关键业务查询。这里的核对是演练验收要求,不是本文已经完成的测试;节点成功启动也不能替代业务数据的完整性检查。
完整恢复配置模板及敏感字段
下列 YAML 保留 3.5 分支原文的结构、示例值和注释。IP、URI、访问键均是原文占位示例,不是可用凭证。它不是一份可以直接投入生产的配置:
# 要下载以进行恢复的集群快照的信息。
cluster_snapshot:
# 快照的 URI。
# 示例 1: s3://defaultbucket/test/f7265e80-631c-44d3-a8ac-cf7cdc7adec811019/meta/image/automated_cluster_snapshot_1704038400000
# 示例 2: s3://defaultbucket/test/f7265e80-631c-44d3-a8ac-cf7cdc7adec811019/meta
cluster_snapshot_path: <cluster_snapshot_uri>
# 存储快照的存储卷名称。您必须在 `storage_volumes` 部分中定义它。
# 注意:它必须与原始集群中的相同。
storage_volume_name: my_s3_volume
# [可选] 要恢复快照的新集群的节点信息。
# 如果未指定此部分,恢复后的新集群仅具有 Leader FE 节点。
# CN 节点保留原始集群的信息。
# 注意:不要在此部分中包含 Leader FE 节点。
frontends:
# FE 主机。
- host: xxx.xx.xx.x1
# FE edit_log_port。
edit_log_port: 9010
# FE 节点类型。有效值:`follower`(默认)和 `observer`。
type: follower
- host: xxx.xx.xx.x2
edit_log_port: 9010
type: observer
compute_nodes:
# CN 主机。
- host: xxx.xx.xx.x3
# CN heartbeat_service_port。
heartbeat_service_port: 9050
- host: xxx.xx.xx.x4
heartbeat_service_port: 9050
# 新集群中存储卷的信息。用于恢复克隆的快照。
# 注意:存储卷的名称必须与原始集群中的相同。
storage_volumes:
# S3 兼容存储卷的示例。
- name: my_s3_volume
type: S3
location: s3://defaultbucket/test/
comment: my s3 volume
properties:
- key: aws.s3.region
value: us-west-2
- key: aws.s3.endpoint
value: https://s3.us-west-2.amazonaws.com
- key: aws.s3.access_key
value: xxxxxxxxxx
- key: aws.s3.secret_key
value: yyyyyyyyyy
# HDFS 存储卷的示例。
- name: my_hdfs_volume
type: HDFS
location: hdfs://127.0.0.1:9000/sr/test/
comment: my hdfs volume
properties:
- key: hadoop.security.authentication
value: simple
- key: username
value: starrocks
cluster_snapshot_path 可指向具体快照目录,也可按原文例子指向 meta 路径;实际选择应与要恢复的材料对应。storage_volume_name 必须在 storage_volumes 中定义,而且存储卷名称必须与原集群一致。不要把 Leader FE 填进 frontends 列表。
节点信息段是可选的。原文提醒,未提供该部分时,新集群只有 Leader FE 的 FE 信息,CN 会保留原集群的信息。因此,演练中不能在省略节点段之后假定所有旧地址都已被替换。
静态审查发现的配置风险:S3 示例把 aws.s3.access_key 和 aws.s3.secret_key 放在 YAML 中。原文仅包含占位符,没有发现真实秘密;但若填入长期密钥,该文件、备份、工单和版本库都会成为泄露面。应限制配置文件访问权限,并根据环境按官方 AWS 认证说明选择合适的认证方式。不要把环境变量写成模板值后就假定 StarRocks 会自动插值;本篇没有核实这种行为。
HDFS 示例使用 hadoop.security.authentication: simple,这是示例配置,不提供强身份认证保障;不能把其中的 username: starrocks 当作安全认证机制。对于需要认证的 HDFS 环境,应使用经该环境和版本验证的认证配置,而不是直接复制这里的 simple 模板。
版本与编辑说明
本次已完整读取原始 3.5 网页,并与 StarRocks 官方仓库 branch-3.5 的中文文档源文件交叉核对。当前默认文档已指向较新版本,快照类型和部分恢复说明发生变化;本文没有把新版本中的 manual 类型写进 3.5 的接口表。
与原文相比,本稿重排了停用操作的位置,增加了独立恢复环境、目录保全、凭证与 HDFS 认证风险、节点验收及“未实测”的说明;技术语法与完整 YAML 示例保持可追溯。本文未执行 SQL、启动脚本、目录清理或数据复制,也未证明任何具体集群可以恢复。静态检查没有发现问题的部分,同样不等于不存在漏洞。
来源:StarRocks 3.5 集群快照文档。Copyright 2021-present StarRocks, Inc. All rights reserved. 官方仓库的 Apache License 2.0 文本与仓库 NOTICE 全文保留于本文末尾。本文为获授权的中文整理与安全批注版,原始内容归 StarRocks 及相关贡献者,原创示意图由未完纪绘制。
版权与许可全文
以下保留本页涉及的来源材料或示例代码的版权、许可条件与免责声明;各自适用范围依原声明。中文翻译及编辑标注:未完纪,2026-10-05。
LICENSE-StarRocks.txt
Copyright 2021-present StarRocks, Inc. All rights reserved.
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
NOTICE-StarRocks.txt
StarRocks Copyright 2021-present, StarRocks Inc. --------------------------- apache-doris-incubating NOTICE --------------------------- Apache Doris (incubating) Copyright 2018-2021 The Apache Software Foundation This product includes software developed at The Apache Software Foundation (http://www.apache.org/). Based on source code originally developed by Baidu (http://www.baidu.com/). --------------------------- apache-impala NOTICE --------------------------- Apache Impala Copyright 2019 The Apache Software Foundation This product includes software developed at The Apache Software Foundation (http://www.apache.org/). Portions of this software were developed at Cloudera, Inc (http://www.cloudera.com/). This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (http://www.openssl.org/) This product includes cryptographic software written by Eric Young (eay@cryptsoft.com). This product includes software written by Tim Hudson (tjh@cryptsoft.com). This product includes software developed by the University of Chicago, as Operator of Argonne National Laboratory. Copyright (C) 1999 University of Chicago. All rights reserved. --------------------------- apache-kudu NOTICE --------------------------- Apache Kudu Copyright 2016 The Apache Software Foundation This product includes software developed at The Apache Software Foundation (http://www.apache.org/). Portions of this software were developed at Cloudera, Inc (http://www.cloudera.com/). This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (http://www.openssl.org/) This product includes cryptographic software written by Eric Young (eay@cryptsoft.com). This product includes software written by Tim Hudson (tjh@cryptsoft.com).












暂无评论内容