在 Druid 中,让一个分段不再参与查询,与把它从存储中永久删除,是两个不同的动作。首先通过 Coordinator 将分段标为 unused;随后提交 kill任务,才会删除对应元数据与深存储文件。只做第一步,磁盘或对象存储中的文件仍然存在。
本文依据 Apache Druid 官方《Tutorial: Deleting data》完整技术流程译编,并核对随教程的 deletion-index.json与 deletion-kill.json。核验日期为 2026 年 10 月 5 日。源地址中的 latest与代码仓库的 master都会变化,复现时应使用与本机发行版匹配的教程文件。本文只做静态审阅,没有启动 Druid、导入数据、标记分段或提交删除任务。

准备一个独立的教程数据源
原教程要求一个运行中的 Druid 实例和命令行 JSON 工具 jq。下面保留 Linux/Bash 命令形式,目标地址来自本机 quickstart:Coordinator/Overlord 使用 localhost:8081,Web 控制台通常为 http://localhost:8888。已有集群的服务位置、认证与 TLS 配置不一定相同,不应直接替换成生产地址运行。
初始数据是 2015 年 9 月 12 日的 Wikipedia 编辑样例。发行包中的 quickstart/tutorial/deletion-index.json创建名为 deletion-tutorial的数据源。已核对的配置采用 index_parallel,时间字段为 time,格式为 iso;关键粒度部分如下:
{
"type": "uniform",
"segmentGranularity": "hour",
"queryGranularity": "none",
"intervals": ["2015-09-12/2015-09-13"],
"rollup": false
}
因此示例按小时生成分段,不做 rollup。输入来自本地目录 quickstart/tutorial/下的 wikiticker-2015-09-12-sampled.json.gz,输入格式是 JSON;配置的 appendToExisting为假,不能把它当作随意向已有同名数据源追加的命令。教程应使用没有业务数据的独立环境。
下列请求是原教程的导入步骤,实际执行会向 Druid 提交写入任务。本文未执行:
curl -X POST http://localhost:8081/druid/indexer/v1/task \
-H "Content-Type: application/json" \
-d @quickstart/tutorial/deletion-index.json
确认导入任务完成后,再在控制台查看数据源与分段。原文指向 unified-console.md#datasources的本地链接是文档路径瑕疵;本稿改为说明从本机控制台入口进入相应视图,不把这个旧路径继续当成可用页面。
第一步:按时间范围标为不用
原教程先处理 UTC 时间 18:00 到 20:00 的分段,范围是 2015-09-12T18:00:00.000Z/2015-09-12T20:00:00.000Z,对应 18 点和 19 点两个小时。发送的是 markUnused请求:
# 会改变分段可用状态;只用于已核对的隔离教程数据
curl -X POST \
-H "Content-Type: application/json" \
-d '{"interval":"2015-09-12T18:00:00.000Z/2015-09-12T20:00:00.000Z"}' \
http://localhost:8081/druid/coordinator/v1/datasources/deletion-tutorial/markUnused
原教程说明,这两个小时的分段随后不再显示于其控制台的已使用分段视图。但此时只是状态发生变化。原文在本地深存储目录 var/druid/segments/deletion-tutorial/的列表中,仍能看到:
2015-09-12T18:00:00.000Z_2015-09-12T19:00:00.000Z
2015-09-12T19:00:00.000Z_2015-09-12T20:00:00.000Z
这些是原文给出的目录示例,不是本次机器上的检查结果。对象存储部署也不会拥有完全相同的本地目录;应到实际深存储后端核对相应对象。
也可以用精确分段 ID 选择目标
第二种方法是提交 segmentIds。在控制台的 Segments 视图打开分段元数据,identifier字段就是完整 ID。原文 23 点分段的示例为:
deletion-tutorial_2015-09-12T23:00:00.000Z_2015-09-13T00:00:00.000Z_2023-05-16T00:04:12.091Z
ID 包含版本等信息,重新导入后可能不同,不能把文档中的这个旧 ID 当成自己的分段 ID。原教程通过 API 拉取 ID 数组,再用 jq '.[-2:]'取最后两项,意图选中 22 点与 23 点:
unusedSegmentIds=$(curl -X GET \
-H 'Content-Type:application/json' \
http://localhost:8081/druid/coordinator/v1/datasources/deletion-tutorial/segments \
| jq '.[-2:]')
静态审阅发现:“数组最后两项”并不表达“最后两个小时”的业务条件。真实集群可能有多个分区、多个版本或不同返回顺序,HTTP 请求失败也可能产生非预期内容。不要仅凭数组位置自动执行删除相关操作。
以下编辑修订把查询、筛选与写入分开。它只针对原教程每小时一个目标分段的前提,按明确时间前缀筛选,并要求两个小时分别恰好匹配一个 ID;不符合前提时让 jq失败。这个片段只读取 API 并生成本地待审阅 JSON,没有发出状态修改请求,也未在本次执行。
curl --fail-with-body --silent --show-error \
http://localhost:8081/druid/coordinator/v1/datasources/deletion-tutorial/segments \
-o segments.json
jq -e '
if type != "array" or any(.[]; type != "string") then
error("Expected an array of segment IDs")
else
[.[] | select(startswith(
"deletion-tutorial_2015-09-12T22:00:00.000Z_2015-09-12T23:00:00.000Z_"
))] as $hour22
| [.[] | select(startswith(
"deletion-tutorial_2015-09-12T23:00:00.000Z_2015-09-13T00:00:00.000Z_"
))] as $hour23
| if ($hour22 | length) == 1 and ($hour23 | length) == 1 then
{segmentIds: ($hour22 + $hour23)}
else
error("Expected exactly one segment for each tutorial hour; inspect manually")
end
end
' segments.json > mark-unused-ids.json
与原文的差别:增加 HTTP 失败处理、JSON 类型检查、按时间条件筛选和每小时数量断言,并把结果保存在文件里供核对。它不能代替人工确认数据源、完整 ID 与备份,也不适用于不满足单分段前提的生产数据。--fail-with-body要求支持该参数的 curl 版本;旧版可采用 --fail并另行保留错误响应。若第一条命令失败,应停止,不得继续使用旧的 segments.json。
核对生成文件中的两个完整 ID 后,才是实际改变状态的请求:
# 此请求会把文件列出的分段标为 unused
curl --fail-with-body -X POST \
-H "Content-Type: application/json" \
--data-binary @mark-unused-ids.json \
http://localhost:8081/druid/coordinator/v1/datasources/deletion-tutorial/markUnused
原教程的预期结果是 22 点与 23 点分段也从已使用分段视图中移出,但它们的深存储目录仍在。到这里一共选择了四个小时,尚未永久删除文件。
第二步:用 kill 任务永久移除
永久删除的危险边界在这里。kill 任务会删除落在任务范围中的 unused 分段的元数据及深存储文件,不能靠把状态改回 used 来撤销。执行前必须核对备份、精确目标和将受影响的所有 unused 分段。本次没有提交该任务。
原教程的 quickstart/tutorial/deletion-kill.json内容是:
{
"type": "kill",
"dataSource": "deletion-tutorial",
"interval": "2015-09-12/2015-09-13"
}
这里覆盖一整天,并没有列出上面那四个分段 ID。因此,如果当天还有其他已标为 unused 的分段,它们也可能落入任务范围。不要把这份 JSON 误读为“只删刚才四个 ID”。
下面是原教程提交 kill 任务的命令,只作为原流程记录。它是不可逆删除操作,不是查询命令:
# 危险:永久删除指定日期内符合条件的 unused 分段
curl -X POST \
-H "Content-Type: application/json" \
-d @quickstart/tutorial/deletion-kill.json \
http://localhost:8081/druid/indexer/v1/task
编辑建议:若真实意图只覆盖教程选定的两段时间,应先检查安装版本的 kill 任务规范,再分别采用 18:00/20:00和 22:00/次日00:00这样的更窄范围,核对范围内所有 unused 分段。窄化时间也不是按 ID 锁定:同一时窗中其他符合条件的版本或分区仍需审阅。本文不把这一建议包装成已经验证的替代任务。
用两层证据确认结果
按原教程,kill 任务完成后,本地深存储列表不再包含 18、19、22、23 点四个小时目录,保留下来的时段是 00 点至 17 点,以及 20、21 点。原文展示的过程可整理为:
| 阶段 | 四个目标小时的使用状态 | 深存储文件 |
|---|---|---|
| 初始导入完成 | used | 存在 |
| markUnused 完成 | unused | 仍存在 |
| kill 任务成功完成 | 对应元数据被删除 | 对应文件被删除 |
这些是官方教程陈述的预期现象,并非本次实测。实际核验应首先确认任务状态与服务端错误,再核对分段元数据及深存储对象。只看控制台中的活动分段减少,无法证明物理文件已经清除;只看一次请求返回,也不能证明异步任务已成功结束。
本地示例用只读目录列表 ls -l1 var/druid/segments/deletion-tutorial/观察文件变化。对于其他深存储后端,应使用对应的只读查询方式,并区分分段清理与对象存储自己的版本、回收或保留策略。本文没有核查任何实际后端。
静态审查范围
已审阅源文请求、导入配置和 kill 配置。没有发现这些片段中的真实硬编码凭据;原文 HTTP 地址、无认证请求仅适用于其本机教程环境,不能据此推断生产服务可免认证或明文管理。也没有执行任何破坏性示例。
本文明确修正了旧控制台路径说明,指出数组末尾取两项的不可靠选择方式、导入配置的非追加语义,以及 kill 的整日范围。编辑修订仍只经过静态阅读,没有经过 Bash、jq 或 Druid 运行验证;未发现更多问题不等于不存在漏洞。
原文:Tutorial: Deleting data。维护及发布方:Apache Software Foundation,原页无可确认的个人作者。Copyright © 2026 Apache Software Foundation。除另有说明外,原文采用 CC BY-SA 4.0;本中文译编、编辑补充与自绘图同样以 CC BY-SA 4.0 提供,修改日期为 2026-10-05。软件及仓库示例另见 Druid 仓库许可证。Apache、Druid 及相关标识的商标权归 Apache Software Foundation;本文不表示其对译编内容背书。
原教程完整导入配置
以下为原仓库 deletion-index.json;与前文小时粒度摘录对应,供完整核对。它会写入教程数据源,不是只读查询。修改与执行须在可丢弃环境审查。
{
"type" : "index_parallel",
"spec" : {
"dataSchema" : {
"dataSource" : "deletion-tutorial",
"timestampSpec": {
"column": "time",
"format": "iso"
},
"dimensionsSpec" : {
"dimensions" : [
"channel",
"cityName",
"comment",
"countryIsoCode",
"countryName",
"isAnonymous",
"isMinor",
"isNew",
"isRobot",
"isUnpatrolled",
"metroCode",
"namespace",
"page",
"regionIsoCode",
"regionName",
"user",
{ "name": "added", "type": "long" },
{ "name": "deleted", "type": "long" },
{ "name": "delta", "type": "long" }
]
},
"metricsSpec" : [],
"granularitySpec" : {
"type" : "uniform",
"segmentGranularity" : "hour",
"queryGranularity" : "none",
"intervals" : ["2015-09-12/2015-09-13"],
"rollup" : false
}
},
"ioConfig" : {
"type" : "index_parallel",
"inputSource" : {
"type" : "local",
"baseDir" : "quickstart/tutorial/",
"filter" : "wikiticker-2015-09-12-sampled.json.gz"
},
"inputFormat" : {
"type" : "json"
},
"appendToExisting" : false
},
"tuningConfig" : {
"type" : "index_parallel",
"maxRowsPerSegment" : 5000000,
"maxRowsInMemory" : 25000
}
}
}
仓库配置版权与许可
以下许可仅说明原仓库配置代码的许可;正文与配图保留前述 CC BY-SA 4.0 声明。引用配置的缩进及相关讲解由未完纪于2026-10-05整理;源配置字段未改变。
Apache Druid
Copyright 2018-2026 The Apache Software Foundation
This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
原文图示
















暂无评论内容