Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 36 additions & 6 deletions docs/en_us/3.1-PipelineProtocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -489,7 +489,7 @@ This algorithm property requires additional fields:

- `order_by`: *string*
How the results are sorted. Optional, default is [`Horizontal`](#horizontal).
Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Random`](#random).
Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Random`](#random).
You can use it with the `index` field.

- `index`: *int*
Expand All @@ -512,6 +512,10 @@ This algorithm property requires additional fields:
If set to true, you can paint the unwanted parts in the image green with RGB: (0, 255, 0), and those green parts won't be matched.
Note: The algorithm itself has strong robustness, so this feature is usually unnecessary for normal background variations. If you do need to use it, only mask the interfering areas and avoid excessive masking that could cause loss of main subject edge features.

- `center`: *array<int, 2>*
Center coordinate [x, y] for Radiation sorting. Optional, defaults to the center of the bounding box of all results.
Only effective when `order_by` is `Radiation`.

### `FeatureMatch`

Feature matching, a more powerful "find image" with better generalization, resistant to perspective and size changes.
Expand All @@ -537,7 +541,7 @@ This algorithm property requires additional fields:

- `order_by`: *string*
How the results are sorted. Optional, default is [`Horizontal`](#horizontal).
Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random).
Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random).
You can use it with the `index` field.

- `index`: *int*
Expand All @@ -563,6 +567,10 @@ This algorithm property requires additional fields:
- `ratio`: *double*
The distance ratio for KNN matching, [0 - 1.0], where larger values make the matching more lenient (easier to connect). Optional, default is 0.6.

- `center`: *array<int, 2>*
Center coordinate [x, y] for Radiation sorting. Optional, defaults to the center of the bounding box of all results.
Only effective when `order_by` is `Radiation`.

### `ColorMatch`

Color matching, also known as "find color."
Expand Down Expand Up @@ -591,7 +599,7 @@ This algorithm property requires additional fields:

- `order_by`: *string*
How the results are sorted. Optional, default is [`Horizontal`](#horizontal).
Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random).
Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random).
You can use it with the `index` field.

- `index`: *int*
Expand All @@ -603,6 +611,10 @@ This algorithm property requires additional fields:
If set to true, after applying color filtering, it will only count the maximum connected block of pixels.
If set to false, it won't consider whether these pixels are connected.

- `center`: *array<int, 2>*
Center coordinate [x, y] for Radiation sorting. Optional, defaults to the center of the bounding box of all results.
Only effective when `order_by` is `Radiation`.

### `OCR`

Text recognition.
Expand All @@ -626,7 +638,7 @@ This algorithm property requires additional fields:

- `order_by`: *string*
How the results are sorted. Optional, default is [`Horizontal`](#horizontal).
Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Area`](#area) | [`Length`](#length) | [`Random`](#random) | [`Expected`](#expected).
Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Area`](#area) | [`Length`](#length) | [`Random`](#random) | [`Expected`](#expected).
You can use it with the `index` field.

- `index`: *int*
Expand All @@ -645,6 +657,10 @@ This algorithm property requires additional fields:
Only the color binarization logic of the referenced node is used; its `count`, `order_by`, and other result parameters are not involved.
OCR nodes with this field set will not participate in batch optimization.

- `center`: *array<int, 2>*
Center coordinate [x, y] for Radiation sorting. Optional, defaults to the center of the bounding box of all results.
Only effective when `order_by` is `Radiation`.

### `NeuralNetworkClassify`

Deep learning classification, to determine if the image in a **fixed position** matches the expected "category."
Expand All @@ -670,13 +686,17 @@ This algorithm property requires additional fields:

- `order_by`: *string*
How the results are sorted. Optional, default is [`Horizontal`](#horizontal).
Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Random`](#random) | [`Expected`](#expected).
Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Random`](#random) | [`Expected`](#expected).
You can use it with the `index` field.

- `index`: *int*
Index to hit. Optional, default is `0`.
If there are N results in total, the value range of `index` is [-N, N - 1], where negative numbers are converted to N - index using Python-like rules. If it exceeds the range, it is considered that there is no result in the current identification.

- `center`: *array<int, 2>*
Center coordinate [x, y] for Radiation sorting. Optional, defaults to the center of the bounding box of all results.
Only effective when `order_by` is `Radiation`.

For example, if you want to recognize whether a cat or a mouse appears in a **fixed position** in the image, and you've trained a model that supports this three-category classification, and you want to click when it recognizes a cat or a mouse but not when it recognizes a dog, the relevant fields would be:

```jsonc
Expand Down Expand Up @@ -720,13 +740,17 @@ This algorithm property requires additional fields:

- `order_by`: *string*
How the results are sorted. Optional, default is [`Horizontal`](#horizontal).
Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) | [`Expected`](#expected).
Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) | [`Expected`](#expected).
You can use it with the `index` field.

- `index`: *int*
Index to hit. Optional, default is `0`.
If there are N results in total, the value range of `index` is [-N, N - 1], where negative numbers are converted to N - index using Python-like rules. If it exceeds the range, it is considered that there is no result in the current identification.

- `center`: *array<int, 2>*
Center coordinate [x, y] for Radiation sorting. Optional, defaults to the center of the bounding box of all results.
Only effective when `order_by` is `Radiation`.

For example, if you want to detect cats, dogs, and mice in an image and only click when a cat or a mouse is detected but not when a dog is detected, the relevant fields would be:

```jsonc
Expand Down Expand Up @@ -1566,6 +1590,12 @@ For `OCR`, sort by the order of regex patterns in the `expected` list; results m
For `NeuralNetworkClassify` and `NeuralNetworkDetect`, sort by the order of class indices in the `expected` list.
Results not matching any expected value are ranked last.

### `Radiation`

Sort from center outward. Results are sorted by the Euclidean distance from each result's center point to a reference center, in ascending order. Closer results are ranked first.
By default, the center of the bounding box of all results is used as the reference point; if the `center` parameter (`[x, y]`) is specified, that coordinate is used instead.
Suitable for scenarios where center-positioned results are preferred.

## Waiting for the Screen to Stabilize

Waits for the screen to stabilize. It exits the action only when there is no significant change in the screen for a certain continuous time.
Expand Down
42 changes: 36 additions & 6 deletions docs/zh_cn/3.1-任务流水线协议.md
Original file line number Diff line number Diff line change
Expand Up @@ -495,7 +495,7 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim

- `order_by`: *string*
结果排序方式。可选,默认 [`Horizontal`](#horizontal)。
可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Random`](#random) 。
可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Random`](#random) 。
可结合 `index` 字段使用。

- `index`: *int*
Expand All @@ -518,6 +518,10 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim
若为 true,可以将图片中不希望匹配的部分涂绿 RGB: (0, 255, 0),则不对绿色部分进行匹配。
注意:算法本身具有较强鲁棒性,常规背景变化通常无需使用此功能。若确需使用,应仅遮盖干扰区域,避免过度涂抹导致主体边缘特征丢失。

- `center`: *array<int, 2>*
Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。
仅当 `order_by` 为 `Radiation` 时有效。

### `FeatureMatch`

特征匹配,泛化能力更强的“找图”,具有抗透视、抗尺寸变化等特点。
Expand All @@ -543,7 +547,7 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim

- `order_by`: *string*
结果排序方式。可选,默认 [`Horizontal`](#horizontal) 。
可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) 。
可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) 。
可结合 `index` 字段使用。

- `index`: *int*
Expand All @@ -569,6 +573,10 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim
- `ratio`: *double*
KNN 匹配算法的距离比值,[0 - 1.0] , 越大则匹配越宽松(更容易连线)。可选,默认 0.6 。

- `center`: *array<int, 2>*
Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。
仅当 `order_by` 为 `Radiation` 时有效。

### `ColorMatch`

颜色匹配,即“找色”。
Expand Down Expand Up @@ -597,7 +605,7 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim

- `order_by`: *string*
结果排序方式。可选,默认 [`Horizontal`](#horizontal) 。
可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) 。
可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) 。
可结合 `index` 字段使用。

- `index`: *int*
Expand All @@ -609,6 +617,10 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim
若为是,在完成颜色过滤后,则只会计数像素点 **全部相连** 的最大块。
若为否,则不考虑这些像素点是否相连。

- `center`: *array<int, 2>*
Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。
仅当 `order_by` 为 `Radiation` 时有效。

### `OCR`

文字识别。
Expand All @@ -632,7 +644,7 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim

- `order_by`: *string*
结果排序方式。可选,默认 [`Horizontal`](#horizontal)。
可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Area`](#area) | [`Length`](#length) | [`Random`](#random) | [`Expected`](#expected) 。
可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Area`](#area) | [`Length`](#length) | [`Random`](#random) | [`Expected`](#expected) 。
可结合 `index` 字段使用。

- `index`: *int*
Expand All @@ -652,6 +664,10 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim
仅使用该节点的颜色二值化逻辑,不依赖其 `count`、`order_by` 等结果参数。
设置了该字段的 OCR 节点不参与 batch 优化。

- `center`: *array<int, 2>*
Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。
仅当 `order_by` 为 `Radiation` 时有效。

### `NeuralNetworkClassify`

深度学习分类,判断图像中的 **固定位置** 是否为预期的“类别”。
Expand All @@ -677,13 +693,17 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim

- `order_by`: *string*
结果排序方式。可选,默认 [`Horizontal`](#horizontal) 。
可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Random`](#random) | [`Expected`](#expected) 。
可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Random`](#random) | [`Expected`](#expected) 。
可结合 `index` 字段使用。

- `index`: *int*
命中第几个结果。可选,默认 0 。
假设共有 N 个结果,则 `index` 的取值范围为 [-N, N - 1] ,其中负数使用类 Python 的规则转换为 N - index 。若超出范围,则视为当前识别无结果。

- `center`: *array<int, 2>*
Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。
仅当 `order_by` 为 `Radiation` 时有效。

举例:例如画面中 **固定位置** 可能出现 猫、狗、老鼠,我们训练了支持该三分类的模型。
希望识别到 猫 或 老鼠 才点击,而识别到 狗 不点击,则相关字段为:

Expand Down Expand Up @@ -728,13 +748,17 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim

- `order_by`: *string*
结果排序方式。可选,默认 [`Horizontal`](#horizontal) 。
可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) | [`Expected`](#expected) 。
可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) | [`Expected`](#expected) 。
可结合 `index` 字段使用。

- `index`: *int*
命中第几个结果。可选,默认 0 。
假设共有 N 个结果,则 `index` 的取值范围为 [-N, N - 1] ,其中负数使用类 Python 的规则转换为 N - index 。若超出范围,则视为当前识别无结果。

- `center`: *array<int, 2>*
Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。
仅当 `order_by` 为 `Radiation` 时有效。

举例:例如画面中可能出现 猫、狗、老鼠,我们训练了支持该三分类的检测模型。
希望检测到 猫 或 老鼠 才点击,而识别到 狗 不点击,则相关字段为:

Expand Down Expand Up @@ -1574,6 +1598,12 @@ graph LR
对于 `NeuralNetworkClassify` 和 `NeuralNetworkDetect`,按 `expected` 分类下标列表的顺序排序。
未匹配到任何 expected 的结果将被排在最后。

### `Radiation`

从中心向外排序。按识别结果中心点到参考中心的欧氏距离升序排列,距离越近的结果排在越前面。
默认以所有结果包围框的中心为参考点;若指定了 `center` 参数(`[x, y]`),则以该坐标为中心。
适用于需要优先选择靠近中心位置结果的场景。

## 等待画面静止

等待画面静止。需连续一定时间 画面 **没有较大变化** 才会退出动作。
Expand Down
2 changes: 1 addition & 1 deletion sample/resource/default_pipeline.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
"OCR": {
"recognition": "OCR",
"threshold": 0.3, // 置信度阈值
"order_by": "Horizontal", // 排序方式:Horizontal | Vertical | Area | Length
"order_by": "Horizontal", // 排序方式:Horizontal | Vertical | Radiation | Area | Length
"replace": [] // 文字替换规则列表,格式:[["待替换字符串", "替换为"]]
},
// ===== 动作类型默认配置 =====
Expand Down
12 changes: 9 additions & 3 deletions source/MaaFramework/Resource/PipelineDumper.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,9 @@ std::string dump_order_by(MAA_VISION_NS::ResultOrderBy order_by)
{
static const std::unordered_map<MAA_VISION_NS::ResultOrderBy, std::string> order_by_map = {
{ MAA_VISION_NS::ResultOrderBy::Horizontal, "Horizontal" }, { MAA_VISION_NS::ResultOrderBy::Vertical, "Vertical" },
{ MAA_VISION_NS::ResultOrderBy::Score, "Score" }, { MAA_VISION_NS::ResultOrderBy::Area, "Area" },
{ MAA_VISION_NS::ResultOrderBy::Length, "Length" }, { MAA_VISION_NS::ResultOrderBy::Random, "Random" },
{ MAA_VISION_NS::ResultOrderBy::Expected, "Expected" },
{ MAA_VISION_NS::ResultOrderBy::Radiation, "Radiation" }, { MAA_VISION_NS::ResultOrderBy::Score, "Score" },
{ MAA_VISION_NS::ResultOrderBy::Area, "Area" }, { MAA_VISION_NS::ResultOrderBy::Length, "Length" },
{ MAA_VISION_NS::ResultOrderBy::Random, "Random" }, { MAA_VISION_NS::ResultOrderBy::Expected, "Expected" },
};
return order_by_map.at(order_by);
}
Expand Down Expand Up @@ -107,6 +107,7 @@ PipelineV2::JRecognition PipelineDumper::dump_reco(Recognition::Type type, const
.index = p.result_index,
.method = p.method,
.green_mask = p.green_mask,
.center = p.center,
};
} break;

Expand All @@ -131,6 +132,7 @@ PipelineV2::JRecognition PipelineDumper::dump_reco(Recognition::Type type, const
.green_mask = p.green_mask,
.detector = kDetectorNameMap.at(p.detector),
.ratio = p.ratio,
.center = p.center,
};
} break;

Expand All @@ -152,6 +154,7 @@ PipelineV2::JRecognition PipelineDumper::dump_reco(Recognition::Type type, const
.order_by = dump_order_by(p.order_by),
.index = p.result_index,
.connected = p.connected,
.center = p.center,
};
} break;

Expand All @@ -177,6 +180,7 @@ PipelineV2::JRecognition PipelineDumper::dump_reco(Recognition::Type type, const
.only_rec = p.only_rec,
.model = p.model,
.color_filter = p.color_filter,
.center = p.center,
};
} break;

Expand All @@ -190,6 +194,7 @@ PipelineV2::JRecognition PipelineDumper::dump_reco(Recognition::Type type, const
.expected = p.expected,
.order_by = dump_order_by(p.order_by),
.index = p.result_index,
.center = p.center,
};
} break;

Expand All @@ -204,6 +209,7 @@ PipelineV2::JRecognition PipelineDumper::dump_reco(Recognition::Type type, const
.threshold = p.thresholds,
.order_by = dump_order_by(p.order_by),
.index = p.result_index,
.center = p.center,
};
} break;

Expand Down
Loading
Loading