diff --git a/docs/en_us/3.1-PipelineProtocol.md b/docs/en_us/3.1-PipelineProtocol.md index f0ca952120..8555562c4d 100644 --- a/docs/en_us/3.1-PipelineProtocol.md +++ b/docs/en_us/3.1-PipelineProtocol.md @@ -489,7 +489,7 @@ This algorithm property requires additional fields: - `order_by`: *string* How the results are sorted. Optional, default is [`Horizontal`](#horizontal). - Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Random`](#random). + Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Random`](#random). You can use it with the `index` field. - `index`: *int* @@ -512,6 +512,10 @@ This algorithm property requires additional fields: If set to true, you can paint the unwanted parts in the image green with RGB: (0, 255, 0), and those green parts won't be matched. Note: The algorithm itself has strong robustness, so this feature is usually unnecessary for normal background variations. If you do need to use it, only mask the interfering areas and avoid excessive masking that could cause loss of main subject edge features. +- `center`: *array* + Center coordinate [x, y] for Radiation sorting. Optional, defaults to the center of the bounding box of all results. + Only effective when `order_by` is `Radiation`. + ### `FeatureMatch` Feature matching, a more powerful "find image" with better generalization, resistant to perspective and size changes. @@ -537,7 +541,7 @@ This algorithm property requires additional fields: - `order_by`: *string* How the results are sorted. Optional, default is [`Horizontal`](#horizontal). - Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random). + Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random). You can use it with the `index` field. - `index`: *int* @@ -563,6 +567,10 @@ This algorithm property requires additional fields: - `ratio`: *double* The distance ratio for KNN matching, [0 - 1.0], where larger values make the matching more lenient (easier to connect). Optional, default is 0.6. +- `center`: *array* + Center coordinate [x, y] for Radiation sorting. Optional, defaults to the center of the bounding box of all results. + Only effective when `order_by` is `Radiation`. + ### `ColorMatch` Color matching, also known as "find color." @@ -591,7 +599,7 @@ This algorithm property requires additional fields: - `order_by`: *string* How the results are sorted. Optional, default is [`Horizontal`](#horizontal). - Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random). + Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random). You can use it with the `index` field. - `index`: *int* @@ -603,6 +611,10 @@ This algorithm property requires additional fields: If set to true, after applying color filtering, it will only count the maximum connected block of pixels. If set to false, it won't consider whether these pixels are connected. +- `center`: *array* + Center coordinate [x, y] for Radiation sorting. Optional, defaults to the center of the bounding box of all results. + Only effective when `order_by` is `Radiation`. + ### `OCR` Text recognition. @@ -626,7 +638,7 @@ This algorithm property requires additional fields: - `order_by`: *string* How the results are sorted. Optional, default is [`Horizontal`](#horizontal). - Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Area`](#area) | [`Length`](#length) | [`Random`](#random) | [`Expected`](#expected). + Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Area`](#area) | [`Length`](#length) | [`Random`](#random) | [`Expected`](#expected). You can use it with the `index` field. - `index`: *int* @@ -645,6 +657,10 @@ This algorithm property requires additional fields: Only the color binarization logic of the referenced node is used; its `count`, `order_by`, and other result parameters are not involved. OCR nodes with this field set will not participate in batch optimization. +- `center`: *array* + Center coordinate [x, y] for Radiation sorting. Optional, defaults to the center of the bounding box of all results. + Only effective when `order_by` is `Radiation`. + ### `NeuralNetworkClassify` Deep learning classification, to determine if the image in a **fixed position** matches the expected "category." @@ -670,13 +686,17 @@ This algorithm property requires additional fields: - `order_by`: *string* How the results are sorted. Optional, default is [`Horizontal`](#horizontal). - Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Random`](#random) | [`Expected`](#expected). + Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Random`](#random) | [`Expected`](#expected). You can use it with the `index` field. - `index`: *int* Index to hit. Optional, default is `0`. If there are N results in total, the value range of `index` is [-N, N - 1], where negative numbers are converted to N - index using Python-like rules. If it exceeds the range, it is considered that there is no result in the current identification. +- `center`: *array* + Center coordinate [x, y] for Radiation sorting. Optional, defaults to the center of the bounding box of all results. + Only effective when `order_by` is `Radiation`. + For example, if you want to recognize whether a cat or a mouse appears in a **fixed position** in the image, and you've trained a model that supports this three-category classification, and you want to click when it recognizes a cat or a mouse but not when it recognizes a dog, the relevant fields would be: ```jsonc @@ -720,13 +740,17 @@ This algorithm property requires additional fields: - `order_by`: *string* How the results are sorted. Optional, default is [`Horizontal`](#horizontal). - Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) | [`Expected`](#expected). + Possible values: [`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) | [`Expected`](#expected). You can use it with the `index` field. - `index`: *int* Index to hit. Optional, default is `0`. If there are N results in total, the value range of `index` is [-N, N - 1], where negative numbers are converted to N - index using Python-like rules. If it exceeds the range, it is considered that there is no result in the current identification. +- `center`: *array* + Center coordinate [x, y] for Radiation sorting. Optional, defaults to the center of the bounding box of all results. + Only effective when `order_by` is `Radiation`. + For example, if you want to detect cats, dogs, and mice in an image and only click when a cat or a mouse is detected but not when a dog is detected, the relevant fields would be: ```jsonc @@ -1566,6 +1590,12 @@ For `OCR`, sort by the order of regex patterns in the `expected` list; results m For `NeuralNetworkClassify` and `NeuralNetworkDetect`, sort by the order of class indices in the `expected` list. Results not matching any expected value are ranked last. +### `Radiation` + +Sort from center outward. Results are sorted by the Euclidean distance from each result's center point to a reference center, in ascending order. Closer results are ranked first. +By default, the center of the bounding box of all results is used as the reference point; if the `center` parameter (`[x, y]`) is specified, that coordinate is used instead. +Suitable for scenarios where center-positioned results are preferred. + ## Waiting for the Screen to Stabilize Waits for the screen to stabilize. It exits the action only when there is no significant change in the screen for a certain continuous time. diff --git "a/docs/zh_cn/3.1-\344\273\273\345\212\241\346\265\201\346\260\264\347\272\277\345\215\217\350\256\256.md" "b/docs/zh_cn/3.1-\344\273\273\345\212\241\346\265\201\346\260\264\347\272\277\345\215\217\350\256\256.md" index efc64fbc91..7461f99ef8 100644 --- "a/docs/zh_cn/3.1-\344\273\273\345\212\241\346\265\201\346\260\264\347\272\277\345\215\217\350\256\256.md" +++ "b/docs/zh_cn/3.1-\344\273\273\345\212\241\346\265\201\346\260\264\347\272\277\345\215\217\350\256\256.md" @@ -495,7 +495,7 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim - `order_by`: *string* 结果排序方式。可选,默认 [`Horizontal`](#horizontal)。 - 可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Random`](#random) 。 + 可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Random`](#random) 。 可结合 `index` 字段使用。 - `index`: *int* @@ -518,6 +518,10 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim 若为 true,可以将图片中不希望匹配的部分涂绿 RGB: (0, 255, 0),则不对绿色部分进行匹配。 注意:算法本身具有较强鲁棒性,常规背景变化通常无需使用此功能。若确需使用,应仅遮盖干扰区域,避免过度涂抹导致主体边缘特征丢失。 +- `center`: *array* + Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。 + 仅当 `order_by` 为 `Radiation` 时有效。 + ### `FeatureMatch` 特征匹配,泛化能力更强的“找图”,具有抗透视、抗尺寸变化等特点。 @@ -543,7 +547,7 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim - `order_by`: *string* 结果排序方式。可选,默认 [`Horizontal`](#horizontal) 。 - 可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) 。 + 可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) 。 可结合 `index` 字段使用。 - `index`: *int* @@ -569,6 +573,10 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim - `ratio`: *double* KNN 匹配算法的距离比值,[0 - 1.0] , 越大则匹配越宽松(更容易连线)。可选,默认 0.6 。 +- `center`: *array* + Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。 + 仅当 `order_by` 为 `Radiation` 时有效。 + ### `ColorMatch` 颜色匹配,即“找色”。 @@ -597,7 +605,7 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim - `order_by`: *string* 结果排序方式。可选,默认 [`Horizontal`](#horizontal) 。 - 可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) 。 + 可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) 。 可结合 `index` 字段使用。 - `index`: *int* @@ -609,6 +617,10 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim 若为是,在完成颜色过滤后,则只会计数像素点 **全部相连** 的最大块。 若为否,则不考虑这些像素点是否相连。 +- `center`: *array* + Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。 + 仅当 `order_by` 为 `Radiation` 时有效。 + ### `OCR` 文字识别。 @@ -632,7 +644,7 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim - `order_by`: *string* 结果排序方式。可选,默认 [`Horizontal`](#horizontal)。 - 可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Area`](#area) | [`Length`](#length) | [`Random`](#random) | [`Expected`](#expected) 。 + 可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Area`](#area) | [`Length`](#length) | [`Random`](#random) | [`Expected`](#expected) 。 可结合 `index` 字段使用。 - `index`: *int* @@ -652,6 +664,10 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim 仅使用该节点的颜色二值化逻辑,不依赖其 `count`、`order_by` 等结果参数。 设置了该字段的 OCR 节点不参与 batch 优化。 +- `center`: *array* + Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。 + 仅当 `order_by` 为 `Radiation` 时有效。 + ### `NeuralNetworkClassify` 深度学习分类,判断图像中的 **固定位置** 是否为预期的“类别”。 @@ -677,13 +693,17 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim - `order_by`: *string* 结果排序方式。可选,默认 [`Horizontal`](#horizontal) 。 - 可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Random`](#random) | [`Expected`](#expected) 。 + 可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Random`](#random) | [`Expected`](#expected) 。 可结合 `index` 字段使用。 - `index`: *int* 命中第几个结果。可选,默认 0 。 假设共有 N 个结果,则 `index` 的取值范围为 [-N, N - 1] ,其中负数使用类 Python 的规则转换为 N - index 。若超出范围,则视为当前识别无结果。 +- `center`: *array* + Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。 + 仅当 `order_by` 为 `Radiation` 时有效。 + 举例:例如画面中 **固定位置** 可能出现 猫、狗、老鼠,我们训练了支持该三分类的模型。 希望识别到 猫 或 老鼠 才点击,而识别到 狗 不点击,则相关字段为: @@ -728,13 +748,17 @@ MaaResourcePostPath(resource, "resource/debug"); // debug 节点使用 rate_lim - `order_by`: *string* 结果排序方式。可选,默认 [`Horizontal`](#horizontal) 。 - 可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) | [`Expected`](#expected) 。 + 可选的值:[`Horizontal`](#horizontal) | [`Vertical`](#vertical) | [`Radiation`](#radiation) | [`Score`](#score) | [`Area`](#area) | [`Random`](#random) | [`Expected`](#expected) 。 可结合 `index` 字段使用。 - `index`: *int* 命中第几个结果。可选,默认 0 。 假设共有 N 个结果,则 `index` 的取值范围为 [-N, N - 1] ,其中负数使用类 Python 的规则转换为 N - index 。若超出范围,则视为当前识别无结果。 +- `center`: *array* + Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。 + 仅当 `order_by` 为 `Radiation` 时有效。 + 举例:例如画面中可能出现 猫、狗、老鼠,我们训练了支持该三分类的检测模型。 希望检测到 猫 或 老鼠 才点击,而识别到 狗 不点击,则相关字段为: @@ -1574,6 +1598,12 @@ graph LR 对于 `NeuralNetworkClassify` 和 `NeuralNetworkDetect`,按 `expected` 分类下标列表的顺序排序。 未匹配到任何 expected 的结果将被排在最后。 +### `Radiation` + +从中心向外排序。按识别结果中心点到参考中心的欧氏距离升序排列,距离越近的结果排在越前面。 +默认以所有结果包围框的中心为参考点;若指定了 `center` 参数(`[x, y]`),则以该坐标为中心。 +适用于需要优先选择靠近中心位置结果的场景。 + ## 等待画面静止 等待画面静止。需连续一定时间 画面 **没有较大变化** 才会退出动作。 diff --git a/sample/resource/default_pipeline.json b/sample/resource/default_pipeline.json index ba50daebba..6dc89e635c 100644 --- a/sample/resource/default_pipeline.json +++ b/sample/resource/default_pipeline.json @@ -32,7 +32,7 @@ "OCR": { "recognition": "OCR", "threshold": 0.3, // 置信度阈值 - "order_by": "Horizontal", // 排序方式:Horizontal | Vertical | Area | Length + "order_by": "Horizontal", // 排序方式:Horizontal | Vertical | Radiation | Area | Length "replace": [] // 文字替换规则列表,格式:[["待替换字符串", "替换为"]] }, // ===== 动作类型默认配置 ===== diff --git a/source/MaaFramework/Resource/PipelineDumper.cpp b/source/MaaFramework/Resource/PipelineDumper.cpp index 2840ae13a7..6384dd904b 100644 --- a/source/MaaFramework/Resource/PipelineDumper.cpp +++ b/source/MaaFramework/Resource/PipelineDumper.cpp @@ -60,9 +60,9 @@ std::string dump_order_by(MAA_VISION_NS::ResultOrderBy order_by) { static const std::unordered_map order_by_map = { { MAA_VISION_NS::ResultOrderBy::Horizontal, "Horizontal" }, { MAA_VISION_NS::ResultOrderBy::Vertical, "Vertical" }, - { MAA_VISION_NS::ResultOrderBy::Score, "Score" }, { MAA_VISION_NS::ResultOrderBy::Area, "Area" }, - { MAA_VISION_NS::ResultOrderBy::Length, "Length" }, { MAA_VISION_NS::ResultOrderBy::Random, "Random" }, - { MAA_VISION_NS::ResultOrderBy::Expected, "Expected" }, + { MAA_VISION_NS::ResultOrderBy::Radiation, "Radiation" }, { MAA_VISION_NS::ResultOrderBy::Score, "Score" }, + { MAA_VISION_NS::ResultOrderBy::Area, "Area" }, { MAA_VISION_NS::ResultOrderBy::Length, "Length" }, + { MAA_VISION_NS::ResultOrderBy::Random, "Random" }, { MAA_VISION_NS::ResultOrderBy::Expected, "Expected" }, }; return order_by_map.at(order_by); } @@ -107,6 +107,7 @@ PipelineV2::JRecognition PipelineDumper::dump_reco(Recognition::Type type, const .index = p.result_index, .method = p.method, .green_mask = p.green_mask, + .center = p.center, }; } break; @@ -131,6 +132,7 @@ PipelineV2::JRecognition PipelineDumper::dump_reco(Recognition::Type type, const .green_mask = p.green_mask, .detector = kDetectorNameMap.at(p.detector), .ratio = p.ratio, + .center = p.center, }; } break; @@ -152,6 +154,7 @@ PipelineV2::JRecognition PipelineDumper::dump_reco(Recognition::Type type, const .order_by = dump_order_by(p.order_by), .index = p.result_index, .connected = p.connected, + .center = p.center, }; } break; @@ -177,6 +180,7 @@ PipelineV2::JRecognition PipelineDumper::dump_reco(Recognition::Type type, const .only_rec = p.only_rec, .model = p.model, .color_filter = p.color_filter, + .center = p.center, }; } break; @@ -190,6 +194,7 @@ PipelineV2::JRecognition PipelineDumper::dump_reco(Recognition::Type type, const .expected = p.expected, .order_by = dump_order_by(p.order_by), .index = p.result_index, + .center = p.center, }; } break; @@ -204,6 +209,7 @@ PipelineV2::JRecognition PipelineDumper::dump_reco(Recognition::Type type, const .threshold = p.thresholds, .order_by = dump_order_by(p.order_by), .index = p.result_index, + .center = p.center, }; } break; diff --git a/source/MaaFramework/Resource/PipelineParser.cpp b/source/MaaFramework/Resource/PipelineParser.cpp index 69ca547356..82f9276bc9 100644 --- a/source/MaaFramework/Resource/PipelineParser.cpp +++ b/source/MaaFramework/Resource/PipelineParser.cpp @@ -501,6 +501,7 @@ bool PipelineParser::parse_template_matcher_param( { MAA_VISION_NS::ResultOrderBy::Horizontal, MAA_VISION_NS::ResultOrderBy::Vertical, + MAA_VISION_NS::ResultOrderBy::Radiation, MAA_VISION_NS::ResultOrderBy::Score, MAA_VISION_NS::ResultOrderBy::Random, })) { @@ -508,6 +509,8 @@ bool PipelineParser::parse_template_matcher_param( return false; } + get_and_check_value(input, "center", output.center, default_value.center); + if (!get_and_check_value_or_array(input, "template", output.template_, default_value.template_)) { LogError << "failed to get_and_check_value_or_array templates" << VAR(input); return false; @@ -552,6 +555,7 @@ bool PipelineParser::parse_feature_matcher_param( { MAA_VISION_NS::ResultOrderBy::Horizontal, MAA_VISION_NS::ResultOrderBy::Vertical, + MAA_VISION_NS::ResultOrderBy::Radiation, MAA_VISION_NS::ResultOrderBy::Score, MAA_VISION_NS::ResultOrderBy::Area, MAA_VISION_NS::ResultOrderBy::Random, @@ -560,6 +564,8 @@ bool PipelineParser::parse_feature_matcher_param( return false; } + get_and_check_value(input, "center", output.center, default_value.center); + if (!get_and_check_value_or_array(input, "template", output.template_, default_value.template_)) { LogError << "failed to get_and_check_value_or_array templates" << VAR(input); return false; @@ -637,6 +643,7 @@ bool PipelineParser::parse_ocrer_param( { MAA_VISION_NS::ResultOrderBy::Horizontal, MAA_VISION_NS::ResultOrderBy::Vertical, + MAA_VISION_NS::ResultOrderBy::Radiation, MAA_VISION_NS::ResultOrderBy::Area, MAA_VISION_NS::ResultOrderBy::Length, MAA_VISION_NS::ResultOrderBy::Random, @@ -646,6 +653,8 @@ bool PipelineParser::parse_ocrer_param( return false; } + get_and_check_value(input, "center", output.center, default_value.center); + if (!get_and_check_value(input, "model", output.model, default_value.model)) { LogError << "failed to get_and_check_value model" << VAR(input); return false; @@ -757,6 +766,7 @@ bool PipelineParser::parse_nn_classifier_param( { MAA_VISION_NS::ResultOrderBy::Horizontal, MAA_VISION_NS::ResultOrderBy::Vertical, + MAA_VISION_NS::ResultOrderBy::Radiation, MAA_VISION_NS::ResultOrderBy::Score, MAA_VISION_NS::ResultOrderBy::Random, MAA_VISION_NS::ResultOrderBy::Expected, @@ -764,6 +774,9 @@ bool PipelineParser::parse_nn_classifier_param( LogError << "failed to parse_order_of_result" << VAR(input); return false; } + + get_and_check_value(input, "center", output.center, default_value.center); + if (!get_and_check_value(input, "model", output.model, default_value.model)) { LogError << "failed to get_and_check_value model" << VAR(input); return false; @@ -799,6 +812,7 @@ bool PipelineParser::parse_nn_detector_param( { MAA_VISION_NS::ResultOrderBy::Horizontal, MAA_VISION_NS::ResultOrderBy::Vertical, + MAA_VISION_NS::ResultOrderBy::Radiation, MAA_VISION_NS::ResultOrderBy::Score, MAA_VISION_NS::ResultOrderBy::Area, MAA_VISION_NS::ResultOrderBy::Random, @@ -808,6 +822,8 @@ bool PipelineParser::parse_nn_detector_param( return false; } + get_and_check_value(input, "center", output.center, default_value.center); + if (!get_and_check_value(input, "model", output.model, default_value.model)) { LogError << "failed to get_and_check_value model" << VAR(input); return false; @@ -863,6 +879,7 @@ bool PipelineParser::parse_color_matcher_param( { MAA_VISION_NS::ResultOrderBy::Horizontal, MAA_VISION_NS::ResultOrderBy::Vertical, + MAA_VISION_NS::ResultOrderBy::Radiation, MAA_VISION_NS::ResultOrderBy::Score, MAA_VISION_NS::ResultOrderBy::Area, MAA_VISION_NS::ResultOrderBy::Random, @@ -871,6 +888,8 @@ bool PipelineParser::parse_color_matcher_param( return false; } + get_and_check_value(input, "center", output.center, default_value.center); + std::vector> default_lower; std::vector> default_upper; @@ -976,6 +995,8 @@ bool PipelineParser::parse_order_of_result( { "horizontal", MAA_VISION_NS::ResultOrderBy::Horizontal }, { "Vertical", MAA_VISION_NS::ResultOrderBy::Vertical }, { "vertical", MAA_VISION_NS::ResultOrderBy::Vertical }, + { "Radiation", MAA_VISION_NS::ResultOrderBy::Radiation }, + { "radiation", MAA_VISION_NS::ResultOrderBy::Radiation }, { "Score", MAA_VISION_NS::ResultOrderBy::Score }, { "score", MAA_VISION_NS::ResultOrderBy::Score }, { "Area", MAA_VISION_NS::ResultOrderBy::Area }, diff --git a/source/MaaFramework/Resource/PipelineTypesV2.h b/source/MaaFramework/Resource/PipelineTypesV2.h index a9ca81b0db..1f74cb52f9 100644 --- a/source/MaaFramework/Resource/PipelineTypesV2.h +++ b/source/MaaFramework/Resource/PipelineTypesV2.h @@ -2,6 +2,7 @@ #include #include +#include #include #include @@ -34,8 +35,9 @@ struct JTemplateMatch int index = 0; int method = 0; bool green_mask = false; + std::optional center; - MEO_TOJSON(roi, roi_offset, MEO_KEY("template") template_, threshold, order_by, index, method, green_mask); + MEO_TOJSON(roi, roi_offset, MEO_KEY("template") template_, threshold, order_by, index, method, green_mask, center); }; struct JFeatureMatch @@ -49,8 +51,9 @@ struct JFeatureMatch bool green_mask = false; std::string detector; double ratio = 0; + std::optional center; - MEO_TOJSON(roi, roi_offset, MEO_KEY("template") template_, count, order_by, index, green_mask, detector, ratio); + MEO_TOJSON(roi, roi_offset, MEO_KEY("template") template_, count, order_by, index, green_mask, detector, ratio, center); }; struct JColorMatch @@ -64,8 +67,9 @@ struct JColorMatch std::string order_by; int index = 0; bool connected = false; + std::optional center; - MEO_TOJSON(roi, roi_offset, method, lower, upper, count, order_by, index, connected); + MEO_TOJSON(roi, roi_offset, method, lower, upper, count, order_by, index, connected, center); }; struct JOCR @@ -80,8 +84,9 @@ struct JOCR bool only_rec = false; std::string model; std::string color_filter; + std::optional center; - MEO_TOJSON(roi, roi_offset, expected, threshold, replace, order_by, index, only_rec, model, color_filter); + MEO_TOJSON(roi, roi_offset, expected, threshold, replace, order_by, index, only_rec, model, color_filter, center); }; struct JNeuralNetworkClassify @@ -93,8 +98,9 @@ struct JNeuralNetworkClassify std::vector expected; std::string order_by; int index = 0; + std::optional center; - MEO_TOJSON(roi, roi_offset, labels, model, expected, order_by, index); + MEO_TOJSON(roi, roi_offset, labels, model, expected, order_by, index, center); }; struct JNeuralNetworkDetect @@ -107,8 +113,9 @@ struct JNeuralNetworkDetect std::vector threshold; std::string order_by; int index = 0; + std::optional center; - MEO_TOJSON(roi, roi_offset, labels, model, expected, threshold, order_by, index); + MEO_TOJSON(roi, roi_offset, labels, model, expected, threshold, order_by, index, center); }; struct JCustomRecognition diff --git a/source/MaaFramework/Vision/ColorMatcher.cpp b/source/MaaFramework/Vision/ColorMatcher.cpp index ecdb2c4530..37b95171ea 100644 --- a/source/MaaFramework/Vision/ColorMatcher.cpp +++ b/source/MaaFramework/Vision/ColorMatcher.cpp @@ -146,6 +146,9 @@ void ColorMatcher::sort_(ResultsVec& results) const case ResultOrderBy::Vertical: sort_by_vertical_(results); break; + case ResultOrderBy::Radiation: + sort_by_radiation_(results, param_.center); + break; case ResultOrderBy::Score: sort_by_count_(results); break; diff --git a/source/MaaFramework/Vision/FeatureMatcher.cpp b/source/MaaFramework/Vision/FeatureMatcher.cpp index d30f677d5e..2c01cdecae 100644 --- a/source/MaaFramework/Vision/FeatureMatcher.cpp +++ b/source/MaaFramework/Vision/FeatureMatcher.cpp @@ -282,6 +282,9 @@ void FeatureMatcher::sort_(ResultsVec& results) const case ResultOrderBy::Vertical: sort_by_vertical_(results); break; + case ResultOrderBy::Radiation: + sort_by_radiation_(results, param_.center); + break; case ResultOrderBy::Score: sort_by_count_(results); break; diff --git a/source/MaaFramework/Vision/NeuralNetworkClassifier.cpp b/source/MaaFramework/Vision/NeuralNetworkClassifier.cpp index b25fd6a087..2cafe75838 100644 --- a/source/MaaFramework/Vision/NeuralNetworkClassifier.cpp +++ b/source/MaaFramework/Vision/NeuralNetworkClassifier.cpp @@ -146,6 +146,9 @@ void NeuralNetworkClassifier::sort_(ResultsVec& results) const case ResultOrderBy::Vertical: sort_by_vertical_(results); break; + case ResultOrderBy::Radiation: + sort_by_radiation_(results, param_.center); + break; case ResultOrderBy::Score: sort_by_score_(results); break; diff --git a/source/MaaFramework/Vision/NeuralNetworkDetector.cpp b/source/MaaFramework/Vision/NeuralNetworkDetector.cpp index ebd28518b7..897836df53 100644 --- a/source/MaaFramework/Vision/NeuralNetworkDetector.cpp +++ b/source/MaaFramework/Vision/NeuralNetworkDetector.cpp @@ -232,6 +232,9 @@ void NeuralNetworkDetector::sort_(ResultsVec& results) const case ResultOrderBy::Vertical: sort_by_vertical_(results); break; + case ResultOrderBy::Radiation: + sort_by_radiation_(results, param_.center); + break; case ResultOrderBy::Score: sort_by_score_(results); break; diff --git a/source/MaaFramework/Vision/OCRer.cpp b/source/MaaFramework/Vision/OCRer.cpp index c8e2fc1ee8..c2c7dac8b4 100644 --- a/source/MaaFramework/Vision/OCRer.cpp +++ b/source/MaaFramework/Vision/OCRer.cpp @@ -389,6 +389,9 @@ void OCRer::sort_(ResultsVec& results) const case ResultOrderBy::Vertical: sort_by_vertical_(results); break; + case ResultOrderBy::Radiation: + sort_by_radiation_(results, param_.center); + break; // case ResultOrderBy::Score: // sort_by_score_(results); // break; diff --git a/source/MaaFramework/Vision/TemplateMatcher.cpp b/source/MaaFramework/Vision/TemplateMatcher.cpp index 3ac850f0fa..f50b31c995 100644 --- a/source/MaaFramework/Vision/TemplateMatcher.cpp +++ b/source/MaaFramework/Vision/TemplateMatcher.cpp @@ -164,6 +164,9 @@ void TemplateMatcher::sort_(ResultsVec& results) const case ResultOrderBy::Vertical: sort_by_vertical_(results); break; + case ResultOrderBy::Radiation: + sort_by_radiation_(results, param_.center); + break; case ResultOrderBy::Score: sort_by_score_(results, low_score_better_); break; diff --git a/source/MaaFramework/Vision/VisionTypes.h b/source/MaaFramework/Vision/VisionTypes.h index b5ea0168a3..c36e325a06 100644 --- a/source/MaaFramework/Vision/VisionTypes.h +++ b/source/MaaFramework/Vision/VisionTypes.h @@ -1,6 +1,7 @@ #pragma once #include +#include #include #include #include @@ -43,6 +44,7 @@ enum class ResultOrderBy { Horizontal, Vertical, + Radiation, Score, Area, Length, // for OCR @@ -72,6 +74,7 @@ struct TemplateMatcherParam : public RoiTargetParamBase ResultOrderBy order_by = ResultOrderBy::Horizontal; int result_index = 0; + std::optional center; }; struct OCRerParam : public RoiTargetParamBase @@ -87,6 +90,7 @@ struct OCRerParam : public RoiTargetParamBase ResultOrderBy order_by = ResultOrderBy::Horizontal; int result_index = 0; + std::optional center; }; struct TemplateComparatorParam : public RoiTargetParamBase @@ -109,6 +113,7 @@ struct NeuralNetworkClassifierParam : public RoiTargetParamBase ResultOrderBy order_by = ResultOrderBy::Horizontal; int result_index = 0; + std::optional center; }; struct NeuralNetworkDetectorParam : public RoiTargetParamBase @@ -128,6 +133,7 @@ struct NeuralNetworkDetectorParam : public RoiTargetParamBase ResultOrderBy order_by = ResultOrderBy::Horizontal; int result_index = 0; + std::optional center; }; struct ColorMatcherParam : public RoiTargetParamBase @@ -143,6 +149,7 @@ struct ColorMatcherParam : public RoiTargetParamBase ResultOrderBy order_by = ResultOrderBy::Horizontal; int result_index = 0; + std::optional center; }; struct FeatureMatcherParam : public RoiTargetParamBase @@ -179,6 +186,7 @@ struct FeatureMatcherParam : public RoiTargetParamBase ResultOrderBy order_by = ResultOrderBy::Horizontal; int result_index = 0; + std::optional center; }; struct RectComparator diff --git a/source/MaaFramework/Vision/VisionUtils.hpp b/source/MaaFramework/Vision/VisionUtils.hpp index 0a96cea56a..ef6fec13b8 100644 --- a/source/MaaFramework/Vision/VisionUtils.hpp +++ b/source/MaaFramework/Vision/VisionUtils.hpp @@ -33,6 +33,31 @@ inline static void sort_by_vertical_(ResultsVec& results) }); } +// From center outward (inspired by FindText.ahk dir==9) +// Sorts results by Euclidean distance from a given center point. +// If center is not specified, computes from the bounding box of all results. +template +inline static void sort_by_radiation_(ResultsVec& results, std::optional center) +{ + if (!center.has_value()) { + if (results.empty()) { + return; + } + cv::Rect union_rect = results.front().box; + for (const auto& res : results) { + union_rect |= res.box; + } + center = cv::Point(union_rect.x + union_rect.width / 2, union_rect.y + union_rect.height / 2); + } + std::ranges::sort(results, [¢er](const auto& lhs, const auto& rhs) -> bool { + double ldx = lhs.box.x + lhs.box.width / 2.0 - center->x; + double ldy = lhs.box.y + lhs.box.height / 2.0 - center->y; + double rdx = rhs.box.x + rhs.box.width / 2.0 - center->x; + double rdy = rhs.box.y + rhs.box.height / 2.0 - center->y; + return (ldx * ldx + ldy * ldy) < (rdx * rdx + rdy * rdy); + }); +} + template inline static void sort_by_score_(ResultsVec& results, bool reverse = false) { diff --git a/source/binding/NodeJS/src/apis/pipeline.d.ts b/source/binding/NodeJS/src/apis/pipeline.d.ts index 30efee19dc..f759d7ac43 100644 --- a/source/binding/NodeJS/src/apis/pipeline.d.ts +++ b/source/binding/NodeJS/src/apis/pipeline.d.ts @@ -19,14 +19,15 @@ declare global { } type OrderByMap = { - TemplateMatch: 'Horizontal' | 'Vertical' | 'Score' | 'Random' - FeatureMatch: 'Horizontal' | 'Vertical' | 'Score' | 'Area' | 'Random' - ColorMatch: 'Horizontal' | 'Vertical' | 'Score' | 'Area' | 'Random' - OCR: 'Horizontal' | 'Vertical' | 'Area' | 'Length' | 'Random' | 'Expected' - NeuralNetworkClassify: 'Horizontal' | 'Vertical' | 'Score' | 'Random' | 'Expected' + TemplateMatch: 'Horizontal' | 'Vertical' | 'Radiation' | 'Score' | 'Random' + FeatureMatch: 'Horizontal' | 'Vertical' | 'Radiation' | 'Score' | 'Area' | 'Random' + ColorMatch: 'Horizontal' | 'Vertical' | 'Radiation' | 'Score' | 'Area' | 'Random' + OCR: 'Horizontal' | 'Vertical' | 'Radiation' | 'Area' | 'Length' | 'Random' | 'Expected' + NeuralNetworkClassify: 'Horizontal' | 'Vertical' | 'Radiation' | 'Score' | 'Random' | 'Expected' NeuralNetworkDetect: | 'Horizontal' | 'Vertical' + | 'Radiation' | 'Score' | 'Area' | 'Random' @@ -48,6 +49,7 @@ declare global { index?: number method?: 10001 | 3 | 5 green_mask?: boolean + center?: Point }, 'template', Mode @@ -64,6 +66,7 @@ declare global { green_mask?: boolean detector?: 'SIFT' | 'KAZE' | 'AKAZE' | 'BRISK' | 'ORB' ratio?: number + center?: Point }, 'template', Mode @@ -90,6 +93,7 @@ declare global { order_by?: OrderByMap['ColorMatch'] index?: number connected?: boolean + center?: Point } type RecognitionOCR = { @@ -103,6 +107,7 @@ declare global { only_rec?: boolean model?: string color_filter?: string + center?: Point } type RecognitionNeuralNetworkClassify = RequiredIfStrict< @@ -114,6 +119,7 @@ declare global { expected?: MaybeArray order_by?: OrderByMap['NeuralNetworkClassify'] index?: number + center?: Point }, 'model', Mode @@ -129,6 +135,7 @@ declare global { threshold?: number order_by?: OrderByMap['NeuralNetworkDetect'] index?: number + center?: Point }, 'model', Mode diff --git a/source/binding/Python/maa/pipeline.py b/source/binding/Python/maa/pipeline.py index 3d49181aa0..498a15db37 100644 --- a/source/binding/Python/maa/pipeline.py +++ b/source/binding/Python/maa/pipeline.py @@ -63,6 +63,7 @@ class JTemplateMatch: index: int = 0 method: int = 5 green_mask: bool = False + center: Optional[Tuple[int, int]] = None @dataclass @@ -76,6 +77,7 @@ class JFeatureMatch: index: int = 0 green_mask: bool = False ratio: float = 0.6 + center: Optional[Tuple[int, int]] = None @dataclass @@ -89,6 +91,7 @@ class JColorMatch: count: int = 1 index: int = 0 connected: bool = False + center: Optional[Tuple[int, int]] = None @dataclass @@ -103,6 +106,7 @@ class JOCR: only_rec: bool = False model: str = "" color_filter: str = "" + center: Optional[Tuple[int, int]] = None @dataclass @@ -114,6 +118,7 @@ class JNeuralNetworkClassify: labels: List[str] = field(default_factory=list) order_by: str = "Horizontal" index: int = 0 + center: Optional[Tuple[int, int]] = None @dataclass @@ -126,6 +131,7 @@ class JNeuralNetworkDetect: threshold: List[float] = field(default_factory=lambda: [0.3]) order_by: str = "Horizontal" index: int = 0 + center: Optional[Tuple[int, int]] = None @dataclass diff --git a/tools/pipeline.schema.json b/tools/pipeline.schema.json index dfac2223f1..34076eb1e3 100644 --- a/tools/pipeline.schema.json +++ b/tools/pipeline.schema.json @@ -654,12 +654,19 @@ "enum": [ "Horizontal", "Vertical", + "Radiation", "Score", "Random" ], - "markdownDescription": "*string*\n\n结果排序方式。可选,默认 `Horizontal`。\n\n可选的值:`Horizontal` | `Vertical` | `Score` | `Random` 。\n\n可结合 `index` 字段使用。", + "markdownDescription": "*string*\n\n结果排序方式。可选,默认 `Horizontal`。\n\n可选的值:`Horizontal` | `Vertical` | `Radiation` | `Score` | `Random` 。\n\n可结合 `index` 字段使用。", "default": "Horizontal" }, + "center": { + "title": "Center Property", + "description": "Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。仅当 `order_by` 为 `Radiation` 时有效。", + "$ref": "#/$defs/jsonPoint", + "markdownDescription": "*array*\n\nRadiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。仅当 `order_by` 为 `Radiation` 时有效。" + }, "index": { "title": "Index Property", "description": "命中第几个结果。可选,默认 0 。", @@ -775,13 +782,20 @@ "enum": [ "Horizontal", "Vertical", + "Radiation", "Score", "Area", "Random" ], - "markdownDescription": "*string*\n\n结果排序方式。可选,默认 `Horizontal` 。\n\n可选的值:`Horizontal` | `Vertical` | `Score` | `Area` | `Random` 。\n\n可结合 `index` 字段使用。", + "markdownDescription": "*string*\n\n结果排序方式。可选,默认 `Horizontal` 。\n\n可选的值:`Horizontal` | `Vertical` | `Radiation` | `Score` | `Area` | `Random` 。\n\n可结合 `index` 字段使用。", "default": "Horizontal" }, + "center": { + "title": "Center Property", + "description": "Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。仅当 `order_by` 为 `Radiation` 时有效。", + "$ref": "#/$defs/jsonPoint", + "markdownDescription": "*array*\n\nRadiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。仅当 `order_by` 为 `Radiation` 时有效。" + }, "index": { "title": "Index Property", "description": "命中第几个结果。可选,默认 0 。", @@ -918,13 +932,20 @@ "enum": [ "Horizontal", "Vertical", + "Radiation", "Score", "Area", "Random" ], - "markdownDescription": "*string*\n\n结果排序方式。可选,默认 `Horizontal` 。\n\n可选的值:`Horizontal` | `Vertical` | `Score` | `Area` | `Random` 。\n\n可结合 `index` 字段使用。", + "markdownDescription": "*string*\n\n结果排序方式。可选,默认 `Horizontal` 。\n\n可选的值:`Horizontal` | `Vertical` | `Radiation` | `Score` | `Area` | `Random` 。\n\n可结合 `index` 字段使用。", "default": "Horizontal" }, + "center": { + "title": "Center Property", + "description": "Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。仅当 `order_by` 为 `Radiation` 时有效。", + "$ref": "#/$defs/jsonPoint", + "markdownDescription": "*array*\n\nRadiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。仅当 `order_by` 为 `Radiation` 时有效。" + }, "index": { "title": "Index Property", "description": "命中第几个结果。可选,默认 0 。", @@ -1034,14 +1055,21 @@ "enum": [ "Horizontal", "Vertical", + "Radiation", "Area", "Length", "Random", "Expected" ], - "markdownDescription": "*string*\n\n结果排序方式。可选,默认 `Horizontal`。\n\n可选的值:`Horizontal` | `Vertical` | `Area` | `Length` | `Random` | `Expected` 。\n\n可结合 `index` 字段使用。", + "markdownDescription": "*string*\n\n结果排序方式。可选,默认 `Horizontal`。\n\n可选的值:`Horizontal` | `Vertical` | `Radiation` | `Area` | `Length` | `Random` | `Expected` 。\n\n可结合 `index` 字段使用。", "default": "Horizontal" }, + "center": { + "title": "Center Property", + "description": "Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。仅当 `order_by` 为 `Radiation` 时有效。", + "$ref": "#/$defs/jsonPoint", + "markdownDescription": "*array*\n\nRadiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。仅当 `order_by` 为 `Radiation` 时有效。" + }, "index": { "title": "Index Property", "description": "命中第几个结果。可选,默认 0 。", @@ -1136,13 +1164,20 @@ "enum": [ "Horizontal", "Vertical", + "Radiation", "Score", "Random", "Expected" ], - "markdownDescription": "*string*\n\n结果排序方式。可选,默认 `Horizontal` 。\n\n可选的值:`Horizontal` | `Vertical` | `Score` | `Random` | `Expected` 。\n\n可结合 `index` 字段使用。", + "markdownDescription": "*string*\n\n结果排序方式。可选,默认 `Horizontal` 。\n\n可选的值:`Horizontal` | `Vertical` | `Radiation` | `Score` | `Random` | `Expected` 。\n\n可结合 `index` 字段使用。", "default": "Horizontal" }, + "center": { + "title": "Center Property", + "description": "Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。仅当 `order_by` 为 `Radiation` 时有效。", + "$ref": "#/$defs/jsonPoint", + "markdownDescription": "*array*\n\nRadiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。仅当 `order_by` 为 `Radiation` 时有效。" + }, "index": { "title": "Index Property", "description": "命中第几个结果。可选,默认 0 。", @@ -1238,14 +1273,21 @@ "enum": [ "Horizontal", "Vertical", + "Radiation", "Score", "Area", "Random", "Expected" ], - "markdownDescription": "*string*\n\n结果排序方式。可选,默认 `Horizontal` 。\n\n可选的值:`Horizontal` | `Vertical` | `Score` | `Area` | `Random` | `Expected` 。\n\n可结合 `index` 字段使用。", + "markdownDescription": "*string*\n\n结果排序方式。可选,默认 `Horizontal` 。\n\n可选的值:`Horizontal` | `Vertical` | `Radiation` | `Score` | `Area` | `Random` | `Expected` 。\n\n可结合 `index` 字段使用。", "default": "Horizontal" }, + "center": { + "title": "Center Property", + "description": "Radiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。仅当 `order_by` 为 `Radiation` 时有效。", + "$ref": "#/$defs/jsonPoint", + "markdownDescription": "*array*\n\nRadiation 排序的中心坐标 [x, y]。可选,默认自动使用所有结果包围框的中心。仅当 `order_by` 为 `Radiation` 时有效。" + }, "index": { "title": "Index Property", "description": "命中第几个结果。可选,默认 0 。", @@ -1638,6 +1680,28 @@ } ] }, + "center": { + "anyOf": [ + { + "$ref": "#/$defs/TemplateMatch/properties/center" + }, + { + "$ref": "#/$defs/FeatureMatch/properties/center" + }, + { + "$ref": "#/$defs/ColorMatch/properties/center" + }, + { + "$ref": "#/$defs/Ocr/properties/center" + }, + { + "$ref": "#/$defs/NeuralNetworkClassify/properties/center" + }, + { + "$ref": "#/$defs/NeuralNetworkDetect/properties/center" + } + ] + }, "method": { "anyOf": [ {