[ET-VK][custom_ops] Standardize test case label format across binaries

Previously the test case labels printed by each custom_ops test binary used wildly inconsistent formats. Examples from a recent on-device pass:

    ACCU  conv1d_dw [1,16,64] K=3 s=1 p=1 d=1  Tex(HP) f32
    ACCU  I=144  Buf(4W)
    correctness_1_64_32_Buffer_Float
    small_1d_linear_weight
    ACCU  In=1,16,4,4  r=2  same_qp  [4W4C->4W4C]
    ACCU  32->3  I=64,64  g=1  k=3  Buf(4C1W) [general]

Standardize all 21 binaries on:

    [ACCU/PERF]  [INPUT->OUTPUT dtype]  [shape data + inline op params]  [Storage(layout)]  [optional suffix]

Concrete examples after this commit:

    ACCU  f32->f32  [1,16,64] k3 s1 p1 d1                Tex(HP)
    ACCU  i8->i8    [144]+[144]                          Buf(4W)
    ACCU  f32->f32  [1,64]x[32,64]                       Buf(WP)
    ACCU  i32->f32  [1,8] lw                             Tex(WP)
    ACCU  i8->i8    [1,16,4,4] r=2                       Buf(4W4C)->Buf(4W4C)  [same_qp]
    ACCU  f32->f32  [1,3,64,64]x[32,3,3,3] s1 p1 d1 g1   Buf(4C1W)  [general]

Adds three helpers to test/custom_ops/utils.h:

- dtype_short(vkapi::ScalarType) -> "f32" / "f16" / "i8" / "i32"
- shape_bracket(std::vector<int64_t>) -> "[N,C,H,W]" form
- make_test_label(prefix, in_dtype, out_dtype, shape_str, storage_str, suffix) assembles the four sections with two-space separators and an optional bracketed suffix.

Each binary's TestCase-builder helper now invokes make_test_label rather than rolling its own ostringstream. Op-specific shape detail (kernel/stride/padding/dilation/groups for conv, ratio for pixel_shuffle, output_padding for transposed conv, layout-transition arrows for clone/qdq) is constructed inline since it varies per op. Information that was load-bearing in the old labels (impl_selector tags, +bias/no_bias flags, same_qp/diff_qp markers, const_b for binary ops, multi-stage Buf->Buf transitions) is preserved as a bracketed suffix.

Two minor format deviations worth flagging:

- choose_qparams_per_row produces two outputs (scale tensor + zero_point tensor). Encoded as out_dtype = "f32,i8" -- a comma-separated tuple after the arrow -- rather than forcing a single token.
- test_embedding_q4gsw loses the free-form prose labels (small_1d_linear_weight, llama_3_2_1b_prefill_*). Replaced with structural shape info + an lw / scales-dtype suffix; readers can still derive the same context but it is less greppable as a "story".

Differential Revision: [D105059941](https://our.internmc.facebook.com/intern/diff/D105059941/)

ghstack-source-id: 381655474
Pull Request resolved: #19570

Author: ssjia

backends/vulkan/test/custom_ops/add.cpp

@@ -45,20 +45,12 @@ std::vector<TestCase> generate_add_test_cases() {
         TestCase test_case;
 
         // Create a descriptive name for the test case
-        std::string size_str = "";
-        for (size_t i = 0; i < sizes.size(); ++i) {
-          size_str += std::to_string(sizes[i]);
-          if (i < sizes.size() - 1)
-            size_str += "x";
-        }
-
-        std::string storage_str =
-            (storage_type == utils::kTexture3D) ? "Texture3D" : "Buffer";
-        std::string dtype_str = (data_type == vkapi::kFloat) ? "Float" : "Half";
-
-        // Add data generation type to the name for clarity
-        std::string test_name =
-            "Add_" + size_str + "_" + storage_str + "_" + dtype_str;
+        std::string shape_str =
+            shape_bracket(sizes) + "+" + shape_bracket(sizes);
+        std::string storage_str = repr_str(storage_type, utils::kWidthPacked);
+        std::string dtype_str = dtype_short(data_type);
+        std::string test_name = make_test_label(
+            "ACCU", dtype_str, dtype_str, shape_str, storage_str);
         test_case.set_name(test_name);
 
         // Set the operator name for the test case

backends/vulkan/test/custom_ops/choose_qparams_per_row.cpp

@@ -37,13 +37,19 @@ TestCase create_test_case_from_config(
     vkapi::ScalarType input_dtype) {
   TestCase test_case;
 
-  // Create a descriptive name for the test case
-  std::string storage_str =
-      (storage_type == utils::kTexture3D) ? "Texture3D" : "Buffer";
-  std::string dtype_str = (input_dtype == vkapi::kFloat) ? "Float" : "Half";
-
+  // Create a descriptive name for the test case. Op produces (scale, zero_pt)
+  // per row; the "output" is conceptually i8 quantization params.
+  bool is_perf =
+      !(config.num_channels < kRefDimSizeLimit &&
+        config.channel_size < kRefDimSizeLimit);
+  std::string prefix = is_perf ? "PERF" : "ACCU";
+  std::string in_dtype = dtype_short(input_dtype);
+  std::string out_dtype = "f32,i8"; // pair: (scale, zero_point)
+  std::string shape_str = "[" + std::to_string(config.num_channels) + "," +
+      std::to_string(config.channel_size) + "]";
+  std::string storage_str = repr_str(storage_type, utils::kWidthPacked);
   std::string test_name =
-      config.test_case_name + "_" + storage_str + "_" + dtype_str;
+      make_test_label(prefix, in_dtype, out_dtype, shape_str, storage_str);
   test_case.set_name(test_name);
 
   // Set the operator name for the test case

backends/vulkan/test/custom_ops/q4gsw_linear.cpp

@@ -50,12 +50,21 @@ TestCase create_test_case_from_config(
   TestCase test_case;
 
   // Create a descriptive name for the test case
-  std::string storage_str =
-      (storage_type == utils::kTexture3D) ? "Texture3D" : "Buffer";
-  std::string dtype_str = (input_dtype == vkapi::kFloat) ? "Float" : "Half";
-
-  std::string test_name =
-      config.test_case_name + "_" + storage_str + "_" + dtype_str;
+  bool is_perf =
+      !(config.M < kRefDimSizeLimit && config.K < kRefDimSizeLimit &&
+        config.N < kRefDimSizeLimit);
+  std::string prefix = is_perf ? "PERF" : "ACCU";
+  std::string dtype_str = dtype_short(input_dtype);
+  std::string shape_str = "[" + std::to_string(config.M) + "," +
+      std::to_string(config.K) + "]x[" + std::to_string(config.N) + "," +
+      std::to_string(config.K) + "] g" + std::to_string(config.group_size);
+  std::string storage_str = repr_str(storage_type, utils::kWidthPacked);
+  std::string suffix = "[" + config.op_name + "]";
+  if (!config.has_bias) {
+    suffix += " no_bias";
+  }
+  std::string test_name = make_test_label(
+      prefix, dtype_str, dtype_str, shape_str, storage_str, suffix);
   test_case.set_name(test_name);
 
   // Set the operator name for the test case

backends/vulkan/test/custom_ops/q8csw_conv2d.cpp

@@ -27,12 +27,27 @@ TestCase create_test_case_from_config(
   TestCase test_case;
 
   // Create a descriptive name for the test case
-  std::string storage_str =
-      (storage_type == utils::kTexture3D) ? "Texture3D" : "Buffer";
-  std::string dtype_str = (input_dtype == vkapi::kFloat) ? "Float" : "Half";
-
-  std::string test_name =
-      config.test_case_name + "_" + storage_str + "_" + dtype_str;
+  bool is_perf = config.channels.out > kRefDimSizeLimit ||
+      config.channels.in > kRefDimSizeLimit ||
+      config.input_size.h > kRefDimSizeLimit ||
+      config.input_size.w > kRefDimSizeLimit;
+  std::string prefix = is_perf ? "PERF" : "ACCU";
+  std::string dtype_str = dtype_short(input_dtype);
+  std::string in_shape = "[1," + std::to_string(config.channels.in) + "," +
+      std::to_string(config.input_size.h) + "," +
+      std::to_string(config.input_size.w) + "]";
+  std::string weight_shape = "[" + std::to_string(config.channels.out) + "," +
+      std::to_string(config.channels.in / config.groups) + "," +
+      std::to_string(config.kernel.h) + "," + std::to_string(config.kernel.w) +
+      "]";
+  std::string shape_str = in_shape + "x" + weight_shape + " s" +
+      std::to_string(config.stride.h) + " p" +
+      std::to_string(config.padding.h) + " d" +
+      std::to_string(config.dilation.h) + " g" + std::to_string(config.groups);
+  std::string storage_str = repr_str(storage_type, utils::kChannelsPacked);
+  std::string suffix = "[" + config.op_name + "]";
+  std::string test_name = make_test_label(
+      prefix, dtype_str, dtype_str, shape_str, storage_str, suffix);
   test_case.set_name(test_name);
 
   // Set the operator name for the test case

backends/vulkan/test/custom_ops/q8csw_linear.cpp

@@ -36,12 +36,24 @@ TestCase create_test_case_from_config(
   TestCase test_case;
 
   // Create a descriptive name for the test case
-  std::string storage_str =
-      (storage_type == utils::kTexture3D) ? "Texture3D" : "Buffer";
-  std::string dtype_str = (input_dtype == vkapi::kFloat) ? "Float" : "Half";
-
-  std::string test_name =
-      config.test_case_name + "_" + storage_str + "_" + dtype_str;
+  bool is_perf =
+      !(config.M < kRefDimSizeLimit && config.K < kRefDimSizeLimit &&
+        config.N < kRefDimSizeLimit);
+  std::string prefix = is_perf ? "PERF" : "ACCU";
+  std::string dtype_str = dtype_short(input_dtype);
+  std::string shape_str = "[" + std::to_string(config.M) + "," +
+      std::to_string(config.K) + "]x[" + std::to_string(config.N) + "," +
+      std::to_string(config.K) + "]";
+  utils::GPUMemoryLayout layout = (storage_type == utils::kTexture3D)
+      ? utils::kChannelsPacked
+      : utils::kWidthPacked;
+  std::string storage_str = repr_str(storage_type, layout);
+  std::string suffix = "[" + config.op_name + "]";
+  if (!config.has_bias) {
+    suffix += " no_bias";
+  }
+  std::string test_name = make_test_label(
+      prefix, dtype_str, dtype_str, shape_str, storage_str, suffix);
   test_case.set_name(test_name);
 
   // Set the operator name for the test case

backends/vulkan/test/custom_ops/test_conv1d_dw.cpp

@@ -37,22 +37,22 @@ static TestCase create_conv1d_dw_test_case(
   bool is_perf = config.C > kRefDimSizeLimit || config.L > kRefDimSizeLimit;
 
   std::string prefix = is_perf ? "PERF" : "ACCU";
-  std::string storage_str = storage_type_abbrev(storage_type);
-  std::string dtype_str = (dtype == vkapi::kHalf) ? "f16" : "f32";
+  std::string storage_str = storage_type_abbrev(storage_type) + "(HP)";
+  std::string dtype_str = dtype_short(dtype);
   std::string bias_str = config.has_bias ? "+bias" : "";
 
   int64_t L_out =
       (config.L + 2 * config.padding - config.dilation * (config.K - 1) - 1) /
           config.stride +
       1;
 
-  std::string name = prefix + "  conv1d_dw" + bias_str + " [" +
-      std::to_string(config.N) + "," + std::to_string(config.C) + "," +
-      std::to_string(config.L) + "] K=" + std::to_string(config.K) +
-      " s=" + std::to_string(config.stride) +
-      " p=" + std::to_string(config.padding) +
-      " d=" + std::to_string(config.dilation) + "  " + storage_str + "(HP) " +
-      dtype_str;
+  std::string shape = "[" + std::to_string(config.N) + "," +
+      std::to_string(config.C) + "," + std::to_string(config.L) + "] k" +
+      std::to_string(config.K) + " s" + std::to_string(config.stride) + " p" +
+      std::to_string(config.padding) + " d" + std::to_string(config.dilation);
+
+  std::string name = make_test_label(
+      prefix, dtype_str, dtype_str, shape, storage_str, bias_str);
 
   test_case.set_name(name);
   test_case.set_operator_name("test_etvk.test_conv1d_dw.default");

backends/vulkan/test/custom_ops/test_conv1d_pw.cpp

@@ -35,15 +35,17 @@ static TestCase create_conv1d_pw_test_case(
       config.C_out > kRefDimSizeLimit || config.L > kRefDimSizeLimit;
 
   std::string prefix = is_perf ? "PERF" : "ACCU";
-  std::string storage_str = storage_type_abbrev(storage_type);
-  std::string dtype_str = (dtype == vkapi::kHalf) ? "f16" : "f32";
+  std::string storage_str = storage_type_abbrev(storage_type) + "(HP)";
+  std::string dtype_str = dtype_short(dtype);
 
   std::string bias_str = config.has_bias ? "+bias" : "";
 
-  std::string name = prefix + "  conv1d_pw" + bias_str + " [" +
-      std::to_string(config.N) + "," + std::to_string(config.C_in) + "," +
-      std::to_string(config.L) + "]x[" + std::to_string(config.C_out) + "," +
-      std::to_string(config.C_in) + ",1]  " + storage_str + "(HP) " + dtype_str;
+  std::string shape = "[" + std::to_string(config.N) + "," +
+      std::to_string(config.C_in) + "," + std::to_string(config.L) + "]x[" +
+      std::to_string(config.C_out) + "," + std::to_string(config.C_in) + ",1]";
+
+  std::string name = make_test_label(
+      prefix, dtype_str, dtype_str, shape, storage_str, bias_str);
 
   test_case.set_name(name);
   test_case.set_operator_name("test_etvk.test_conv1d_pw.default");

backends/vulkan/test/custom_ops/test_conv2d_dw.cpp

@@ -59,9 +59,8 @@ static TestCase create_conv2d_dw_test_case(
       config.dims.H > kRefDimSizeLimit || config.dims.W > kRefDimSizeLimit;
 
   std::string prefix = is_perf ? "PERF" : "ACCU";
-  std::string storage_str = storage_type_abbrev(storage_type);
-  std::string layout_str = layout_abbrev(memory_layout);
-  std::string dtype_str = (dtype == vkapi::kHalf) ? "f16" : "f32";
+  std::string storage_str = repr_str(storage_type, memory_layout);
+  std::string dtype_str = dtype_short(dtype);
   std::string bias_str = config.has_bias ? "+bias" : "";
 
   int64_t H_out = calc_out_size(
@@ -76,22 +75,27 @@ static TestCase create_conv2d_dw_test_case(
       config.stride.w,
       config.padding.w,
       config.dilation.w);
+  (void)H_out;
+  (void)W_out;
 
+  // groups for depthwise conv2d == number of input channels
   std::string shape = "[" + std::to_string(config.dims.N) + "," +
       std::to_string(config.dims.C) + "," + std::to_string(config.dims.H) +
       "," + std::to_string(config.dims.W) + "] k" +
-      std::to_string(config.kernel.h) + "x" + std::to_string(config.kernel.w) +
-      " s" + std::to_string(config.stride.h) + " p" +
-      std::to_string(config.padding.h) + " d" +
-      std::to_string(config.dilation.h) + "->[" +
-      std::to_string(config.dims.N) + "," + std::to_string(config.dims.C) +
-      "," + std::to_string(H_out) + "," + std::to_string(W_out) + "]";
-
-  std::string selector_str =
-      impl_selector.empty() ? "" : " [" + impl_selector + "]";
-
-  std::string name = prefix + "  conv2d_dw" + bias_str + " " + shape + "  " +
-      storage_str + "(" + layout_str + ") " + dtype_str + selector_str;
+      std::to_string(config.kernel.h) + " s" + std::to_string(config.stride.h) +
+      " p" + std::to_string(config.padding.h) + " d" +
+      std::to_string(config.dilation.h) + " g" + std::to_string(config.dims.C);
+
+  std::string suffix = bias_str;
+  if (!impl_selector.empty()) {
+    if (!suffix.empty()) {
+      suffix += " ";
+    }
+    suffix += "[" + impl_selector + "]";
+  }
+
+  std::string name =
+      make_test_label(prefix, dtype_str, dtype_str, shape, storage_str, suffix);
 
   test_case.set_name(name);
   test_case.set_operator_name("test_etvk.test_conv2d_dw.default");

backends/vulkan/test/custom_ops/test_conv2d_pw.cpp

@@ -38,19 +38,18 @@ static TestCase create_conv2d_pw_test_case(
       config.W > kRefDimSizeLimit;
 
   std::string prefix = is_perf ? "PERF" : "ACCU";
-  std::string storage_str = storage_type_abbrev(storage_type);
-  std::string layout_str = layout_abbrev(memory_layout);
-  std::string dtype_str = (dtype == vkapi::kHalf) ? "f16" : "f32";
+  std::string storage_str = repr_str(storage_type, memory_layout);
+  std::string dtype_str = dtype_short(dtype);
   std::string bias_str = config.has_bias ? "+bias" : "";
 
+  // Pointwise conv2d: kernel 1x1, stride 1, pad 0, dilation 1, groups 1
   std::string shape = "[" + std::to_string(config.N) + "," +
       std::to_string(config.C_in) + "," + std::to_string(config.H) + "," +
-      std::to_string(config.W) + "]->[" + std::to_string(config.N) + "," +
-      std::to_string(config.C_out) + "," + std::to_string(config.H) + "," +
-      std::to_string(config.W) + "]";
+      std::to_string(config.W) + "]x[" + std::to_string(config.C_out) + "," +
+      std::to_string(config.C_in) + ",1,1] s1 p0 d1 g1";
 
-  std::string name = prefix + "  conv2d_pw" + bias_str + " " + shape + "  " +
-      storage_str + "(" + layout_str + ") " + dtype_str;
+  std::string name = make_test_label(
+      prefix, dtype_str, dtype_str, shape, storage_str, bias_str);
 
   test_case.set_name(name);
   test_case.set_operator_name("test_etvk.test_conv2d_pw.default");

backends/vulkan/test/custom_ops/test_embedding_q4gsw.cpp

@@ -115,7 +115,35 @@ void embedding_4bit_reference(TestCase& tc) {
 
 TestCase create_test_case(const EmbeddingConfig& config) {
   TestCase test_case;
-  test_case.set_name(config.test_case_name);
+
+  // Compute the output shape for label: indices_shape + [embed_dim]
+  std::vector<int64_t> output_shape_for_label = config.indices_shape;
+  output_shape_for_label.push_back(config.embed_dim);
+
+  // Treat any case that uses the large llama vocab/embed sizes as PERF.
+  // Otherwise default to ACCU (no PERF distinction is exercised by this op).
+  bool is_perf = config.vocab_size > 1024 || config.embed_dim > 1024;
+  std::string prefix = is_perf ? "PERF" : "ACCU";
+
+  std::string in_dtype = dtype_short(vkapi::kInt); // indices are i32
+  std::string out_dtype = dtype_short(config.dtype);
+
+  std::string storage_str = repr_str(config.storage_type, utils::kWidthPacked);
+
+  std::string shape_str = shape_bracket(config.indices_shape) + "x[" +
+      std::to_string(config.vocab_size) + "," +
+      std::to_string(config.embed_dim) + "]";
+  shape_str += " g" + std::to_string(config.group_size);
+  if (config.is_linear_weight) {
+    shape_str += " lw";
+  }
+  std::string suffix;
+  if (config.scales_dtype == vkapi::kFloat) {
+    suffix = "[f32_scales]";
+  }
+  std::string name = make_test_label(
+      prefix, in_dtype, out_dtype, shape_str, storage_str, suffix);
+  test_case.set_name(name);
   test_case.set_operator_name("et_vk.embedding_q4gsw.default");
   test_case.set_shader_filter({});