Optimize TsFileWriter::do_check_schema CPU overhead for repeated wide-tablet writes

[kkzi]

Problem

TsFileWriter::do_check_schema shows noticeable CPU overhead when writing repeated tablets with the same device and schema.

In our workload, each writer repeatedly writes tablets for a fixed device with a fixed set of measurements. However,
write_tablet() calls do_check_schema() for every tablet write. This appears to repeatedly perform schema lookup and
measurement-name matching even though the schema has already been registered and does not change during the file lifecycle.

Evidence

Using Windows Performance Analyzer on a 60-second TsFile archive workload, the active processing window showed:

• TsFileArchive.dll!storage::TsFileWriter::do_check_schema<MeasurementNamesFromTablet> as the top TsFileArchive function
  hotspot
• std::string::compare also appeared as a related hotspot
• The workload writes tablets with the same device and the same schema repeatedly
• TsFile writing took longer than the input data duration

Example hotspot:

TsFileArchive.dll!storage::TsFileWriter::do_check_schema<storage::MeasurementNamesFromTablet>

Suspected Cause

write_tablet() creates or looks up schema information on every call:

• creates a StringArrayDeviceID from tablet.insert_target_name_
• looks up the device in schemas_
• iterates all measurement names
• looks up each measurement in measurement_schema_map_
• fills chunk_writers and data_types

For wide tablets and high-frequency writes, this repeated per-tablet schema validation becomes expensive.

Suggested Optimization

Add a cached or prepared schema path for repeated tablet writes.

Possible approaches:

1. Cache the resolved schema for the last used device/tablet schema inside TsFileWriter.
2. Add an explicit prepared API, for example:

PreparedTabletSchema prepare_tablet_schema(device_id, schema_vec);
int write_tablet_prepared(const Tablet& tablet, const PreparedTabletSchema& prepared);

The cached/prepared data could include:

• MeasurementSchemaGroup*
• resolved ChunkWriter* list
• resolved TSDataType list

This would allow repeated writes with the same device and schema to skip per-column name lookup.

Expected Benefit

Reduce CPU overhead in high-throughput TsFile writing workloads, especially for wide schemas where the same tablet schema is
written repeatedly.

Notes

This should preserve the existing validation behavior for dynamic schemas or changing devices. The optimization can be limited
to cache hits where both device and schema identity match.

[ColinLeeo]

Thank you for the feedback. This issue raises a valuable point.

The current implementation performs device parsing and measurement lookup every time a Tablet is written. This can become a CPU hotspot when the schema is wide or the Tablet contains relatively few rows, especially when repeatedly writing with a fixed schema.

Introducing a caching mechanism is a promising optimization direction. Repeated access patterns generally exhibit strong locality in device resolution, schema lookup results, and measurement access paths. Caching these results could significantly reduce redundant computation and improve overall write throughput.

We will explore this direction further and evaluate more fine-grained optimization strategies based on caching, including the appropriate cache granularity, invalidation strategy, and integration with the write path. We will also analyze representative workloads to compare the benefits and costs of different approaches.

In addition, we will add benchmarks covering different write patterns to ensure that the optimization is stable and its benefits are measurable.

Thank you again for raising this issue. It is very helpful in identifying and investigating this performance problem.