Improve performance of h5trav interfaces for links to objects#6400
Improve performance of h5trav interfaces for links to objects#6400jhendersonHDF wants to merge 3 commits into
Conversation
For objects with multiple hard links, use hash table to map between object tokens and pathnames during traversal to avoid linear scan over all previous objects for each hard link seen Use separate hash table for h5trav "table" interface to map between object tokens and an index into the table of visited objects. This facilitates quick lookups of objects when adding hard link name aliases for h5repack processing
jhendersonHDF
added
the
Component - Tools
| #include "hdf5.h" | ||
|
|
||
| /* Typedefs for visiting objects */ | ||
| typedef herr_t (*h5trav_obj_func_t)(const char *path_name, const H5O_info2_t *oinfo, const char *first_seen, |
There was a problem hiding this comment.
These typedefs had to be moved down for the new trav_seen_t parameter to be available
| trav_obj_t *objs; | ||
|
|
||
| /* Private data for this trav_table_t */ | ||
| void *priv_data; |
There was a problem hiding this comment.
This is really a pointer to a structure with a UT_hash_handle, but exposing the uthash API at the h5trav.h level is problematic due to its header-only nature and already being used in H5private.h. This is just a quick hack to hide the implementation details.
| * where a visited object was placed to facilitate quicker | ||
| * lookups when adding path aliases | ||
| */ | ||
| typedef struct trav_table_hash_t { |
There was a problem hiding this comment.
This structure is mostly for h5repack, which now has its own hash table separate from the main one used by other h5trav interfaces. I considered modifying the h5trav interfaces to allow h5repack to share a single hash table, but it would have been fairly awkward to do so and would have imposed unnecessary memory overhead for other tools that wouldn't need the extra information that would have been stored, so I instead just used a separate hash table specifically for the h5trav "table" interface.
| *------------------------------------------------------------------------- | ||
| */ | ||
| static int | ||
| trav_token_visited_cmp(hid_t loc_id, const H5O_token_t *token1, const H5O_token_t *token2) |
There was a problem hiding this comment.
This function is called by HASH_FIND and is just used to wrap around H5Otoken_cmp() instead of a plain memcmp of object token bytes
| udata.fields = fields; | ||
|
|
||
| /* Check for multiple links to top group */ | ||
| if (oinfo.rc > 1) |
There was a problem hiding this comment.
Moved this down below the udata initialization in case the hash table gets initialized by the call to trav_token_add().
jhendersonHDF
requested review from
bmribler,
brtnfld,
derobins,
fortnern,
glennsong09,
lrknox,
mattjala,
qkoziol and
vchoi-hdfgroup
as code owners
mattjala
left a comment
mattjala
left a comment
There was a problem hiding this comment.
Only a couple minor issues
3 participants
For objects with multiple hard links, use hash table to map between object tokens and pathnames during traversal to avoid linear scan over all previous objects for each hard link seen
Use separate hash table for h5trav "table" interface to map between object tokens and an index into the table of visited objects. This facilitates quick lookups of objects when adding hard link name aliases for h5repack processing.
See the linked issue for context