perf: 简化 QMTSource.is_available(),避免每次数据请求都探测 xtdata#31
Open
DrIsDr wants to merge 3 commits into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
本 PR 旨在优化 HybridDataManager 在优先尝试 QMT 数据源时的性能表现,通过将 QMTSource.is_available() 从“每次真实探测 xtdata API”改为“轻量连接状态检查”,避免在 QMT 不可用/响应慢时反复阻塞后才进入 fallback 逻辑。
Changes:
- 将
QMTSource.is_available()简化为直接返回self.is_connected,移除每次调用的xtdata.get_market_data_ex()探测。 - 补充
is_available()的说明,明确该变更的性能动机与 fallback 行为背景。
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
b8ed87a to
ba1697d
Compare
- 验证 is_available() 仅依据 is_connected 返回状态 - 验证 is_available() 不再调用 xtdata API 做真实探测 - 验证 close() 正确重置连接状态、清空缓存、重置 _last_used - 配置 pytest 入口:pyproject.toml [tool.pytest.ini_options] - gitignore 保留 unit_tests/ 目录,忽略 .worktrees/
ba1697d to
52e974f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题描述
QMTSource.is_available()每次被调用时都会执行一次真实的xtdata.get_market_data_ex()API 探测:而
HybridDataManager在每次获取价格、基本面、交易日历前都会调用source.is_available()(hybrid_manager.py第 164、279、390 行等)。当 QMT 未启动、未安装或响应较慢时,每次数据请求都会先阻塞在 QMT 探测上,失败后再 fallback 到 Tushare,造成明显的卡顿。实际运行日志表现为:
每次请求都要等待 QMT 探测超时/失败后才会继续,严重影响回测和策略运行体验。
改动内容
将
QMTSource.is_available()简化为只返回连接状态:connect()方法在初始化时已经做过真实的连接探测并设置self.is_connected,因此is_available()无需重复探测。为什么这是安全的
语义一致:
is_available()在其他数据源中都是轻量状态检查:DuckDBSource.is_available()->return self.connection is not NoneTushareSource.is_available()-> 检查is_connected和 tokenQMT 原本的实现与该方法的轻量语义不符。
fallback 机制健全:
HybridDataManager对实际数据调用有try-except保护:即使 QMT 在运行中突然断开,
is_available()返回 True 后实际调用失败,也会被捕获并 fallback 到下一个数据源,行为与当前一致。性能收益:消除每次数据请求前的多余 API 探测,QMT 不可用时可立即进入 fallback,策略运行更流畅。
验证
python -m py_compile core/data_manager/sources/qmt_source.py通过from core.data_manager.sources.qmt_source import QMTSource导入正常QMTSource.is_available(),不影响其他数据源或HybridDataManager的数据获取逻辑