YY/T 1907-2023 in English
VALIDArtificial intelligence medical device―Coronary CT image processing software―Algorithm performance test methods
- Issued on:2023-09-05
- Implemented on:2024-09-15
- File Format:PDF
- Delivery:Via email within 5 business days
$301.00
本文件描述了采用人工智能技术的冠状动脉CT影像处理软件算法性能测试方法。
本文件适用于采用人工智能技术对冠状动脉CT影像进行后处理的软件产品。
本文件不适用于影像前处理和过程优化。
Introduction
Analysis of the core framework of the standard
| Test dimensions | Key indicators | Calculation method | Pass criteria |
|---|---|---|---|
| Regional segmentation | Dice coefficient, Hausdorff distance | Formula (1)-(2) | Mean ± standard deviation of vascular layer |
| Calcification score | ICC, Bland-Altman analysis | Formula (3)-(5) | 95% consistency limit |
| Stenosis assessment | Area under the ROC curve | Confusion matrix analysis | ≥0.85 (ideal value) |
Clinical scenario testing methodology
1. CT blood flow reserve fraction verification
Invasive pressure guidewire measurements were used as the gold standard, and the test position needed to be fixed at 2-3cm distal to the stenosis. Key verification steps:
- Construct a three-category classification confusion matrix (FFR>0.8/0.75-0.8/<0.75)
- When calculating specificity, the impact of the false positive rate on clinical decision-making should be emphasized
- The test set must contain at least 30 gray zone samples
2. Two-dimensional verification of plaque assessment
Detection verification:
The intersection over union (IoU) threshold is used for judgment. Manufacturers must specify in the document:
- Cross-sectional images are used as the reference plane
- Matching rules for cross-layer plaques (matching at any layer is considered TP)
- Vascular segmentation is used as the smallest statistical unit
Classification verification:
A three-dimensional confusion matrix needs to be established, with special attention to:
- Characteristics of vulnerable plaques (low density/positive reconstruction/punctate calcification)
- Definition criteria for mixed plaques and calcified plaques
Specifications for building test datasets
| Dimensions | Specific requirements | Examples |
|---|---|---|
| Device diversity | ≥3 brands of CT equipment data | Siemens, GE, United Imaging, etc. |
| Lesion coverage | 6 categories of stenosis severity | Table B.4 Definitions |
| Image quality | CT value range 200-450HU | Aortic root SD <30HU |
Note: The test set must be independent of the training set. The sample size calculation must refer to the statistical formula in Appendix B. The sample size formula for the Pearson correlation coefficient test is shown in formulas (B.1)-(B.6).
Implementation recommendations and risk control
1. Algorithm iteration management
It is recommended to establish a version control mechanism, and each update requires re-verification:
- Core function indicator fluctuation range ≤5%
- Special verification report for new functions
2. Clinical boundary conditions
Must be clearly marked in the instructions:
- Not applicable to diameter <1.5mm vessel assessment
- Special treatment procedures for patients after stent surgery
- Automatic identification threshold for motion artifacts
Loading PDF document...
Error loading PDF. Please make sure the file is valid and try again.
We also recommend
-
YY/T 1949-2024 in English
Special requirements for AI medical device datasets: Color fundus photos of diabetic retinopathy
2024-09-29
