UMIN-CTR Clinical Trial
| Unique ID issued by UMIN | UMIN000060250 |
|---|---|
| Receipt number | R000068879 |
| Scientific Title | The diagnostic accuracy of large language models in screening citations in orthopaedic literatures |
| Date of disclosure of the study information | 2026/01/06 |
| Last modified on | 2026/01/04 15:42:12 |
* This page includes information on clinical trials registered in UMIN clinical trial registed system.
* We don't aim to advertise certain products or treatments
Basic information
Public title
The diagnostic accuracy of large language models in screening citations in orthopaedic literatures
Acronym
ORTHOSCREEN
Scientific Title
The diagnostic accuracy of large language models in screening citations in orthopaedic literatures
Scientific Title:Acronym
ORTHOSCREEN
Region
| Japan |
Condition
Condition
the musculoskeletal literature
Classification by specialty
| Orthopedics |
Classification by malignancy
Others
Genomic information
NO
Objectives
Narrative objectives1
This study aims to elucidate the diagnostic accuracy, time expenditure, and associated costs of LLMs in screening citations for meta-analyses focused on hip fractures, in comparison to conventional manual screening methods.
Basic objectives2
Others
Basic objectives -Others
This study aims to elucidate time expenditure, and associated costs of LLMs in screening citations for meta-analyses focused on hip fractures, in comparison to conventional manual screening methods.
Trial characteristics_1
Trial characteristics_2
Developmental phase
Assessment
Primary outcomes
The primary endpoint of this study is the sensitivity and specificity of LLM screening for title and abstract screening as well as full-text screening.
Key secondary outcomes
The secondary endpoint of this study is the time and costs for LLM screening compared to manual screening.
Base
Study type
Observational
Study design
Basic design
Randomization
Randomization unit
Blinding
Control
Stratification
Dynamic allocation
Institution consideration
Blocking
Concealment
Intervention
No. of arms
Purpose of intervention
Type of intervention
Interventions/Control_1
Interventions/Control_2
Interventions/Control_3
Interventions/Control_4
Interventions/Control_5
Interventions/Control_6
Interventions/Control_7
Interventions/Control_8
Interventions/Control_9
Interventions/Control_10
Eligibility
Age-lower limit
| Not applicable |
Age-upper limit
| Not applicable |
Gender
Male and Female
Key inclusion criteria
Ten reviewers for five SRMAs (Two main reviewers for each SRMA) will be recruited by online meeting of Japanese orthopaedic trauma association systematic review committee.The inclusion criteria for participants are as follows: (1) being a member of the systematic review committee in JOTA; (2) availability to participate in citation screening; and (3) willingness to provide informed consent.
Key exclusion criteria
We will exclude committee members who cannot give informed consent and are unavailable for citation screening.
Target sample size
10
Research contact person
Name of lead principal investigator
| 1st name | Natsumi |
| Middle name | |
| Last name | Saka |
Organization
Teikyo University, School of Medicine
Division name
Department of Orthopaedics
Zip code
1738606
Address
2-1-1 Kaga, Itabashi
TEL
0339641211
natsumi613@gmail.com
Public contact
Name of contact person
| 1st name | Natsumi |
| Middle name | |
| Last name | Saka |
Organization
Teikyo University, School of Medicine
Division name
Department of Orthopaedics
Zip code
1738606
Address
Kaga2-11-1, Itabashi
TEL
0339641211
Homepage URL
natsumi613@gmail.com
Sponsor or person
Institute
Teikyo University
Institute
Department
Personal name
Funding Source
Organization
Ministry of Education, Culture, Sports, Science and Technology
Organization
Division
Category of Funding Organization
Japanese Governmental office
Nationality of Funding Organization
Japan
Other related organizations
Co-sponsor
Name of secondary funder(s)
IRB Contact (For public release)
Organization
Teikyo University Medical Research Ethics Committee
Address
2-1-1 Kaga, Itabashi
Tel
0339647256
turb-office@teikyo-u.ac.jp
Secondary IDs
Secondary IDs
NO
Study ID_1
Org. issuing International ID_1
Study ID_2
Org. issuing International ID_2
IND to MHLW
東京都
Institutions
Institutions
Other administrative information
Date of disclosure of the study information
| 2026 | Year | 01 | Month | 06 | Day |
Related information
URL releasing protocol
Publication of results
Unpublished
Result
URL related to results and publications
Number of participants that the trial has enrolled
Results
Results date posted
Results Delayed
Results Delay Reason
Date of the first journal publication of results
Baseline Characteristics
Participant flow
Adverse events
Outcome measures
Plan to share IPD
IPD sharing Plan description
Progress
Recruitment status
Preinitiation
Date of protocol fixation
| 2025 | Year | 11 | Month | 14 | Day |
Date of IRB
| 2025 | Year | 12 | Month | 01 | Day |
Anticipated trial start date
| 2026 | Year | 01 | Month | 15 | Day |
Last follow-up date
| 2027 | Year | 12 | Month | 31 | Day |
Date of closure to data entry
Date trial data considered complete
Date analysis concluded
Other
Other related information
Search Strategy and Data Acquisition: Initially, search queries will be formulated for five Clinical Questions designated for new Systematic Reviews based on the Japanese Orthopedic Association Guidelines for Management of Hip Fractures. Literature will be retrieved from literature database as well as clinical trial registries.
Random Sampling: From the retrieved literature, 500 records will be randomly selected. This process will be conducted by the Principal Investigator using a dedicated software application.
Human Screening Protocol For the SR committee screening, two primary reviewers will independently assess the records using screening application to determine eligibility based on inclusion and exclusion criteria for each CQ. Discrepancies will be resolved by a third reviewer. Reviewers will remain blinded to each other's decisions and to the results of the Large Language Model screening described below.
LLM Screening and Time Tracking: Simultaneously, the LLM will be utilized to perform screening on the same dataset. Additionally, the time required for the human reviewers to complete the screening- automatically logged within screening application will be extracted for analysis.
Management information
Registered date
| 2026 | Year | 01 | Month | 04 | Day |
Last modified on
| 2026 | Year | 01 | Month | 04 | Day |
Link to view the page
Value
https://center6.umin.ac.jp/cgi-open-bin/ctr_e/ctr_view.cgi?recptno=R000068879
