Comparative evaluation of the reliability and validity of three data extraction programs: UnGraph, GraphClick, and DigitizeIt

Rakap S., Rakap S., Evran D., Cig O.

COMPUTERS IN HUMAN BEHAVIOR, vol.55, pp.159-166, 2016 (SSCI) identifier identifier

  • Publication Type: Article / Article
  • Volume: 55
  • Publication Date: 2016
  • Doi Number: 10.1016/j.chb.2015.09.008
  • Journal Indexes: Social Sciences Citation Index (SSCI), Scopus
  • Page Numbers: pp.159-166
  • Keywords: Data extraction programs, Reliability and validity, Single-subject experimental research designs, UnGraph, GraphClick, DigitizeIt
  • Ondokuz Mayıs University Affiliated: Yes


The use of evidence-based practices in education has been gaining a lot of attention in recent years. Researchers often use meta-analyses to identify evidence-based practices. To conduct meta-analyses of studies employing single-subject experimental research (SSER) designs for the purpose of identifying evidence base for a practice, a necessary step is to obtain raw data from published graphs. One method for obtaining raw data from published SSER graphs is the use of computer programs specifically designed to extract data from graphs. Purpose of the present study was to examine the reliability and validity of three data extraction programs, Ungraph, GraphClick, and DigitizeIt, using 60 graphs obtained from 15 SSER studies focused on a practice. Three coders extracted data from the graphs using the three programs. Values extracted by each coder were compared to (a) each other (reliability) and (b) values reported in the original articles in which the graphs were obtained from (validity). Results showed that raw data from SSER graphs can be obtained reliably using all three data extraction programs and values obtained using the three programs are highly valid. These results suggest that researchers can use data extracted using these programs with a high level of confidences while concluding meta-analyses of studies employing SSER designs. Authors make recommendations for improving the accuracy of data extraction using the three programs. (C) 2015 Elsevier Ltd. All rights reserved.