[en] The need for Receiver Operating Characteristic (ROC) analysis is indicated by a discussion of the limitations of accuracy and of sensitivity and specificity as indices of diagnostic detection or discrmination performance. The concept of a variable decision threshold is shown to lead in a natural way to the ROC curve as a means for specifying diagnostic performance. Practical techniques for measuring ROC curves are described, and directions for possible generalizations of conventional ROC analysis are indicated

[en] Effort in this project during the past year has focused on the development, refinement, and distribution of computer software that will allow current Receiver Operating Characteristic (ROC) methodology to be used conveniently and reliably by investigators in a variety of evaluation tasks in diagnostic medicine; and on the development of new ROC methodology that will broaden the spectrum of evaluation tasks and/or experimental settings to which the fundamental approach can be applied. Progress has been limited by the amount of financial support made available to the project

[en] Receiver Operation Characteristic (ROC) methodology is now widely recognized as the most satisfactory approach to the problem of measuring and specifying the performance of a diagnostic procedure. The primary advantage of ROC analysis over alternative methodologies is that it seperates differences in diagnostic accuracy that are due to actual differences in discrimination capacity from those that are due to decision threshold effects. Our effort during the past year has been devoted to developing digital computer programs for fitting ROC curves to diagnostic data by maximum likelihood estimation and to developing meaningful and valid statistical tests for assessing the significance of apparent differences between measured ROC curves. FORTRAN programs previously written here for ROC curve fitting and statistical testing have been refined to make them easier to use and to allow them to be run in a large variety of computer systems. We have attempted also to develop two new curve-fitting programs: one for conventional ROC data that assumes a different functional form for the ROC curve, and one that can be used for ''free-response'' ROC data. Finally, we have cooperated with other investigators to apply our techniques to analyze ROC data generated in clinical studies, and we have sought to familiarize the medical community with the advantages of ROC methodology. 36 ref

[en] A test for the statistical significance of observed differences between two measured Receiver Operating Characteristic (ROC) curves has been designed and evaluated. The set of observer response data for each ROC curve is assumed to be independent and to arise from a ROC curve having a form which, in the absence of statistical fluctuations in the response data, graphs as a straight line on double normal-deviate axes. To test the significance of an apparent difference between two measured ROC curves, maximum likelihood estimates of the two parameters of each curve and the associated parameter variances and covariance are calculated from the corresponding set of observer response data. An approximate Chi-square statistic with two degrees of freedom is then constructed from the differences between the parameters estimated for each ROC curve and from the variances and covariances of these estimates. This statistic is known to be truly Chi-square distributed only in the limit of large numbers of trials in the observer performance experiments. Performance of the statistic for data arising from a limited number of experimental trials was evaluated. Independent sets of rating scale data arising from the same underlying ROC curve were paired, and the fraction of differences found (falsely) significant was compared to the significance level, α, used with the test. Although test performance was found to be somewhat dependent on both the number of trials in the data and the position of the underlying ROC curve in the ROC space, the results for various significance levels showed the test to be reliable under practical experimental conditions

No abstract available

[en] It is attempted to show how an internal noise source (darklight and threshold jitter) would tend to explain experimental data concerning the visual detection of noise-limited signal in diagnostic imaging. The interesting conclusions can be drawn that the internal noise sets the upper limit to the utility of data processing techniques designed to reduce image noise. Moreover, there should be instances where contrast enhancement techniques may be far more useful to the human observer than corresponding reductions in noise amplitude, especially at high count rates (sigma/sub p/ less than or equal to sigma/sub D/). Then too, the limitations imposed on the human observer by an internal noise source, may point towards the need for additional methods (e.g. computer/microdensitometer) of interpreting images of high photon density so that the highest possible signal to noise ratio might be obtained

[en] The limitations of diagnostic accuracy as a measure of decision performance require introduction of the concepts of the sensitivity and specificity of a diagnostic test. These measures and the related indices, true positive fraction and false positive fraction, are more meaningful than accuracy, yet do not provide a unique description of diagnostic performance because they depend on the arbitrary selection of a decision threshold. The receiver operating characteristic (ROC) curve is shown to be a simple yet complete empirical description of this decision threshold effect, indicating all possible combinations of the relative frequencies of the various kinds of correct and incorrect decisions. Practical experimental techniques for measuring ROC curves are described, and the issues of case selection and curve-fitting are discussed briefly. Possible generalizations of conventional ROC analysis to account for decision performance in complex diagnostic tasks are indicated. ROC analysis is shown to be related in a direct and natural way to cost/benefit analysis of diagnostic decision making. The concepts of average diagnostic cost and average net benefit are developed and used to identify the optimal compromise among various kinds of diagnostic error. Finally, the way in which ROC analysis can be employed to optimize diagnostic strategies is suggested

[en] One of the problems posed by the astigmatic collimator is an accurate and effective description of its imaging properties. Since the image of an object can change drastically with its position relative to the collimator, the traditional method of describing a collimator in terms of its resolution and sensitivity is inadequate and must be extended to include the effects of source position. The authors have developed the generalized collimator transfer function (GCTF) to describe the non-stationary effects of collimator hole pattern on imaging. This concept is ideally suited to deal with the non-stationary effects of convergent and astigmatic collimation. The performance of a stationary imaging system can be described mathematically in terms of its point source response function (PSRF). The PSRF describes the image resulting from a point source located anywhere in front of the camera. the underlying assumption is that if the image of a point source is known, then the image of any extended source distribution can be reconstructed by convolution of the point source response function with the extended source distribution. For stationary systems the imaging processes is assumed to be shift-invariant. However, the authors goal is to analyze systems that are not necessarily shift-invariant. In order to accommodate such systems, they introduce the generalized point source response function (GPSRF). During the past year they have applied the GPSRF to generate SPECT images of analytical phantoms constructed as a collection of Gaussian source distributions

[en] During the last four years the authors have worked extensively on an algorithm to reconstruct the source distribution within a scattering medium from a single point of view. Their initial results four years ago using a linear reconstruction algorithm were very disappointing. Although two channels of source depth information were recovered from the scattered radiation, the reconstruction algorithm produced a significant degradation in the resolution within the imaging plane. Three years ago they developed a new nonlinear algorithm which solved this problem. The solution involved coupling the information about source depth which is available only in the low spatial frequencies of the images with the detailed information about the spatial distribution of source which is available in the high spatial frequencies. Using this algorithm they have found that two (and perhaps) three channels of depth information can be extracted from the scattered radiation generated by a ^99mTc source in water. The new algorithm does not degrade spatial resolution in the imaging plane, provides depth resolution with standard deviation of 4 cm without requiring any camera motion, and produces significant attenuation correction in the reconstructed source distribution. Although this depth resolution is still quite modest, the technique has the potential for significantly better quantitation when combined with standard SPECT reconstruction algorithms

[en] An ROC curve provides an empirical description of the trade-offs which are possible among the various types of correct and incorrect decisions as the human decision-maker varies one or more confidence thresholds. Conventional ROC curves measured in simple decision-making situations can, in some cases, be used to predict human decision performance in more complex situations. By considering both the consequences of the various types of diagnostic decisions and the overhead cost of a diagnostic study, one can use the ROC curve to evaluate the diagnostic usefulness of a study in any particular clinical context. Since the ROC curve describes the possible relationships among the probabilities of the various types of correct and incorrect decisions, it plays a central role in optimizing diagnostic strategies using the general techniques of decision analysis. Applications in radiographic image evaluation are described