The measurement of image quality plays an important role in all acquisition systems. In many medical applications, such as in the presented Near Infrared (NIR) imaging of the finger vascular system, there is no possibility of comparing the system outcomes with a reference data set. As a solution for this problem, a range of available methods have been presented, which try to reflect or imitate subjective human operator assessment. In this paper, we present a review of such quality metrics and introduce a novel approach based on distance transformations. The proposed method was compared with the state-of-the-art metrics on NIR vascular system images obtained by our constructed acquisition device. By changing the intensity of the finger backlight and by examining intentionally blurred images, we proved that our approach was capable of providing most sensible quality outcomes, while the others turned out to be much less reliable. The proposed evaluation technique may also be employed in any other medical application, where the assessment of the image quality has a direct impact on the final diagnosis.