Retinal AI trained on Canon images lost accuracy on cheaper Nikon cameras

A retinal AI model trained on photos from one brand of eye camera performed noticeably worse on images from another, according to a preprint posted to arXiv on October 21 by Lisa Y.W. Tang of the University of British Columbia. The cheaper Nikon camera in the study also produced inadequate images about twice as often as the Canon one.

The paper argues that the camera a clinic buys may be an overlooked source of error in AI eye screening, though it does not measure who gets screened on which device.

Two cameras, one dataset

Tang used BRSET, a Brazilian dataset of retinal photos from three outpatient eye centres in São Paulo collected between 2010 and 2020, covering 8,524 patients. Images came from two fundus cameras: a Canon CR-2 and a Nikon NF5050.

Image quality alone differed sharply. About 18.6% of Nikon images were rated inadequate, against 8.8% of Canon images. The biggest gap was in whether the image captured the right field of the retina: 15.84% of Nikon images failed that check, compared with 4.76% for Canon.

The abstract says refurbished Canon systems cost more than twice as much as Nikon ones. The prices given in the paper's footnote suggest a smaller gap in US dollars, so the exact ratio is not settled.

Where the model stumbled

The study used RETFound-Green, a retinal foundation model with 22.2 million parameters, kept frozen as a feature extractor. Tang trained simple classifiers on top of it using Canon images only, then tested them on held-out Canon images and on the Nikon images the model had never seen.

Across 12 tasks, from diabetic retinopathy grading to myopic fundus, the size of the drop varied a lot from task to task. The worst case was hemorrhage detection, where AUROC fell from 0.804 on Canon to 0.569 on Nikon. Hypertensive retinopathy dropped from 0.864 to 0.690.

Neither of the obvious fixes helped consistently. Feeding the model higher-resolution images or switching between five downstream classifiers did not reliably shrink the cross-camera gap.

Why it matters for screening

Diabetic retinopathy is one of the main reasons people lose sight they could have kept, and AI screening is often pitched for places with few specialists. If accuracy depends on camera hardware, the paper says, results reported as a single average can hide weak spots. Tang calls the cost and quality trade-off "an important consideration for reliable and equitable AI deployment."

The study has clear limits. It covers only two cameras in one dataset, tests one direction (Canon to Nikon) and cannot fully separate the effect of the camera from the effect of image quality. The paper has not yet been peer reviewed. Tang recommends reporting retinal AI performance separately for each camera used in practice.