RetinalNet technology, justification, and work to date

Since the beginning of the SARS-COV-2 pandemic until now there has been an acute shortage of SARS-COV-2 testing capability in the U.S. and worldwide.  Therefore, in April 2020 FSU in collaboration with its partner FG Labs we began research for the purpose of developing an (AI) artificial intelligence based, rapid, non-invasive SARS-COV-2 screening solution.

The diagnostic test for SARS-COV-2 infection is a reverse transcription polymerase chain reaction (RT-PCR) test. However, there has been a severe shortage of test-kits worldwide and laboratories in most countries have struggled to process the available tests within a reasonable timeframe. The US Dept of Health, Inspector General conducted a SARS-CoV-2 Hospital Experience Survey [52] from March 23-27, 2020, with hospital administrators from 323 hospitals across 46 States. The ensuing IG report released on 3 April stated that: “Hospitals reported that their most significant challenges centered on testing and caring for patients with SARS-COV-2 and keeping staff safe. Hospitals said that severe shortages of testing supplies and extended waits for test results limited hospitals' ability to monitor the health of patients and staff”. Furthermore, RT-PCR has limited sensitivity for SARS-COV-2 infection at 71% [28]. The paucity of testing kits warrants a system of ‘triaging’ patients most needful of a confirmatory RT-PCR test for SARS-COV-2. A rapid, non-invasive and readily available screening tool based on machine learning and computer vision could fill this gap by identifying patients who need further testing. It should detect both symptomatic and asymptomatic subjects therefore will be more useful for contract tracing and access control purposes.  Identifying and isolating asymptomatic, infected “carriers” remains well beyond the capabilities of current tests.  The ability to  readily identify such “silent carriers” will dramatically improve our approach to contagion management. The US testing capacity is currently approximately 800,000 test per day, however, the current demand for tests is 1.5 million tests/day and growing (WJS reference)

Due to the SARS-COV-2 testing shortage FG Labs and its partner FSU made the decision to explore options for SARS-COV-2 testing using artificial intelligence (AI).  After researching a range of AI solutions, it was decided to develop “RetinalNet” a non-invasive, rapid, means of risk assessment for SARS-CoV-2.  Our RetinalNet  proof-of-concept (POC) research uses fundus eye images captured using FDA approved handheld portable non-mydriatic camera technology.  The eye images are then processed through our (CNN) Convolutional Neural Network model that employs transfer learning with deep neural network architecture - Inception Resnet to re-train and fine-tune the model.  This approach has been implemented in our POC and evaluated to effectively detect Diabetic Retinopathy from retinal scans. An overall validation accuracy of 92.58% has been achieved by our POC over the validation dataset of retinal scans. Our deep neural network has been trained over a dataset of 1368 retinal scans split into two classes - “No Diabetic Retinopathy” and “Diabetic Retinopathy”. In addition to the retinal scans, our RetinalNet POC was re-trained, fine-tuned and evaluated on Chest X-Ray images to detect normal, pneumonia and SARS-CoV-2 cases. Our POC model trained and evaluated over 1196 Chest X-Ray images resulted in a validation accuracy of 89.29% over the validation dataset. We are encouraged that RetinalNet.01 differentially diagnosed SARS-CoV-2 pneumonia based on imaging analysis alone.

We hypothesize that our Convolutional Neural Network based approach could prove useful for SARS-CoV-2 risk assessment from not only retinal scans but scleral and iris scans as well.  The success of this approach for retinal imaging Diabetic Retinopathy Detection as well as X-Ray imaging SARS-COV-2 detection reinforces our hypothesis of using CNNs for effective risk assessment for SARS-CoV-2 from retinal as well as scleral and iris scans using handheld portable non-mydriatic camera technology.

Significant datasets needed:

The data set used to train our POC model was provided by Kaggle.  The CNN model was trained to detect diabetic retinopathy in retinal images and COVID-19 in X-ray images.  We implemented a dataset of 1368 retinal scans split into two classes - “No Diabetic Retinopathy” and “Diabetic Retinopathy”. An overall validation accuracy of 92.58% has been achieved by our POC model over the validation dataset of retinal scans. We also implemented a dataset of 1196 Chest X-Ray images split into 3 classes – “normal”, “pneumonia” and “SARS-CoV-2 cases”.  Our POC model achieved a validation accuracy of 89.29% over the validation dataset.

In collaboration with FSU we are currently collecting a data set of COVID-19 negative and positive eye images of both the retina and the eye surface (iris and sclera).  This data set is being collected from 3000 subjects beginning mid July 2020 ending December 2020 or sooner depending on the date of the last of the 3000 COVID-19 tests administered.  We estimate the FSU viral testing and FG Labs imaging process will render between 150 and 300 SARS-COV-2 positive subject eye images.  We hypothesize that a minimum of 750 to 1,500 SARS-COV-2 positive eye images, and an additional 750 to 1,500 SARS-COV-2 negative eye images will be needed to validate our hypothesis by training the RetinalNet CNN to be able to recognize a SARS-COV-2 signature in the eye images with a reasonable degree of accuracy.

Our goal is to collect the additional eye imaging datasets needed by partnering with UNC affiliates who are scheduled to conduct an additional 7500 to 15,000 COVID-19 viral tests.

Risks and proposed mitigation.

There is a risk that a much larger data set of 10,000+ sample eye images may be required to train and test our CNN and achieve a reasonable degree of accuracy,  In this event it will require us to partner with an increased number of COVID-19 testing partners to gather the necessary eye images.

We may also determine that there is a fidelity issue with the eye image datasets as a result of the capability of the currently available retinal imaging devices.  In that regard a much deeper retinal scan may be required using hand-held OCT (optical coherence tomography) retinal scanning devices which are still in the prototype phase and commercially unavailable.  In this event we will need to purchase and/or construct OCT prototypes.


Abstract    Methodology


(910) 584-6193   •

Fortem Genus, Inc.

427 Franklin St.

Fayetteville, NC 28301




(910) 584-6193   •