Can AI be Used to Improve Medication Adherence?

Inspiration

The median age has been increasing worldwide. In fact, Canadian researchers estimated that by the year 2036, the proportion of individuals aged 65 and older will make up a quarter of the population. In the discussion of health and longevity, a crucial yet often underestimated cause of concern lies in the suboptimal adherence of medications prescribed to individuals by health care practitioners. Between a patient and a doctor, even with the utmost care, information will be lost throughout the medical jargon and foreign language spoken by a trained medical professional. Due to the gap and disconnect between the carer and the cared, the patient will likely not receive the optimal care intended by the doctor. We wish to close this gap one step at a time, to ensure that the increasing number of future patients will have an easy time receiving the care that was intended by ensuring proper administration of medication. Researchers estimated that individuals with chronic illness might be more likely to take their medications. However, according to the World Health Organization, adherence among this group averages only about 50%. Poor adherence to prescription medication can lead to individual drawbacks such as an infection not fully clearing up, bacterial infection building resistance to the medication in the individual. Worse yet, the medication resistant bacteria could be spread and become a dominant strain as we have seen in MRSA in hospitals. There are many reasons for poor adherence to medication, including mistrust of doctors, cost, too many medications, misunderstanding, and a lack of symptoms. We hope to eliminate a few of these factors and assist individuals in bettering caring for themselves by following through with their doctor’s prescription and finishing their medicine reference url

What it does

Daily dose is an optical character recognition based web app that helps you schedule medication, so you don't forget and finish the prescribed course. It employs Google’s image recognition API and scans a image of your prescription. Afterwards the important information will be extracted and used to create recurring events in your Google calendar. This will ensure that you consistently follow through with your medication. How to use it:

Sign in with Google+.
Upload a photo of your medication, or use your webcam.
Connect to your Google Calendar.
Make sure the information is correct

How we built it

Daily Dose was created using plain HTML, CSS, and JavaScript for the front-end, and Node.js and Express for the back-end. Wireframing was first performed in Adobe XD. Then, after tinkering with the colors, fonts, and layout, a mobile-first prototype was created. With a prototype finished, a basic express skeleton project was generated and the prototype was translated from design to HTML and CSS. At this stage, four routes and their corresponding pages were set up for mobile and desktop (home, dashboard, form, and upload). This was accomplished using media queries in Sass. Then, on the back-end, additional routes were created for Google+ sign-up and log-in. This was implemented using various npm libraries such as passport and cookie-session, crucial for ensuring user security and fast development time. Additionally, mongoose and mongoDB were used to store user data in a collection. At this point, important keys were hidden using dotenv. Next, external libraries like Uppy and Multer were again central for the creation of a local drop-box system with webcam access. Upon further assessment, it was decided that the upload route would be removed, and the drop box directly be inserted into the user’s dashboard. Getting this step right was important since user medication photos were then fed through Google vision, an optical character recognition API that performed the bulk of the heavy lifting. The API returned a list of words in the image, which were filtered using regular expressions and various functions. Next, a form that the user was redirected to was autocomplete using the data parsed from earlier. Upon user verification, the form was submitted to the Google+ calendar API that scheduled for the user, when to take their medication. Finally, the front-end and back-end were cleaned up a bit, the domain, daily-dose.online was purchased on domain.com, and the project was hosted on Replit.

Challenges we ran into

Finding the right path The development process may seem straightforward, but there were many challenges and bugs along the way. Although the hackathon started out strong, with quick iteration of the prototype and design, the first major roadblock came when trying to connect Google vision API with the uploaded image. First, the API was not able to identify the location of the image. Then, the API was not able to identify the location of the secret keys required for the API. But after many attempts at finding the right paths, the API was finally able to return some text that had been uploaded as an image.

How do you identify the name of a drug? The second roadblock came when trying to parse the text returned by the Google vision API. All you have to do is identify which words returned correspond to your key parameters, and then format it in a way that can be used in the pipeline later. Regular expressions provide a robust albeit primitive means of parsing string data in JavaScript, but the real issue lies not in the code per-say, but in the variation between prescription labels. With the variation in pill bottles, and their accompanying labels, how do you identify the name of a drug, when there are seemingly little formatting rules? The solution is to train a machine learning model so that it will identify key parameters on a bottle of medication, or perhaps to look into NLP algorithms that are designed specifically for processing human language. Instead, a solution composed of, if and else statements, hope and determination allow us to move onto the next step.

Can I get a date? The third roadblock came finally, when trying to upload the parsed medication data to google calendar. First, there is the challenge of connecting to a user's calendar, which involves OAuth and the passing of special tokens between our server and Google’s API. Then, there is the true goal of scheduling a medication across multiple days and weeks, according to the direction on the bottle. The convoluted documentation of Google’s API pages seemed to form a loop, and finding the structure of RULES needed to set up a recurring event was a challenge in itself. These tasks were immensely challenging, although interesting but I wanted to give up because it was very late into the project at this point. Eventually, a few tests passed as expected, and the project was cleaned up.

What's next for Daily Dose

As previously mentioned, there were many challenges in the construction of Daily Dose. it would be interesting to see the application of NLP and machine learning solutions for parsing medical labels. Although regex was used, there may be more attractive and advantageous techniques, particularly due to the variability of language and placement of text on medical labels. Daily Dose, and models similar, demonstrate remarkable potential for technical growth. Optical Character Recognition are widely used in the distribution of mail across the globe. However, the same software has yet to be widely adopted at a consumer level. For instance, such software has the potential to track product receipts, which would help customers reliably obtain refunds. They could also be used to upload written or typed lecture notes, so that students and educators could later access them electronically.

Resources

See the devpost here

👋 Thanks for making it to the end!