Docviser Generative AI Assistant.

A powerful AI LLM tool for doctors to leverage on the full scale of the generative AI for their patients. It uses a wide knowledge base formed by all the medical information of the patients: active/past treatments, epicrisis, files with medical tests and examinations.

It supports any types of files from structured documents like .pdf, .docx, .xlsx, .csv, .json, .txt to images that contain relevant text like .jpg and .png and even PDF scans that in the most of the cases would be impossible to work with.

Doctors can now easily scan or take photos using Docviser Mobile app on their phones to any relevant documents and upload them to their patients file base.

Doctors can then hold conversations with Docviser Assistant and ask anything about their patient to get an answer in record time while staying compliant and respecting the full privacy of a medical application.

1. Implement a ChatGPT like conversation tool that can answer any kind of question about a specific patient by using the entire knowledge base and medical history of the patient.

2. Choosing the right LLM (Large Language Model) in order to retrieve the best possible answers in a timely manner

3. Secure environment to be able to use OpenAI API and leverage on the power of OpenAI models to ensure a wider view on other aspects of medicine like side effects to certain molecules or connections with different pathologies.

4. Process files like images or PDF scans in order to extract text and use them together with the rest of the information to create the vectorized database. In most of the cases doctors still work with papers and medical tests and examinations coming from other sources on a piece of paper into their patient's hands.

1. In order to implement a ChatGPT like conversational tool we had to take in consideration the exact use case that a doctor will require in order to quickly find useful information about a patient without generating confusions. At the same time we had to make sure we respect privacy and we won’t create conflicting information. This is the reason why at the moment Docviser Assistant (Beta) is per patient and will provide information about a single patient at once.

2. Choosing the right LLM was a good challenge, as we tried several ones that are private and we can maintain into our system without worrying about privacy, a private gpt would have saved us a lot of time. We tried several open source models that we can potentially use and develop further like GPT4All ( https://gpt4all.io/ ) Unfortunately during our tests the private-gpt model had a very modest behavior in comparison with gpt-4 or even gpt-3.5-turbo from OpenAI. Especially that in our case it has to work flawless in different languages like English as well as Romanian, Deutsch, Turkish and other supported languages. We had to make sure that the model will be able to execute correctly in any of these languages and will be able to provide relevant answers in a timely manner. Therefore we made a decision to move further with GPT-4 as the default model.

3. Now that we made a decision about a default model, we wanted to make sure we remain compliant and we won’t just send information to open-ai to create context. Even if we did that and supposingly open-ai doesn’t store your information and it still remains private(as they claim to do so), we still had to handle a full scale knowledge base that would definitely exceed OpenAi GPT-4 context maximum length. Therefore we used Langchain (https://www.langchain.com/) in order to create a vectorized database using Open AI Embeddings. Vector database is a very important tool and Langchain is a great tool in manipulating & training your models to handle all the connections via embeddings without compromising information. It can be used in order to train models and execute queries using your models combined with your vectorized database. Now in environments where privacy is extremely important, this would still not be enough and you want to make sure that data like PII, PHI etc. stays private and you have full control over it. In order to handle this properly we then used another AI tool, this time completely private and isolated within our AWS Infrastructure. We used AWS Comprehend. Comprehend will detect any information like PII, PHI, card numbers etc. or even custom information that we provide so that we can further encrypt and redact in real time while interacting with your AI model.

4. If Langchain’s Documents Loader was a great tool to help creating embeddings from structured files, this tool wouldn't work with files like images with text (.jpg, .png) or PDF scans that contain a lot of text. Here in order to handle it properly we used another AI tool in order to detect text from images and extract it properly with high accuracy.