Developed Medical Language Model (MedLMs)
Finetuned multiple LLMs for Medical answering system and achieved remarkable performance.
The integration of AI in healthcare is revolutionizing how we approach Medical Question Answering (MedQA) systems, providing clinicians, patients, and researchers with quick and reliable responses. In this project, we explore the fine-tuning of several pre-trained large language models and adapting them to the medical domain through efficient methods like LoRA. The final models were deployed using LangChain and Netlify for better infernece speed.
By experimenting with datasets like WikiDoc and MedQuad, we uncover valuable insights into how these models perform in terms of speed, scalability, and task-specific accuracy. Comparing the models reveals key trade-offs. Larger models like Llama 3 excel at handling complex medical queries, while smaller models such as DistilGPT2 prioritize speed and efficiency for simpler tasks. Our findings emphasize how fine-tuning and careful model selection can create precise, scalable solutions tailored to the unique challenges of healthcare applications.
Models:
- Llama 3
- DistilGPT2
- Gemma 1.1
- Mistral 7B
- Llama 2
All the models fine-tuned in our project can be found at Apurva’s Hugging Face and here. Please feel free to use them and test it.
Usage and feedback:
Please try our model(s) and give feedback so we can improve and tune our parameters and possibly adapt to different scenarios.
You can make a pull request in this repo: Repository
Thanks !!