Impact of Context on Large Language Models for Clinical Named Entity Recognition

This study explores the impact of context on Large Language Model (LLM) performance in clinical named entity recognition (NER), focusing on three context levels—sentence, section, and document—and two methods of context incorporation: (1) embedded context - expanding the input to include broader context (e.g., replacing a sentence with its full section or entire document) and (2) detached context - incorporating context as independent material before the input. Using the MTSamples and i2b2 datasets, we evaluated GPT-4o models across varying context conditions. Our analysis reveals that detached context at the section level enhanced model performance, achieving 0.594 exact match F1 on MTSamples and 0.485 on i2b2, while embedded context generally reduced performance.