The Human Condition in Structured and Unstructured Data

 

Do you prefer the Sears Tower or a Jackson Pollock painting? Do you prefer the USRDS data tables or poems by e. e. cummings? Do you delight in structured data or unstructured data? Do you have to choose?

 

I am hoping in Health IT we will not have to choose between structured and unstructured data, which, like sturdy skyscrapers and creative paintings, each have benefits. Much of healthcare data now being reported in Meaningful Use and other quality programs is focused on structured data elements, but in a recent Hospitals & Health Networks interview Doug Fridsma, Director of the Office of Science and Technology in the Office of the National Coordinator for Health Information Technology (ONCHIT), said, “Data standards are an important part of Meaningful Use, but so is flexibility.” Maybe we can have the best of both worlds.

 

Structured vs. unstructured

 

Structured data is the easiest type of data to capture and categorize in a database. Accounting data, an example of structured data, includes numbers with a specific value in a particular column. Structured data in healthcare would be a lab value or patient demographic data that is entered from a dropdown box or radio button. Structured data is consistent and resides in pre-defined fields within the record.

 

Unstructured data is unorganized, may have irregularities or be ambiguous, and is typically “text-heavy.” A prime example of unstructured data in Health IT is a paragraph about the history of present illness. It is hard to condense patient complaints or physician assessments into a series of checkboxes and radio buttons and yet there is great value in analyzing patient information without having to manage free text. It is helpful to know if there is pertinent information about a cardiac procedure in the physician note even if it is not listed in the structured data of the problem list.

 

It is estimated that 95% of the worlds’ data is unstructured, so healthcare is not unusual. Today e-mails, text messages, Word documents, videos, and pictures are all unstructured data. The document or file has structure, but the content or text within it is unstructured. Solutions are being developed to parse unstructured data into patterns that improve the value and usability of the data. Natural Language Processing tools and other data-mining tools create opportunity to glean structure from free text.

 

Standards vs. story

 

ONCHIT is working to create standards around structured data capture that are essential to interoperability of healthcare systems, but that has not helped providers at the point of care. Standards for data capture benefit from structured data, but structured data capture at the site of data entry is not a good partner to User Centered Design where providers need the flexibility of unstructured data capture or free text.

 

In his interview, Doug Fridsma says that healthcare is about the human condition and the human condition is a story, an unstructured narrative. As Stuart Lewis puts it, “patients don’t speak template.”¹ Or as Fridsma says, “Just because we can structure, doesn’t mean we should.”

 

Joy of use for EHRs may come when the technology values both structured and unstructured data. As a provider I would like to capture the patient story to document an individual, unique event. I would also like to collect and analyze every piece of data from that patient encounter that can be used to track, monitor, and provide the best quality of care for that patient.

 

We need a digital world where structured and unstructured data can coexist. We need standards and interoperability side by side with narrative and flexibility. There is no other way to capture the human condition.

 

The Sky Was
by e.e. cummings

 

1. Lewis, S (2011). Brave new EMR. Annals of Internal Medicine. 154: 368-369.