What Is Natural Language Processing?

The United States faces a looming healthcare worker shortage as doctors age out and retire at just the same time so many other Americans are doing the same. The Association of American Medical Colleges projects a physician shortfall in the US between 37,000 and 124,000 by 2033, while Mercer predicts a shortage of nurses and other healthcare workers in the hundreds of thousands by 2026.

What does all this have to do with natural language processing? Nearly three-quarters of physicians cite unwieldy clerical requirements as a source of job dissatisfaction. According to Tashfeen Suleman, CEO of CloudMedx, many doctors report “spending two hours or more completing documentation outside of work hours each day.” It’s enough to drive some healthcare professionals out of the field prematurely.

CloudMedx is among the players working on a solution. The company is developing a streamlined medical records platform that incorporates natural language processing via speech recognition; the same technology that powers digital assistants like Alexa and Siri has the potential to take on the most tedious and burdensome parts of healthcare documentation, substituting automation for tedious repetition and summarization for reading through old charts. As Suleman explains, “Healthcare providers should be able to speak into their phones or other devices and have the information they provide automatically entered into healthcare records through natural language processing.”

Healthcare is just one of many fields in which natural language processing can streamline processes and improve efficiency. What is natural language processing? This article explores that question and also discusses:

• How is natural language processing used?
• Benefits and drawbacks of natural language processing
• Natural language processing careers
• What degree do you need to work in natural language processing?

What is natural language processing?

Machine learning, a branch of artificial intelligence (AI), fuels natural language processing (NLP). The technology allows computer systems to complete such NLP tasks as decoding the meaning and structure of text, understanding voice commands, and responding appropriately. Doing so requires a combination of computational linguistics (modeling language with computers) and models drawn from statistics, machine learning, and deep learning. As the SAS website notes, “Human language is astoundingly complex and diverse… [NLP] helps resolve ambiguity in language and adds useful numeric structure to the data for many downstream applications, such as speech recognition or text analytics.”

NLP is Siri, Alexa, and so much more. If you’ve dictated a text message, asked your car to switch from FM to Bluetooth, used search engines, or answered questions on the phone from an automated customer-service assistant, you’ve encountered NLP.

How is natural language processing used?

NLP helps sort your email inbox, streamline your Google search, and autocomplete your tweets. You may use NLP algorithms to communicate with someone who doesn’t speak English or ask a GPS app for directions. NLP can help a researcher or corporation analyze data.

Tableau, an analytics software company, cites eight key NLP processes:

• Data analysis
• Digital phone calls
• Email filters
• Language translation
• Predictive text
• Search results
• Smart assistants
• Text analytics

Benefits and drawbacks of natural language processing

Natural language processing has made it far easier to interact with technology using only human language. NLP systems have learned to read and interpret data, gathering important information in new and faster ways and promising relief from tedious, time-consuming tasks. But languages are complex, and NLP algorithms never perform flawlessly.

Here’s a look at some advantages and disadvantages of natural language processing.

Chatting with machines

Anyone who uses modern technology has experienced NLP, from smart speakers to chatbots. NLP allows people to trigger sophisticated technological processes using everyday language. “The next enhancement for these applications is question answering, the ability to respond to our questions—anticipated or not—with relevant and helpful answers in their own words,” IBM reports.

Languages are complicated

Some words have many divergent meanings. When you encounter the word “down,” for example, does it indicate a physical direction or a state of mind? Humans have a lifetime of experience parsing such semantics, but the process doesn’t come easily to computers. Throw in curveballs such as sarcasm and tone and things can really get tricky. Programmers must put a great deal of time and effort into giving NLP systems the tools to make such distinctions.

Humans and machines alike find some kinds of texts more challenging than others. While NLP can readily handle basic commands, engineers have long strived for higher goals in natural language understanding. It’s one thing to read and comprehend a recipe, quite another to read and comprehend a poem or a mathematical word problem.

Analytics Insight, a website that covers data-driven technologies, lists a series of hurdles to creating a fully functioning NLP:

• Development time
• Innate biases
• Keeping a conversation moving
• Language differences
• Misspellings
• Phrases with multiple intentions
• Phrasing ambiguities
• Training data
• Uncertainty and false positives
• Words with multiple meanings

Digging through data

Suleman asserts that computers should do the heavy lifting for healthcare providers accessing a patient’s medical history. NLP can provide sophisticated data analysis in fields such as finance and marketing as well, boiling down large quantities of data and highlighting important details. “Whether you’re doing research on a company or mining some vast data sets on a country you’re interested in that no single human being could ever read, you start to need those same types of technologies,” says Georg Kucsko, an MIT lecturer and head of machine learning research and development at Kensho, a company that specializes in AI and machine learning.

On the surface, finance seems much more oriented toward numbers than words, but NLP can play a crucial role in the field. As Mikey Shulman, Kensho’s head of machine learning, explains: “A company will release its report in the morning, and it will say, ‘Our earnings per share were $1.12.’ That’s text. By the time that data makes its way into a database of a data provider where you can get it in a structured way, you’ve lost your edge. Hours have passed.” NLP can provide transcriptions in minutes, providing a competitive advantage.

In-depth analysis

Technology has reached a stage where it can not only plow through vast quantities of text but also comprehend and report on it. NLP has the potential to revolutionize healthcare, marketing, finance, government, and other fields.

Tableau gives this example from the world of retail sales: “While sentiment analysis sounds daunting to brands–especially if they have a large customer base–a tool using NLP will typically scour customer interactions, such as social media comments or reviews, or even brand name mentions to see what’s being said” to “help brands determine how well a marketing campaign is doing or monitor trending customer issues before they decide how to respond.”

Accuracy not guaranteed

Natural language processing has progressed dramatically in recent years, providing services that few anticipated at the beginning of the century. Even so, NLP still has its limitations. “There is always a possibility of errors in predictions and results that need to be taken into account,” advises Encora, a company that offers differentiated innovation services and software engineering solutions.

Some of the hurdles listed above from Analytics Insight also deal with NLP accuracy. False positives, for example, which occur “when an NLP detects a term that should be intelligible and/or addressable but can’t be adequately replied to… The idea is to create an NLP system that can identify its own limits and clear up uncertainty using questions or hints.”

Bridging language divides

Language-learning apps like Duolingo and Babbel help people broaden their knowledge and prepare for communication with speakers of others languages, such as for international work situations or vacation travel. Machine translation tools like Google Translate and Microsoft Translator serve a more immediate purpose, translating written text and verbal conversations on the spot. NLP has played a crucial role in the development of such apps.

Natural language processing careers

It takes plenty of human minds to get machines to use language effectively, and the expanding NLP sphere has opened up many career options. Recent job listings include:

• NLP engineer/scientist
• NLP processing specialist
• NLP data scientist
• Natural language generation developer
• NLP research assistant
• Machine learning and LPR researcher
• Customer experience chat room operator
• Machine learning engineer
• NLP engineer
• Conversational AI designer
• Language technologist
• Educational data scientist
• Applied quantitative researcher
• Linguist

The average annual salary for NLP-related jobs is between $75,000 and $100,000, according to IT Chronicles.

What degree do you need to work in natural language processing?

Anyone interested in an NLP-related career should consider a degree in computer science, computational linguistics, mathematics, or a related field. Looking to expand your career options and maximize your income potential? Consider a master’s degree from a school such as Southern Methodist University or Case Western Reserve University. Their computer science programs include a variety of specializations, including cybersecurity, software engineering, and artificial intelligence and machine learning.

.

Questions or feedback? Email editor@noodle.com