Megaputer Blog | Мегапьютер

Manual validation of entities in Entity Extraction

15/05/2018Jeff Palan, , Text Analytics

Algorithmic extraction of entities from text is a powerful tool and a core feature of PolyAnalyst, but it can be difficult to get results with absolutely no false positives. Previously, the user would have to reword the algorithm little by little to remove each false positive, or find some other way to filter the results. Fortunately PolyAnalyst supports manual validation of entity extraction results. In effect this means that a user can mark each extraction as being invalid, valid or null, and only those that are not invalid will make their way into the actual dataset. This has a number of advantages beyond not needing to write additional Extended Pattern Definition Language (XPDL) code. For one, it makes it easier for multiple users to collaborate and manipulate the results. Additionally, other users can see what types of extractions are being marked as invalid, which may give them a deeper understanding…

Why We Use Dependency Parsing

15/05/2018Rebecca Hale, Featured, Text Analytics

For beginners, text analysis often begins with a simple keyword search. Analysts can start by searching for “great service”. They may then begin adding other searches like “service was great” or “service is so great”. This, of course, is a very rudimentary approach that will miss a lot of relevant comments, so users who want to get more results quickly will advance to proximity searches. For example, they may search for “great” within three words of “service”. While some may use word lists (“awesome”, “good”, or “excellent” service) or parts of speech (adjectives near “service”) to improve results, most text analysis systems stop right there. For advanced text searching, however, you need to have information about word relationships. For example, “the insured driver hit a bus” and “a bus hit an insured driver” are two very different events that might help a text analysis system determine who was at fault…

Prepping the XPDL Seminar Series

09/05/2018Rebecca Hale, Featured, Text Analytics

Registration for the XPDL webinar series opens June 1, 2018. Following numerous requests from our customers for more XPDL training material, I’ve been developing an XPDL seminar series. We receive a lot of questions about how best to use XPDL to extract information from text, because it is such an important task for text analysis. Data analysts, survey conductors, customer experience managers… anyone working with text data needs a way to extract the information they need. After testing these presentations on some of our analysts, we’ll begin preparing brand-new training modules for our customers. This week, we had a lot of great questions from our analysts, making sure they understand the more complex rule types. Practicing the series at our weekly meetings has given us a lot of great tips and improvements— for example, we learned that users want lots of practical text examples, showing exactly the effect of the…