Sintellisys has built a range of different AI applications, from image recognition to natural language processing. Generally, we collect a number of examples. Split them into, say 90% for training a machine learning model and 10% for testing it. When the model passes the test, we are done. However, the reality is:

• You need a VERY large number of examples to train the model for real world use. Still, you are not guaranteed 100% consistent results. We have implemented a machine learning model to extract information from shipping instruction documents. Currently, the model has predicted more than 9000 different formats of this document type based on close to 100,000 examples trained. See below illustrations of the different formats. We have models for Master & House Bill of Lading including other key freight documents.  To overcome this issue, many people leverage transfer learning. Use pre-trained models (like ours) as a basis to build your own.
• You need a good process to tell which models work and which won’t in the real world. We have created an example below. We have crawled a few articles online and sent them through different NLP models to analyse the sentiment. Many of these capable models produce different results. If you don’t know any better, your team will choose one randomly. To overcome this issue, we need to run these models through stress tests designed to highlight specific differences in their performances to help us decide which fits our needs better. Some models are good at something but very often are bad at something else. It might not always be possible to train a single model for everything. We use a mix of models in our applications to achieve desired outcomes.