Train HighQ AI classifiers
To get the most out of the HighQ AI engine, you need to actively train it. Training means that you help the engine to understand what types of files you are uploading (i.e. helping it understand and recognise a type of document).
When you enable AI training and assign a classifier to files, the files in the selected folder are mapped to the selected classifier name in the AI hub. The AI Hub automatically uses this mapping to train a new File Classifier. The new file classifier is then available as a classification type across the whole HighQ instance (unless you create a 'Private' classifier, which is only used in a single site). If the HighQ AI finds any examples of this file anywhere in the system, it will label it with the new classification.
note
In addition to classifiers, you can train your own clauses, or re-train HighQ-provided clauses.
The information below will help you train your AI engines to better understand file metadata.
note
We highly recommend you only turn on AI training for only one or two sites. This makes it easier to administer training and to manage the files that are used as training examples in a single place, rather than have them scattered across many sites.
Enabling AI engine training
To enable AI engine training, navigate to
Admin
> AI Hub>Configure
and scroll down to Advanced settings
:
Select the
Enable AI engine training
checkbox and select Save
.note
This training is for the HighQ AI engine. To train other AI engines, please refer to how to use their training capabilities on their platforms. It is not currently possible to train other AI engines (such as Kira) using the HighQ AI Hub.
Manage your AI training
Files that belong to a classification can be used to train the HighQ AI so it is better equipped to recognise similar files.
Training HighQ AI from the Files module (select multiple files)
You can train the AI with one or more files on your site. This allows you to target a set of new or relevant files for analysis.
Open the
Files
module.Select the files to use to train a classifier. Click
Action
, Assign AI
, then select Train HighQ AI
.
note
The
Assign AI
> Train HighQ AI
and Assign contract template
options are not available if a folder is selected.Training HighQ AI from the Files module (select the contents of a folder)
You can train the HighQ AI with the contents of a folder. This allows you to use every file in a folder as an example for training.
Select the folder and click
More actions
> Edit details
:
In the
Edit folder
screen, navigate to the Settings
tab and click Select...
(under the AI Training
heading) to open the Classifiers screen (see below).
Selecting an existing classification
The
Classifiers
window displays the names of all available classifiers, with a short description and a tag to show the language used by the classifier.
Select the classifier you want to associate with the folder and select
Done
.note
See Classifiers provided by HighQ AI for a description of the classifiers packaged with the HighQ AI engine.
Adding new classifiers
If no classifier matches your needs, you can create your own, using your own documents for analysis.
Select
New classifier
:
The
New classifier
screen opens:
Enter a name and description for your classifier and also choose the scope of the classifier: if you choose a private scope, the AI will only search for documents in that specific site and not instance-wide.
You have the option to create a new classifier in English or German. Please contact your account manager should you wish to add classifiers in German.
Once you have entered the details, select
Add
to save your new classifier. Select your new classifier in the Classifiers window and select Done
.Training the AI with new files
If you assigned a folder to train the AI, any files uploaded to the folder will qualify as 'training files' for the assigned classification in the AIML (AI Modelling Language) database. This means that the AI understands that the type of document uploaded to the folder is a certain type, and will learn to identify this type if it sees it again.
note
When a new classification is created or a new version of an existing classification is added, the system administrator should check the list of HighQ AI models to ensure that the best AI versions are deployed to their sites. They do this in the System Administration for Document Analysis, by managing the HighQ AI.
Training the AI
Once you have set your classifier for your folder, you can start to train the AI. You can train the AI from the
Files
module, an iSheet
or the Admin
module.Manage AI training in the Admin module
Navigate to
Admin
> AI Hub
> Configure
:
In the HighQ section, click
More actions
> Manage Training
:
It shows information about each classifier and the current version used. Select
Train
to update the trained version with any new files or folders that the classifier has been assigned to.Click
Choose version
to see versions that have been trained - in this screen, you can revert to an older version of the trained classifier (click More actions
> Rollback to here
), or delete a version (click More actions
> Delete
).note
A system administrator can use rollback to delete multiple versions at the same time.
note
Only a system administrator can delete a version created by another user.
Select
Done
and Save
if you made any changes.Show training examples for HighQ AI training
As of version 5.6, the train button for a classifier will not be enabled until enough training examples have been provided.
This makes it easier to see how many more examples are required before a new version of the AI model can be trained.
To access this, within your site, select
Site admin
:
The
Site admin
screen opens. Within Site admin
, navigate to AI Hub
> Configure
:
The list of all available AI will be shown in the
AI Hub
. Navigate to HighQ AI
and select More actions
. The new Manage training
option opens:
Select
Manage training
. The Manage - HighQ AI
screen opens. Within the Manage - HighQ AI
screen, click the Classifier
tab:
You can now see many more examples are required to train a new version of the AI model.
note
Please note that due to our "Examples cumulative" enhancement, all newly trained or out-of-the-box AI models will show the label "Add 1 example" to train, as they can be trained by adding a single new example
note
Please also note that when you have trained a new classifier or clause, you must enable it on any site that you want to use. It is not automatically enabled, in case you are not ready to make use of that classifier or you do not wish to use it on other sites
Minimum number of example documents
If you do not have enough examples of documents to train the AI, you will receive the following message:
Ideally, you should have at least a minimum of
50 new
files in folders where the same classifier is applied, instance-wide, however, you only need one example to train a new version of a model, as the new version will be trained on the original examples plus any additional (new) examples. Once you have checked that there is at least one file of the correct classification (more is better), navigate back to the admin panel and click More actions
> Train AI
.Using the information
Once you have set up your AI engine and have trained it, you can insert it into a file metadata iSheet, and use the data in data visualisation.