#include <abstractrecogniser.h>
Inheritance diagram for AbstractRecogniser:

Signals | |
| void | trainingProgressed (int progress) |
Public Member Functions | |
| AbstractRecogniser (JavaVM *jvm, DigestDbModel *digestDbModel, QObject *parent=0) | |
| virtual | ~AbstractRecogniser () |
| virtual QByteArray | key () const=0 |
| virtual QString | title () const=0 |
| virtual QString | description () const=0 |
| JavaVM * | jvm () const |
| DigestDbModel * | digestDbModel () const |
| const QSet< int > & | trainingSet () const |
| const QList< QByteArray > & | orderedFeatures () const |
| const QString & | modelFilePath () const |
| const QHash< QString, QVariant > & | params () const |
| virtual QHash< QString, QVariant > | defaultParams () const=0 |
| int | currentTrainingProgress () const |
| virtual bool | loadRecord (const DTrainedRecogRecord &record) |
| virtual bool | train (const QSet< int > &trainingSet, const QList< QByteArray > &orderedFeatures, const QString &modelFilePath, const QHash< QString, QVariant > ¶ms) |
| virtual ClassProbabilities | classify (const StrokeList &strokes) |
Protected Member Functions | |
| virtual bool | initTraining (const QList< QByteArray > &featureKeys, const QHash< QString, QVariant > ¶ms)=0 |
| virtual bool | examineSample (const FeatureVec &featureVec, const QSet< int > &classes)=0 |
| virtual bool | finaliseTraining ()=0 |
| virtual bool | writeModelFile (const QString &fileName)=0 |
| virtual bool | readModelFile (const QString &fileName)=0 |
| virtual Stroke | flatten (const StrokeList &strokes) |
| virtual ClassProbabilities | classify (const FeatureVec &featureVec)=0 |
| virtual FeatureVec | extractFeatures (const Stroke &stroke) |
Once an instance of this class has been created, the readModelFile() method will be called, followed by classifyGestureImp() a multitude of times.
In order to make a recogniser implementation visible to the user in the application, its class must be added to the RecogniserFactory.
See also: AbstractRecogniserTrainer and RecogniserFactory.
TODO: FIX DOC!
Once an instance of this class has been created, the train() method will be called. The training is implemented in the form of a "template method". (By the way, a Template Method is where we "Define the skeleton of an algorithm in an operation, deferring some steps to subclasses. Template Method lets subclasses redefine certain steps of an algorithm without changing the algorithm's structure." - Design Patterns, Gamma et al.) When train() is called, the following pattern of calls will be made:
initTraining() examineSample() examineSample() examineSample() ... examineSample() finalizeTraining() writeModelFile()
In order to make a recogniser trainer implementation visible to the user in the application, its class must be added to the RecogniserTrainerFactory.
See also: AbstractRecogniser and RecogniserTrainerFactory.
| AbstractRecogniser::AbstractRecogniser | ( | JavaVM * | jvm, | |
| DigestDbModel * | digestDbModel, | |||
| QObject * | parent = 0 | |||
| ) |
The jvm is required, as it need to instantiate feature extractors, which may be written in Java - TODO: remove this - make the factory map the JVM with Java features automatically!
| AbstractRecogniser::~AbstractRecogniser | ( | ) | [virtual] |
| QByteArray AbstractRecogniser::key | ( | ) | const [pure virtual] |
Returns the recogniser's "key". This is used for identifying recognisers for purposes such as recording the type of a given trained recogniser and for use in "factories".
This string should only contain lower-case characters. All string matching performed with it will be case-insensitive.
Note: This does not return a (const) reference for the sake of "wrappers" (e.g. JavaRecogniserWrapper and StlRecogniserWrapper), which may have to convert the string from a different structure, usually with the use of a temporary variable. Qt's implicit sharing will help performance.
Implemented in JavaRecogniserWrapper, StlRecogniserWrapper, and LinearRecogniser.
| QString AbstractRecogniser::title | ( | ) | const [pure virtual] |
Returns the recogniser's "title". This is used in the application's user interface to label the recogniser in a human friendly / readable manner.
Note: This does not return a (const) reference for the sake of "wrappers" (e.g. JavaRecogniserWrapper and StlRecogniserWrapper), which may have to convert the string from a different structure, usually with the use of a temporary variable. Qt's implicit sharing will help performance.
Implemented in JavaRecogniserWrapper, StlRecogniserWrapper, and LinearRecogniser.
| QString AbstractRecogniser::description | ( | ) | const [pure virtual] |
Returns the recogniser's "description". This is used in the application's user interface to provide more information to the user about the recogniser.
A subset of HTML may be used, such as to make the text bold and change its colour, size and alignment. Please see the Qt documentation for more details on its HTML support.
Note: This does not return a (const) reference for the sake of "wrappers" (e.g. JavaRecogniserWrapper and StlRecogniserWrapper), which may have to convert the string from a different structure, usually with the use of a temporary variable. Qt's implicit sharing will help performance.
Implemented in JavaRecogniserWrapper, StlRecogniserWrapper, and LinearRecogniser.
| JavaVM * AbstractRecogniser::jvm | ( | ) | const |
| DigestDbModel * AbstractRecogniser::digestDbModel | ( | ) | const |
| const QSet< int > & AbstractRecogniser::trainingSet | ( | ) | const |
| const QList< QByteArray > & AbstractRecogniser::orderedFeatures | ( | ) | const |
| const QString & AbstractRecogniser::modelFilePath | ( | ) | const |
TODO: re-write this doc
Returns the file name for the output model file, including the path (e.g. "/Users/bob/Library/Digest/linear_recog52.model").
This should be used by the implementation of writeModelFile().
Note: The reason that "Path" is part of the method name is because an outputPath() method exists. Thus, if this method were called outputFilename(), then it might appear that it did not include the path and would need to be concatenated with outputPath().
| const QHash< QString, QVariant > & AbstractRecogniser::params | ( | ) | const |
| virtual QHash<QString, QVariant> AbstractRecogniser::defaultParams | ( | ) | const [pure virtual] |
Implemented in JavaRecogniserWrapper, StlRecogniserWrapper, and LinearRecogniser.
| int AbstractRecogniser::currentTrainingProgress | ( | ) | const |
Returnns the current training progress - that is, a value between 0 and trainingSet().count().
| bool AbstractRecogniser::loadRecord | ( | const DTrainedRecogRecord & | record | ) | [virtual] |
| bool AbstractRecogniser::train | ( | const QSet< int > & | trainingSet, | |
| const QList< QByteArray > & | orderedFeatures, | |||
| const QString & | modelFilePath, | |||
| const QHash< QString, QVariant > & | params | |||
| ) | [virtual] |
TODO: re-write me
Trains a new recogniser, using the provided gestureIds, featureKeys and writes a model file out to outputFilePath.
The outputFilePath is the file name for the output model file, including the path (e.g. "/Users/bob/Library/Digest/linear_recog52.model"). See also outputFilePath() and outputPath().
| ClassProbabilities AbstractRecogniser::classify | ( | const StrokeList & | strokes | ) | [virtual] |
| void AbstractRecogniser::trainingProgressed | ( | int | progress | ) | [signal] |
| bool AbstractRecogniser::initTraining | ( | const QList< QByteArray > & | featureKeys, | |
| const QHash< QString, QVariant > & | params | |||
| ) | [protected, pure virtual] |
This is used primarily for ensuring that the trainer's state is ready for training, as train() can be called multiple times.
The featureKeys are provided here, as they may be required to initialise one or more data structures.
Implemented in JavaRecogniserWrapper, and StlRecogniserWrapper.
| AbstractRecogniser::examineSample | ( | const FeatureVec & | featureVec, | |
| const QSet< int > & | classes | |||
| ) | [protected, pure virtual] |
This is called every time a sample/example is to be processed (or logged, where processing could be delayed until finaliseTraining() is called).
The featureVec is the feature vector that has been extracted from the sample.
Note: The FeatureResultT typedef is currently a double.
The featureVec can be converted to an STL vector:
vector<FeatureResultT> v = featureVec.toStdVector()
Note: If you wan't the stoke data for some reason, then store a copy of what you return from flatten().
Implemented in JavaRecogniserWrapper, and StlRecogniserWrapper.
| bool AbstractRecogniser::finaliseTraining | ( | ) | [protected, pure virtual] |
This is used primarily for building the model if that wasn't (completely) done by examineSample().
This should also be used to free any allocated memory and to clear the state, remembering that train() may be called multiple times.
Implemented in JavaRecogniserWrapper, StlRecogniserWrapper, and LinearRecogniser.
| bool AbstractRecogniser::writeModelFile | ( | const QString & | fileName | ) | [protected, pure virtual] |
This must be implemented and must write a file to fileName. The fileName is the file name for the output model file, including the path (e.g. "/Users/bob/Library/Digest/linear_recog52.model"). If the model must be written as multiple files, then fileName should be the master/index/reference file (which will be passed to the corresponding AbstractRecogniser::readModelFile() method) and any support files should be placed along side it in the same path.
There are no restrictions on the file being binary or ascii, nor on its contents.
See also fileName()
Implemented in JavaRecogniserWrapper, and StlRecogniserWrapper.
| bool AbstractRecogniser::readModelFile | ( | const QString & | fileName | ) | [protected, pure virtual] |
This method is called during initialisation, as to load in the recogniser specific model file that was created by the corresponding AbstractRecogniserTrainer::writeModelFile() method. Thus, this method will be called only once.
The fileName is the file name of the model file, including the path (e.g. "/Users/bob/Library/Digest/linear_recog52.model").
Because some recognisers may not require a training phase, or may not use a GestureLab managed model file for some other reason, this method could be implemented to just return true.
There are no restrictions on the file being binary or ascii, nor on its contents.
Implemented in JavaRecogniserWrapper, StlRecogniserWrapper, and LinearRecogniser.
| Stroke AbstractRecogniser::flatten | ( | const StrokeList & | strokes | ) | [protected, virtual] |
This is used to merge/join/flattern a (potentially) multi-stroke gesture before feature extraction is performed (and thus examination or classification).
If any filtering needs to be performed on the stroke(s), then it should be done by this method. This includes smoothing the stroke(s) or even reducing the number of points in them.
This default implementation simply joins all of the strokes together. It does not join strokes together based on the location of their start and end points.
Note: This is non-const, as the recogniser may need to examine the stroke's structure and record information about it, ready for use by either classify() or examineSample().
Reimplemented in JavaRecogniserWrapper, and StlRecogniserWrapper.
| ClassProbabilities AbstractRecogniser::classify | ( | const FeatureVec & | featureVec | ) | [protected, pure virtual] |
This is called every time a given gesture is to be classified / recognised.
The featureVec is the feature vector that has been extracted from the gesture. Hence, the gesture parameter may not need to be used at all.
Note: The FeatureResultT typedef is currently a double.
The featureVec can be converted to an STL vector:
vector<FeatureResultT> v = featureVec.toStdVector()
Note: If you wan't the stoke data for some reason, then store a copy of what you return from flatten().
The ClassProbabilities results to be returned is simply a hash-table that maps gesture classes to the recogniser's "confidence" that the given gesture belongs to the given class. The sum of the confidence values across all the classes (those known and trained-on) should be equal to 1.0. Thus, scaling may need to be performed before returning the results.
Note: The ClassProbabilities typedef is a QHash (hash-table - see the Qt doc), mapping the class ID ClassIndexT (currently an int) to the confidence ClassProbabilityT (currently a double).
It does not matter if there are fewer classes in the returned ClassProbabilities than there exists in the current database (new classes may be added to or removed from the database at any given time). Thus, a recogniser could simply return a hash that only has one item in it, which would have to have a confidence of 1.0.
Example - A recogniser has trained on three classes: X, Y and Z. If the recogniser has a confidence of 70% that the given gesture was an X, 20% that it was a Y and 10% that it was a Z, then the following would return the corresponding ClassProbabilities:
ClassProbabilities results; results[X] = 0.7; results[Y] = 0.2; results[Z] = 0.1; return results;
Implemented in JavaRecogniserWrapper, and StlRecogniserWrapper.
| FeatureVec AbstractRecogniser::extractFeatures | ( | const Stroke & | stroke | ) | [protected, virtual] |
Warning: You shouldn't need to call this directly, nor re-implement it. However, for some reason you want to - hence, it's allowed.
1.5.2