Source: texcla/models/token_model.py#L0


TokenModelFactory


TokenModelFactory.__init__

__init__(self, num_classes, token_index, max_tokens, embedding_type="glove.6B.100d", \
    embedding_dims=100, embedding_path=None)

Creates a TokenModelFactory instance for building various models that operate over (samples, max_tokens) input. The token can be character, word or any other elementary token.

Args:

  • num_classes: The number of output classes.
  • token_index: The dictionary of token and its corresponding integer index value.
  • max_tokens: The max number of tokens across all documents. This can be set to None for models that allow different word lengths per mini-batch.
  • embedding_type: The embedding type to use. Set to None to use random embeddings. (Default value: 'glove.6B.100d')
  • embedding_dims: The number of embedding dims to use for representing a word. This argument will be ignored when embedding_type is set. (Default value: 100)