models.lsimodel – Latent Semantic Indexing — gensim documentation

### Navigation

- index
- modules |
- next |
- previous |
- Gensim home|
- API reference|
- Tutorials »
- API Reference »

# `models.lsimodel` – Latent Semantic Indexing

Module for Latent Semantic Indexing.

class `gensim.models.lsimodel.` `LsiModel`(corpus=None, id2word=None, numTopics=200, chunks=None, decay=1.0, serial_only=None)

  Objects of this class allow building and maintaining a model for Latent Semantic Indexing (also known as Latent Semantic Analysis).

  The main methods are:

  1. constructor, which initializes the projection into latent topics space,
  2. the `[]` method, which returns representation of any input document in the latent space,
  3. the addDocuments() method, which allows for incrementally updating the model with new documents.

  Model persistency is achieved via its load/save methods.

  numTopics is the number of requested factors (latent dimensions).

  After the model has been trained, you can estimate topics for an arbitrary, unseen document, using the `topics = self[document]` dictionary notation. You can also add new training documents, with `self.addDocuments`, so that training can be stopped and resumed at any time, and the LSI transformation is available at any point.

  If you specify a corpus, it will be used to train the model. See the method addDocuments for a description of the chunks and decay parameters.

  The algorithm will automatically try to find active nodes on other computers and run in a distributed manner; if this fails, it falls back to serial mode (single core). To suppress distributed computing, set the serial_only constructor parameter to True.

  Example:

      >>> lsi = LsiModel(corpus, numTopics = 10)
      >>> print lsi[doc_tfidf]
      >>> lsi.addDocuments(corpus2) # update LSI on additional documents
      >>> print lsi[doc_tfidf]

  `addDocuments`(corpus, chunks=None, decay=None)

    Update singular value decomposition factors to take into account a new corpus of documents.

    Training proceeds in chunks of chunks documents at a time. If the distributed mode is on, each chunk is sent to a different worker/computer. Size of chunks is a tradeoff between increased speed (bigger chunks) vs. lower memory footprint (smaller chunks).

    Setting decay < 1.0 causes re-orientation towards new data trends in the input document stream, by giving less emphasis to old observations. This allows SVD to gradually “forget” old observations and give more preference to new ones. The decay is applied once after every chunks documents.

  classmethod `load`(fname)
    Load a previously saved object from file (also see save).
  `printDebug`(numTopics=5, numWords=10)

    Print (to log) the most salient words of the first numTopics topics.

    Unlike printTopics(), this looks for words that are significant for a particular topic and not for others. This should result in a more human-interpretable description of topics.

  `printTopic`(topicNo, topN=10)

    Return a specified topic (=left singular vector), 0 <= topicNo < self.numTopics, as string.

    Return only the topN words which contribute the most to the direction of the topic (both negative and positive).

        >>> lsimodel.printTopic(10, topN = 5)
        '-0.340 * "category" + 0.298 * "$M$" + 0.183 * "algebra" + -0.174 * "functor" + -0.168 * "operator"'

  `save`(fname)
    Save the object to file via pickling (also see load).

`gensim.models.lsimodel.` `clipSpectrum`(s, k)
  Given singular values s, return how many factors should be kept to avoid storing spurious values. The returned value is at most k.

`gensim.models.lsimodel.` `iterSvd`(corpus, numTerms, numFactors, numIter=200, initRate=None, convergence=0.0001)

  Perform iterative Singular Value Decomposition on a streaming corpus, returning numFactors greatest factors U,S,V^T (ie., not necessarily the full spectrum).

  The parameters numIter (maximum number of iterations) and initRate (gradient descent step size) guide convergency of the algorithm.

  The algorithm performs at most numFactors*numIters passes over the corpus.

  See Genevieve Gorrell: Generalized Hebbian Algorithm for Incremental Singular Value Decomposition in Natural Language Processing. EACL 2006.

  Use of this function is deprecated; although it works, it is several orders of magnitude slower than our own, direct (non-stochastic) version (which operates in a single pass, too, and can be distributed).

  I keep this function here purely for backup reasons.

`gensim.models.lsimodel.` `svdUpdate`(U, S, V, a, b)

  Update SVD of an (m x n) matrix X = U * S * V^T so that [X + a * b^T] = U’ * S’ * V’^T and return U’, S’, V’.

  The original matrix X is not needed at all, so this function implements flexible online updates to an existing decomposition.

  a and b are (m, 1) and (n, 1) matrices.

  You can set V to None if you’re not interested in the right singular vectors. In that case, the returned V’ will also be None (saves memory).

  This is the rank-1 update as described in Brand, 2006: Fast low-rank modifications of the thin singular value decomposition, but without separating the basis from rotations.

### Navigation

- index
- modules |
- next |
- previous |
- Gensim home|
- API reference|
- Tutorials »
- API Reference »