The Indexing Subsystem.

The first indexing dispatcher to register with the Name Service becomes active. The Name Service guarantees that only one indexing dispatcher succeeds. The backup dispatchers monitor the active one, stepping in to take over if it dies. Periodically, the backup indexers connect to the master and ping it.

Incoming operations are rewritten to basic I/O operations

e.g., invalidate file X @ position Y.

e.g., blacklist doc X in index Y_Z.

Index and Search nodes are arranged in a matrix. Each column holds a subset of the content to distribute the load. This allows scale for volume and indexing performance.

Every row holds a replica of the full content of the index which enables an increased number of queries per second.
Rows able to replay operations internally to re-establish synchronization after downtime. Multiple indexing rows add indexer resilience. Rows are in sync both with respect to content and indices.

The column master is elected at run-time. If the master fails, a new master is elected.

The column master synchronizes content operations and indexing to all of its backups.

During indexing of new content there will be 2 indices,

1 that houses the active index against which the search service searches. And.

1 that houses the incremenatal index that is being built from the newly added content. This is added to active index in batches.

This needs to be as large as the active index to allow for scenarios when we need to reset the entire index.

The active index will be divided into 3 partitions - 0, 1 and 2. These vary in ascending size from 25% to 50% to 100%. As content is added, the indexer service will send it to the smallest of the partitions, partition 0. When this reaches maximum capacity the index dispatcher is copied to the next largest partition 1. And so on.

The advantage of this, given that all new content is send to partition 0, is that we can now quickly re-build the smaller index partition.

The column master ensures that column contents are always in sync. By copying first the FIXML to all backup nodes and then copying the index when it has been created on the master.

A new index is not activated until the index is copied to all rows. That is, the new index is not activated until all columns have the same content.

The master indexer is the only one to receive operations and initiate indexing, synchronizing.

Currently, all search controllers connect to the master indexer for guidance. Only the master indexer builds indices. Backups only store the FIXML. If the master fails the crown is passed to a backup, which assumes the master role.

A failover will require the index to be rebuilt from FIXML. This may take several minutes to a couple of hours depending on the volume of content. During this time indexing of new content is not possible. However, search is uninterrupted.

Processing subsystem.

Multiple processor servers provide resilience and throughput.
Multiple content distributors provide for resilience and throughput.