Types of distributed deployments
You can customize your Splunk Enterprise deployment in a wide variety of ways. There are, however, some fundamental groupings into which most deployments fall. This topic discusses some key characteristics and considerations for various types of deployments.
Key factors that determine the type of deployment
These are the main issues that determine the type and scale of your deployment:
- Indexing volume. How much data are planning to index on a daily basis? To handle increased indexing loads, you might need multiple indexers.
- Number and type of searches. How frequently will you be running searches, either scheduled or ad hoc? What type of searches will you be running? Large numbers of searches, or frequent process-intensive searches, can tax both search head and indexer resources.
- Number of concurrent users. How many users will be viewing dashboards or running searches concurrently? To handle increased numbers of users, you might need to add search heads, usually through a search head cluster.
- Data fidelity requirements. If you must ensure that the system never loses data, an indexer cluster is a necessity.
- Availability requirements. What requirements do you have for data availability? If you must always have access to the full set of data, you might need to deploy both an indexer cluster and a search head cluster.
- Disaster recovery requirements. How important is fast disaster recovery? A multisite indexer cluster can ensure fast failover to identical sets of data across geographically dispersed data centers.
Other considerations can also enter into your overall deployment plans, such as security requirements and the location of the data.
Representative deployment types
These are some of the main types of deployments, based on size:
- Departmental. A single instance that combines indexing and search management functions.
- Small enterprise. One search head with two or three indexers.
- Medium enterprise. A small search head cluster, with several indexers.
- Large enterprise. A large search head cluster, with large numbers of indexers.
These deployment types are just points on a continuous scale, ranging from single-instance deployments to deployments that provide enterprise-wide coverage for a vast number of use cases.
In addition, you can deploy an indexer cluster in an enterprise deployment of any size. An indexer cluster offers advantages such as high availability, disaster recovery, and simplified scaling.
It is also possible to combine topologies in various ways. For example, you can deploy a search head that searches across both an indexer cluster and a set of independent indexers.
Note: The terms "small enterprise," "medium enterprise," and so on, do not specifically address the size of the enterprise using the Splunk platform. Instead, they are indicators of the breadth and depth of the functions that the Splunk platform supports in the enterprise. As awareness of the value of the Splunk platform for handling a wide range of use cases grows with continued success, the size of a deployment also typically grows. So, for example, a Fortune 500 company might start with a departmental-level, single-instance Splunk Enterprise installation for a very specific use case, and then, over time, transition through small enterprise and medium enterprise deployments, to eventually adopt a large enterprise deployment that provides key value to organizations and use cases distributed throughout the company.
Get started with your deployment
Read the rest of this topic to get a clear sense of the type of deployment you want to implement. Then turn to one of the following topics, accordingly:
- "Departmental deployment: Single indexer"
- "Small enterprise deployment: Single search head with multiple indexers"
- "Medium to large enterprise deployment: Search head cluster with multiple indexers"
- "High availability deployment: Indexer cluster"
These topics provide further details on each deployment type, including a diagram of the basic architecture. Most importantly, each includes a high-level, end-to-end guide to the implementation process, with links to the specific procedures to follow to implement the deployment.
Primary characteristics of deployments at representative scaling levels
The characteristics of a deployment change as it grows in size. This table gives you some idea of what to expect, with information on the Splunk components that you need to deploy to meet your needs.
|Departmental||Small enterprise||Medium enterprise||Large enterprise|
|Indexing volume (daily)||0-20GB||20-100GB||100-300GB||300GB-1TB+|
|# of forwarders||Median < 10; maximum 100||Median in the 10's; maximum in the 100's||Median in the 10's; maximum in the low 1000's||Median in the 10's; maximum in the 1000's|
|# of users||Median < 10||Median in the 10's||Median in the 10's; maximum in the low 100's||Median in the 10's; maximum 500+|
|# of apps (pre-packaged and customer-developed, combined)||1-10||1-10||1-20+||10-50|
|Indexing tier||1 indexer||2-3 indexers, possibly in a cluster||4-9 indexers, possibly in a cluster||10+ indexers, possibly in a cluster|
|Search management tier||Combined with indexer||1 standalone search head||3 search heads in a cluster||3+ search heads in a cluster|
|Configuration management function||Manual configuration or deployment server||Manual configuration or deployment server||Deployment server or 3rd party tool for forwarders and indexers. Deployer for search head cluster.||Deployment server or 3rd party tool for forwarders and indexers. Deployer for search head cluster.|
Design considerations also change as the deployment scales. This table summarizes some of the issues you need to consider when designing your deployment.
|Departmental||Small enterprise||Medium enterprise||Large enterprise|
|Forwarder issues||Management, monitoring||Load balancing, management, monitoring||Load balancing, management, monitoring, intermediate forwarders||Load balancing, management, monitoring, intermediate forwarders|
|Search issues||User counts, alerts, apps||Search head/indexer knowledge management, user counts||Search head/indexer knowledge management, user counts, search head clustering, job servers||Search head/indexer knowledge management, user counts, search head clustering, job servers|
|Scheduled search workload||Alerts, app/dashboard dependent, summary searches||Alerts, app/dashboard dependent, summary searches||Alerts, app/dashboard dependent, summary searches, job server||Alerts, app/dashboard dependent, summary searches, job server, API/SDK|
|Input types||Network, scripted||Network, scripted, batch, integrations||Network, scripted, batch, integrations||Network, scripted, batch, integrations|
|Availability||Platform-dependent (RAID, power supplies)||Data fabric (forwarder load balancing, storage, index replication)||User interface (search head clustering, load balancers); data fabric (forwarder load balancing, storage, index replication)||User interface (search head clustering, load balancers); data fabric (forwarder load balancing, storage, index replication)|
|Recoverability||Backup, retention||Backup, index replication, bucket/index restoration||Backup, index replication, bucket/index restoration||Backup, index replication, bucket/index restoration|
|Accessibility||Local vs. enterprise authentication||Authentication method||Authentication method||Authentication method|
|Staffing||Admin: 0.5-1 person; search/dashboard/appdev/ knowledge manager: 0.25-1 person||Admin: 0.5-1 person; search/dashboard/appdev/ knowledge manager: 0.5-1.5 persons||Admin/architect: 1-2 persons; knowledge manager: 0.5-2 persons; search/dashboard/appdev: 1-3 persons; program/project manager: 1 person||Admin: 2-4+ persons; architect: 1+ persons; knowledge manager: 2-5+ persons; search/dashboard/appdev: 2-6+ persons; program manager: 1 person; project manager: 0.5-2 persons|
For information regarding training opportunities and Professional Services offerings appropriate to your deployment scale, contact your Splunk sales representative.
For more guidance in determining the size and type of your deployment:
- For details on hardware capacity planning and deployment scaling, see the Capacity Planning manual.
- For a discussion of the benefits and trade-offs of implementing a high availability deployment, see "About indexer clusters and index replication" in the Managing Indexers and Clusters of Indexers manual.
Start implementing your distributed deployment
Departmental deployment: Single indexer
This documentation applies to the following versions of Splunk® Enterprise: 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11, 6.5.0, 6.5.1, 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.5.8, 6.5.9, 6.5.10, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 6.6.5, 6.6.6, 6.6.7, 6.6.8, 6.6.9, 6.6.10, 6.6.11, 6.6.12, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10, 8.1.0, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.1.6, 8.1.7, 8.2.0, 8.2.1, 8.2.2, 8.2.3