![]() You end up with more capacity than you actually might need.Unfortunately, it’s not quite that simple. You buy a 1TB hard drive, and never think of Splunk storage again.You math those two numbers together (yes, I’m using math as a verb here) and determine you need 900gb of disk space.Your compliance requirement stipulates that you need 90 days of logs immediately available.Without knowing any better, you might think that a Splunk disk calculation would work something like this: So we just do some math and buy some disks, right? Planning the storage around the typical search behavior can result in better system performance due to the tiered method that Splunk uses. You may have other reporting or business analytics use cases that require a larger set of data to be available quickly. For Splunk Enterprise Security Suite (ES) or Splunk IT Service Intelligence (ITSI) use cases, Splunk runs tons of searches nearly constantly over a relatively small range of data. In terms of Splunk, immediately available would mean that the data must be immediately searchable (and not frozen or otherwise archived).Īnother consideration around retention would be the intended use for the data from a search perspective. Many compliance standards, such as PCI, include verbiage regarding how long logs must be maintained and how much data must be immediately available (PCI DSS, 10.7). The retention period is typically at least somewhat dictated by compliance. However, there are some nuances to this value to consider. Retention PeriodĪt face value, retention period is simple: how long data is available in Splunk. This would be a situation where your calculations might not be based on simply the license size. In this case, it may make sense to purchase storage based on your anticipated ingestion rate and budge to increase the storage allocated in subsequent years. This could mean that your organization ends up purchasing a much larger Splunk license than what you might anticipate using for a few years. For more details on this, talk to someone who drives a fancier car and uses larger words with less actual meaning than me (aka your sales rep). In many cases, it can make more sense to buy a larger license with the intent to grow into that license to result in a lower average cost per unit of data in Splunk. This means that the cost per gigabyte of daily ingestion per day decreases with a larger license. However, there are cases where this is not always the best method.Īccording to Splunk’s current Pricing FAQ, volume discounts are offered for larger license sizes. ![]() Generally, it’s safe to use the licensed capacity as an upper limit for this value, as you should not consistently be exceeding your license. The daily ingestion rate is simple – how much data is Splunk consuming each day, before compression. These two values work together to account for the bulk of the calculation of how much space you will need allocated. The two most important factors impacting Splunk storage are the daily ingestion rate and the desired retention period. While storage seems simple at face value, there are a few factors to consider when sizing a Splunk environment that are important for ensuring that Splunk performs well and maintains data as searchable for an appropriate period of time. There’s enough to consider that this topic warrants an entirely separate blog post. ![]() ![]() However, one topic that I did not focus on was the storage requirements. In my previous blog post I discussed some considerations for designing and sizing a Splunk environment, and the resources that servers will need for a mock deployment. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |