Hi there 👋

This article summarises the process for using the Splunk TA OTel collector with a custom built OpenTelemetry (OTel) collector. Examples are provided to reduce the size of the overall application with the intention to run OS host monitoring using a smaller custom built OTel collector. The full post is available on medium, Building a custom OpenTelemetry collector (OTel), and on Splunk Lantern as Building a custom OpenTelemetry collector.

The pstack and eu-stack utilities do not exist inside the Splunk container, and therefore if your using the Splunk operator for Kubernetes it can be difficult to gather stacks for support purposes. This article demonstrates how to build a container that includes eu-stack’s for debugging purposes and a script you can use to obtain stacks when working with Splunk support. Stacks are useful for any situation where performance of the instance is an issue. ...

A year has passed since the article, Splunk Operator for Kubernetes (SOK) — Lessons from our implementation, this article discusses new learnings using the SOK. The full post is available on medium, Splunk Operator for Kubernetes (SOK) — Lessons from our implementation part 2 . A Splunk lantern version is available as Splunk Operator for Kubernetes: Advanced operational learnings.

This series of articles provides performance tuning tips for the Splunk platform at search head, indexing and forwarder tiers. Each article contains links to practical examples of how the tips can be implemented in another environment. The original posts are availabe on medium as Splunk — performance tuning tips — forwarding tier, Splunk — performance tuning tips — search head tier, and Splunk — performance tuning tips — indexing tier The lantern versions of the article are available as Performance tuning the forwarding tier, Performance tuning the search head tier and Performance tuning the indexing tier.

A discussion around Splunk metrics indexes and how using different ingestion methods changes the bucket. In our production environment, we encountered challenges with high cardinality metrics indexes, specifically those receiving metric data with a large number of unique dimension values. The full post is available on medium, Splunk indexers — metrics data — HEC fields vs INDEXED_EXTRACTIONS changes the bucket, and on Splunk lantern, Preventing premature bucket rolling in metrics indexes.

In addition to the Splunkbase apps, I have written various automation components related to Splunk. Each repository has a unique purpose as described in the following sections. Splunk This repository contains various scripts that I have built over the years, the most popular would be the transfersplunkknowledgeobjects.py script. This first script is documented in the README.md of the repository and exists to assist in search head to search head (or search head cluster to search head cluster) migrations using the REST API. Note that POST’ing too many searches to the savedsearches REST API can be very disruptive to the Splunk scheduler and this works best for small migrations. ...

Over the years I have created a number of Splunk-related applications. This post exists to summarise the applications I have created, along with why they might be useful within your environment. Alerts For Splunk Admins GitHub link First published in 2017, this application was created after previous Splunk conf presentations would demonstrate an interesting platform-focussed dashboard, and then, they would not provide any source code or example searches to build the dashboard. I decided to create a presentation catered to Splunk administrators with all searches and dashboards available after the presentation. ...

We ran bare metal indexers for years, but we rarely exceeded 30% CPU. This post summarises improvements seen by switching to the Splunk Operator for Kubernetes The full post is available on medium, Splunk Operator for Kubernetes (SOK) — Improvements on the indexing tier, and as a Splunk Lantern article

When creating indexers with the Splunk Operator for Kubernetes (SOK) the assumption is search heads will be within K8s. We managed to get our search heads to run outside K8s The full post is available on medium, Splunk Operator for Kubernetes (SOK) — Indexers on K8s, Search Heads outside the K8s cluster, and also available as a Splunk Lantern article

The SOK is a Splunk-built K8s operator to run Splunk indexer clusters, search head clusters and standalone instances. This article discusses our lessons from production implementation The full post is available on medium, Splunk Operator for Kubernetes (SOK) — Lessons from our implementation, and as a Splunk Lantern article