Self-service monitoring checks in Sensu Go

Now that we’ve officially launched Sensu Go and announced the EOL schedule for Sensu 1.x, I want to address some of the ways in which we’re either ensuring backwards compatibility between Sensu 1.x and Sensu Go or — in the case of standalone check definitions — are creating something even better.

In Sensu 1.x., standalone check definitions enabled a degree of self-service monitoring through decentralized configuration; Sensu Go’s built-in role-based access control (RBAC) takes it to the next level, empowering more teams through accessible interfaces and self-service monitoring.

In this post, I’ll take a look back at standalone checks — sharing what was good (and not so good) about them — and illustrate how we’ve matched (and exceeded!) their functionality in Sensu Go.

Standalone checks in Sensu 1.x

The good

Standalone checks pushed configuration to the edge; if a particular team or person owned a machine or application, they could easily create checks for that application on that machine, locally. Said another way: the source of truth for a configuration lived locally within the monitoring agent. Say you were wondering what an agent is monitoring — you could easily look up the local configuration and get your answer.

Overall, standalone checks meant decentralized configuration, and therefore meshed well with some teams and their practices, in particular large organizations that have siloes or clean separation of ownership over resources. They also made it easier for tools like Puppet to drive configuration, as they were already inherently good at driving config files on those machines.

In combination with Sensu 1.x’s Safe Mode, standalone checks also made for an absolute secure way to lock down your configuration by making it so they would only ever execute locally defined check configurations.

The not so great

Standalone checks meant Sensu could not provide a central, auditable, and defined source of truth. Since there was no centralized record, you’d have to go to each individual local machine to figure out what’s configured — as you might imagine, this wasn’t very scalable, and made it difficult to audit for security.

Another downside to standalone checks is they caused you to rely very heavily on configuration management to distribute changes. You had to use tools like Puppet, Chef, or Ansible to get effective distribution and combat configuration drift (and therefore maintain your sanity). While config management tools are great, you can’t always rely on them — environments like Kubernetes’ config management is not through the traditional approach (i.e., it’s done through ConfigMaps, etc.), and some organizations don’t have it as part of their practice. Either way, you want to be able to solve for those problems in a consistent way.

Standalone checks were all or nothing in terms of configuration — you’d either have the ability to change local config or you didn’t. And, users that did have that ability could change anything about your configuration, not just checks, leading to all kinds of change management and security concerns.

True self-service monitoring with Sensu Go

With Sensu Go, we’ve built features to better address and solve your problems. Standalone checks don’t work in Sensu Go because we wanted better experiences for operators and developers — how can you have a centrally defined, managed, and granular system if you also have the ability to configure on the edge, which can create inconsistencies with what you see centrally?

Here are new features in Sensu Go that better address what standalone checks were solving for:

• A centralized configuration API. As the name implies, it’s centralized, so you can get that single source of truth. Here’s an overview of the Sensu Go API.
• The new API also offers a proper authentication and authorization model with RBAC, so you can get that granularity and control over who’s making what changes. Instead of it being all or nothing in terms of config changes, you can grant particular people the ability to change specific configurations. Read more about RBAC.
• Our overall goal with Sensu Go’s new API is to provide better user experiences across your team, not just the one experience for the UNIX sysadmin. Now users can use CLI tools and web UIs to manage configuration, increasing accessibility across your team and empowering everyone to effectively contribute to monitoring. Now, folks no longer need to know how to jump on a particular box and change a config file in order to configure Sensu. Here’s a quick reference for sensuctl, the Sensu CLI and an overview of the Sensu web interface.
• Target individual machines with monitoring agent unique subscriptions. With Sensu Go, you can have it so checks only target a particular host without having to configure them locally. Much like RBAC, unique subscriptions give you granular scheduling to target single machines. Read more about check scheduling here.
• Assets for plugin packaging and delivery. One argument for standalone over centralized configuration is you’d have to install plugins ahead of time locally anyways, so why not configure them at the same time? With Sensu Go, we’ve solved the packaging and delivery problem with assets — now, you can configure checks centrally and dynamically deliver the check dependencies (i.e., plugins) on demand. Updates to assets automatically get pushed to all machines that need to execute that check, solving any management issues that arose out of relying wholly on config management. Discover and share assets with Bonsai and here’s how to install plugins using assets.
• Coming soon in Sensu Go, you’ll be able to lock down check executions to specific commands with the agent check command allow list. If configured, the agent compares received check commands with its allow list. If a check command is not in the allow list, the agent does not execute the command and responds with an event declaring this fact. Follow this feature’s progress here.

Next steps

If you haven’t already, we invite you to take Sensu Go for a spin either by installing it or trying it out in the Sensu Go sandbox, which comes pre-configured with InfluxDB and Grafana.

If you’re using config management tools, take a look at the Sensu Puppet module and Chef Cookbook for installing and configuring Sensu Go. Both of these maintain similar experiences to what you had with Sensu 1.x while leveraging the new capabilities outlined above.

In an upcoming post, we’ll show you how to publish a Sensu Ruby check as an asset, so stay tuned! In the meantime, we invite you to join our Community Forum to ask questions (and get answers) on upgrading to Sensu Go (feel free to tag your posts with 1x-migration).