Looking back at one year of Sensu assets & announcing a new Sensu plugins SDK

Assets were one of the biggest additions in Sensu Go. How have new and experienced Sensu users reacted to Sensu assets as a workflow? And what’s on the roadmap for Sensu contributors?

Background

One of the most important improvements that Sensu Go introduced to the Sensu platform are assets, a solution for real-time distribution of monitoring instrumentation across hybrid/multi-cloud infrastructure. Assets effectively eliminate the need for pre-installing monitoring scripts or plugins on host nodes, facilitating a unified monitoring workflow across bare metal servers, virtual machines, and containers (it’s monitoring automation without the heavy lift of traditional configuration management). Assets are also platform aware, so we can automatically handle distribution to hybrid Linux/Windows/Mac/BSD/Solaris/etc. environments. And the best part is – they’re built entirely on GNU tar, HTTP, and some intelligent management of the monitoring agent environment (i.e., dynamically setting $PATH, $LD_LIBRARY_PATH, and $CPATH), so it works with all of the languages and tools you’re already using. Assets can even be used to package and distribute third-party instrumentation such as Nagios plugins, Prometheus exporters, and that handful of shell scripts you still use to monitor certain systems.

To make assets even easier to use, we also launched a free companion service called Bonsai, which is equivalent to a Docker Hub for Sensu assets. In the same way that Docker Hub (and other container registries) make it possible to do things like docker run sensu/sensu:latest – prompting your container runtime to look up where the sensu/sensu:latest container image is hosted and automatically download the image – Bonsai lets you run sensuctl asset add sensu/monitoring-plugins and Sensu Go will look up, register the asset, and take care of distribution to Sensu agents.

Bonsai makes it easy to share, discover, and distribute Sensu assets and has been a popular resource since we made it public nearly a year ago. In January 2020 alone, there have been over 5 million asset downloads from Bonsai, and over 1,500 individual asset binaries contributed into Bonsai. As community member Landon Dao puts it, “Instead of having to manage all these checks, you have assets that are more seamless.”

Example

With Bonsai, using assets to extend you monitoring workflows becomes trivially simple:

Resources

Want to learn more about assets? You’re in luck – we have already published quite a bit of content on the subject, including:

• Sensu Go asset reference: Learn the basics of building a Sensu asset.
• Bonsai source repository: The source code — still under active development — that powers bonsai.sensu.io, so you can host your own private asset index if you need it; we even published a Bonsai public API spec in case you want to develop compatible integrations.
• Sensu asset utils repository: Scripted automation to help you build assets for executables using shared libraries.
• Contributing assets for existing Sensu plugins: Procedure for building Sensu assets for existing community plugins.
• Strategies for Ruby assets with shared libraries: Shared libraries require extra care.
• Sensu Ruby runtime: Used to build and run Ruby-based Sensu plugin assets.
• Sensu Go asset automation repository: Helper repository containing reusable scripts intended to work with CI deployments.
• Asset sharing tips: Make sure your assets are reusable by others.
• [Webinar] Assets in Sensu Go: A walkthrough of how assets are packaged up and used.

This first year with assets has been a valuable learning experience for everyone. A big thank you to the Sensu Community members that have embraced the Sensu asset technology early on, and provided invaluable feedback on the asset tooling to help us make the technology better with each Sensu Go release. Our asset building resources are still a bit of a mishmash — what’s missing is official, opinionated tooling and best practices, which we’re working on now!

Announcing new contributor resources for the Sensu Community

Because assets are a significant change in how operators share Sensu solutions, we thought it best to spin up a new organized collection of tooling and resources where we can collaborate on the maturing asset workflows. Here’s what we’ve put together so far:

• New sensu-community GitHub organization for Sensu users and contributors
• New sensu-plugins-sdk Golang library
• New Sensu plugin template repositories with GitHub Actions CI pipelines
• New sensu-plugin-tool to help use the template repositories
• New “Sensu Plugins” Discourse categories, to facilitate discussion for further best practices

The current focus of the sensu-community resources is creating a great experience for first-time asset contributors. We are focusing on the Golang SDK, as statically compiled Golang binaries are a great fit for the asset workflow for both contributors and users. The template repositories give you all the necessary boilerplate programming to make a functional ‘hello world’ asset, so you can focus on writing the unique business logic for your specific monitoring needs. The sensu-plugin-tool puts this all together and helps you initialize your own project using the template repository and GitHub Actions CI pipeline.

We’ll still keep the capability to use Ruby-based Sensu plugins as assets via the Ruby runtime and related resources. We’ll be able to discuss evolving best practises for all software languages in Discourse for those in the community who want to help make a great experience for other languages as well.

Sensu Contributor Roadmap: working with the Sensu Community to continue improving assets

The new tools are just the start, we also want to make sure we get a chance to engage directly with our community around assets so we can continue to get feedback and make improvements. Here’s what we have planned in the next couple of months:

• Webinar: Getting started with assets (details below)
• Contributor sessions at the virtual Sensu Community Summit, coming up on May 19-22 — register now (it’s free)!

Watch our webinar on getting started with Sensu assets & Bonsai. We cover the following topics:

• How to publish your first asset using the new sensu-plugins-sdk and template plugin project(s)
• How to package simple scripts as assets (e.g., bash and Powershell scripts)
• How to package compiled binary plugins (including dynamically linked libraries) as assets

This webinar is great for anyone who finds it useful to have guided instruction using the new tooling, and some of the other asset building resources we’ve developed over the last year. And if you are planning on attending the upcoming Community Summit in May, the webinar gives you a great introduction to asset building prior to our hands-on hacking sessions at the Summit.

Recap / one more thing…

Our last couple of blog posts covered a lot of ground, so let’s recap.We’re rebranding/relaunching the Sensu Summit with a renewed focus on end-users and our amazing contributor community. The first Sensu Community Summit will happen this May 19 in San Francisco, with a US East event to follow in October 2020. One of the themes we hope to promote around the event is “Contributors”, so to help us prepare for this event, we’re going to host a contributor-oriented webinar around creating your own assets (for private/on-prem use, or for sharing with the community via Bonsai). But there’s also one more thing:

Announcing the sensu-plugins-sdk and new template plugin project(s)

We’re very excited to announce a brand new collection of tools for developing Sensu plugins in the Golang programming language, including an officially supported SDK that makes it easy to develop great Sensu monitoring plugins (the same SDK we use to develop our commercial plugins for Sensu Go), a set of template repositories that includes a templatized continuous integration solution (via GitHub Actions), and a CLI tool for initializing new plugin projects. Publishing your first asset to Bonsai is now as easy as these three steps:

1. Download sensu-plugin-tool and start a new plugin project.

    sensu-plugin-tool new check helloworld
    cd helloworld
    git tag -a 0.1.0

2. Push your plugin to GitHub. The built-in GitHub actions CI pipeline will trigger a build and publish a release via GitHub Releases! You’re now ready to share your first Sensu plugin with the world.
3. Register your asset in Bonsai. Login to Bonsai using your GitHub account, select your plugin repository, and publish.

Publishing your first asset to Bonsai is just the beginning. These new SDK and plugin templating tools give you everything you need for a complete end-to-end workflow for building functional Sensu plugins. Just modify your plugin project code, add a GitHub tag (SemVer only!), and commit your changes. GitHub Actions will automatically build the new release, and Bonsai will automatically register the new version of your asset.

We’ll explore these new tools in greater detail during the upcoming webinar and a sensu-plugins-sdk workshop at the Sensu Community Summit. In the meantime, we look forward to your feedback & suggestions for how to improve the SDK – PRs are always welcome!