Dead Man's Snitch

Posts About Alerts

Announcing: Error Logs with Field Agent

Last year we introduced Field Agent, an important addition to your toolkit that gathers extra data on your critical tasks, packages it up, and radios it back to Dead Man's Snitch. Since then, we've been working on new ways to leverage the data that Field Agent collects to give you more insight into your Snitches and their failures.

Diagnosing why a job failed often involves jumping onto servers and parsing through logs to find out what happened. Starting today, we're making it easy to see the job's output to help get problems resolved faster with new error logs.

error logs for dead man's snitch
Error logs, a feature of Error Notices, are available right now to anyone using Field Agent and are on the Surveillance Van plan or higher. To get started you need to set up your job to be monitored by Field Agent. Our documentation is a great place to start. As always, if you have questions let us know.

For those of you who use integrations for alerts, don't worry. You'll be able to access error logs from Slack and PagerDuty, which will include a link right to the logs when available.

Slack error log

To start being alerted immediately when your job runs but errors, and access to the error log output, you may want to consider upgrading to the Surveillance Plan or higher.

Posted on 2017-09-25 in alerts, announcement, feature, release

Smart Alerts and Error Notices

Level up your Snitches with our two newest features: Smart Alerts and Error Notices. Both features are focused on letting you know as soon as possible that a critical job ran into problems. They also make it easier for you to get those issues fixed a lot sooner.

Together, Smart Alerts and Error Notices create a powerful combo. Error Notices let you know as soon as a job runs but errors, and Smart Alerts gives you a safety net when/if that job fails to run at all.

Smart Alerts

Smart Alerts empower your Snitches to let you know sooner when they go missing. Dead Man's Snitch will learn from your snitch's check-in behavior and adjust the expected check-in time as it learns more about when your snitch checks-in. This means no longer having to wait until the end of the day to find out your snitch failed to check-in 23 hours earlier.

Smart Alerts are available for weekly and monthly Snitches for all paid plans. Hourly and daily Snitches are supported on the Surveillance Van plan.

Error Notices

It's now possible (and highly recommended) to send your job's exit status along with your check-in rather than checking in only on success. If your job runs but errors, we will immediately notify you if something went wrong.

If you're already using the Dead Man's Snitch Field Agent, then there are no changes required and you can start getting Error Notices today.

If you're using an HTTP client (like Curl) to check-in, you can send the status as the "s" parameter in either the query or a form. We will treat an exit status of "0" as a success and anything else as a failure. When using Cron it's also important to always check-in whenever your job runs and not only on success. To do this replace "&&" with ";" and use "$?" to get the exit status of your command.

Example; curl$?

To get started with Error Notices you can read up on the documentation. This feature is available for users on the Surveillance Van plan

Field Agent

Field AgentIf you haven't already, you may want to add our Field Agent to your tool set. Error Notices is just the first of a host of features we have planned that take advantage of the Field Agent. Start using the Field Agent today so you’ll be ready when we roll those features out.

Posted on 2016-10-25 in alerts, announcement, documentation, feature, release