<p>Starting with version 4.2, <a href="http://www.splunk.com/">Splunk</a> provides alerting not only by polling and running searches on a scheduled basis but also in <a href="http://docs.splunk.com/Splexicon:Realtimealert">real-time</a>. In the previous blog post, I had discussed the benefits of integrating Splunk and OpsGenie. In this post, I’ll go over the use case of sending Splunk alerts to iPhone via push notifications as an example. Here are the steps:</p><p><strong>Step 1: Setup OpsGenie</strong></p><p>Naturally, an <a href="http://www.opsgenie.com/">OpsGenie</a> account is needed. You can create a <a href="https://www.opsgenie.com/customer/signUp">free trial account</a> for your organization to check it out. Once the account is created, you can <a href="http://support.opsgenie.com/customer/portal/articles/551148-adding-users">invite users</a> to OpsGenie. Each user you’d like to send alerts to will need an OpsGenie account. Users can set their own notification preferences to specify how they’d like to be notified (push notifications, SMS, email, voice call, etc). Users can receive notifications on any mobile phone via SMS or phone calls, and OpsGenie also provides an iPhone app (and soon apps for Android, etc. as well) that allows users to receive notifications via <a href="http://support.apple.com/kb/HT3576">iPhone push notifications</a>. <a href="http://itunes.apple.com/app/opsgenie/id528590328">OpsGenie iPhone app</a> is available in Apple App Store.</p><p><img alt="image" src="http://media.tumblr.com/9312927939bed5d146344e812baaeb2f/tumblr_inline_mu3anbnQgl1soq1dj.png" width="262"/></p><p>In this example above, the user has configured to receive via email and iPhone push notification immediately, and if the user does not see the alert within 5 minutes, OpsGenie will send a notification via SMS, and another one via a phone call, if the user still has not seen the notification within 10 minutes.</p><p>You will need to know the usernames (email of the user) of the users that will be notified. Alternatively, you can create a group in OpsGenie(for example web_operations), make users member of the group and specify the group as the recipient when creating the alert. To create an alert in <a href="http://support.opsgenie.com/customer/portal/articles/565170-web-api---create-alert">OpsGenie via the Web API</a>, you will need OpsGenie the API Key for your account. You can get see the API key (aka customerKey) at the <a href="https://www.opsgenie.com/customer/settings">account settings</a> page.</p><p><strong>Step 2: Configure Splunk to run a script as an alert action</strong></p><p>Splunk alerting (used Splunk Enterprise v4.3 on a linux server for this example) supports running a script as an “alert action”, and Splunk embeds python as a scripting engine, hence we can execute a python script as an alert action without having to install any other software. Splunk documentation describes <a href="http://docs.splunk.com/Documentation/Splunk/latest/Admin/Configurescriptedalerts">how to configure “scripted alerts”</a>. For any alert that you’d like to notify users, you can configure “Run a script” alert action and specify the name of the python script, for example, splunk2opsgenie.py. The script needs to be stored in $SPLUNK_HOME/bin/scripts directory as stated in the UI.</p><p><img alt="image" src="http://media.tumblr.com/12983138889b0143ab8478581f5a3057/tumblr_inline_mu3ansvLx21soq1dj.png"/></p><p><strong>Step 3: Alert Action Script</strong></p><p>As described in Splunk documentation, Splunk passes alert data to the scripts as environment variables. We can access these variables in the script using python sys module and use them to set OpsGenie alert properties. To <a href="http://support.opsgenie.com/customer/portal/articles/565170-web-api---create-alert">create an alert in OpsGenie</a>, we will make a web request (HTTP POST) to OpsGenie. We can use python json module to format the data and urllib2 module to make the web request. Below is a sample script that creates an alert in OpsGenie. It sets the mandatory properties customerKey, recipients and message. It also sets optional properties to store the parameters passed by Splunk (optional parameters are set in “details”” as a map)</p><pre>#!/opt/splunk/bin/python<br/>import sys<br/>import urllib2<br/>import json<br/># store params passed Splunk as optional alert properties<br/>details = {<br/> "numberOfEvents":sys.argv[1],<br/> "terms":sys.argv[2],<br/> "query":sys.argv[3],<br/> "url":sys.argv[6],<br/> "reason":sys.argv[5],<br/> "searchName":sys.argv[4]<br/>}<br/># populate the map that contains alert properties<br/>alertProps = {<br/> "customerKey":"444ad5de-b0e3-123d-a6dc-e0ca0ae381ff",<br/> "message":sys.argv[5],<br/> "recipients":"web_operations",<br/> "source":"Splunk",<br/> "details":details<br/>}<br/>jdata = json.dumps(alertProps)<br/>response = urllib2.urlopen("https://api.opsgenie.com/v1/json/alert", jdata)</pre><p>At this point we’re all set to receive Splunk alerts on our mobile device via mobile push notifications, SMS, etc. Here is what a Splunk alert would like on iPhone:</p><p><img alt="image" src="http://media.tumblr.com/81e472df8796f55a939af4123c91cb60/tumblr_inline_mu3aoxq7a31soq1dj.png" width="262"/></p><p><img alt="image" src="http://media.tumblr.com/cf49b23ec21a8c04133fd0d50e6ecee2/tumblr_inline_mu3ap0yXR71soq1dj.png" width="262"/></p><p>Recipients section shows the status for each recipient, whether they have viewed the alert, commented on the alert, executed an action, etc.</p><p><img alt="image" src="http://media.tumblr.com/f11110fa28362461664c217b54564e37/tumblr_inline_mu3apyXfbI1soq1dj.png" width="262"/></p><p>All alert activity is tracked by OpsGenie and can be seen in the Logs tab for the alert.</p><p><img alt="image" src="http://media.tumblr.com/ca2674dacc440ad4662f902d8e978ced/tumblr_inline_mu3arlVpJm1soq1dj.png" width="262"/></p><p><span>Sending Splunk alerts to an iPhone is only the first step. You can also attach relevant files (like search results) to an alert, and configure actions to be executed by the recipients. Stay tuned for future blog posts or take a look at our </span><a href="http://support.opsgenie.com/">support site</a><span> for more information. Sign up for a </span><a href="https://www.opsgenie.com/customer/signUp">free trial</a><span> and give it a shot. We’re looking forward to</span><a href="http://support.opsgenie.com/"> hear from you</a><span>. </span></p><p><br/>OpsGenie also provides another integration which can be used to create alerts and also attach Splunk search results to the alert : <a href="http://support.opsgenie.com/customer/portal/articles/714183-all-downloads#splunk-integration">Splunk Integration Download.</a></p>![]()
↧
