Splunk® Enterprise

Module System User Manual

Download manual as PDF

Download topic as PDF

Include 3rd-party libraries

Important notice: As part of Advanced XML deprecation, the Module System is officially deprecated beginning with Splunk Enterprise 6.3. For more information, see Advanced XML Deprecation.
To add your module code
To define the treemap module
To import the protovis treemap visualization library
To handle the search request
To render the treemap
To integrate the TreeMap module into the app view

SplunkIcon.pngExample 6


This recipe extends the Render JSON data recipe by using the same search to render a treemap visualization, instead of an HTML table. For treemap rendering, we've selected the Protovis graphical library, which provides a way to demonstrate the integration of non-Splunk software in your application. You can use this mechanism to include any custom JavaScript.

Note: The example for this tutorial, Example 6, currently only renders in the Firefox browser.

Details

The module template file, TreeMap.html, added to the TreeMap module shows the mechanism for importing third-party libraries.

Additionally, a request to get CPU utilization data by both name and processor was added to the search string in Example6.xml.

 index=_internal source=*metrics.log group=pipeline |
     stats sum(cpu_seconds) as totalCPU by name, processor

Hierarchically showing processors grouped by indexer stage makes the visualization more interesting, and is the way treemaps are intended to be used.

Cb 01.png

 

See the Treemapping reference to learn about treemaps and their application.

To add your module code

Create the $SPLUNK_HOME/etc/apps/Dev_tutorial/appserver/modules/TreeMap directory and add the following files:

  • TreeMap.conf
  • TreeMap.html
  • TreeMap.js
  • TreeMap.py

The HTML template file is added to the module as the mechanism for linking the protovis.js graphical library.

To define the treemap module

In your TreeMap.conf file, define your module class name and your module base class.

[module] className = Splunk.Module.TreeMap superClass = Splunk.Module.DispatchingModule

To import the protovis treemap visualization library

Your template, TreeMap.html custom template provides the mechanism for linking third-party libraries. Custom templates are Mako templates, packaged within an app, that emit dynamic HTML. You can specify custom or third-party JavaScript in the template.

  1. Define the %page, %namespace, and %lib tags. <%page args="module" expression_filter="h"/><%namespace name="lib" file="//lib.html" import="*"/><%lib:script_tags files="${['/static/app/%s/protovis.js' % APP['id']]}" />

    Use the %page expression for all templates to enforce the Mako expression filter, h, for all content rendered by the template. Any variable passed to this template is escaped before rendering.

    Tip: This is a much more scalable practice than trying to apply expression filters on a per-variable basis. If double-escaping is needed, manually unescape variables using the n expression filter: <div>Something = ${ pre_escaped n}</div>

    The %namespace tag defines the lib namespace and imports from the lib.html base template (see Mako Namespaces ), inheriting all the def s from lib.html.

    The %lib:script_tags tag includes the protovis.js reference in a script tag when TreeMap.html is rendered.


    Tip: Although not used in this example, you can use templates to include custom CSS and JavaScript in the HTML rendered by the template, using:
    • script_tags() for JavaScript
    • stylesheet_tags() for CSS
    The best practice is to use the css(), custom_css(), or js() methods inherited from layout/base.html. This ensure that dependencies and load orders are respected.

    Implement the script_tags() and stylesheet_tags() calls inside custom_css(), as shown:
    <%def name="custom_css()"> <%lib:stylesheet_tags files="${['/static/app/<appname>/<custom>.css']}" /> <%lib:script_tags files="${['/static/app/<appname>/<custom>.js']}" />
  2. Protovis requires a DOM element ID where it renders the treemap. Define a <DIV> element with ID TreeMapID. <div id="TreeMapID" class="TreeMapResults"></div>

    Note: See the Mako documentation for more about Mako templates.

Note: Load order is important, here. Because the module template loads the protovis.js library after the module is loaded, you cannot reference library functions in the module constructor.

To handle the search request

The TreeMap.py file handles the search request, formatting a JSON string and transmitting the data to the client.

Note: This only differs from the Render JSON data example in adding processor field data to the results.

  1. Define the treemap module class, inheriting from ModuleHandler. class TreeMap(module.ModuleHandler)
  2. Override the generateResults() method, making the following code additions. def generateResults(self, host_app, client_app, sid, count=1000, offset=0, entity_name='results'):

    Validate parameters:

    count = max(int(count), 0) offset = max(int(offset), 0) if not sid: raise Exception('TreeMap.generateResults - sid not passed!')

    Get the job associated with this search instance:

    try: job = splunk.search.getJob(sid) except splunk.ResourceNotFound, e: logger.error('TreeMap could not find job %s. Exception: %s' % (sid, e)) return _('<p class="resultStatusMessage">Could not get search data.</p>')

    Get the search results data associated with the job:

    dataset = getattr(job, entity_name)[offset: offset+count]

    Build a dictionary of CPU utilization, iterating through the search results and using field name literals to index the data:

    outputJSON = {} for i, result in enumerate(dataset): tdict = {} tdict[str(result.get('processor', None))] = str(result.get('totalCPU', None)) name = str(result.get('name', None)) if name not in outputJSON: outputJSON[name] = dict() outputJSON[name].update(tdict)

    For each indexer stage, the data structure includes the processor-CPU utilization, name-value pairs, for indexer processors.

    Set the text/json data format in the response header and format a JSON string for transfer to the client:

    cherrypy.response.headers['Content-Type'] = 'text/json' return json.dumps(outputJSON, sort_keys=True)

    Note: Beginning with release 4.3, you can use render_json() to transmit JSON data.

The generateResults() method now formats CPU utilization data similar to the following JSON example:

dev-null:

   {
   nullqueue:"0.000000"
   },

fschangemanager:

   {
   fschangemanager:"0.000000",
   sendindex:"0.000000"
   },

indexerpipe:

   {
   http-output-generic-processor:"0.000000",
   index_thruput:"0.074836",
   indexandforward:"0.000000",
   indexer:"11.016261"
   indexin:"0.000000"
   signing:"0.000000"
   syslog-output-generic-processor:"0.022083"
   tcp-output-generic-processor:"0.000000"
   },

merging:

   {
   aggregator:"0.732066",
   readerin:"0.000000",
   sendout:"0.031409"
   },

parsing:

   {
   header:"0.000000",
   linebreaker:"0.017684",
   readerin:"0.000000",
   sendout:"0.031909"
   utf8:"0.000000"
   },

scheduler:

   {
   livesplunks:"0.000000"
   },

tcp:

   {
   tcp:"0.000000"
   },

typing:

   {
   annotator:"0.000000",
   previewout:"0.000000",
   readerin:"0.000000",
   regexreplacement:"0.000000"
   sendout:"0.140565"
   }

To render the treemap

Client-side code is implemented in the TreeMap.js file.

Note: Refer to the Create a custom module example for the general discussion of client processing.

Here, we override the renderResults() method with calls to the Protovis API for rendering the treemap.

  1. Define the client component, subclassing Splunk.Module.DispatchingModule. Splunk.Module.TreeMap = $.klass(Splunk.Module.DispatchingModule, {...}
  2. Override the renderResults() method, making the following code changes.

    Verifying data are available from the server.

    renderResults: function($super, results) { if(!results) { this.resultsContainer.html('No content available.'); return; }

    Set up treemap rendering properties for a Protovis panel and render the treemap.

    The Protovis namespace is pv. You might find it helpful to review the Protovis API.

    var re = "", color = pv.Colors.category19().by(function(d) d.parentNode.nodeName) nodes = pv.dom(results).root("flare").nodes(); var vis = new pv.Panel() .width(920) .height(420) .canvas(document.getElementById('TreeMapID')); /* .canvas($('#TreeMapID')[0]); */ var treemap = vis.add(pv.Layout.Treemap) .nodes(nodes) .round(true); treemap.leaf.add(pv.Panel) .fillStyle(function(d) color(d).alpha(1)) .strokeStyle("#fff") .lineWidth(1) .antialias(false); treemap.label.add(pv.Label) .textStyle(function(d) pv.rgb(0, 0, 0, 1)); vis.render(); }

    Note: You can learn more about the Protovis treemap implementation and API at the GitHub repository.

    The panel origin is set to the <DIV id=TreeMapID> element previously defined in our template file.

    The input data used to build the treemap is the raw JSON data passed in the results parameter, which is assigned to the nodes variable.

    Tip: To handle browser dependencies, it is recommended that you use jQuery where possible, as shown in the code comment, above, for getting the desired <DIV> element by ID, for rendering.

To integrate the TreeMap module into the app view

In the $SPLUNK_HOME/etc/apps/Dev_tutorial/default/data/ui/views/Example6.xml file, add the following line in the view hierarchy where you want your module data rendered:

<module name="TreeMap"></module>

The treemap visualization is rendered in the bottom-most panel, displaying search results initiated by the HiddenSearch module and forwarded to the TreeMap module.

Related recipes

Render JSON data shows you how to transfer your data as JSON and format the display on the on the client.

Parameterize your module shows you how to statically parameterize your module.

Setup your app shows you how to make your app configurable.

PREVIOUS
Render JSON data
  NEXT
Parameterize your module

This documentation applies to the following versions of Splunk® Enterprise: 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.5.0, 6.5.1, 6.5.1612 (Splunk Cloud only), 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.5.8, 6.5.9, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 6.6.5, 6.6.6, 6.6.7, 6.6.8, 6.6.9, 6.6.10, 6.6.11, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.2.0, 7.2.1


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters