Create a new Jupyter notebook (.ipynb) in JupyterHub to develop code for visualizing, analyzing, and processing data from your tests, measurements, assets, and more.

Before you create a Jupyter notebook, complete the following tasks:
  • Install Jupyter Notebooks for SystemLink from NI Package Manager to access predefined Jupyter notebooks that can leverage SystemLink Python APIs.
  • Append /niapis/python/index.html to your SystemLink application URL to access the Python APIs.
  • Learn about the user interface and structure of a Jupyter notebook.
  • Install Python libraries you want to use to enhance your data analysis and processing. NI recommends using the Python Package Index (PyPI) to obtain packages.
  • Learn about the built-in magic commands that the IPython kernel enables.
  1. In the SystemLink web application, click Jupyter.
  2. Under Notebook, click Python 3.
    A Jupyter notebook launches and starts a kernel to run Python in the notebook.
  3. Rename the notebook to describe the type of report the notebook executes.
  4. In a code cell, import the Python modules, libraries, and widgets for your interactive notebook.
    For example, you can implement code to import Pandas and Scrapbook for the following goals:
    • To build and to handle dataframes (Pandas).
    • To execute a notebook (Scrapbook).
    • To capture notebook results (Scrapbook).
    import copy
import datetime
import dateutil.parser
import pandas as pd
import scrapbook as sb
from dateutil import tz
  5. Use one of the SystemLink Python APIs to access the data you want to visualize, analyze, or process.
    To access data from the SystemLink Test Monitor service, use the following command.
    from systemlink.clientconfig import get_configuration
from systemlink.clients.nitestmonitor import *
  6. Establish a connection to your SystemLink server.
    To connect a Test Monitor client to the server, use the following command. 
    http_client_config = get_configuration(route_name='nitestmonitor')
  7. Define the parameters and metadata for the notebook.
    1. In a code cell, define the parameters.
      You can implement code to accomplish the following goals:
      • Filter your test results by a specific time range.
      • Group your test results by the day the test system acquired the result.
      filter = 'startedWithin <= "30.0:0:0"'
group_by = 'Day'
    2. On the right sidebar, open the Property Inspector pane.
    3. In the Cell Metadata code block, add the parameters, the parameter default values, and the outputs.
      To add this information, you can implement the following code:
      "papermill": {
        "parameters": {
            "group_by": "Day",
            "results_filter": "startedWithin <= \"30.0:0:0\"",
        }
    },
    "systemlink": {
        "namespaces": [
            "ni-testmanagement"
        ],
        "outputs": [
            {
                "display_name": "This will show in dashboard output selector",
                "id": "data_frame_output",
                "type": "data_frame"
            },
            {
                "display_name": "This will show in dashboard output selector",
                "id": "scalar_output",
                "type": "scalar"
            }
        ],
        "parameters": [
            {
                "display_name": "Group By",
                "id": "group_by",
                "options": [
                    "Day",
                    "System",
                    "Test Program",
                    "Operator",
                    "Product"
                ],
                "type": "string"
            },
            {
                "default_display": {
                    "startedWithin": {
                        "unit": "DAYS",
                        "value": 30
                    }
                },
                "display_name": "Results Filter",
                "id": "results_filter",
                "type": "test_monitor_result_query"
            },
        ],
        "version": 2
    },
    "tags": [
        "parameters"
    ]
}
    4. For namespaces, enter one or more of the following namespaces depending on where you want to view your report:
      Namespace Report display location
      ni-assetmanager Asset Manager
      ni-testmanagement Test Monitor
      Any namespace you define Any other client that uses namespaces to query notebooks
    5. For version, enter one of the following version numbers depending on how you want to use your notebook:
      Goal Version
      • You want to view your data in Test Insights under Reports.
      • You are customizing a notebook that is installed with a previous version of SystemLink.
      		 1
      • You want to return multiple outputs with one notebook.
      • You want to use a scalar output type.
      • You want to see your data in a dashboard.
      		 2
  8. Query a SystemLink data service for the data you want to visualize, analyze, or process.
    To query the Test Monitor service for test results in ascending order, you can use the following code: 
    results_api = ResultsApi(api_client=ApiClient(http_client_config))
query = ResultsAdvancedQuery(filter=filter, order_by=[ResultSortDefinitionObject(field=ResultField.STARTED_AT)])
query_response = await results_api.query_results_v2(post_body=query)
results = query_response.results

results_list = [result.to_dict() for result in results]
  9. Format Pandas dataframe data based on how you want to group the data returned from the query.
    To group test results by their status, you can use the following code:
    group_names = []
for result in results_list:
    if grouping in result:
        group_names.append(result[grouping])

formatted_results = {
    'id': [result['id'] for result in results_list],
    'status': [result['status']['status_type'] for result in results_list],
    grouping: group_names
}

df_results = pd.DataFrame.from_dict(formatted_results)
  10. Configure how you want to visualize, analyze, or process the data.

    The following examples are some of the ways you can configure the data.

    • Filter out results you do not want to include in the report.
    • Convert timestamps.
    • Compute data within a timeframe.
    • Group data in a specific way.
  11. Convert the Pandas dataframe into the output format of a SystemLink report.
    To convert the Pandas dataframe, you can use the following code:
    result.append({
    'type': 'data_frame',
    'id': 'data_frame_output', 
    data': [{
        'format': 'XY',
        'x': ['2018-11-17T00:00:00', '2018-11-18T00:00:00', ...],
        'y': [94.0, 89.9, ...]
    }],
    'config': {
        'title': 'Title',
        'graph': {
            'axis_labels': ['x-axis-label', 'y-axis-label'],
            'tick_labels': [{'x': 0, 'label': 'tick label 0', ... }],
            'orientation': 'VERTICAL',
            'plot_style': ['SCATTER'],
            'plot_color': ['blue']
        }
    }
})
    Tip To ensure Test Monitor supports the dataframe output, verify the value of the ID (data_frame_output). The value must be correct in the cell and in the Code Metadata for the parameter code cell. For more information, refer to the previous step 7.
    Note If you use the V2 report format in a Jupyter notebook, you can only visualize the report results on a dashboard.
  12. Add a new scalar output.
    To add a scalar output, you can use the following code: 
    result.append({
    'type': 'scalar',
    'id': 'scalar_output',
    'config': {
        'title': 'Scalar Output Title'
    },
    'value': 3
})
    Note For more information on adding a scalar output, refer to the SystemLink examples repository in GitHub.
  13. Record results with Scrapbook.
    To parse the SystemLink web application for results, you can use the following code.
    sb.glue('result', result)
  14. Add a new cell in the notebook.
  15. On the toolbar, choose Markdown from the dropdown menu to add documentation about your code to the notebook.
  16. On the menu bar, select Run » All Cells to verify the notebook correctly returns and processes data. If the notebook returns an error, add import pdb; pdb.set_trace() to the code cell with the error. Adding this code activates the built-in Python debugger.
  17. Click Save.
  18. Optional: To run your notebook at a scheduled time, create an Analysis Automation procedure and add a scheduled task.
After you create your notebook, you can bind the notebook as a data source to a dashboard tile. By adding this tile to a dashboard, you can monitor the notebook results. For more information on sharing a notebook, refer to Sharing a Jupyter Notebook.