Creating a New Jupyter Notebook

Create a new Jupyter notebook (.ipynb) in JupyterHub to develop code for visualizing, analyzing, and processing data from your tests, measurements, assets, and more.

Before you create a Jupyter notebook, complete the following tasks.

• Install Jupyter Notebooks for SystemLink in NI Package Manager to access predefined Jupyter notebooks that leverage SystemLink Python APIs.
• Append /niapis/python/index.html to your NI SystemLink Web Application URL to access SystemLink's Python APIs.
• Learn about the user interface and structure of a Jupyter notebook.
• Install Python libraries you want to use to enhance your data analysis and processing. NI recommends using PyPi to obtain packages.
• Learn about the built-in magic commands the IPython kernel enables.

Complete the following steps to create a Jupyter notebook.

1. In NI SystemLink Web Application, click Jupyter.
2. On the toolbar, click +.
3. Under Notebook, click Python 3. A Jupyter notebook launches and starts a kernel to run Python in the notebook.
4. Rename the notebook to describe the type of report it will execute.
5. In a code cell, import the Python modules, libraries, and widgets you need to build your notebook and make it interactive. For example, if you want to import Pandas to build and handle dataframes and Scrapbook to execute the notebook and capture results, you need to implement the following code.

   import copy
   import datetime
   import dateutil.parser
   import pandas as pd
   import scrapbook as sb
   from dateutil import tz

6. Use one of the SystemLink Python APIs to access the data you want to visualize, analyze, or process. For example, if you want to access data from the SystemLink Test Monitor service, your code would look like the following command:

   from systemlink.clientconfig import get_configuration
   from systemlink.clients.nitestmonitor import *

7. Establish a connection to your SystemLink Server. For example, if you want to connect a Test Monitor client to the server, use the following command:

   http_client_config = get_configuration(route_name='nitestmonitor')

8. Define the parameters and metadata for the notebook.
   1. In a code cell, define the parameters. For example, if you wanted to filter your test results to a specific time range and group them by the day your test system acquired them, your code may look like the following:

      filter = 'startedWithin <= "30.0:0:0"'
      group_by = 'Day'

   2. On the left pane, click Cell Inspector ( ) and add the parameters, their default values, and outputs to the Cell Metadata code block. For example, your code may look like the following:

      "papermill": {
              "parameters": {
                  "group_by": "Day",
                  "results_filter": "startedWithin <= \"30.0:0:0\"",
              }
          },
          "systemlink": {
              "namespaces": [
                  "ni-testmanagement"
              ],
              "outputs": [
                  {
                      "display_name": "This will show in dashboard output selector",
                      "id": "data_frame_output",
                      "type": "data_frame"
                  },
                  {
                      "display_name": "This will show in dashboard output selector",
                      "id": "scalar_output",
                      "type": "scalar"
                  }
              ],
              "parameters": [
                  {
                      "display_name": "Group By",
                      "id": "group_by",
                      "options": [
                          "Day",
                          "System",
                          "Test Program",
                          "Operator",
                          "Product"
                      ],
                      "type": "string"
                  },
                  {
                      "default_display": {
                          "startedWithin": {
                              "unit": "DAYS",
                              "value": 30
                          }
                      },
                      "display_name": "Results Filter",
                      "id": "results_filter",
                      "type": "test_monitor_result_query"
                  },
              ],
              "version": 2
          },
          "tags": [
              "parameters"
          ]
      }

   3. For namespaces, enter one or more of the following namespaces depending on where you want to view your report:

      Namespace	Report display location
      ni-assetmanager	Asset Manager
      ni-testmanagement	Test Monitor
      Any namespace you define	Any other client that uses namespaces to query notebooks

   4. For version, enter one of the following version numbers depending on how you want to use your notebook:

      Goal	Version
      You want to view your data in Test Monitor under Reports. You are customizing a notebook installed with a previous version of SystemLink.	1
      You want to return multiple outputs with one notebook. You want to use a scalar output type. You want to see your data in a dashboard.	2

9. Query a SystemLink data service for the data you want visualize, analyze, or process. For example, if you want to query the Test Monitor service for test results in ascending order, your code may look like the following:

   results_api = ResultsApi(api_client=ApiClient(http_client_config))
   query = ResultsAdvancedQuery(filter=filter, order_by=[ResultSortDefinitionObject(field=ResultField.STARTED_AT)])
   query_response = await results_api.query_results_v2(post_body=query)
   results = query_response.results
   
   results_list = [result.to_dict() for result in results]

10. Format the data in a Pandas dataframe based on how you want to group the data returned from the query. For example, if you want to group your test results by their status, your code may look like the following:

    group_names = []
    for result in results_list:
        if grouping in result:
            group_names.append(result[grouping])
    
    formatted_results = {
        'id': [result['id'] for result in results_list],
        'status': [result['status']['status_type'] for result in results_list],
        grouping: group_names
    }
    
    df_results = pd.DataFrame.from_dict(formatted_results)

11. Configure how you want to visualize, analyze, or process the data. The following examples are some of the ways you can configure the data.
    • Filter out results you do not want to include in the report.
    • Convert timestamps.
    • Compute data within a timeframe.
    • Group data in a specific way.
12. Convert the Pandas dataframe into the SystemLink reports output format. For example, the code may look like the following:

    result.append({
        'type': 'data_frame',
        'id': 'data_frame_output', 
        data': [{
            'format': 'XY',
            'x': ['2018-11-17T00:00:00', '2018-11-18T00:00:00', ...],
            'y': [94.0, 89.9, ...]
        }],
        'config': {
            'title': 'Title',
            'graph': {
                'axis_labels': ['x-axis-label', 'y-axis-label'],
                'tick_labels': [{'x': 0, 'label': 'tick label 0', ... }],
                'orientation': 'VERTICAL',
                'plot_style': ['SCATTER'],
                'plot_color': ['blue']
            }
        }
    })

    

    Tip  

    To ensure Test Monitor supports the dataframe output, verify the value of the id (data_frame_output) is correct in the cell and in the Code Metadata for the parameter code cell. Refer to step 8b for more information.

    

    Note  

    If you use the V2 report format in a Jupyter notebook, you can only visualize the report results on a dashboard.

13. Add a new scalar output. For example, your code may look like the following:

    result.append({
        'type': 'scalar',
        'id': 'scalar_output',
        'config': {
            'title': 'Scalar Output Title'
        },
        'value': 3
    })

14. Record results with Scrapbook. Refer to the following code example for optimal parsing in the NI SystemLink Web Application.

    sb.glue('result', result)

15. Add a new cell in the notebook.
16. On the toolbar, select Code»Markdown to add documentation about your code to the notebook.
17. On the menu bar, select Run»All Cells to verify the notebook returns results or processes data as expected.
18. If the notebook returns an error, add import pdb; pdb.set_trace() to the code cell with the error to use the built-in Python debugger.
19. Click Save.
20. Optional: If you want the notebook to execute on a recurring basis, create a scheduled task with the Windows Task Scheduler. Refer to Task Scheduler documentation for more information.

After you create your notebook, you can bind it as a data source to a tile and monitor the results on a dashboard. If you need to share your notebook, refer to Sharing a Jupyter Notebook for more information.