You can use notebook services to author and automate notebook workflows.
License:
This capability is available at two licensing levels:
- An ArcGIS Notebook Services license is provided with the ArcGIS Enterprise on Kubernetes license. This license allows organization members to author notebooks using ArcGIS Notebook Python 3 Standard runtime and automate notebook workflows using scheduled tasks, webhooks, and web tools.
- An ArcGIS Notebook Services Advanced license is available as a premium capability and provides access to the ArcGIS Notebook Python 3 Advanced runtime with additional ArcPy capabilities beyond what is provided in the ArcGIS Notebook Services license.
Note:
To support notebook services, any external reverse proxies or load balancers must be able to pass websocket traffic to the backend targets and have requirements set so that sessions remain open for 10 minutes.
Introduction to notebook services
Notebook services provide a versatile web-based Python notebook interface for geospatial data analysis. With notebooks, you can perform analysis, automate workflows, and immediately visualize data and analysis results in a geographic context.
Notebooks are an efficient, modern environment that combine code, on-the-fly visualizations and maps, and data tools. In the notebook editor, you can write, document, and run Python code in one place.
Before enabling notebook services, ensure that your administrator has allocated sufficient resource quota and worker nodes to support this premium capability.
Configure notebook services
The following configuration steps may require changes to the way you've deployed ArcGIS in your organization; review them carefully before proceeding.
Notebook services require the pod runAsUser to have read and write permissions on the associated network share. In some providers, this may require allowing others to have those permissions, while in other instances the shared location can be updated to allow the default user ID (117932853) to own the parent directory and any subdirectories or files. If others are allowed read or write access, proper IP ranges should be used as a security control on the NFS configuration. For clusters that enforce arbitrary user ID assignment during pod admission and scheduling, the user ID should be set to a static value and permissions adjusted accordingly.
Enable notebook services
To enable notebook services, you must configure a notebook workspace and enable the capability in ArcGIS Enterprise Manager.
- In ArcGIS Enterprise Manager, click the Capabilities button in the sidebar.
The capabilities page appears.
- Turn on the Notebook services toggle button.
The Configure workspace dialog box appears and prompts you to specify required values. The workspace will be used to store files associated with each organization member's notebooks.
- Provide the required information:
- For Volume type, choose Dynamic or Static to specify how you want to provision a persistent volume for the workspace. Use Static when the PVC should bind to an existing PV. Use Dynamic when a new PV should be provisioned through the specified storage class. See considerations for Dynamic provisioning and Static provisioning to learn more.
Note:
The PV for this workspace must be of type FileSystem and support ReadWriteMany (RWX) access mode. Before configuring a notebook workspace, verify that the PV you allocate to this workspace supports RWX access mode. Additionally, to support the PV, you may be required to install a supporting driver in your Kubernetes cluster.
- Provide Volume name text.
If Static is chosen, you must provide a volume name.
- For Storage class name, specify your own storage class name or if you've set a storage class, specify the storage class name for the volume.
Note:
Storage class name is required for Dynamic and optional for Static.
Note:
When binding to an existing PV that has no storage class defined, leave the storage class name blank. If a default storage class is configured in the cluster, the DefaultStorageClass admission controller adds the default storage class and prevents the PVC from binding. In this case, administrators should either add a storage class specification to the PV or remove the default storage class configuration.
- For Size (GiB), define the size of the PV.
The minimum size is 100 GiB. The value must match the size of the existing PV when using static binding. If the value is higher than the size of the existing PV, the PVC does not bind with the PV.
Note:
Only one notebook workspace can be configured for your organization. Its storage size is limited to the value specified at the time it was created. Ensure that you choose a sufficient size for the workspace to meet your current and future needs.
- Click Save.
You are prompted with a message that indicates the process to enable may take some time.
A request to enable notebook services is submitted. This process validates prerequisites and activates supporting resources. A notebook automation service is automatically started.
- For Volume type, choose Dynamic or Static to specify how you want to provision a persistent volume for the workspace. Use Static when the PVC should bind to an existing PV. Use Dynamic when a new PV should be provisioned through the specified storage class. See considerations for Dynamic provisioning and Static provisioning to learn more.
If the capability fails to enable, repeat the above steps to ensure that the workspace has been configured, the notebook services license is valid and available, and the system services have been started. Review the logs to identify the requirements for this capability.
Optionally, you can deploy a deamonSet to preload notebook services images. This will help improve the time it takes to open a notebook for the first time. The initial loading of a notebook is set to timeout after 1 minute.
Notebook services is now configured. You can begin to use notebooks in your organization. See how to create a notebook and publish a notebook as a web tool.
Additionally, see how to adjust settings for the notebook services capability.
Notebook services settings
After you've enabled notebook services, you can view and edit related settings such as the supporting resources for the applicable runtimes, WebSocket size, maximum snapshots per notebook item, and more.
The notebook service capability includes three main tabs where you can view or edit settings: Overview, Properties, and Runtimes.
- Overview—Determine the status for your notebook automation service status, the number of running notebook deployments, and the storage class assigned to the dedicated notebook workspace.
- Properties—Manage the limits related to both interactive and automated notebook deployments.
- Runtimes—View and edit the settings for available notebook service runtimes.
View and edit properties
The following properties can be viewed and edited in ArcGIS Enterprise Manager:
- WebSocket size (MB)—The maximum size (in megabytes) of a packet that can be passed between the server and client browsers using the WebSocket protocol. WebSocket is used to facilitate efficient, secure two-way communication with clients. The default value of this property is 16 MB. You may need to raise this limit if a particularly large piece of data is being passed by a notebook to the server site.
- Idle notebook threshold (minutes)—This property governs when idle notebooks are closed. Notebooks that are open but have been inactive for this amount of time (in minutes) are automatically closed. The notebook's kernel is stopped. The default is 7 minutes.
- Empty deployment threshold—This property sets how long a notebooks deployment can remain open (in minutes) without running any active notebooks. After this value is reached, the deployment is automatically closed. The default is 60 minutes. As with the idle notebook threshold, you may want to reduce this value to conserve resources.
- Maximum snapshots per notebook item—This property limits the number of snapshots allowed for a notebook item. The default value is 5.
To view and edit properties, complete the following steps:
- Sign in to ArcGIS Enterprise Manager as an administrator.
- Click the Capabilities page.
- On the Notebook services card, click Settings.
- Click the Properties page.
If you change any of the setting values, the Save button becomes available.
- Click Save to persist your edits.
View and edit runtimes
You can perform the following runtime management tasks in ArcGIS Enterprise Manager:
- View available runtimes.
- Duplicate a system created runtime to make a user created runtime. Each runtime can be configured separately, which enables different types of tasks to use runtimes with different settings.
- View and edit the system resources available per runtime deployment. If there are multiple users authoring notebooks concurrently, you can monitor usage and adjust these settings as necessary.
- Control the nodes where pods for a runtime can be scheduled. Pod placement control is useful when some tasks require resources, such as a GPU, that are only available on some nodes in the cluster.
To view or edit a runtime, complete the following steps:
- Sign in to ArcGIS Enterprise Manager as an administrator.
- Click the Capabilities page.
- On the Notebook services card, click Settings.
- Click the Runtimes page to see the list of available runtimes.
- To create a new runtime, click Duplicate next to the system created runtime you want to duplicate.
- Click the name of the runtime you want to view or edit.
- To view or edit the resources that each runtime deployment can access, complete the following steps:
- Click Runtime settings.
- View or edit CPU requests and CPU limits.
- View or edit the Memory requests, Memory limits, Maximum swap memory, and Shared memory.
- Use the Enable GPU toggle button and the GPU units value to view or edit the runtime's access to GPU resources. GPU Units represents the number of GPUs per node.
Note:
Your Kubernetes cluster must be configured to enable GPU resources. See Configure GPU-enabled nodes for more information.
- Click Save to persist any edits.
- To view or edit the pod placement settings for a runtime, complete the following steps:
- Click Pod placement.
- View or edit Node affinity and Tolerations.
If you are configuring node affinity for GPU-enabled notebook services, use the following values:
- Type—Required
- Key—Specify the key used for labeling the GPU node, for example notebook.
- Operator—In
- Value—Specify the value used for labeling the GPU node, for example NotebooksGPUNode.
If you are configuring tolerations for GPU-enabled notebook services, use the following values:
- Effect—No Execute
- Key—nvidia.com/gpu
- Operator—Exists
- If configuring new Node affinity or Tolerations, click Add.
- Click Save to persist any edits.
- To view the libraries that are included in the runtime, click Manifest.