ArcGIS Enterprise Manager provides several options for managing your organization's data stores, including viewing folder data stores registered with your organization and adding and removing persistent volume (PV)-based folder data stores. You can also update cloud object and relational store credentials, increase the volume size of stores, and add storage to support raster analytics in your organization.
Manage folder data stores
From the User storage tab of the Data stores page in ArcGIS Enterprise Manager, you can view the folder data stores registered with your organization. If your organization has file- or folder-based GIS data in PVs, you can add the PVs as folder data stores and unregister them if needed. Adding PV-based folder data stores is only supported for administrators from ArcGIS Enterprise Manager. Other user managed data stores can be added from the portal.
Persistent volumes are a native Kubernetes solution for exposing data. For more information, refer to the Kubernetes documentation. ArcGIS Enterprise on Kubernetes supports using NFS shares, Windows (SMB) shares, or local persistent volumes as PV-based folder data stores.
Benefits of adding PVs as folder data stores
There are a variety of reasons why organizations may want to expose folder-based GIS data as PVs and add the PVs as folder data stores:
- The flexibility of choosing Network File System (NFS) versions and mount options
- The ability to register a Windows share directly with ArcGIS Enterprise on Kubernetes using SMB protocol
- To meet requirements for cases in which data must be more local to service pods to have better performance—for example, local volumes are the recommended option for routing and geocoding services
- To add a folder data store in organizations enforcing the restricted pod security standard, as registration of folder data stores is limited to PV-based volumes.
Common scenarios for organizations adding a PV-based folder data store include the following:
- Windows share as a persistent volume—An organization has folder data for map image services in a Windows share that is used by publishers to connect to data from ArcGIS Pro. Instead of configuring an NFS share separately with duplicate data or configuring dual protocol on their file server for ArcGIS Pro and ArcGIS Enterprise to have access to the data, the administrator registers the Windows share directly to ArcGIS Enterprise on Kubernetes.
- NFS share as a persistent volume—An organization has folder data for map and image services in an NFS share. The administrator registers this share with ArcGIS Enterprise on Kubernetes, using their desired NFS version and specifying additional mount options. Organization members can now publish map and image services that reference the data in the share.
- Local persistent volume—An organization wants to share a geocode service as a locator to ArcGIS Enterprise on Kubernetes. To ensure the best performance, the locator data, including ArcGIS StreetMap Premium data, must be in close proximity to the compute-geocode service pods. To do this, the administrator creates a local PV, places locator data there, then adds the local PV as a PV-based folder data store. For more information see Configure the organization to geocode addresses.
Prerequisites for adding a PV-based folder data store
You must create the associated Persistent Volume before adding a PV-based folder data store. Each type of PV-based folder data store also has additional prerequisites that must be met.
- Before adding a PV-based folder data store for an NFS share, complete the following steps:
- Ensure the NFS server is accessible to the cluster.
Note:
For an AKS cluster, the cluster needs to be on the same or peered Azure virtual network (VNet) as the NFS Server. The cluster must be created on an existing VNet, which can be the same VNet as your NFS Server VM.
- Create the PV. See the sample NFS share yaml file for more information.
- Ensure the NFS server is accessible to the cluster.
- Before adding a PV-based folder data store for a Windows (SMB) share, complete the following steps:
- Install the SMB CSI Driver for Kubernetes in your cluster.
- Create the PV. See the sample Windows share yaml file for more information.
- Before adding a PV-based folder data store for routing or geocoding, compete the following steps:
- Copy the data (for example, a locator for geocoding services) to the nodes in your cluster.
Note:
It is recommended that you use pod placement settings for services that will use the local PV so that they are only scheduled to nodes in a dedicated node pool. Then you can automate copying data to nodes in that pool using launch templates or other automation strategies.
- Create the PV. See the sample local PV yaml file for more information.
- Copy the data (for example, a locator for geocoding services) to the nodes in your cluster.
Add a PV-based folder data store
To add a PV-based folder data store, complete the following steps:
- Sign in to ArcGIS Enterprise Manager as an administrator.
- Click the Data stores button in the sidebar.
The data stores page appears.
- Click the User storage tab.
- Click Add store and do the following:
- Enter a name for the folder data store.
- In the Publisher folder path field, provide individual shared locations where source data can be accessed and published by clients, such as ArcGIS Pro. This is usually a Windows path.
- If using a Publisher folder path with a drive letter, provide the Publisher folder hostname.
- If you are adding an NFS share, complete the following steps:
- Choose Linux for Share type.
- For NFS share path, provide an NFS path on the host machine where shared data can be stored and accessed across the organization. The NFS share path is available in the persistent volume definition file.
- For NFS hostname, provide the machine name where shared data is stored and accessed through an NFS path.
- If you are adding a Windows share, complete the following steps:
- Choose Windows for Share type.
- For Volume path, provide the source path of the Windows share. The volume path is available in the persistent volume definition file.
- If you are adding a local PV-based folder data store for routing or geocoding services, complete the following steps:
- Check the box next to This is a local persistent volume for routing or geocoding services.
- Choose Linux for Share type.
- Provide the local path on the nodes for the local PV in the Volume path field. The volume path is available in the persistent volume definition file.
- Provide the following information about the volume specification of the PV:
- Access mode
- Storage class name (optional)
- Size (GiB)
- Persistent volume name
- Persistent volume claim name (optional)
- Click Add store.
View details of a PV-based folder data store
For a PV-based folder data store that has been previously added, you can view its details by clicking the More options button next to the data store. Details include the following information:
- Publisher folder path
- Volume path (local persistent volumes and Windows shares only)
- NFS share path and NFS hostname (Linux only)
- Persistent Volume name
- Persistent Volume Claim name (if applicable)
Unregister a PV-based data store
If you unregister a PV-based folder data store, the services that reference it will be unable to connect to the data.
To unregister a data store, complete the following steps:
- Sign in to ArcGIS Enterprise Manager as an administrator.
- Click the Data stores button in the sidebar.
The data stores page appears.
- Click the User managed tab.
- Click the More options button
next to the PV-based data store you want to unregister.
- Click Unregister and click Unregister again to confirm.
Manage required storage
You may need to update the credentials of cloud object stores and cloud relational stores or increase the volume size of object stores, relational stores, and spatiotemporal index stores.
Update credentials
To update credentials for cloud object stores and cloud relational stores, complete the following steps:
- Sign in to ArcGIS Enterprise Manager as an administrator.
- Click the Data stores button in the sidebar.
The data stores page appears.
- Click the More options button
for the store and click Update credentials.
- If you are updating a cloud object store, provide the following information:
- If using Amazon S3, choose Access key or IAM role as the authentication type and provide the following information:
- If you selected Access key, paste or type the access key and secret key for the IAM user.
- If using Azure Blob, choose Storage account key or Managed identity as the authentication type and provide the following information:
- If you selected Storage account key, enter the storage account name and paste or type the storage account key.
- If you selected Managed identity, enter the storage account name and select System-assigned or User-assigned as the identity type.
- If you selected user-assigned identity, provide the client ID.
- If using Google Cloud Storage, paste or type the access key and the secret key for the service account.
- If using Amazon S3, choose Access key or IAM role as the authentication type and provide the following information:
- If you are updating a cloud relational store, provide the database username and password.
- Click Update.
The credentials have been updated.
Increase volume size
You can increase the volume size of object stores, relational stores, and spatiotemporal and index stores.
Note:
This option applies when storage was provisioned as persistent volumes. It is not supported for storage that was configured with cloud services.
To update the volume size, complete the following steps:
- Sign in to ArcGIS Enterprise Manager as an administrator.
- Click the Data stores button in the sidebar.
The data stores page appears.
- Click the More options button
for the data store and click Increase volume size.
- Increase the volume size and click Update.
If a failure occurs when updating the volume size, the Increase volume size window will contain information about that attempt. The Requested size column lists the previously requested volume size. To perform this workflow again, the new value must be greater than the previously requested size.
Manage raster stores
Add, update the properties of, or unregister raster stores.
Add a raster store
To add storage to support raster analytics, complete the following steps:
- Sign in to ArcGIS Enterprise Manager as an administrator.
- Click the Data stores button in the sidebar.
The data stores page appears.
- In the Raster stores section, click Add store.
- Enter the name of the new raster store.
- Select the storage service from the drop-down menu:
- Amazon S3
- Azure Blob
- Google Cloud Storage
- Relational store
- If using Amazon Simple Storage Service (S3), provide the following information:
- Bucket name—The name of the bucket created in Amazon S3.
- Region—The region where the bucket exists.
Note:
To connect to a private S3 endpoint, choose Custom as the region. Type or paste the URL to access the private cloud in the Private URL field.
- Folder (optional)—The name of an existing or nonexisting root directory in Amazon S3 that can be used for the raster store.
- Authentication type—Choose Access key or IAM role.
- If using access key as the authentication type, provide the following information:
- Access key—Paste or type the access key for the IAM user.
- Secret key—Paste or type the secret key for the IAM user.
- If using Azure Blob, provide the following information:
- Container name—The name of the container created in Azure.
- Storage account—The name of the parent storage account the container exists under.
- Storage domain—The Azure storage domain where the Blob storage container exists.
Note:
To connect to a custom Azure Blob endpoint, choose Other as the storage domain and provide the URL or DNS endpoint to access the storage location.
- Folder (optional)—The name of an existing or nonexisting root directory in Azure Blob that can be used for the raster store.
- Authentication type—Choose Shared key, Microsoft Entra ID, or Shared Access Signature.
- If you selected Microsoft Entra ID, and choose user-assigned managed identity as the identity type, provide the client ID. If you choose service principal, also include the tenant ID and client secret.
Note:
User-assigned identities must have at least read-only access to the storage account container. If more than one user-assigned managed identity is assigned to an instance or storage container, you must specify a client ID.
- If you selected Shared key, paste or type the primary or secondary account key for the associated storage account.
- If you selected Shared Access Signature, enter the token.
- If using Google Cloud Storage, provide the following information:
- Bucket name—The name of the bucket created in Google Cloud Storage.
- Folder (optional)—The name of an existing or nonexisting root directory in Google Cloud Storage that can be used for the raster store.
- Access key—Paste or type the access key for the service account.
- Secret key—Paste or type the secret key for the service account.
- If using a relational store to store the mosaic datasets for hosted imagery layers, choose a database connection file.
- Click Add store.
The raster store is added to your organization and appears in the Raster stores section.
Update raster store properties
It may become necessary to update the properties you specified when adding the raster store. For example, you may need to change the authentication type, specify a new region or storage domain, or update relational database user passwords.
When updating cloud raster store properties, the bucket or container name and the folder, if applicable, are populated from the existing connection. The remaining values are not populated, meaning you must provide information for all raster store properties, not just the ones you are changing. For example, to update the bucket or container name, you also need to provide the region or storage domain, authentication type, and any other properties required for the storage service.
To update raster store properties, complete the following steps:
- Sign in to ArcGIS Enterprise Manager as an administrator.
- Click the Data stores button in the sidebar.
The data stores page appears.
- In the Raster stores section, click the More options button
next to the raster store you want to update.
- If you are updating a cloud raster store, update the properties available for the storage service:
- Amazon S3—Optionally, enter a new bucket name or folder name. Select the region of the bucket, choose an authentication type, and provide the information required for the authentication type.
- Azure Blob—Optionally, enter a new container name or folder name. Provide the storage account name, select a storage domain, select an authentication type, and provide the information required for the authentication type.
- If using Google Cloud Storage—Optionally, enter a new bucket name or folder name. Provide the access key and secret key.
For descriptions of the properties, review the step for your storage service in the Add a raster store section above.
Caution:
Do not update the bucket or container name without an exact replica in the updated location. Similarly, do not change the region or storage domain without an exact replica in the updated region. Doing so could break the existing published services associated with this raster store.
- If you are updating a hosted raster store, choose the updated database connection file.
Caution:
Choosing the incorrect connection file or changing the database username could break the existing published services associated with this raster store.
- Click Save.
Validate a raster store
To determine whether a cloud raster store is healthy or unhealthy, complete the following steps:
- Sign in to ArcGIS Enterprise Manager as an administrator.
- Click the Data stores button in the sidebar of ArcGIS Enterprise Manager.
- Click the Validate button next to the raster store you want to validate.
Validation will succeed if the raster store is accessible and can be used. Validation will fail if the raster store is inaccessible, cannot be written to, or has some other issue, such as expired credentials. In the case of system failure or other unforeseen event, validation may also fail because the status of the raster store could not be determined.
Validation for relational raster stores is not currently available in ArcGIS Enterprise Manager. Use a database client like ArcGIS Pro to validate the username and password for a relational raster store. Use network management tools to ensure network traffic is open between the cluster and the database.
Unregister a raster store
To unregister a raster store, complete the following steps:
- Sign in to ArcGIS Enterprise Manager as an administrator.
- Click the Data stores button in the sidebar of ArcGIS Enterprise Manager.
The data stores page appears.
- Click the More options button
next to the raster store you want to unregister.
- Click Unregister and click Unregister again to confirm.