Storage APIs
  • 22 Sep 2020
  • 3 Minutes To Read
  • Contributors
  • Print
  • Share
  • Dark
    Light

Storage APIs

  • Print
  • Share
  • Dark
    Light

Storage Usage API

The Storage Usage API will return the amount of storage consumed in any destination that has been generated from a given source path. This API will return the physical size of the backup data in the destination that is currently specified in the backup policy applied to any source path.

All values returned are in GiB, and rounded to the nearest whole number. The API is an HTTP GET using the end point /x/igneous/v1/cloud-buckets/usage as outlined below.

Input Parameters

The Storage Usage API has two input parameters:

  1. systemName - the name of the source system that hosts the path. This input parameter is required, and is a string.

  2. exportPath - the rest of the source path. This input parameter is required, and is a string.

Outputs:

The Storage Usage API has six possible outputs:

  1. SourceSize - The size of the path on the source system during the most recent backup (in GiB).
  2. RemoteSize - This is an array of the remaining five outputs.
  3. Location - This is the location of the backup data. The possible outputs are:
    1. aws - Amazon Web Service
    2. azs - Microsoft Azure
    3. gcs - Google Cloud Storage
    4. s3 - Other S3 storage
    5. nfs - NFS storage
  4. CloudBucket: - The name of the cloud bucket where the backup resides.
  5. Usage: - The size of the backup data in the destination, rounded to the nearest GiB.
  6. Account - The name of the cloud account used to backup the data (only applicable to cloud destinations).
  7. Host: - Host information (Only applicable for Other s3 destinations).

Error outputs

There are two possible outputs when an error has occured:

  1. Failure(HTTP/1.1 401 Unauthorized) response: - When API is accessed without creating the token and authorization.
  2. Failure(HTTP/1.1 404 Not Found) response:
    a. {} Empty response is returned when an invalid exportPath is specified.
    b. {"Error":"GetSystemByName: name "<>" not found"}
    Error response is returned when invalid value for systemName is provided.

API Examples:

For these examples, we will be querying the size of the path /smallvol/uw, which is stored on the smallvol.iggy.bz system. This example dataset is 100 GiB on the source.

The API call would be constructed as follows:

export AUTH="Authorization: <token>"
curl -H "$AUTH" -X GET 'http://<IP>/x/igneous/v1/cloud-buckets/usage?systemName=smallvol.iggy.bz&exportPath=/smallvol/uw' 

Output examples:

The following are a few examples of outputs that would be expected under the given circumstances:

  1. The source path has been backed up to Google Cloud Storage, consuming 90 GiBs in the 1991-national-champions bucket using the Google account donjames.
{
  "SourceSize": 100,
  "RemoteSize": [
    {
      "Location": "gcs",
      "CloudBucket": "1991-washington-champions",
      "Usage": 90,
      "Account": "donjames"
    }
  ]
}
  1. The source path has been backed up to a custom S3 storage destination, consuming 114 GiBs in the 2001-rosebowl-champions bucket, and also had been backed up to Microsoft Azure, consuming 93 GiBs in the 2016-cfp bucket using the chrispetersen account.
"SourceSize": 100,
  "RemoteSize": [
    {
      "Location": "s3",
      "CloudBucket": "2001-rosebowl-champions",
      "Usage": 114
    },
    {
      "Location": "azs",
      "CloudBucket": "2016-cfp",
      "Usage": 93,
      "Account": "chrispetersen"
    }
  ]
}
  1. The source path is valid, but there is no backup policy assigned to the path.
{
  "SourceSize": 0,
  "RemoteSize": null
}
  1. The API is accessed without creating the token and authorization
HTTP/1.1 401 Unauthorized
Date: Tue, 15 Sep 2020 22:41:33 GMT
Content-Length: 0
  1. Invalid systemName is specified
HTTP/1.1 404 Not Found
Content-Length: 60
Content-Type: text/plain; charset=utf-8
Date: Tue, 15 Sep 2020 22:40:06 GMT
X-Request-Id: 78b795a7-f401-4a39-87ff-d0e3114636d4
{"Error":"GetSystemByName: name smallvl.iggy.bz not found"}
  1. Invalid exportPath is specified
HTTP/1.1 404 Not Found
Content-Length: 3
Content-Type: text/plain; charset=utf-8
Date: Tue, 15 Sep 2020 22:39:23 GMT
X-Request-Id: c438c021-d00a-4894-a666-f4d852484d78
{}

Get Bucket Information

This API is used to obtain information on all the buckets on a particular Igneous instance. It is literally a HTTP GET operation thus no additional JSON body is required.

To obtain a list of all buckets and their associated status
HTTP GET

https://<IP>/x/igneous/v1/buckets/

To obtain a list of all buckets from a specific system

GET https://<IP>/x/igneous/v1/buckets/?system=<system name>

To obtain status of a specific bucket

GET https://<IP>/x/igneous/v1/buckets/<bucket name>

API Examples

  1. Example to list all buckets using CURL
curl --insecure https://<IP>/x/igneous/v1/buckets \
-H 'Authorization: 2WXRXD92X2PRWZH57V9Z' \
-H 'Content-Type: application/json'

Example Output

{
            "Buckets": {
                "backup-export-22540": {
                    "Primary": false,
                    "BackupPolicy": "",
                    "SourcePath": "/mixed-native",
                    "System": "flashblade.iggy.bz"
                },
                "archive-export-22507": {
                    "Primary": true,
                    "LastSize": 2157,
                    "SourcePath":
                    "/ifs/pipeline-molly/pipeline_molly_source_nfs",
                    "System": "data03.isilon.iggy.bz"
                },
                "backup-export-22373": {
                    "Primary": false,
                    "BackupPolicy": "",
                    "SourcePath": "/ifs/ericltest/wpsmbbktest",
                    "System": "data03.isilon.iggy.bz"
                },
                "backup-export-22467": {
                    "Primary": false,
                    "BackupPolicy": "",
                    "SourcePath": "/homedirs2",
                    "System": "netapp-admin.iggy.bz"
               }
      }
}
  1. Example to list all buckets on the system isilon42.iggy.bz
curl --insecure https://<IP>/x/igneous/v1/buckets/?system=isilon42.iggy.bz \
-H 'Authorization: 2WXRXD92X2PRWZH57V9Z' \
-H 'Content-Type: application/json'

Example Output

{
    "Buckets": {
        "backup-export-3459": {
            "System": "isilon42.iggy.bz",
            "SourcePath": "/ifs",
            "BackupPolicy": "",
            "Primary": false
        },
        "backup-export-3446": {
            "System": "isilon42.iggy.bz",
            "SourcePath": "/ifs/bigfiles",
            "BackupPolicy": "",
            "Primary": false
        },
        "backup-export-4007": {
            "System": "isilon42.iggy.bz",
            "SourcePath": "/ifs/imagery",
            "BackupPolicy": "",
            "Primary": false
        }
    }
}
Was This Article Helpful?