I’ve been working a fair bit with Qualys APIs using PowerShell past few months. Never got a chance to blog about it, which sucks coz this blog is what I refer to when I am stuck on something myself. Anyways, the other week I had to query the CSAM API and so before I forget it all I’d better blog it here. :)
The thing with Qualys is separate and different, like the various modules themselves in the web portal. At least on the web you authenticate once and it lets you access all modules even though they look different; but with the API you got to authenticate separately, and each has a different way. So that’s the first hoop to just over always.
This PDF is their guide. I am not a fan of any of their API guides, it’s just so dry reading with not many examples.
Anyways, first thing to do is figure out the API URL to use. That’s on pages 5 and 6; you figure it out from the portal. Mine was https://gateway.qg2.apps.qualys.com. Next thing is to authenticate. That’s on page 7. Here’s a screenshot:
So we must send a POST request to the API base URL, to the /auth endpoint, and the body (that’s what curl -d does) must contain the username, password, and a token field set to true. Notice it’s just username & password, and since most firms typically have MFA turned on for Qualys you must use a different account with MFA disabled and restricted to just API access.
I created a PowerShell function for this. (That wasn’t what I did initially, but when polling the API over long periods I ran into the above issue of the token expiring. So I created a function and this way I can re-authenticate as needed. More details later).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
function Authenticate-QualysAPI { # https://www.qualys.com/docs/qualys-gav-csam-api-v2-user-guide.pdf $header = @{ "ContentType" = "application/x-www-form-urlencoded" } $qg_server = "https://gateway.qg2.apps.qualys.com" $authBodyHash = @{ "username" = $username "password" = $password "token" = 'true' } $authUrl = "${qg_server}/auth" $signon = Invoke-RestMethod -Uri $authUrl -Method $method -Headers $header -Body $authBodyHash -UseBasicParsing return $signon } |
I set $username and $password separately. From a Key Vault or Automation Account variable etc. The function assumes they exist globally.
The function returns a token. I put into a header variable and use everywhere.
|
1 2 3 |
$authToken = @{ "Authorization" = ("Bearer " + $signon) } |
Getting a specific asset
Page 15 goes into getting the details of a specific asset. This is based on asset Id.
The URL is the same base URL as before, but to a different endpoint. And you got to pass the asset Id. Also, the method is a ‘GET’ instead of ‘POST’ as is usual. Here’s what I do:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
$assetId = "12344521" $assetsUrl = "${qg_server}/rest/2.0/get/am/asset?assetId=$assetId" try { $results = Invoke-RestMethod -Uri $assetsUrl -Headers $authToken -Method 'GET' -ContentType 'application/json' -ErrorAction Stop } catch { # Catch re-auth requirement if ($_ -match "Unauthorized") { $authToken = Authenticate-QualysAPI $authHeader = @{ "Authorization" = ("Bearer " + $authToken) } $results = Invoke-RestMethod -Uri $assetsUrl -Headers $authToken -Method 'GET' -ContentType 'application/json' -ErrorAction Stop } else { # Catch any other errors; sleep and retry Start-Sleep -Seconds 10 $results = Invoke-RestMethod -Uri $assetsUrl -Headers $authToken -Method 'GET' -ContentType 'application/json' -ErrorAction Stop } } |
I make the call. Check if there’s an error and if the error contains the word “Unauthorized”. If so, I authenticate and call the API again. For any other errors I sleep 10 seconds and retry; sometimes the API simply errors and a retry works.
The output variable looks like this:
|
1 2 3 4 5 6 7 8 |
$results responseMessage : Valid API Access count : 1 responseCode : SUCCESS lastSeenAssetId : hasMore : 0 assetListData : @{asset=System.Object[]} |
The responseCode is how you can check the call worked. Apart from any HTTP errors itself which you’d have to capture via try/ catch. For example, if I weren’t authorized I’d get an error like this:
Invoke-RestMethod: {"timestamp":"2023-03-11T08:59:48.328+00:00","path":"/rest/2.0/get/am/asset","status":401,"error":"Unauthorized","message":"Not authenticated","requestId":"cf63dc9b-195070503"}
This is what I match against earler to re-authenticate.
The assetListData contains the actual results. This is a hash table. Has a single key called asset which is an array of results. In this case, a single result. So $results.assetListData.asset is what one needs to look at.
To put the single result above into a JSON file, for instance, I’d do:
|
1 |
$results.assetListData.asset | ConvertTo-Json -Depth 5 | Out-File blah.json |
Searching for assets
I had to search for all assets with an agent installed, so we could make a report with their Agent Id and names etc.
Page 27 has the details on that.
Key things to note with this are that it only returns 100 results per call. So you have to keep calling the API on a loop. Observe the output of the single asset query above:
|
1 2 3 4 5 6 7 8 |
$results responseMessage : Valid API Access count : 1 responseCode : SUCCESS lastSeenAssetId : hasMore : 0 assetListData : @{asset=System.Object[]} |
Here hasMore is 0 coz ther are no more results. But if I was doing a search, and say there were 102 results, the API would return 100 results and set hasMore to 1 and lastSeenAssetId to the 100th asset Id of those results. I’ll then have to call the API again, but pass it this last seen asset Id and I will get the remaining results from there on. Since there’s only 2 results this time hasMore will be set to 0 and lastSeenAssetId is empty.
In terms of the actual search itself, you can use filters (page 29, 30).
Luckily, with PowerShell I don’t need to create a file etc.
To simply get all assets here’s what I do:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
$assetsUrl = "${qg_server}/rest/2.0/search/am/asset?pageSize=300" try { $results = Invoke-RestMethod -Uri $assetsUrl -Headers $authToken -Method 'POS' -ContentType 'application/json' -ErrorAction Stop } catch { # Catch re-auth requirement if ($_ -match "Unauthorized") { $authToken = Authenticate-QualysAPI $authHeader = @{ "Authorization" = ("Bearer " + $authToken) } $results = Invoke-RestMethod -Uri $assetsUrl -Headers $authToken -Method 'POS' -ContentType 'application/json' -ErrorAction Stop } else { # Catch any other errors; sleep and retry Start-Sleep -Seconds 10 $results = Invoke-RestMethod -Uri $assetsUrl -Headers $authToken -Method 'POST' -ContentType 'application/json' -ErrorAction Stop } } |
This first part is pretty much the same as before, except that I use a different URL, and I set the number of results I want to be 300, and I use ‘POST’.
Let’s add these to an array:
|
1 2 3 4 5 6 7 |
$assetsArray = [System.Collections.Generic.List[object]]@() if ($results.responseCode -eq "SUCCESS") { foreach ($entry in $results.assetListData.asset) { $assetsArray.Add($entry) } } |
In the next part though I keep looping while hasMore is 1, and keep adding each result to the above array. Only difference between this part and the code above is I use a different variable for the API
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
while ($results.hasMore -eq 1) { $lastSeenAssetId = $results.lastSeenAssetId $assetsUrl2 = "${assetsUrl}&lastSeenAssetId=$lastSeenAssetId" try { $results = Invoke-RestMethod -Uri $assetsUrl2 -Headers $authToken -Method 'POST' -ContentType 'application/json' -ErrorAction Stop } catch { # Catch re-auth requirement if ($_ -match "Unauthorized") { $authToken = Authenticate-QualysAPI $authHeader = @{ "Authorization" = ("Bearer " + $authToken) } $results = Invoke-RestMethod -Uri $assetsUrl2 -Headers $authToken -Method 'POST' -ContentType 'application/json' -ErrorAction Stop } else { # Catch any other errors; sleep and retry Start-Sleep -Seconds 10 $results = Invoke-RestMethod -Uri $assetsUrl2 -Headers $authToken -Method 'POST' -ContentType 'application/json' -ErrorAction Stop } } if ($results.responseCode -eq "SUCCESS") { foreach ($entry in $results.assetListData.asset) { $assetsArray.Add($entry) } } } |
And that’s it.
If I wanted to filter results when polling the API I don’t have to use a file like in the Qualys examples. Can use a hash table like this:
|
1 2 3 4 5 6 7 8 9 |
$filterQuery = @{ "filters" = @( @{ "field" = "inventory.source" "operator" = "EQUALS" "value" = "QAGENT" } ) } |
The Appendix (page 75) has a list of supported operators and which operator works with which field. It’s kind of limited in that you can’t search for something being empty, for instance.
Once you define the query though, the rest is easy. Same two part code as above just that I use this line instead:
|
1 |
$results = Invoke-RestMethod -Uri $assetsUrl -Headers $authToken -Method $method -ContentType 'application/json' -ErrorAction Stop -Body ($filterQuery | ConvertTo-Json) |
And in the second part, where I am looping, change $assetsUrl to $assetsUrl2 of course.