{"id":5900,"date":"2021-10-16T14:19:20","date_gmt":"2021-10-16T13:19:20","guid":{"rendered":"https:\/\/rakhesh.com\/?p=5900"},"modified":"2021-11-29T16:21:22","modified_gmt":"2021-11-29T16:21:22","slug":"reading-an-event-hub-from-azure-functions-and-or-logic-apps","status":"publish","type":"post","link":"https:\/\/rakhesh.com\/azure\/reading-an-event-hub-from-azure-functions-and-or-logic-apps\/","title":{"rendered":"Reading an Event Hub from Azure Functions and\/ or Logic Apps"},"content":{"rendered":"

Continuing from the previous post<\/a> where I exported data from Log Analytics to an Event Hub I now want to capture events from an Event Hub and do an HTTP trigger to an external REST API. I can do this via Logic Apps or Azure Functions (and others too I suppose but these are the two I tried).<\/p>\n

Update<\/strong> (19 Oct 2021): The Functions way doesn’t really work so if you are reading this post for that feel free to skip this one and read my later post<\/a> instead. Of course this one’s a good read to know what stupid mistakes I made. \ud83e\udd26\u200d\u2642\ufe0f<\/p>\n

Logic Apps<\/h3>\n

I started with this as I figure this would be easier. I am going to skip the basic steps here like how to create a Logic App etc.<\/p>\n

First get stuff from the Event Hub. For this we need to create a connection<\/a> to it. While doing this the first time I thought I could get away with an access policy that has only the Listen<\/code> claim but that didn’t help. Looks like you need an access policy with all three rights, similar to the default one. So that’s something to bear in mind.<\/p>\n

When creating the trigger initially I left things at the default and just ran it to see what the output looks like. It was like this:<\/p>\n

\"\"<\/p>\n

That seems to be some base64 encoded data. Then I realized I can change the content type in the trigger to application\/json<\/code>.<\/p>\n

\"\"<\/p>\n

Did that and I get better results.<\/p>\n

\"\"<\/p>\n

The output is now JSON. It contains a single “record<\/code>” property that is an array. Each element of the array is the following:<\/p>\n

{\r\n      \"TenantId\": \"xxxxx\",\r\n      \"TimeGenerated\": \"xxxxx\",\r\n      \"SourceSystem\": \"xxxxx\",\r\n      \"Account\": \"xxxxx\",\r\n      \"AccountType\": \"xxxxx\",\r\n      \"Computer\": \"xxxxx\",\r\n      \"EventSourceName\": \"xxxxx\",\r\n      \"Channel\": \"xxxxx\",\r\n      \"Task\": \"xxxxx\",\r\n      \"Level\": \"xxxxx\",\r\n      \"EventID\": \"xxxxx\",\r\n      \"Activity\": \"xxxxx\",\r\n      \"SourceComputerId\": \"xxxxx\",\r\n      \"EventOriginId\": \"xxxxx\",\r\n      \"MG\": \"xxxxx\",\r\n      \"TimeCollected\": \"xxxxx\",\r\n      \"ManagementGroupName\": \"xxxxx\",\r\n      \"PrivilegeList\": \"xxxxx\",\r\n      \"SubjectAccount\": \"xxxxx\",\r\n      \"SubjectDomainName\": \"xxxxx\",\r\n      \"SubjectLogonId\": \"xxxxx\",\r\n      \"SubjectUserName\": \"xxxxx\",\r\n      \"SubjectUserSid\": \"xxxxx\",\r\n      \"Type\": \"xxxxx\",\r\n      \"_Internal_WorkspaceResourceId\": \"xxxxx\",\r\n      \"_ResourceId\": \"\/subscriptions\/xxxxxx\/resourceGroups\/yyyyyy\/providers\/Microsoft.Compute\/virtualMachines\/aaaaaa\"\r\n}<\/pre>\n
I copied everything in that Content box in the output. Then I went back to the Logic App and added a Parse JSON<\/code> action. I pasted the text I copied above into the sample payload section and so Logic Apps helpfully generated a schema.<\/p>\n
\"\"<\/p>\n
Here’s the schema just in case anyone’s following along:<\/p>\n
{\r\n    \"properties\": {\r\n        \"records\": {\r\n            \"items\": {\r\n                \"properties\": {\r\n                    \"Account\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"AccountType\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"Activity\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"AuthenticationPackageName\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"Channel\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"Computer\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"EventData\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"EventID\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"EventOriginId\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"EventSourceName\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"ImpersonationLevel\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"IpAddress\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"IpPort\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"KeyLength\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"Level\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"LmPackageName\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"LogonGuid\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"LogonProcessName\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"LogonType\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"LogonTypeName\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"MG\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"ManagementGroupName\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"PrivilegeList\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"Process\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"ProcessId\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"ProcessName\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"SourceComputerId\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"SourceSystem\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"SubjectAccount\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"SubjectDomainName\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"SubjectLogonId\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"SubjectUserName\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"SubjectUserSid\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"TargetAccount\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"TargetDomainName\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"TargetLogonId\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"TargetUserName\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"TargetUserSid\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"Task\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"TenantId\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"TimeCollected\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"TimeGenerated\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"TransmittedServices\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"Type\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"WorkstationName\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"_Internal_WorkspaceResourceId\": {\r\n                        \"type\": \"string\"\r\n                    },\r\n                    \"_ResourceId\": {\r\n                        \"type\": \"string\"\r\n                    }\r\n                },\r\n                \"required\": [\r\n                    \"TenantId\",\r\n                    \"TimeGenerated\",\r\n                    \"SourceSystem\",\r\n                    \"Computer\",\r\n                    \"EventSourceName\",\r\n                    \"Channel\",\r\n                    \"Task\",\r\n                    \"Level\",\r\n                    \"EventID\",\r\n                    \"Activity\",\r\n                    \"SourceComputerId\",\r\n                    \"EventOriginId\",\r\n                    \"MG\",\r\n                    \"TimeCollected\",\r\n                    \"ManagementGroupName\",\r\n                    \"Type\",\r\n                    \"_Internal_WorkspaceResourceId\",\r\n                    \"_ResourceId\"\r\n                ],\r\n                \"type\": \"object\"\r\n            },\r\n            \"type\": \"array\"\r\n        }\r\n    },\r\n    \"type\": \"object\"\r\n}<\/pre>\n
Next I added a For each<\/code> action. I know the records<\/code> property is an array so I added that as the input to the For each<\/code> (that is the only option anyways).<\/p>\n
\"\"<\/p>\n
Now I want to parse the individual entry. So I added an action for that too. For the schema I copy pasted an individual entry from the earlier output I captured from the Content box above and pasted that in. This gave me a schema:<\/p>\n
{\r\n    \"type\": \"object\",\r\n    \"properties\": {\r\n        \"TenantId\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"TimeGenerated\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"SourceSystem\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"Account\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"AccountType\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"Computer\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"EventSourceName\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"Channel\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"Task\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"Level\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"EventID\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"Activity\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"SourceComputerId\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"EventOriginId\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"MG\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"TimeCollected\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"ManagementGroupName\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"AuthenticationPackageName\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"ImpersonationLevel\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"IpAddress\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"IpPort\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"KeyLength\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"LmPackageName\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"LogonGuid\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"LogonProcessName\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"LogonType\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"LogonTypeName\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"Process\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"ProcessId\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"ProcessName\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"SubjectAccount\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"SubjectDomainName\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"SubjectLogonId\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"SubjectUserName\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"SubjectUserSid\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"TargetAccount\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"TargetDomainName\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"TargetLogonId\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"TargetUserName\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"TargetUserSid\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"TransmittedServices\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"Type\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"WorkstationName\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"_Internal_WorkspaceResourceId\": {\r\n            \"type\": \"string\"\r\n        },\r\n        \"_ResourceId\": {\r\n            \"type\": \"string\"\r\n        }\r\n    }\r\n}<\/pre>\n
 <\/p>\n
\"\"<\/p>\n
Right ho! Let’s run that to see how things look.<\/p>\n
\"\"<\/p>\n
Perfect! Now I can bung this off in an HTTP Request:<\/p>\n
\"\"<\/p>\n
And that’s it!<\/p>\n
(In reality I added an additional step to get the API key from a Key Vault and also some filtering of the JSON to skip records I am not interested in… but that doesn’t matter here)<\/em>.<\/p>\n
Azure Function<\/h3>\n
Create a new function app. PowerShell’s all I know so that’s what I am going to use:<\/p>\n
\"\"<\/p>\n
 <\/p>\n
Go with the serverless plan.<\/p>\n
Then create a new function. Select the Event Hub trigger template. Add a connection to the Event Hub (a policy using just the Listen<\/code> claim is sufficient). Put in the Event Hub name.<\/p>\n
\"\"<\/p>\n
You now end up with a function like this but it doesn’t work:<\/p>\n
param($eventHubMessages, $TriggerMetadata)\r\n\r\nWrite-Host \"PowerShell eventhub trigger function called for message array: $eventHubMessages\"\r\n\r\n$eventHubMessages | ForEach-Object { Write-Host \"Processed message: $_\" }\r\n<\/pre>\n
The output is like this:<\/p>\n
2021-10-16T10:40:20.364 [Information] Executed 'Functions.EventHubTrigger1' (Succeeded, Id=eeea7a72-2b03-4c45-9f71-48e312e83659, Duration=146ms)\r\n2021-10-16T10:40:20.434 [Information] Executing 'Functions.EventHubTrigger1' (Reason='(null)', Id=2c47176a-e485-422e-81ae-9597962695a2)\r\n2021-10-16T10:40:20.434 [Information] Trigger Details: PartionId: 2, Offset: 73025121728-73025121728, EnqueueTimeUtc: 2021-10-16T10:40:20.2040000Z-2021-10-16T10:40:20.2040000Z, SequenceNumber: 6329-6329, Count: 1\r\n2021-10-16T10:40:20.453 [Information] INFORMATION: PowerShell eventhub trigger function called for message array: System.Collections.Hashtable\r\n2021-10-16T10:40:20.454 [Information] INFORMATION: Processed message: System.Collections.Hashtable\r\n2021-10-16T10:40:20.455 [Information] Executed 'Functions.EventHubTrigger1' (Succeeded, Id=2c47176a-e485-422e-81ae-9597962695a2, Duration=27ms)\r\n2021-10-16T10:40:20.792 [Information] Executing 'Functions.EventHubTrigger1' (Reason='(null)', Id=9e84634b-797b-40ae-9d88-a198d5a17835)\r\n2021-10-16T10:40:20.792 [Information] Trigger Details: PartionId: 2, Offset: 73025445776-73025445776, EnqueueTimeUtc: 2021-10-16T10:40:20.7540000Z-2021-10-16T10:40:20.7540000Z, SequenceNumber: 6330-6330, Count: 1\r\n2021-10-16T10:40:20.825 [Information] INFORMATION: PowerShell eventhub trigger function called for message array: System.Collections.Hashtable\r\n2021-10-16T10:40:20.825 [Information] INFORMATION: Processed message: System.Collections.Hashtable\r\n2021-10-16T10:40:20.826 [Information] Executed 'Functions.EventHubTrigger1' (Succeeded, Id=9e84634b-797b-40ae-9d88-a198d5a17835, Duration=42ms)\r\n2021-10-16T10:40:21.199 [Information] Executing 'Functions.EventHubTrigger1' (Reason='(null)', Id=d98c5525-5e3b-45f6-bb57-b512ce8dc8fa)\r\n2021-10-16T10:40:21.199 [Information] Trigger Details: PartionId: 2, Offset: 73025768160-73025768160, EnqueueTimeUtc: 2021-10-16T10:40:21.1640000Z-2021-10-16T10:40:21.1640000Z, SequenceNumber: 6331-6331, Count: 1<\/pre>\n
Having no idea what’s wrong here, but knowing the objects seem to be hash tables I tried the following:<\/p>\n
param($eventHubMessages, $TriggerMetadata)\r\n\r\nWrite-Host \"PowerShell eventhub trigger function called for message array: $eventHubMessages\"\r\n\r\n# $eventHubMessages | ForEach-Object { Write-Host \"Processed message: $_\" }\r\n\r\n$eventHubMessages | ForEach-Object { Write-Host \"Processed message: $($_ | Get-Member) \" }<\/pre>\n
That gave the following error:<\/p>\n
2021-10-16T10:42:58.840 [Error] Executed 'Functions.EventHubTrigger1' (Failed, Id=4379689c-e1fb-44ab-8609-586e7389384c, Duration=6ms)Binding parameters to complex objects (such as 'Object') uses Json.NET serialization.1. Bind the parameter type as 'string' instead of 'Object' to get the raw values and avoid JSON deserialization, or2. Change the queue payload to be valid json. The JSON parser failed: Unexpected character encountered while parsing value: T. Path '', line 0, position 0.<\/pre>\n
Trying to access a single property didn’t help either:<\/p>\n
# Since I know the JSON has the Computer property (based on what I did with Logic Apps above)\r\n$eventHubMessages | ForEach-Object { Write-Host \"Processed message: $($_.Computer) \" }<\/pre>\n
Hmm.<\/p>\n
Googling on this latest error got me to this GitHub issue though<\/a>. Looks like I should specify the binding as string.<\/p>\n
That’s under the “Integration” section, so I did just that:\"\"<\/p>\n
Select “String”:<\/p>\n
\"\"<\/p>\n
I reverted the code back to the default:<\/p>\n
param($eventHubMessages, $TriggerMetadata)\r\n\r\nWrite-Host \"PowerShell eventhub trigger function called for message array: $eventHubMessages\"\r\n\r\n$eventHubMessages | ForEach-Object { Write-Host \"Processed message: $_\" }\r\n<\/pre>\n
Any better now? Yes:<\/p>\n
2021-10-16T12:38:14.344 [Information] INFORMATION: Processed message: {\"records\": [{ ....<\/pre>\n
I truncated the output as there’s a lot. As before though I was looking at a records<\/code> property that was an array. Thing is, since I said this was a string I figured I’ll have to convert it to JSON first and then try to manipulate. So I did the following:<\/p>\n
param($eventHubMessages, $TriggerMetadata)\r\n\r\n# Commenting this out to reduce the noise\r\n# Write-Host \"PowerShell eventhub trigger function called for message array: $eventHubMessages\"\r\n\r\n$eventHubMessages | ConvertFrom-JSON | ForEach-Object { \r\n    Write-Host \"==== Message Array ====\"  \r\n    foreach ($record in $_.records) {\r\n\tWrite-Host \"**** JSON ****\"\r\n        ConvertTo-JSON $record -Depth 10\r\n    } \r\n}<\/pre>\n
That works!<\/p>\n
2021-10-16T13:05:26.890 [Information] INFORMATION: **** JSON ****\r\n2021-10-16T13:05:26.891 [Information] OUTPUT: {\"TenantId\": \"xxxxx\",,\"TimeGenerated\": \"xxxxx\",,\"SourceSystem\": \"xxxxx\",,\"Account\": \"xxxxx\",,\"AccountType\": \"xxxxx\",,\"Computer\": \"xxxxx\",,\"EventSourceName\": \"xxxxx\",,\"Channel\": \"xxxxx\",,\"Task\": \"xxxxx\",,\"Level\": \"xxxxx\",,\"EventID\": \"xxxxx\",,\"Activity\": \"xxxxx\",,\"SourceComputerId\": \"xxxxx\",,\"EventOriginId\": \"xxxxx\",,\"MG\": \"xxxxx\",,\"TimeCollected\": \"xxxxx\",,\"ManagementGroupName\": \"xxxxx\",,\"AuthenticationPackageName\": \"xxxxx\",,\"ImpersonationLevel\": \"xxxxx\",,\"IpAddress\": \"xxxxx\",,\"IpPort\": \"xxxxx\",,\"KeyLength\": \"xxxxx\",,\"LmPackageName\": \"xxxxx\",,\"LogonGuid\": \"xxxxx\",,\"LogonProcessName\": \"xxxxx\",,\"LogonType\": \"xxxxx\",,\"LogonTypeName\": \"xxxxx\",,\"Process\": \"xxxxx\",,\"ProcessId\": \"xxxxx\",,\"ProcessName\": \"xxxxx\",,\"SubjectAccount\": \"xxxxx\",,\"SubjectDomainName\": \"xxxxx\",,\"SubjectLogonId\": \"xxxxx\",,\"SubjectUserName\": \"xxxxx\",,\"SubjectUserSid\": \"xxxxx\",,\"TargetAccount\": \"xxxxx\",,\"TargetDomainName\": \"xxxxx\",,\"TargetLogonId\": \"xxxxx\",,\"TargetUserName\": \"xxxxx\",,\"TargetUserSid\": \"xxxxx\",,\"TransmittedServices\": \"xxxxx\",,\"Type\": \"xxxxx\",,\"WorkstationName\": \"xxxxx\",,\"_Internal_WorkspaceResourceId\": \"xxxxx\",,\"_ResourceId\": \"xxxxx\",}\r\n2021-10-16T13:05:26.891 [Information] INFORMATION: **** JSON ****\r\n2021-10-16T13:05:26.891 [Information] OUTPUT: {\"TenantId\": \"xxxxx\",,\"TimeGenerated\": \"xxxxx\",,\"SourceSystem\": \"xxxxx\",,\"Account\": \"xxxxx\",,\"AccountType\": \"xxxxx\",,\"Computer\": \"xxxxx\",,\"EventSourceName\": \"xxxxx\",,\"Channel\": \"xxxxx\",,\"Task\": \"xxxxx\",,\"Level\": \"xxxxx\",,\"EventID\": \"xxxxx\",,\"Activity\": \"xxxxx\",,\"SourceComputerId\": \"xxxxx\",,\"EventOriginId\": \"xxxxx\",,\"MG\": \"xxxxx\",,\"TimeCollected\": \"xxxxx\",,\"ManagementGroupName\": \"xxxxx\",,\"LogonType\": \"xxxxx\",,\"LogonTypeName\": \"xxxxx\",,\"TargetAccount\": \"xxxxx\",,\"TargetDomainName\": \"xxxxx\",,\"TargetLogonId\": \"xxxxx\",,\"TargetUserName\": \"xxxxx\",,\"TargetUserSid\": \"xxxxx\",,\"Type\": \"xxxxx\",,\"_Internal_WorkspaceResourceId\": \"xxxxx\",,\"_ResourceId\": \"xxxxx\",}<\/pre>\n
I added those extra text markers in there so I can easily identify the individual JSON pieces. In the final code I can remove those:<\/p>\n
param($eventHubMessages, $TriggerMetadata)\r\n\r\n# Commenting this out to reduce the noise\r\n# Write-Host \"PowerShell eventhub trigger function called for message array: $eventHubMessages\"\r\n\r\n$eventHubMessages | ConvertFrom-JSON | ForEach-Object { \r\n    foreach ($record in $_.records) {\r\n        ConvertTo-JSON $record -Depth 10\r\n    } \r\n}<\/pre>\n
All I need to do now is post this to a REST API. That’s easy stuff.<\/p>\n
Update<\/strong> (19 Oct 2021): Turns out I might have mistaken coz when I got to the supposedly easy stuff of posting this via a REST API I realized that the output isn’t formatted well. Here’s an example of what it looked like on the receiving side:<\/p>\n
[\r\n  \"{\\r\",\r\n  \"  \\\"AccountType\\\": \\\"User\\\",\\r\",\r\n  \"  \\\"Level\\\": \\\"8\\\",\\r\",\r\n  \"  \\\"MG\\\": \\\"00000000-0000-0000-0000-000000000001\\\",\\r\",\r\n  \"  \\\"ImpersonationLevel\\\": \\\"%%1833\\\",\\r\",\r\n  \"  \\\"LogonGuid\\\": \\\"00000000-0000-0000-0000-000000000000\\\",\\r\",\r\n  \"  \\\"ProcessId\\\": \\\"0x274\\\",\\r\",\r\n  \"  \\\"SubjectLogonId\\\": \\\"0x3e7\\\",\\r\",\r\n  \"  \\\"TargetLinkedLogonId\\\": \\\"0x0\\\",\\r\",<\/pre>\n
When I output it to the logs it looks fine, but when POSTing it it’s messed up. I think something about the conversion from JSON to hash table and back is messing things up. I tried alternatives like ConvertFrom-JSON -AsHashTable\u00a0 <\/code>(because the default is to convert it to a PSCustomObject<\/code>) but that didn’t help either.<\/p>\n
Another thing I noticed is that each $eventHubMessages<\/code> object itself could contain many records<\/code> JSON arrays. So I tried the following but no luck:<\/p>\n
foreach ($eventHubMessage in $eventHubMessages) {\r\n    $recordArray = ConvertFrom-JSON -AsHashTable -Depth 10 $eventHubMessage\r\n    foreach ($record in $recordArray.records) \r\n    {\r\n        Write-Host $record.Computer \" \" $record.EventID\r\n\r\n        if ($interestedComputers -contains $record.Computer -and $interestedEvents -contains $record.EventID) {\r\n            Write-Host \"Found interested EventID ($record.EventID) from computer ($record.Computer)\"\r\n            \r\n            $request = Invoke-WebRequest -Uri $Url -Method \"POST\" -Headers $headers -ContentType $contentType -Body (ConvertTo-JSON -Depth 10 $record)\r\n            if ($request.StatusCode -eq 200) { Write-Host \"Successfully pushed to Qomplx\" } else { Write-Host \"Error \" $request.StatusCode }\r\n        } else {\r\n            # Write-Host \"Skipping record\"\r\n        }\r\n    }\r\n}<\/pre>\n
I am converting the incoming object from a JSON array to an array of hash tables and then iterating through it. It works fine on paper – I can get the Computer name and EventID etc. but when I push it out it’s f**ed up. :-\/<\/p>\n
Anyways, I don’t have too much time to spend on this (already wasted a day and weekend) so I think I’ll stick to Logic Apps for now.<\/p>\n
Update<\/strong> (20 Oct 2021): I wonder if it’s related to this PowerShell Core issue on GitHub<\/a>. There’s a SO post too<\/a> I found from the GitHub issue.<\/p>\n
Update<\/strong> (25 Oct 2021): I now have this working. It was just a case of me not understanding how Functions work. I first talk about my folly in this blog post<\/a> and later in this blog post<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"
Continuing from the previous post where I exported data from Log Analytics to an Event Hub I now want to capture events from an Event Hub and do an HTTP trigger to an external REST API. I can do this via Logic Apps or Azure Functions (and others too I suppose but these are the … Continue reading Reading an Event Hub from Azure Functions and\/ or Logic Apps<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[887],"tags":[383,105,503,1046],"jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/rakhesh.com\/wp-json\/wp\/v2\/posts\/5900"}],"collection":[{"href":"https:\/\/rakhesh.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rakhesh.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rakhesh.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rakhesh.com\/wp-json\/wp\/v2\/comments?post=5900"}],"version-history":[{"count":0,"href":"https:\/\/rakhesh.com\/wp-json\/wp\/v2\/posts\/5900\/revisions"}],"wp:attachment":[{"href":"https:\/\/rakhesh.com\/wp-json\/wp\/v2\/media?parent=5900"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rakhesh.com\/wp-json\/wp\/v2\/categories?post=5900"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rakhesh.com\/wp-json\/wp\/v2\/tags?post=5900"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}