Graph API group delta changes

Long time no posts!

The other day at work I had to setup an automation that synced group membership changes in one group to another.

Typically, I would have read the membership of the source group, membership of the destination group, and done a quick compare of them to add/ remove elements. I script in PowerShell and thanks to a StackOverflow post I came across in the past I use the following code to do this:

This is slow though because I have some 100+ groups to do with this and while the actual comparision is fast I still have to make all those Graph API calls to get membership and such, which eventually leads to throttling and disconnects etc. It’s not bad, and I don’t plan on replacing a working thing with something else, but with the new task I had to do I wanted to try something different.

For one, while previously I wanted to do a full sync – i.e. remove anything in the destination group that’s not present in the source group – this time I only wanted a one way sync. Any additions/ deletions in the source group must be made in the destination, but I don’t care if anyone adds/ removes entries in the destination group itself. That is to say, if my source group contains users A, B, and C; and the destination group too contains the same. Then I add D and E, I want these to sync over to the destination; but now if someone removes E from the destination I am OK with that. The source group will thus contain A, B, C, D, E, while the destination contains A, B, C, D. If someone adds F to the destination group, I am fine with that stayin on… even though the source group doesn’t have it.

The best approach for this seems to be the group: delta Graph API call. It’s a bit of a weird call coz you don’t actually run this against a particular group. You could filter against a particular group, but by default it just gives everything. For example:

The output contains value contains everything. Here’s an example entry from the results:

This is a group that currently exists in my tenant. Notice the members@delta entry? Here’s what that contains:

If I look at the group, it has just one user and its id matches the output of members@delta.

This gives us an idea of the output of this API. It shows every group addition and deletion in the tenant, along with any group membership additions and deletions as well as any change in other information. One can “replay” this in a way to get to the same state as things are currently.

Here’s what another entry looks like:

I couldn’t find that id anywhere in my tenant. Until I went through the audit logs and realized this is a group I deleted a few days ago (oddly it didn’t appear in the “Deleted Groups” section either). This is what I was talking about – the output includes even groups that don’t exist any more.The @removed property tells me the reason too – the group was deleted.

The output seems to be limited to additions/ deletions only, not other changes. For instance, I renamed a group but there was no property capturing this fact.

Going back to the original output:

There’s also an @odata.nextLink property. This is a link that can be called to get the next set of results. If I do that, I see the following (I am doing the first step again so I capture @odata.nextLink):

In the second run, I don’t get a @odata.nextLink any more. Instead I get @odata.deltaLink. What does this mean?

The way the delta API works is that the output includes these tokens – a skip token ($skiptoken) and a delta token ($deltatoken). The skip token is an indicator of how much output was returned; so calling the API again with the skip token gives you the next set of output with a new skip token. The idea is you keep calling the API with skip tokens until you get all the output and skip token is empty.

To make things easy, the API gives you a new URL each time with the skip token added. That’s what @odata.nextLink is, which is why the URL looks like https://graph.microsoft.com/v1.0/groups/delta?$skiptoken=eKgfrPQSvBw5VPEZGw9FfwaG4... – the API is being helpful.

Similarly, the delta token is a token that captures the point in time or state of the output. You only get the delta token when you get the final set of results – i.e. when there is no more skip token. So essentially, you keep polling the API with skip tokens until you get no skip token; at which point you should have a delta token and so you capture that somewhere. The next time you call the API – which can be immediately or after how many every hours or days – if you do so with the delta token, it will give you all the changes since the previous invocation. Nice, huh? And at the end of that you will get a new delta token which can be used in subsequent invocations to get all the changes since then.

This is what @odata.deltaLink contains – which is why the URL looks like https://graph.microsoft.com/v1.0/groups/delta?$deltatoken=eKgfrPQSvBw5VPEZGw9Ff...

All this and more can be found in the delta API docs  by the way.

Here’s me invoking the URL with the delta token.

No changes, hence value is empty.

Let me go and rename a group now and run the same URL.

Nice! There’s the group Id. Let’s look at the value:

Hmm, not super helpful. I know what I changed in this case – the displayName – but there’s no way to glean that from the output. For comparison, here’s the output from the original delta query with none of the tokens. Since I stored that in a variable I can refer to it here:

Turns out there’s a way to get just the difference.

I was hoping it might tell me what the previous value is, but it doesn’t. Instead all this one does is show the changed properties, along with standard output like the id.

Here’s the same after I changed the description too:

Now let me add someone and try with the same URL:

Kind of obvious, but like with other Graph API calls one can focus on the properties to be tracked. Say I am not interested in any other changes except the display name. In this case my initial query will have to mention that:

And subequent delta queries will only show changes to that.

Ditto for just group membership changes, which is what I am interested in. $select=members

What is the actual output in the value field?

How can I do this better? Need a loop! 😊

Here’s a sample output:

So far I have been dealing with multiple groups. Let’s focus on one now. To do that you filter by the group id.

I have to do $filter= id eq 'a6131882-b110-4dc6-b1ea-e8c3c5cfccdb' but since I am already within single quotes I got to escape these. Thus I put $filter= id eq ''a6131882-b110-4dc6-b1ea-e8c3c5cfccdb'' (to escape the single quote put it twice).

A brief digression on the Get-MgGroupDelta cmdlet.

It looks to be useful but is pretty useless actually. Here’s how its default output looks like:

Nice. The equivalent to get a single group would be:

So far so good. The beauty of using this cmdlet is that one doesn’t have to bother with @odata.nextLink any more. The cmdlet will loop over that and return all results. Any deltas are under AdditionalProperties.

But where do I get the delta link for the next invocation? No idea. Or how do I invoke this cmdlet in a delta run? No idea. The help page is useless, and I couldn’t find anything in Google or GitHub. Eventually I gave up.

Back to what we were doing. Here’s the equivalent code for a single group id.

In my case I had to do this for about 40+ groups. Not a problem, I can loop over the ids. But what do I do with the delta links? How do I store it for the next time around?

Typically I would use SharePoint for such a task. In fact, I would read the group names from there and store the link in one of the columns. But this was an Azure Function I was expecting to run quite often and for some reason I didn’t want to depend on SharePoint. I felt it could be an additional point of failure – Graph or PnP.PowerShell might make a call and fail and I don’t want the group syncs to fail because of that. I wanted something local to the Function itself where I store the delta links temporarily.

I suppose I could have gone with storing them as files in the storage account associated with the Function. But then I thought let me use Azure Table storage. This too is present in the storage account after all.

I am not going to go into the details of that here. I’ll probably blog it later. As of now I spent most of my weekend morning writing the current post. 😱

Update: Continued…