Connecting Azure Function (PowerShell) to Cosmos DB

There are numerous blog posts on this but I couldn’t quite figure out how to do this. I guess it’s a Sunday, my brain isn’t working… 😊

Also, this blog post is more like notes for myself as I am pretty much creating it as I figure this stuff. Usually I take notes in Bear  and then write a blog post, this time I am skipping that to save me some double work. Instead, I am creating a test version of the items below so I get a hang of what needs doing; and then I’ll recreate it for real later. This also means, I will be going with defaults for most of the settings as the aim is to get the two talking to each other as quickly as possible.

Cosmos DB

First off, creating a Cosmos DB account. I want a NoSQL one, this is just for storing a bunch of JSON files. I don’t need high performance etc. currently, so it’s mostly going to be default settings. Serverless.

This is a helpful document to understand the elements of Cosmos DB.

The database account contains a database, which in turn contains one or more containers, which contain the items.

(Both images are from the article I linked to above). Note that what a “database”, “container”, or “item” actually refers to depends on what type of Cosmos DB is in use.

Once it is created, I will create a container.

After clicking “Add Container” I am presented with the following screen. I fill it thus:

I have to give a Database id – the name of the database, basically. I will call it “rakcosmosdb”. Within the Cosmos DB account I can have more than one database. I have to give a Container id. I will call this “appRegistrations-Prod”. This contain will contain a bunch of App Registrations, and will be used by my production instance – hence I name it that way. Finally, for the partition key I will choose “objectId”. I know this is a property that exists in the App Registration. It is unique. And using this Cosmos can distribute the data efficiently.

The result is the following.

Azure Function

Next, I’ll create the Azure Function App. Again, this is just a basic setup, going with the defaults.

Go through the rest of the screens and create the Function App.

I found this overview document that I am going to follow now.

First off, it’s for Function runtime versions 2.x and above; and using extension bundle 4.x. Looks like I am on 4.x runtime, so that’s good.

Now for extension bundles.

Something to note, only Cosmos DB for NoSQL is supported via extensions in Azure Functions. Cosmos DB for Table too is supported via Table bindings; for the rest one must use other ways. Let’s go with version 4.x of the extension. I need to add the following code to host.json.

The host.json can be found under the App Files section. Here’s the default as of now:

I must replace the highlighted section.

Next, I think I have to add the following too to host.json:

I am not sure to be honest, let’s add and see. Here’s the final result:

I set the userAgentSuffix to something that identifies the Function App. Could have just used the Function App name.

Now I will create an HTTP triggered function, just to test things out. Going with the default settings for now. After creating, I went to the Integration section and tried adding an output binding.

Click on “Add output above. Select Cosmos DB:

Click “New” to create a new connection:

It shows my Cosmos DB already. I select that and click OK.

Here’s what the screen looks like now:

I will leave the document parameter name as is. From when I created the Cosmos DB account, I know my database name is “rakcosmosdb”. And the collection name is “appRegistrations-Prod”. Fill these and click OK.

What the creation of a new connection has done is that it’s created the following in the Configuration section.

This contains the connection string to the Cosmos DB.

And in the Function itself, I now have the following in function.json.

Here’s what run.ps1 currently looks like:

Let me add something to write to the DB too.

Then I clicked the Test/Run button to see what happens.

Hmm, not good.

This stumped me for a while. Initially, I didn’t find anything additional in the “Monitor” section either; but after about 2-3 minutes I found errors there too. And those had more details:

This is “CosmosDB”, not “cosmosDB”. The latter is what was added to host.json and function.json. Where is it picking that from?

Googling a bit, I came across this post. Sounds like I have to use a Managed Identity? The bundle install page did say v4.x introduces the ability to connect with a Managed Identity, but I figured that’s just an optional thing… especially since the portal itself created the connection string for me. But I guess not. I think once I use the 4.x version I must use a Managed Identity.

Ok, step 1 enable the identity for the Function App.

Step 2 grant it permissions.

What the heck, I need to create a role using the Cosmos DB built-in RBAC system? Can’t do that via the portal! 😡

So, open Cloud Shell. Switch to PowerShell.

I know the id of my Function App Managed Identity from the portal as it’s shown on the screen.

But just in case I didn’t want to lookup via the portal, I can use the following to find it:

I found the following snippet in the RBAC document:

Before I fill that I need to get the role definition Id. I know I want the Cosmos DB Built-in Data Contributor role.

Now I have:

Copy paste that into the Cloud Shell, wait up to a minute (coz it just hung for me), and boom its in place!

And finally, step 3, add these to the app settings. I replaced the existing connection string that was automatically created, with this:

The “rakcosmosdb01224_DOCUMENTDB” part is my connection string in function.json. To that I added two underscores, and the word “credential”. And changed the value to “managedidentity”.

I think that’s all I need, coz according to the doc the other two are optional if I am using the system managed identity.

Did that work? Now I got a different error, but that’s a good thing I think:

I know what to do for this. It’s the missing module. In the App Files section I must edit requirements.psd1 and uncomment the line for the Az module.

Restart the Function App after that, and let’s try again.

Waiting… 🥃

Still waiting… 🥃😴

🥱

Sighs.

Let’s try again.

This is the part I hate with Function Apps and modules. The first few times are a torture while it downloads.

I did the Test/Run again and it’s again downloading.

Finally it succeeded. But it kept erroring upon pushing. Same error:

I remember this being a PITA when I was figuring out Managed Identities and Event Hubs in the past. I tried to read up on the extenstion itself and found this blog post introducing it. So I need a second setting: rakcosmosdb01224_DOCUMENTDB__accountEndpoint with value https://rakcosmosdb01224.documents.azure.com:443/.

Silly me, of course it needs to know the details of the Cosmos DB account. Previously this was in the connection string I deleted.

Added these. Saved.

Try again!

🤬

Still no luck, after that completes. What the heck.

Ooh, one more thing from the blog post.

Huh. Ok, so I rename that in function.json.

Almost missed above, “Anything named Collection is now Container“. This one’s my bad, it is there in the docs I was following too. (And I realized, further below in the docs, outside of the example given, they do use “connection” instead of “connectionStringSetting”).

Screenshot of the v4.x settings:

And… test!

Still I get the error.

Here’s a thought, maybe it’s not happy with the connection string name? Let me change it from “rakcosmosdb01224_DOCUMENTDB” to “rakcosmosdb01224” both in function.json and the two configuration settings.

Nope, no luck.

And then I noticed a weird thing. Even though I had updated function.json, when I went to a different page and came back the changes had reverted. Huh. It wasn’t a one-time – I tried it couple of times with the same result.

So I removed the following from the file (this is in the incorrect bit):

Saved it. Went to the Integration section. Ensured it’s removed, and then I added it back correctly.

Oddly, this time when I ran the Function it didn’t complain… but looks like everything I added to function.json disappeared. What the heck. When I ran the Function again it complained that [Error] ERROR: The specified name ‘outputDocument’ cannot be resolved to a valid output binding of this function.

I don’t know what’s going on here. I repeated the steps, and again it kept disappearing.

At this point I am just going to delete the Function and recreate it. Not the Function App, just the Function. Here’s the code as a backup:

Ok, new Function created. Copy pasted the stuff. But nope, now it complains it cannot find the output binding! Coz it has disappeared.

If I re-add it, restart the Function App, the part I added stays. Yet, doing a Test/Run seems to nuke it.

At this point I went into a stupid cycle of copy pasting it again, trying to restart the Function App a few times etc… until I think I finally made it stick. I have no idea what I did. It was just stupid persistence and frustration I think.

On the plus side, now that it sticks, while the console continues to give an error the message different.

Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: fb0616b6-9c09-4be9-bb3f-0b6c034c4ae9; Reason: (Message: {“Errors”:[“One of the specified inputs is invalid”]} ActivityId: fb0616b6-9c09-4be9-bb3f-0b6c034c4ae9, Request URI: /apps/0b12014b-f854-44a4-8f6f-c5cedc3481fe/services/4919abae-b1f5-4b75-9186-772877134ddd/partitions/78009af3-c91e-4793-b1f9-77ad07aa126b/replicas/133345776092212266p/, RequestStats: RequestStartTime: 2023-07-23T19:08:18.6420113Z, RequestEndTime: 2023-07-23T19:08:18.6435872Z, Number of regions attempted…

A quick Google pointed me to the answer: I should be adding an Id property, of type string.

Did that, and finally finally finally it works!

Phew.

To Summarize?

I’ve lost the plot of things I did here. 😃 So here’s a summary for next time:

• Create the Cosmos DB as usual (detailed above).
• On the Function App side, if you are using the v4.x extension, then add the following to host.json.

• Since we’ll be using a Managed Identity, enable that (Azure Function).
• Modify requirements.psd1 to install the Az module.

• Run the following in Cloud Shell to grant the Function App Managed Identity Write access to the Cosmos DB account.

• Add two app settings under the Configuration section of the Function App.

rakcosmosdb01224__accountEndpoint	https://rakcosmosdb01224.documents.azure.com:443/
rakcosmosdb01224__credential	managedidentity

All the higlighted ones need changing with the correct URL and replacement of the “rakcosmosdb01224” bit.

• Create a new Function. Do not add the Cosmos DB output binding via the Portal. Instead, edit function.json and add it.

I added the highlighted lines, including a comma above it in the JSON. Replace the “connection”, “databaseName”, and “containerName” as needed.

• And that’s it! Now one can push to the DB thus:

One should wrap this around a try/ catch block, so you can retry or put it elsewhere to retry. By default it does not retry.

Le fin!