Nine Knights of Azure: #1 Web App – Tesco Grocery Look Up

Background

I want to learn more Azure and C#, and I want to learn by doing. The other day I listened to the .NET Rocks podcast talking with Adam Cogan about his Nine Knights of Azure talk. The talk lists nine Azure services to learn first. I’m going to follow that framework while building a project.

The Project

Here’s the vision for the project. I use the Tesco online shopping site for most of our household shopping.  I dislike the user interface. What I really want is a touch screen display in the kitchen that allows me to add on items to restock the moment we use up the last one.

Knight #1: Computing: App Services

The first Azure Knight is “Computing: App Services”. To exercise these services I will make a simple web app and host it on the Azure App Services. The app will do only one thing: accept the name of a product and list the resulting hits back from the Tesco groceries web site.

Covering All the Bases

While the app functionality is simple I do want to cover all the steps that I would take on a more complicated product:

  1. Design documentation
  2. Automated functional testing
  3. Automated unit testing
  4. Continuous integration/deployment

Development Flow

The flow of development I like to follow is:

  • Write just enough documentation to be clear what you are building and why
  • Create the list of high-level tests that if they all pass then you will go live
  • For each test:
    • Write up the tests; automate the tests were possible (“functional testing”)
    • Implement the code to pass the test using test driven development (“unit testing”)
    • Refactor the code to make sure that it is clean
    • Push to source control and make sure continuous integration tests all pass
    • Automatically deploy to testing environment
  • Once all the code is built and automated tests are passing complete any manual testing
  • Push to production

Building the Pipeline

I want to learn the Azure DevOps  services so I’m going to build my development pipeline with these tools.

Setting Up the Repo

I’ve created a public Azure DevOps project here: https://dev.azure.com/davegoopot/tesco_home_automation

Project Documentation

I’ve set up the project wiki to use mark down files under the “docs” folder in the git repo. See: https://dev.azure.com/davegoopot/tesco_home_automation/_wiki/wikis/Tesco%20Home%20Automation .

Azure CosmosDB is Too Expensive for Experimenting. Alternative: MongoDB Atlas

Background

Recently I’ve been experimenting with Azure Functions, and I’ve got to the point where I wanted to play with Functions interacting with a storage layer. Azure CosmosDB was the obvious choice. I went through the process of setting up a database and turning all the performance dials down to their minimum settings. Nonetheless, after four days playing with my experimental DB I realised that it was going to cost me more than £20 (UK) per month to keep my toy DB running. I needed an alternative.

MongoDB Atlas

MongoDB Atlas (https://www.mongodb.com/cloud/atlas) is the hosted DB service from the people behind MongoDB. For my purposes it is attractive as it provides a free tier for up to 500MB of storage (https://www.mongodb.com/cloud/atlas/pricing).

It set up an account and downloaded the Compass DB management tool to work with the data. One snag I ran into was that if you had a password with special characters in it then this needed to be URI encoded before entering into the Compass login screen. This held me up for a good couple of hours as the error message back from Compass was the cryptic, “Missing delimiting slash between hosts and options”.

Finally I had my database running and had entered some test data:

2018-05-04 08_22_47-MongoDB Compass Community - sleepsuntil-shard-00-00-vyqwg.mongodb.net_27017_slee

Connecting Azure Functions to MongoDB Atlas

I pulled my connection string information over from Atlas and stored it in the application keys (See my previous post on storing API keys for Azure Functions).

Now I needed to open up the Atlas firewall to allow inbound connections from Azure. This is non-trival since Azure will allocate an outbound IP from Functions from any of the range of IP addresses for their whole data centre. See the Microsoft article explaining outbound IP addresses. I’m hosting in “UK West” and at the time of writing the data centre had 24 different IP ranges. Given the fact that I only have toy data in my DB I decided to allow access from all IPs. If you have a real world example you will need to implement some process to lock this down some more.

With this setup complete I now have some working code, see below, to show Azure Functions connecting to MongoDB Atlas…and it is free!

module.exports = function (context, req) {
    context.log("Starting Atlas example");
    const mongoClient = require("mongodb").MongoClient;

    function opendb() {
        const url = process.env["atlasurl"];
        context.log("Attempting to connect"); 
        const db_promise =  mongoClient.connect(url);
        return db_promise;
    }

    function readdata(db) {
        context.log("Accessing sleepsuntil");
        let dbo = db.db("sleepsuntil");
        context.log("Got DBO " + dbo);
        let query = { key: "example1" };
        context.log("Starting query");
        let results = dbo.collection("testing").findOne(query);
        context.log("Got results");
        return results;
    }

    opendb()
        .then((db) => {
            context.log("Return from open was: " + db);
            return readdata(db);
        })
        .then((results) => {
            context.log("Read: " + JSON.stringify(results));
            context.res.body = `
            <table>
            <tr><td>_id<td>${results._id}</tr>
            <tr><td>key<td>${results.key}</tr>
            <tr><td>value<td>${results.value}</tr>
            </table>`;
            context.res.headers = { "Content-Type": "text/html" };

            context.done();
        })
        .catch((msg) => {
            context.log("Error caught: " + msg);
            context.done();
        });
};

Managing Azure Functions API Keys

Context

I’ve been working on example code to use the JavaScript MongoDB driver to work with the Azure Cosmos DB. To connect to the DB I’ve had to manage my DB API keys – the secrets that allow only me to get at my data. Here’s how to do that in Azure Functions.

Azure Functions Environment Variables

The simplest way to store API keys for use in Azure functions is to write them to the environment variables using the Portal web interface and then read in the environment variable as the script runs.

Setting Environment Variables

To set an environment variable open up the Azure Portal, navigate to your Function app and click the “Applications Settings” link.

2018-04-30 06_38_43-sleepsuntil - Microsoft Azure

Once you open applications settings tile, go down to the “Applications settings” area and click the plus sign to add a new setting. Give the new setting the key “example” and the value anything you like. You then must scroll back to the top of the page and click “Save”.

In your JavaScript function you read the application setting value from the process.env array.

Here is some code to show you the value that you just set.

module.exports = function (context, req) {
    context.res.body = process.env["example"];

    context.done();
};

API Keys From Applications Settings

Now we have a simple way to look after API keys: set the value in an application setting and read into our code using process.env. My code to connect to my Azure CosmosDB is something like this:

let url = 
  `mongodb://${process.env["cosmosdb_name"]}:${process.env["primary_master_key"]}@${process.env["cosmosdb_name"]}.documents.azure.com:10255/mean-dev?ssl=true&sslverifycertificate=false`;

How to Setup Git Deployment of Azure Web Apps

The Azure Portal (https://portal.azure.com/) has had me stumped on an apparently simple task for the past couple of days. I became lost in the UI of are trying to create a new Azure Web App that I could deploy changes to by pushing to a git repository. Here are the steps you actually need to get this done…

  1. Login to https://portal.azure.com/
  2. Top left of the screen click “Create a resource”
  3. In the search box type for “node empty” and pick the “Node JS Empty Web App”
    2018-04-20 07_47_22-Everything - Microsoft Azure

  4. Fill in details for the new web app and optionally click “App service plan/location” if you want to change the size, and therefore cost, of the instance, and where it is located in the world

  5. Press “Create” and wait for your app to deploy
  6. Open the app and click the URL to make sure the deployment is actually now serving web pages
    2018-04-19 07_55_02-Dashboard - Microsoft Azure
  7. This is where I became stuck. Just how do you get the git deployment working from here?
  8. In the “Deployment” sidebar menu click “Deployment credentials” and make sure you have a username and password set up
  9. Click the “Deployment options” two menus down. Pick “Choose source” then “Local Git Repository”. You may have to disconnect any existing options using the button on the top menu of the deployment options tile
    2018-04-19 07_58_45-Choose source - Microsoft Azure
  10. Now you need to scroll halfway down the sidebar menu to find the “Properties” title. In that tile is the git url for your web app
    2018-04-19 08_01_01-Properties - Microsoft Azure
  11. On your local PC git clone that git url. You’ll need the username and password you created earlier to login
  12. Git should then clone you a directory structure something like this:
    2018-04-20 07_39_48-sleepsuntil-web
  13. Edit the file server.js changing the text res.end('Hello, world!'); to something of your choosing
  14. Commit the change in git then do a git push
  15. Reload the URL of Azure app and you should see your changes