Sitecore on Azure… With Unicorn

I’ve been trying to get our internal demo website running on Azure, and have it deployable in a very easy manner. Provisioning Sitecore onto Azure is really easy these days, and the code itself is also easily deployed to my environments (see these previous posts for more information).

The issue I was having was with Unicorn – or more specifically the locations of my data folder. On my local machine, I have Sitecore running in a specific location, and can simply point my Unicorn to a specific location on my machine (i.e. c:\development\demo\data\unicorn). Since all my colleagues working on the projects have the same file structure and we never have to deploy to an actual server that works perfectly.

However, I’m now adding a new deployment server to the mix, with a different file structure. I can’t simply use the same path as it might not exist there. I also can’t use the $(dataFolder) option that Unicorn provides since I don’t work in the Webroot as I have my Unicorned items in source control in that specific location – the actual demo site datafolder lives on c:\sites\demo\data in my case.

SlowCheetah to the rescue! I wrote about SlowCheetah earlier as well, and could actually create a pretty simple transformation: All I had to do was add a Transform to the Unicorn.config file in Visual Studio, to rewrite the data folder path.

addtransform

For each of my transforms (in my case I just use one CM and one CD, but it’ll work for multiple other deploy targets) I’d put in the XML transformation:

<?xml version="1.0" encoding="utf-8" ?>
<!-- For more information on using transformations 
 see the web.config examples at http://go.microsoft.com/fwlink/?LinkId=214134. -->
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <sitecore>
    <unicorn>
      <defaults>
        <targetDataStore physicalRootPath="$(dataFolder)\Unicorn\$(configurationName)" useDataCache="false" type="Rainbow.Storage.SerializationFileSystemDataStore, Rainbow" singleInstance="true" xdt:Transform="Replace" xdt:Locator="Match(type)"/>
      </defaults>
    </unicorn>
  </sitecore>
</configuration>

Because I use the Publish way of deploying my code, I do need to make sure the Unicorn files get deployed as well, so I have to include all my .yml files in my solution (not just source control).

However, after this it just… works :-).

Automatically scale Sitecore on Azure PaaS

Since Sitecore 8.2 update 1, we can simply scale our App Service plans up or out. You can find these options on the App Service Plan as well as the App Service themselves.

Scaling up means selecting a different tier for our subscriptions – more RAM and cores for instance.

scaleup

As you can see, I’m currently running on the S1 tier – which I can’t recommend to anyone as it’s quite slow (of course, it is way below the minimum specs found in the installation guides)…

Our other option is to scale out. That means we can add (or remove) a number of instances based on… something. Let’s take a look.

When scaling out, we have a couple of options:

  1. Scale manually. I can do this by setting the ‘scale by’ field to the option ‘an instance count that I enter manually’. I can select a number of instances using a slider.
    scalemanually
    In the above image, I’m simply making sure that we’re going to run on 3 instances.
  2. Scale on CPU percentage. This option is a bit smarter, we can now automatically scale up.
    scalecpu.png
    As the above image shows, I’ve set Azure up to use 2 instances at minimum and 7 maximum. Azure will automatically scale this based on the target range of the CPU – if that falls between 80 and 100, a new instance will be added (up to the maximum of 7 instances). If it falls below that minimum in the target range, it will automatically scale down as well.
  3. Scale on schedule and performance rules. The performance rules are pretty similar to the previous option, but gives us a bit more flexibility and power.
    scaleout
    As you can see in the above image, we’ve set a Default profile up, with a scale of 1 as minimum and 10 as maximum. We then have 2 rules: One to scale up, every time the CPU percentage hits 80% on average, over the past 10 minutes (see the right hand side), it will auto-scale up. In addition to that, it will also auto-scale down when on average over the past 10 minutes the CPU percentage is <1. Which might be a bit too low…
    There are other metrics that we can use rather than CPU percentage there as well,  and these rules can be set to scale up differently, for instance spinning up a different number based on the metric selected.
    We can also scale on schedule. If I know that I have very high traffic during the week and lower during the weekend, or if my traffic is high during work hours but low outside of them, or even if I have a fixed date that I know I have high traffic I can make sure I scale based on that as well. For this we need to select the scaling profile.
    recurrence
    For the first couple of options mentioned above I can use the ‘recurrence’ tab. For the fixed date, I can (obviously) use the ‘fixed date’ tab, where I can set a start date/time and end date/time. I’ll need to create a profile for my high traffic and low traffic as well, and build out the rules for both.
    schedulerules

There is one really important point here that I have yet to mention: Licensing. If you are on a perpetual licensing model and you want this auto scaling functionality, I would highly recommend putting in the maximums to make sure you won’t be in breach of the license.

Application Insights for Sitecore (or, where have my logfiles gone?)

So with Sitecore’s new Azure PaaS offering, you might’ve noticed in your Azure resource group that there’s an Application Insights resource there.

Application Insights can be used to monitor the live web applications and helps with diagnosing issues and figuring out what users are doing with your app. More information here. Application Insights is quite a big and powerful tool, and I won’t be able to cover everything juts quite yet. I will however run through some of the cool features.

health

Even by itself that’s a pretty cool (and useful) tool to have. But the Application Insights have actually also been integrated with Sitecore.

In the Application Insights overview blade we can get some additional information by clicking the Search button. We can filters the results there as well, by type (such as trace, request or page view) or property. For Sitecore’s log entries we are interested in the Trace filter.

trace

To query the log files in more depth, we can actually hit the Analytics button in that same overview blade. This will open a new window, where we can begin writing queries to find issues logged in our Sitecore logs.

insights

Any of the properties you see in the bottom right we can filter on – you can find those properties on the left hand side as well, in the Schema. There’s also some very useful documentation on writing queries.

In the Performance blade, I can also create my own charts based on Sitecore statistics, as well as see how my application is performing.

performance

We can also do things like set up alerts when there’s over 5 server exceptions in the application (see image below), and if this happens send an email to the administrators.

alerts

This type of reactive analytics isn’t the only thing you can do with Application Insights though – we can use some more advanced, proactive features. In the linked document, it states “For example, if your app loads slowly on some OS versions, or has slower responses in particular geographical location, you’ll get an email about it. Proactive Detection uses machine learning and data mining to help you find issues that would be hard to detect – and are often only discovered when customers complain.”

For more information about this type of diagnostics, see this Microsoft documentation.

Lastly, there’s also the Application Map, which allows you to see a diagram of your setup, showing the dependencies of your components.

appmap.png

By clicking through on the various components, we can see which parts of our setup are working correctly and which have issues. As you can see in the image above, there seems to be something wrong with my search. I can then click through on that to see some detailed information.

searchfailures.png

By clicking through on in the chart, I can also see even more detail (in the below image, I clicked the ‘Dependency Failures’ chart)

searchfailuredetail.png

From there I can then even create a new Work Item, either in Visual Studio Team Services or in GitHub, making sure the Work Items have all relevant data embedded.

I’ve covered only a small portion of what Application Insights can do for us, but I do hope I’ve shown its power and its usefulness. I’m pretty excited, in any case 🙂

Deploying code to Sitecore on PaaS

Now that we have our Sitecore environment on Azure PaaS, lets get some code on there!

I’m going to use Visual Studio to deploy my code, in the same way we can publish code locally to the website if we’re not working in the web root of the project. Of course, you can have your deployment process fully automated using things like FTP, PowerShell and more.

In the Azure portal, I can select the server I’d like to deploy to and select the ‘…More’ option to be able to download the PublishSettings file.

publishsettings

Little side note, if you wanted to upload your code through FTP you can get the username and password to use from the PublishSettings file as well – just open the file in your favourite text editor.

publishsettings2.png

All details required for the FTP connection are there. Also, if you were silly enough to put your username and password on the Internet, you can reset your publish settings by going going back to the Azure portal and in the server blade select the ‘Reset publish profile’ options.

Back to deploying our code through Visual Studio: We now can open (or create) our solution in Visual Studio and select ‘<New Custom Profile>’ in the Publish dropdown. Just give the profile a name, go back to the Profile button and select the ‘Import’ option. Here, you can select the PublishSettings file we downloaded earlier.

importprofile

At this point you’ll get an overview of the connection Visual Studio created for you, which we can then validate.

validateprofile

Now all that’s left is to click the Publish button, and we’ll have our Sitecore environment all up-to-date with the latest and greatest code!

Awesome. So now I have my code online – but oh no, an error. How can I debug this, when I just get a simple yellow screen? Well, first of all you could dive into the logfiles to get some more information. In addition I could turn off remote errors, but neither of those might actually tell me what’s happening within my code. What I’d really like to do is to start debugging. This first needs to be enabled in the Azure portal. You can find it in the Application Settings of your app service.

debug

Simply set Remote Debugging in the Debugging section to On, and set your Remote Visual Studio version to the version you use. Alternatively, Visual Studio also asks if it can enable remote debugging if you skip setting  it manually.

Then all that’s left is to publish your website with the Debug Configuration. In your Server Explorer in Visual Studio you can then right click on the Azure website and select Attach Debugger. The browser will then automatically open to the home page, so you might have to browse around a bit to find your error page :-).

Don’t forget to deploy your code using the Debug profile rather than the Release profile if you do want to debug.

PS. If any of your code has a dependency on the data folder, it has set it to the /App_Data folder by default.

Update

It has been pointed out to me that we don’t actually need to download the publish settings from the portal. Instead, when creating a new Publishing Profile, click the ‘Microsoft Web Apps’ publishing target. Sign in the pop up window, and there’s an option to select a web app (or create a new one)

publishprofile

Getting started with Sitecore on Azure PaaS

So now that Sitecore 8.2 update 1 has been released, we can finally stop using the Sitecore Azure module and go use the true power of Azure PaaS.

Be aware that at this point in time Sitecore only has an XP1 and XM1 delivery using the Azure Resource Manager (ARM) templates – more on ARM here – and XM1 only on the Azure Marketplace. Keep your eyes peeled for further updates on this, as other options will be released.

Before we get into this, make sure you have the following installed on your machine:

By now, you should have downloaded the ARM templates for Sitecore – or alternatively you can also use the Azure Marketplace to install Sitecore on Azure as well, so lets see how this works.

First of all, Sitecore’s ARM templates will not create the MongoDBs for us, so we need to start by getting that set up. I’m going to use mLab‘s free Sandbox tier for that in this blog post, but feel free to use other options (such as the xDB Cloud offering by Sitecore themselves).

mongodbsetup

At the moment of writing I can’t select Azure’s West Europe (Amsterdam) location, as mLabs does not provide a single node there (the only option that’s free).

mlabsdb

I’ll need to create the database for each of the MongoDBs. After the creation of my database is complete, I just add a user to each as well and copy and paste the connectionstring it’s giving me into the appropriate xX.Template.params.json file. In this file, I need to change:

  • The connection strings to all 3 MongoDB connection strings: analytics, tracking_live and tracking_contact – these need to be set to the correct connection strings given to us in the mLabs overview
  • sqlserver.login – the username for the sql server user. This user will be created in the databases automatically
  • sqlserver.password – the password to use for the user created (make sure this has 8 or more characters or an error will be thrown)

Lastly, I need to update the PowerShell script Run.ps1 to use the correct parameters. I need to set the following options:

  • $ArmTemplatePath – needs to be set to the correct xX.Template.json file location
  • $ArmParametersPath – needs to be set to the file location of our updated .params.json file
  • $licenseFileContent – this needs to point to a valid license.xml file
  • $Name – the name for the deployment
  • $location – the Azure region to deploy in. A good overview can be found here, including which functionality is or isn’t available in the region. Make sure the region you select supports Azure Search – not all of them do.
  • $AzureSubscriptionId – Your subscription ID to use for the deployment. You can find this in the ‘Billing’ blade in the Azure portal.

As an additional (optional) change, one could set up a principle service and use that. I just want to get up-and-running ASAP, so I’ll take the manual login option instead for which I don’t have to make any changes.

I also noticed that in my xP1.Template.json file the Application Insights were set to the location of “Central US”:

{
  "type": "Microsoft.Insights/Components",
  "name": "[variables('appInsightsNameTidy')]",
  "apiVersion": "[variables('appInsightsApiVersion')]",
  "location": "Central US",
  "properties": {
    "ApplicationId": "[variables('appInsightsNameTidy')]",
    "Application_Type": "web"
  },
  "tags": {
    "provider": "[parameters('sitecoreTags').provider]"
  }
}

That’s technically fine, but I’d rather have it in the same region as my other resources, so changed the “location” line to the following instead:

"location": "[variables('resourceGroupLocation')]",

Of course, you’ll need to make sure that location supports the Application Insights.

After this I’m all set so I can run the PowerShell Script. It is at this point that you can safely go get a cup of tea, as this will take a little while. In fact, my deployment took 23 minutes and 49 seconds. I didn’t bother moving the blobs used to install Sitecore in my own data center so I could’ve had an even faster deployment – but just under 24 minutes is still pretty impressive, no?

environment