Category Archives: Real-Time

Real-time in Sitecore.Habitat with SignalR and MongoDB

RealTimeYo

Personalizing content has always been a big fascination of mine and I just love Sitecore’s very elegant solution by using the rules engine. Thanks to this very cool tool the editors can do the following:

Rule-based personalization
Rule-based personalization uses logic-based rules to determine the content that is displayed on a webpage. For example, you can set rules based on the IP address or physical location of your visitors, the keywords they use to reach your site, their mobile device, or the goals that they achieve on your website to determine the content that is displayed.

Adaptive personalization

Adaptive personalization is a feature that dynamically changes the content of your website based on the visitor’s behavior during a visit. Adaptive personalization uses visitor profiles and pattern-card matching to dynamically adapt the content shown to visitors in real time. You can set adaptive personalization rules in the Rules Set Editor.

Historical personalization
You can use rules that personalize content based on a contact’s historical or past behavior, rather than their actions from the current session. The Key Behavior Cache contains information about a contact’s recent activities across all channels. The Contact Behavior Profile contains information about a contact’s past profile matches.

You can create and implement personalization rules based on the information in the Key Behavior Cache and Contact Behavior Profile, enabling you to provide contacts with content that is relevant based on their past behavior, rather than just their current interaction with your brand.

We just need one more thing to make the personalization even more interesting – Personalization in real-time!
How can we do that? By using SignalR of course 🙂 (There are of course other ways to do this but SignalR has a special place in my heart)

There are a number of Sitecore “frameworks” out there but there is one who stands out – Sitecore.Habitat. Thanks to the Modular Architecture approach everything is clean. Anders Laub made a great post about this – The groundbreaking Sitecore Habitat

If you guys are going to the Sitecore Symposium 2016 New Orleans, then you must attend on the Habitat Master Class. The course is held by mr Habitat himself – Thomas Eldblom 🙂

First we need to setup the SignalR and hook it up with MongoDB, that is what this post will be about. For some time ago I did a post about real-time in Sitecore, Real-time in Sitecore using SignalR and MongoDB. I’ve decided to migrate the old code to the best ever Sitecore “framework” – Sitecore.Habitat.

Here is how the project look like after the migration. It’s a typical “Foundation” project, since it probably will be used by “Feature” projects.
I’ve highlighted the ones that I will take up in the post.
Realtime für die Leute
Check it out on the GitHub – GoranHalvarsson/Habitat

web.config.transform

In order to make SignalR work properly we need to do some changes/updates in webconfig.
This guy was indeed tricky, working with Xdt transform is not easy but thanks to this little puppy it made my life so much easier – Web.config Transformation Syntax for Web Project Deployment Using Visual Studio.

<configuration  xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">

  <appSettings xdt:Transform="InsertIfMissing">
    <add key="ValidationSettings:UnobtrusiveValidationMode" value="None" xdt:Transform="InsertIfMissing" xdt:Locator="Match(key)"/>
  </appSettings>
  <system.web>
	<httpRuntime targetFramework="4.5" requestValidationMode="2.0" xdt:Transform="SetAttributes(targetFramework,requestValidationMode)" />
  </system.web>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly xdt:Transform="InsertIfMissing" xdt:Locator="Condition(./_defaultNamespace:assemblyIdentity/@name='MongoDB.Bson')">
        <assemblyIdentity name="MongoDB.Bson" publicKeyToken="f686731cfb9cc103" culture="neutral" />
        <codeBase version="2.2.4.26" href="bin\MongoDB.Bson.dll" />
        <codeBase version="1.10.0.62" href="bin\Sitecore\MongoDB.Bson.dll" />
      </dependentAssembly>
      <dependentAssembly xdt:Transform="InsertIfMissing" xdt:Locator="Condition(./_defaultNamespace:assemblyIdentity/@name='MongoDB.Driver')">
        <assemblyIdentity name="MongoDB.Driver" publicKeyToken="f686731cfb9cc103" culture="neutral" />
        <codeBase version="2.2.4.26" href="bin\MongoDB.Driver.dll" />
        <codeBase version="1.10.0.62" href="bin\Sitecore\MongoDB.Driver.dll" />
      </dependentAssembly>
      <dependentAssembly  xdt:Transform="InsertIfMissing" xdt:Locator="Condition(./_defaultNamespace:assemblyIdentity/@name='Microsoft.AspNet.SignalR.Core')">
        <assemblyIdentity name="Microsoft.AspNet.SignalR.Core" publicKeyToken="31bf3856ad364e35" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-2.2.0.0" newVersion="2.2.0.0" />
      </dependentAssembly>
      <dependentAssembly xdt:Transform="InsertIfMissing" xdt:Locator="Condition(./_defaultNamespace:assemblyIdentity/@name='Microsoft.Owin')">
        <assemblyIdentity name="Microsoft.Owin" publicKeyToken="31bf3856ad364e35" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-3.0.1.0" newVersion="3.0.1.0" />
      </dependentAssembly>
      <dependentAssembly xdt:Transform="InsertIfMissing" xdt:Locator="Condition(./_defaultNamespace:assemblyIdentity/@name='Microsoft.Owin.Security')">
        <assemblyIdentity name="Microsoft.Owin.Security" publicKeyToken="31bf3856ad364e35" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-3.0.1.0" newVersion="3.0.1.0" />
      </dependentAssembly>
      <dependentAssembly xdt:Transform="Replace" xdt:Locator="Condition(./_defaultNamespace:assemblyIdentity/@name='Newtonsoft.Json')">
        <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" />
        <bindingRedirect oldVersion="0.0.0.0-9.0.0.0" newVersion="9.0.0.0" />
      </dependentAssembly>
    </assemblyBinding>
  </runtime>

</configuration>

Lets break it down, we will start with the changes to make SignalR work properly.

The request validation feature in ASP.NET provides a certain level of default protection against cross-site scripting (XSS) attacks. In ASP.NET 4, by default, request validation is enabled for all requests. As a result, request validation errors might now occur for requests that previously did not trigger errors. To revert to the behavior of the ASP.NET 2.0 request validation feature we need to add requestValidationMode=”2.0″ in the httpRuntime.

To prevent the Websocket error, 500, which means that the server does not support websockets. In this specific case server means the server (IIS) supports WebSockets but ASP.NET version of your application which hosts SignalR does not support websockets. In the httpRuntime set targetframework to 4.5.

Finally the UnobtrusiveValidationMode setting is really not necessary since this is for webforms only, but hey lets put that one there too 🙂

  <appSettings xdt:Transform="InsertIfMissing">
    <add key="ValidationSettings:UnobtrusiveValidationMode" value="None" xdt:Transform="InsertIfMissing" xdt:Locator="Match(key)"/>
  </appSettings>
  <system.web>
	<httpRuntime targetFramework="4.5" requestValidationMode="2.0" xdt:Transform="SetAttributes(targetFramework,requestValidationMode)" />
  </system.web>

Next part is the dll’s. Since I’m using the 2.0 .Net Driver for MongoDB with Async support but Sitecore is using the previous version of the driver. We need to do the following in config:

  <dependentAssembly xdt:Transform="InsertIfMissing" xdt:Locator="Condition(./_defaultNamespace:assemblyIdentity/@name='MongoDB.Bson')">
    <assemblyIdentity name="MongoDB.Bson" publicKeyToken="f686731cfb9cc103" culture="neutral" />
    <codeBase version="2.2.4.26" href="bin\MongoDB.Bson.dll" />
    <codeBase version="1.10.0.62" href="bin\Sitecore\MongoDB.Bson.dll" />
  </dependentAssembly>
  <dependentAssembly xdt:Transform="InsertIfMissing" xdt:Locator="Condition(./_defaultNamespace:assemblyIdentity/@name='MongoDB.Driver')">
    <assemblyIdentity name="MongoDB.Driver" publicKeyToken="f686731cfb9cc103" culture="neutral" />
    <codeBase version="2.2.4.26" href="bin\MongoDB.Driver.dll" />
    <codeBase version="1.10.0.62" href="bin\Sitecore\MongoDB.Driver.dll" />
  </dependentAssembly>

You will also have to create a Sitecore folder in your bin folder in the project, here you will place the “old” dll’s that Sitecore is using – MongoDB.Bson.dll and MongoDB.Driver.dll.

There will also be some gulp scripts that will copy over the specific dll’s when it’s time to publish to the website:

gulp.task("Realtime-Copy-MongoDB-dll's", function () {
    console.log("Copying mongodb assemblies to website");
    var root = "./src/Foundation/Realtime/code/bin/Sitecore";
    var binFiles = root + "/*.{dll,pdb}";
    var destination = config.websiteRoot + "/bin/Sitecore/";
   
    console.log("copying to " + destination);


    return gulp.src(binFiles, { base: root })
      .pipe(gulp.dest(destination));
});

gulp.task("Realtime-Copy-Newtonsoft-dll", function () {
    console.log("Copying newtonsoft assemblies to website");
    var root = "./src/Foundation/Realtime/code/bin";
    var binFiles = root + "/Newtonsoft.Json.{dll,pdb,xml}";
    var destination = config.websiteRoot + "/bin/";

    console.log("copying to " + destination);


    return gulp.src(binFiles, { base: root })
      .pipe(gulp.dest(destination));
});

gulp.task("Realtime-Xml-Transform", function () {
    return gulp.src("./src/Foundation/Realtime/**/code/*.csproj")
      .pipe(foreach(function (stream, file) {
          return stream
            .pipe(debug({ title: "Applying transform realtime:" }))
            .pipe(msbuild({
                targets: ["ApplyTransform"],
                configuration: config.buildConfiguration,
                logCommand: false,
                verbosity: "normal",
                maxcpucount: 0,
                toolsVersion: 14.0,
                properties: {
                    WebConfigToTransform: config.websiteRoot + "\\web.config"
                }
            }));
      }));

});

The last one is to make sure that we run the xml transform for the project.

RealtimeHub.cs

This is the brain of the SignalR, the hub will take care of incoming(javascript calls from the client) and outgoing(calling javascript methods on the client) calls.

It’s very well described in my older post: Create a real-time connection.
You can check out the the code on GitHub – RealtimeHub.cs

Realtime.js

This is the heart of the SignalR, the javascript will feed and listen to the “brain”(hub).

It’s also very well described in my older post: Gather visitor data.
You can check out the the code on GitHub – Realtime.js

RegisterSignalrProcessor.cs

The pipeline will replace the old startup class. We will need to mark it with OwinStartup(SignalR requires Owin).

using Microsoft.Owin;
using VisionsInCode.Foundation.Realtime.Pipelines;

[assembly: OwinStartup(typeof(RegisterSignalrProcessor))]
namespace VisionsInCode.Foundation.Realtime.Pipelines
{
  using Microsoft.AspNet.SignalR;
  using Owin;
  using Sitecore.Diagnostics;
  using Sitecore.Pipelines;
  using VisionsInCode.Foundation.Realtime.Infrastructure;
  using VisionsInCode.Foundation.Realtime.Repositories;

  public class RegisterSignalrProcessor
  {
    public void Configuration(IAppBuilder app)
    {
      Log.Info("OwinStartup has started", this);

      
      GlobalHost.HubPipeline.AddModule(new ErrorHandlingHubPipelineModule());


      GlobalHost.DependencyResolver.Register(
            typeof(RealtimeHub),
            () => new RealtimeHub(new RealtimeVisitorRepository(), new GeoCoordinateRepository(), new HubContextService(), new GeocoderService()));

      app.MapSignalR();

    }

    public virtual void Process(PipelineArgs args)
    {
      Log.Info("Pipeline RegisterSignalrProcessor called", this);

    }

  }
}

You can check out the the code on GitHub – RegisterSignalrProcessor.cs

Foundation.Realtime.config

And finally the config patch file:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:set="http://www.sitecore.net/xmlconfig/set/">
  <sitecore>
    <settings>
      <setting name="IgnoreUrlPrefixes">
        <patch:attribute name="value">/sitecore/default.aspx|/trace.axd|/webresource.axd|/sitecore/shell/Controls/Rich Text Editor/Telerik.Web.UI.DialogHandler.aspx|/sitecore/shell/applications/content manager/telerik.web.ui.dialoghandler.aspx|/sitecore/shell/Controls/Rich Text Editor/Telerik.Web.UI.SpellCheckHandler.axd|/Telerik.Web.UI.WebResource.axd|/sitecore/admin/upgrade/|/layouts/testing|/sitecore/service/xdb/disabled.aspx|/signalr|/signalr/hubs</patch:attribute>
      </setting>
    </settings>
    <pipelines>
      <initialize>
        <processor type="VisionsInCode.Foundation.Realtime.Pipelines.RegisterSignalrProcessor, VisionsInCode.Foundation.Realtime"
       patch:before="processor[@type='Sitecore.Mvc.Pipelines.Loader.InitializeRoutes, Sitecore.Mvc']" ></processor>
      </initialize>
      <mvc.getPageRendering>
        <processor
          patch:before="*[@type='Sitecore.Mvc.Pipelines.Response.GetPageRendering.GetLayoutRendering, Sitecore.Mvc']"
          type="Sitecore.Foundation.Assets.Pipelines.GetPageRendering.AddAssets, Sitecore.Foundation.Assets">
          <defaultAssets hint="raw:AddAsset">
            <asset type="JavaScript" file="/Scripts/Realtime/jquery.signalR-2.2.1.js" location="Body" />
            <asset type="JavaScript" file="/signalr/hubs" location="Body" />
            <asset type="JavaScript" file="/Scripts/Realtime/ClientTracker.js" location="Body" />
            <asset type="JavaScript" file="/Scripts/Realtime/Realtime.js" location="Body" />
          </defaultAssets>
        </processor>
      </mvc.getPageRendering>
    </pipelines>
  </sitecore>
</configuration>

Lets break it down.
We need to stop Sitecore to capture SignalR requests for that we need to update IgnoreUrlPrefixes by adding signalr|/signalr/hubs:

    <settings>
      <setting name="IgnoreUrlPrefixes">
        <patch:attribute name="value">/sitecore/default.aspx|/trace.axd|/webresource.axd|/sitecore/shell/Controls/Rich Text Editor/Telerik.Web.UI.DialogHandler.aspx|/sitecore/shell/applications/content manager/telerik.web.ui.dialoghandler.aspx|/sitecore/shell/Controls/Rich Text Editor/Telerik.Web.UI.SpellCheckHandler.axd|/Telerik.Web.UI.WebResource.axd|/sitecore/admin/upgrade/|/layouts/testing|/sitecore/service/xdb/disabled.aspx|/signalr|/signalr/hubs</patch:attribute>
      </setting>
    </settings> 

Next is to place the “startup” pipeline in the correct “call order”:

      <initialize>
        <processor type="VisionsInCode.Foundation.Realtime.Pipelines.RegisterSignalrProcessor, VisionsInCode.Foundation.Realtime"
       patch:before="processor[@type='Sitecore.Mvc.Pipelines.Loader.InitializeRoutes, Sitecore.Mvc']" ></processor>
      </initialize>

Finally add the javascripts in the correct order.

      <mvc.getPageRendering>
        <processor
          patch:before="*[@type='Sitecore.Mvc.Pipelines.Response.GetPageRendering.GetLayoutRendering, Sitecore.Mvc']"
          type="Sitecore.Foundation.Assets.Pipelines.GetPageRendering.AddAssets, Sitecore.Foundation.Assets">
          <defaultAssets hint="raw:AddAsset">
            <asset type="JavaScript" file="/Scripts/Realtime/jquery.signalR-2.2.1.js" location="Body" />
            <asset type="JavaScript" file="/signalr/hubs" location="Body" />
            <asset type="JavaScript" file="/Scripts/Realtime/ClientTracker.js" location="Body" />
            <asset type="JavaScript" file="/Scripts/Realtime/Realtime.js" location="Body" />
          </defaultAssets>
        </processor>
      </mvc.getPageRendering>

You can check out the the code on GitHub – Foundation.Realtime.config

Ops, I almost forgot we also need to set the connection string to the new mongodb database, signalr, in the connectionStrings.config:

<add name="signalr" connectionString="mongodb://localhost:27017/habitat_local_signalr" />

There will be a second post where we will use SignalR to change content on the fly(in real-time).

That’s all for now folks 🙂

View visitors in real-time in Sitecore using SignalR and MongoDB

VisitorsInRealtimeMap

In this post I would like to continue on the real-time subject, I see a great potential in using data in real-time. So why not combine my previous posts, Real-time in Sitecore using SignalR and MongoDB(where I showed you guys how to setup real-time and storing the data in Mongodb) and Make a Google Map SPEAK component in Sitecore.

How about show current visitors on a map in real-time. It’s almost like watching popcorn popping 🙂
RealtimeVisitorsAnimated

Add methods to the SignalR Hub – RealTimeHub

In SignalR we need to let Sitecore users be able to listen to the visitor changes. In the class RealTimeHub we will add the AuthenticateListener() method which will be called from the client script in the new speak component. The method will authenticate the Sitecore user( called Listeners) and add the user to the “Listeners” group.

public async Task<bool> AuthenticateListener()
{
    IHubContextService hubService = this._hubContextService.Resolve(Context);

    if (!hubService.IsUserAuthenticated())
        return false;

    await SubscribeToChannel(RealTimeConstants.Signalr.Groups.Listeners);

    return true;
}

public async Task SubscribeToChannel(string channel)
{
    await Groups.Add(Context.ConnectionId, channel);
}

Next will be to create the “notify” methods when the visitor changes in the website. The methods will make client calls to the new SPEAK component.

this.Clients.Group(RealTimeConstants.Signalr.Groups.Listeners).onVisitorChanged(visitor);
this.Clients.Group(RealTimeConstants.Signalr.Groups.Listeners).onVisitorHasLeft(visitor);

The onVisitorChanged will be called in SendClientLocationData() method in class RealTimeHub when we get the geodata from the visitor.

The onVisitorHasLeft will be called when the visitors session has ended.

We will also add a method that presents current visitors for the “Listeners”.

public async Task ShowAllVisitors()
{
    IHubContextService hubService = this._hubContextService.Resolve(Context);

    if (!hubService.IsUserAuthenticated())
        return;
        
    await this._realTimeUserRepository.Collection().Find(v => !v.IsToBeDeleted).ForEachAsync((visitor, i) =>
    {
        this.Clients.Group(RealTimeConstants.Signalr.Groups.Listeners).onVisitorChanged(visitor);
    });
}

Create SPEAK component

The SignalR hub is now ready, lets start doing the SPEAK component. First we need to create a SPEAK page(VisitorsInRealtimeLayout):
VisitorsInRealTimeLayout
Here is the VisitorsInRealtimeLayout in “design layout”:
VisitorsInRealTimeLayoutDesign
We will use the map component, GoogleMapSpeak to present the visitors. The SubPageCode(the js file) will do the magic, connecting to SignalR to get visitor data.

To make SignalR work in SPEAK I added its “file paths” to the require.config.

require.config({
    paths: {
        "signalr.core": "/Scripts/jquery.signalR-2.2.0.min",
        "signalr.hubs": "/signalr/hubs?"
    },
    shim: {
        "signalr.core": {
            deps: ["jquery"],
            exports: "$.connection"
        },
        "signalr.hubs": {
            deps: ["signalr.core"]
        }
    }
});


define(["sitecore", "jquery", "signalr.hubs"], function (sitecore, jQuery) {

    var activeInfoWindow = null;

    var visitorsInRealtimePageCode = sitecore.Definitions.App.extend({

        initialized: function () {

            var self = this;

            self.setupRealtime();

        },
        setupRealtime: function () {

            var self = this;

            self.contactIconPath = "/sitecore/api/ao/v1/contacts/{0}/image?w=40&h=40";

            self.connection = jQuery.hubConnection();

            self.realTimeConnector = self.connection.createHubProxy("RealTimeConnector");

            self.realTimeConnector.on("onVisitorChanged", function (data) {

                if (activeInfoWindow != null)
                    activeInfoWindow.close();

                var infoWindow = new google.maps.InfoWindow();

                var visitor = data;

                if (visitor == null) {
                    console.log("No visitors");
                    return;
                }

                var marker =self.GoogleMapSpeak1.addMarkerToMap(visitor.ContactId,
                    visitor.RealTimeMetaData.GeoCoordinates.Coordinates[0],
                    visitor.RealTimeMetaData.GeoCoordinates.Coordinates[1],
                    self.stringFormat(self.contactIconPath, visitor.ContactId));


                infoWindow.setContent(self.renderHtmlContactInfo(visitor.ContactId));

                infoWindow.open(self.GoogleMapSpeak1.get("speakMap"), marker);

                marker.addListener('click', function () {
                    if (activeInfoWindow != null)
                        activeInfoWindow.close();

                    infoWindow.open(self.GoogleMapSpeak1.get("speakMap"), marker);
                });

                activeInfoWindow = infoWindow;
                
                console.log(visitor.ContactId);
                console.log(visitor.RealTimeMetaData.GeoCoordinates.Coordinates);

            });

            self.realTimeConnector.on("onVisitorHasLeft", function (data) {

                var visitor = data;

                if (visitor == null) {
                    console.log("No visitor");
                    return;
                }

                self.GoogleMapSpeak1.removeMarkerFromMap(visitor.ContactId);


                console.log(visitor.ContactId);
                console.log(visitor.RealTimeMetaData.GeoCoordinates.Coordinates);

            });

            self.realTimeConnector.on("onWhoIs", function () {

                self.realTimeConnector.invoke("AuthenticateListener").done(function (result) {

                    if (!result) {
                        console.log("Not authenticated, not allowed to see visitors");
                        return;
                    }

                    self.realTimeConnector.invoke("ShowAllVisitors").done();

                });

            });

            self.connection.start().done();

        },
        renderHtmlContactInfo: function (contactId) {

            var self = this;

            return self.stringFormat("<div><h4>{0}</h4>", contactId) +
                self.stringFormat("<p><a href='/sitecore/client/Applications/ExperienceProfile/contact?cid={0}'>Get detailed info</a></p>", contactId) +
                "</div></div>";

        },
        stringFormat: function () {
            var s = arguments[0];
            for (var i = 0; i < arguments.length - 1; i++) {
                var reg = new RegExp("\\{" + i + "\\}", "gm");
                s = s.replace(reg, arguments[i + 1]);
            }
            return s;
        }
    });

    return visitorsInRealtimePageCode;
});

Let me give you a quick explanation on how the script works. 🙂
When the client is connected to the RealtimeHub(On the server side), the hub will make a client call to the WhoIs method. Right now nothing is done here but in the future we could log/store the “Listeners”. Next the AuthenticateListener method will be called(which is described in the beginning of the post) and when the “Listener” is authenticated it will make a server call to ShowAllVisitors(which is described in the beginning of the post). The ShowAllVisitors method will loop through the visitors and make a client call to onVisitorChanged. Method onVisitorChanged is also called when a new visitor connects.

In onVisitorChanged we will present the visitor on a map by using the GoogleMapSpeak control.

When the visitors session has ended, a client call will be made to onVisitorHasLeft. The visitor will be removed from the map by using the GoogleMapSpeak control.

That’s all for now folks 🙂

Real-time in Sitecore using SignalR and MongoDB

55305-uncle-milton-giant-ant-farm7

I think it’s really cool that Sitecore is using MongoDB for storing visitor data(The data is being stored when the session ends)
Adam Conn explains it well(as always) in his post, INTRODUCING THE SITECORE ANALYTICS INDEX

But would it not be cool to make a snapshot of your visitors in Real-Time? Like the good old ant farm 🙂

Real-Time for me is the use of WebSockets and here is a great post/slide presentation that explains it very well.

As usually there are some great frameworks you can use – my favorites are Pubnub and Microsoft’s SignalR – I went for the SignalR.

I also wanted to store the “real-time” data and MongoDB is indeed perfect for that, especially now with the latest MongoDB .NET Driver which is completely async.

The idea is to show what the user is doing right now and store that data in a database. I don’t want to store the visitors history data, Sitecore is doing that so well.

I will create a visitor object which will contain Sitecore’s Contact Id, some metadata and the current connections that the user is having on the website.
A connection is a page view, for instance if the user has 5 pages opened in his/her browser – that means 5 opened connections.
When the user is leaving a page the connection will be removed from the visitor object and if the user is leaving the website and have no opened connections, the visitor object will be removed from the database.

It’s all about here and now!

Here is how the visitor object will be stored in MongoDB.
MongoDB

So this is what I would like to to:
1. Setup SignalR and make it work with Sitecore
2. Create a real-time connection between visitors and website
3. Gather visitor data and send it to the website in real-time
4. Extract and deserialize the client data
5. Store the data in MongoDB in real-time (Like a backplane for SignalR)
6. Send data back to the client
7. Client leaves website

Let’s do it!

Setup SignalR

Here are some great tutorial on how to setup the SignalR:
Getting Started with SignalR 2
Getting Started with SignalR 2 and MVC 5

I had to do some minor tweaks/changes in order to make it work properly:

  • The request validation feature in ASP.NET provides a certain level of default protection against cross-site scripting (XSS) attacks. In ASP.NET 4, by default, request validation is enabled for all requests. As a result, request validation errors might now occur for requests that previously did not trigger errors. To revert to the behavior of the ASP.NET 2.0 request validation feature add the following attribute in the httpRuntime:

     <httpRuntime requestValidationMode="2.0">
    
  • Websocket error: If this request has failed with error 500 it means that the server does not support websockets. In this specific case server means the server (IIS) supports WebSockets, but ASP.NET version of your application which hosts SignalR does not support websockets. In the webconfig set targetframework to 4.5.

    <httpRuntime  targetFramework="4.5">
    
  • We also need to do some minor tweaks to make it play with Sitecore.
    The problem is that Sitecore will actually try to handle the URL pattern in their routes instead of allowing the proxy to take action. You will want to make sure that your CMS has an ignore specified for the SignalR. In the web.config, update your IgnoreUrlPrefixes:

    <setting name="IgnoreUrlPrefixes" value="/sitecore/default.aspx|/trace.axd|...../signalr|/signalr/hubs" />
    
  • Next thing will be to create a working connection.

    Create a real-time connection

    First we need to create the hub, it needs to derive from the Microsoft.AspNet.SignalR.Hub class and the HubName attribute specifies how the Hub will be referenced in the JavaScript.
    Never mind the IRealTimeVisitorRepository and IGeoCoordinateRepository, I will come back to them later.

    [HubName("RealTimeConnector")]
    public class RealTimeHub : Hub
    {
        private readonly IRealTimeVisitorRepository _realTimeVisitorRepository;
        private readonly IGeoCoordinateRepository _geoCoordinateRepository;
        private readonly IHubContextService _hubContextService;
    
        public RealTimeHub(IRealTimeVisitorRepository realTimeVisitorRepository,
            IGeoCoordinateRepository geoCoordinateRepository,
            IHubContextService contextService)
        {
            this._realTimeVisitorRepository = realTimeVisitorRepository;
            this._geoCoordinateRepository = geoCoordinateRepository;
            this._hubContextService = contextService;
        }
    
        public override Task OnConnected()
        {
            this.Clients.Caller.onWhoIs();
            return base.OnConnected();
        }
    }
    

    When the hub is up and connected it will call the JavaScript method onWhoIs.

    The IHubContextService contains the HubCallerContext(which is the context that is relative to the current request)

    public class HubContextService : IHubContextService
    {
        private readonly HubCallerContext _hubContext;
    
        private HubContextService(HubCallerContext hubContext)
        {
            this._hubContext = hubContext;
        }
    
        public HubContextService()
        {}
    
        public HubContextService Resolve(HubCallerContext callerContext)
        {
            return new HubContextService(callerContext);
        }
    
        public string ContactId
        {
            get
            {
                string value = GetCookieValue("SC_ANALYTICS_GLOBAL_COOKIE");
                return value.Substring(0,value.IndexOf("|", StringComparison.Ordinal));
            }
        }
    
        public string ConnectionId
        {
            get { return this._hubContext.ConnectionId; } 
        }
    
        public string Headers
        {
            get { return this._hubContext.Request.Headers["User-Agent"]; }
        }
    
        private string GetCookieValue(string cookieKey)
        {
            Cookie cookieValue;
            this._hubContext.RequestCookies.TryGetValue(cookieKey, out cookieValue);
    
            return cookieValue == null ? string.Empty : cookieValue.Value;
        }
        
    }
    

    Since it is the current request we can get the SC_ANALYTICS_GLOBAL_COOKIE and obtain the ContactId. This will be used when we store the data in MongoDB.
    The ConnectionId is unique for that specific connection which will also be stored in the MongoDB.

    To make the hub work we need to add in the Startup class and mark it with OwinStartup(SignalR requires Owin).

    [assembly: OwinStartup(typeof(Sitecore8MVC.Web.Startup))]
    namespace Sitecore8MVC.Web
    {
        class Startup
        {
            public void Configuration(IAppBuilder appBuilder)
            {
                GlobalHost.HubPipeline.AddModule(new ErrorHandlingHubPipelineModule());
    
                var container = new Container();
                container.RegisterSingle<IHubContextService, HubContextService>();
                container.RegisterSingle<IGeoCoordinateRepository, GeoCoordinateRepository>();
                container.RegisterSingle<IRealTimeVisitorRepository, RealTimeVisitorRepository>();
                
                HubConfiguration config = new HubConfiguration
                {
                    EnableJSONP = true, 
                    Resolver = new SignalRSimpleInjectorDependencyResolver(container)
                };
    
               ConfigureSignalR(appBuilder, config);
            }
    
            public static void ConfigureSignalR(IAppBuilder app, HubConfiguration config)
            {
                app.MapSignalR(config);
            }
        }
    }
    

    I’m using SimpleInjector and the class SignalRSimpleInjectorDependencyResolver works like the SimpleInjectorDependencyResolver, it needs to derive from the Microsoft.AspNet.SignalR.DefaultDependencyResolver.

    The ErrorHandlingHubPipelineModule is for tracing “hub” errors.

    public class ErrorHandlingHubPipelineModule : HubPipelineModule
    {
        protected override void OnIncomingError(ExceptionContext exceptionContext,
            IHubIncomingInvokerContext invokerContext)
        {
            Log.Error(String.Format("Error accessing hub {0}",exceptionContext.Error.Message), this );
                
            if (exceptionContext.Error.InnerException != null)
                Log.Error(String.Format("Error accessing hub => Inner Exception {0}", exceptionContext.Error.InnerException.Message), this);
              
            base.OnIncomingError(exceptionContext, invokerContext);
        }
    }
    

    Let’s take a look at the javascript.

    Gather visitor data

    Here is the JavaScript that communicates with the hub.

    var RealTime = RealTime || {};
    
    jQuery(document).ready(function () {
        RealTime.Connector.DomReady();
    });
    
    RealTime.Connector = {
        DomReady: function () {
            RealTime.Connector.Init();
        },
        Init: function () {
    
            if (jQuery("body").data("isinpageeditor").toLowerCase() === "true")
                return;
    
            var connection = $.hubConnection();
    
            var realTimeConnector = connection.createHubProxy("RealTimeConnector");
    
            realTimeConnector.on("onWhoIs", function () {
    
                var jsonObject = RealTime.Connector.GetUserMetaData();
    
                realTimeConnector.invoke("SendClientMetaData", jsonObject).done();
            });
    
            realTimeConnector.on("onWhereIs", function () {
                RealTime.ClientTracker.TrackPerRequest(function (position) {
    
                    if (!position.coords)
                        return;
    
                    var locationUserObject = {};
                    locationUserObject["Coordinates"] = RealTime.ClientTracker.StringFormat("{0},{1}", position.coords.latitude, position.coords.longitude, "1");
    
                    var jsonObject = {};
                    jsonObject["Container"] = locationUserObject;
    
                    realTimeConnector.invoke("sendClientLocationData", jsonObject).done();
                });
            });
    
            realTimeConnector.on("onSetClientData", function (text) {
                $("#geoData").text(text);
            });
    
            connection.start().done();
        },
        GetUserMetaData: function () {
    
            var baseUserObject = {};
            baseUserObject["Language"] = jQuery("body").data("currentlanguage");
            baseUserObject["SiteName"] = jQuery("body").data("currentsite");
            baseUserObject["IpAddress"] = jQuery("body").data("currentipaddress");
            baseUserObject["PageUrl"] = window.location.pathname;
    
            var jsonObject = {};
            jsonObject["Container"] = baseUserObject;
    
            return jsonObject;
        }
    
    }
    

    When method onWhoIs is called it will gather the current user data(by calling the GetUserMetaData method) and send it back(to the hub) through method SendClientMetaData.

    Extract and deserialize

    So we are back to the hub, where method SendClientMetaData was called from the client.

    public async Task SendClientMetaData(Object jsonData)
    {
        KeysAndValuesContainer visitorMetaDataContainer = JsonConvert.DeserializeObject<KeysAndValuesContainer>(jsonData.ToString());
    
        RealTimeVisitor currentUser = await this._realTimeVisitorRepository.Get(Context) ??
                                        await this._realTimeVisitorRepository.Create(Context, visitorMetaDataContainer);
    
        await this._realTimeVisitorRepository.UpdateMetaData(Context, visitorMetaDataContainer);
    
        if (currentUser.RealTimeConnections.All(conn =>
            conn.ConnectionId != this._hubContextService.Resolve(Context).ConnectionId))
            await this._realTimeVisitorRepository.AddConnection(Context);
    
        //Get geodata
        this.Clients.Caller.onWhereIs();
    }
    

    KeysAndValuesContainer is just a POCO class holding the visitors meta data(deserialized json we got from the client).

    RealTimeVisitor contains the visitor data and it’s current connections.

    public class RealTimeVisitor
    {
        [BsonRepresentation(BsonType.ObjectId)]
        public string Id { get; set; }
        public string ContactId { get; set; }
        public IEnumerable<RealTimeConnection> RealTimeConnections { get; set; }
        public RealTimeMetaData RealTimeMetaData { get; set; }
        public DateTime CreatedDate { get; set; }
    }
    
    public class RealTimeConnection
    {
        public string ConnectionId { get; set; }
        public string UserAgent { get; set; }
        public DateTime ConnectedAt { get; set; }
        [BsonElement("loc")]
        public GeoJson GeoCoordinates { get; set; }
    }
    
    public class RealTimeMetaData
    {
        public Dictionary<string, string> MetaData { get; set; }
        [BsonElement("loc")]
        public GeoJson GeoCoordinates { get; set; }
        public DateTime LastUpdateDate { get; set; }
    }
    
    public class GeoJson
    {
        public string Type { get; set; }
        public Double[] Coordinates { get; set; }
    }
    

    RealTimeConnection and RealTimeMetada will have the visitors coordinates and if we want to use MongoDB’s very cool feature “Geospatial Queries” we need to name the GeoCoordinates as “loc” in MongoDB.

    Store the data in MongoDB

    First we need to create a connection string for our new MongoDB database in the ConnectionStrings.config. Lets call it signalr 🙂
    Best would be to have the MongoDB in the cloud, why not in Azure.

     <add name="signalr" connectionString="mongodb://localhost:27017/Sitecore8_signalr" />
    

    Recently MongoDB released the 2.0 .NET Driver with Async support, how cool is that! I wanted of course to use that but Sitecore is using the previous version of the driver, so what to do? After some searching I found the solution – we just have to put the older MongoDB assemblies in a separate folder and in the webconfig update the dependent assemblies in the runtime node:

    <dependentAssembly>
      <assemblyIdentity name="MongoDB.Bson" publicKeyToken="f686731cfb9cc103" culture="neutral"/>
      <codeBase version="2.0.0.828" href="bin\MongoDB.Bson.dll"/>
      <codeBase version="1.8.3.9" href="bin\Sitecore\MongoDB.Bson.dll"/>
    </dependentAssembly>
    <dependentAssembly>
      <assemblyIdentity name="MongoDB.Driver" publicKeyToken="f686731cfb9cc103" culture="neutral"/>
      <codeBase version="2.0.0.828" href="bin\MongoDB.Driver.dll"/>
      <codeBase version="1.8.3.9" href="bin\Sitecore\MongoDB.Driver.dll"/>
    </dependentAssembly>
    

    Now we can use the very cool stuff in the latest driver and Sitecore will still work with the previous version.

    Anders Laub’s brilliant post Working with custom MongoDB collections in Sitecore 8 using WebApi helped me a lot and gave me the idea how to store the data in MongoDB.

    The RealTimeVisitorRepository will handle the RealTimeVisitor in MongoDB.

    public class RealTimeVisitorRepository : IRealTimeVisitorRepository
    {
        private const string CollectionName = "RealTimeVisitor";
    
        private readonly IMongoCollection<RealTimeVisitor> _realTimeVisitorCollection;
        private readonly IHubContextService _hubContextService;
    
        public RealTimeVisitorRepository(IHubContextService hubContextService)
        {
            this._realTimeVisitorCollection = GetCollection<RealTimeVisitor>(ConfigurationManager.ConnectionStrings["signalr"].ConnectionString, CollectionName);
    
            this._hubContextService = hubContextService;
        }
    
        public IMongoCollection<RealTimeVisitor> Collection()
        {
            return this._realTimeVisitorCollection;
        }
    
        public async Task<RealTimeVisitor> Get(string id)
        {
            return await this._realTimeVisitorCollection.Find(visitor => visitor.Id == id).FirstOrDefaultAsync();
        }
    
        public async Task<RealTimeVisitor> Get(HubCallerContext hubContext)
        {
            return await this._realTimeVisitorCollection.Find(visitor => visitor.ContactId == this._hubContextService.Resolve(hubContext).ContactId).FirstOrDefaultAsync();
        }
    
        public async Task<bool> Delete(string id)
        {
            DeleteResult deleteResult = await this._realTimeVisitorCollection.DeleteOneAsync(Builders<RealTimeVisitor>.Filter.Eq(v => v.Id, id));
    
            return deleteResult.IsAcknowledged;
        }
    
        public async Task<bool> AddConnection(HubCallerContext hubContext)
        {
    
            FilterDefinition<RealTimeVisitor> filter = GetContactIdFilter(this._hubContextService.Resolve(hubContext).ContactId);
    
            UpdateDefinition<RealTimeVisitor> update = Builders<RealTimeVisitor>.Update.Push(user => user.RealTimeConnections, new RealTimeConnection()
                {
                    ConnectionId = this._hubContextService.Resolve(hubContext).ConnectionId,
                    ConnectedAt = DateTime.Now,
                    UserAgent = this._hubContextService.Resolve(hubContext).Headers,
                    GeoCoordinates = new GeoJson() { Type = "Point", Coordinates = null }
                });
    
            return await Update(filter, update);
        }
    
        public async Task<bool> UpdateGeoLocation(HubCallerContext hubContext, GeoCoordinate? geoCoordinate)
        {
            if (!geoCoordinate.HasValue)
                return false;
    
            FilterDefinition<RealTimeVisitor> filter = GetContactIdFilter(this._hubContextService.Resolve(hubContext).ContactId);
    
            UpdateDefinition<RealTimeVisitor> update = Builders<RealTimeVisitor>.Update.Set("RealTimeMetaData.loc.Coordinates",
                    new BsonArray { geoCoordinate.Value.Longitude, geoCoordinate.Value.Latitude });
    
    
            return await Update(filter, update);
        }
    
        public async Task<bool> UpdateGeoLocationOnConnection(HubCallerContext hubContext, int indexOfRealtimeConnection, GeoCoordinate? geoCoordinate)
        {
            if (!geoCoordinate.HasValue)
                return false;
    
            FilterDefinition<RealTimeVisitor> filter = GetContactIdFilter(this._hubContextService.Resolve(hubContext).ContactId);
    
            UpdateDefinition<RealTimeVisitor> update = Builders<RealTimeVisitor>.Update.Set(
                String.Format("RealTimeConnections.{0}.loc.Coordinates", indexOfRealtimeConnection),
                new BsonArray { geoCoordinate.Value.Longitude, geoCoordinate.Value.Latitude });
    
    
            return await Update(filter, update);
        }
    
        public async Task<bool> RemoveConnection(HubCallerContext hubContext)
        {
    
            UpdateDefinition<RealTimeVisitor> updateFilter = Builders<RealTimeVisitor>.Update.PullFilter(p => p.RealTimeConnections,
                                                r => r.ConnectionId == this._hubContextService.Resolve(hubContext).ConnectionId);
    
            UpdateResult updateResult = await
                this._realTimeVisitorCollection.UpdateOneAsync(
                    user => user.ContactId == this._hubContextService.Resolve(hubContext).ContactId, updateFilter);
    
            return updateResult.IsAcknowledged;
        }
    
        public async Task<bool> UpdateMetaData(HubCallerContext hubContext, KeysAndValuesContainer metaDataContainer)
        {
            FilterDefinition<RealTimeVisitor> filter = GetContactIdFilter(this._hubContextService.Resolve(hubContext).ContactId);
    
            UpdateDefinition<RealTimeVisitor> update = Builders<RealTimeVisitor>.Update.Set(user => user.RealTimeMetaData.MetaData,
                metaDataContainer.Container)
                .Set(user => user.RealTimeMetaData.LastUpdateDate,
                    DateTime.UtcNow);
    
    
            return await Update(filter, update);
        }
    
        public async Task<RealTimeVisitor> Create(HubCallerContext hubContext, KeysAndValuesContainer keysAndValuesContainer)
        {
            //Will be moved to a factory...
            RealTimeVisitor realTimeUser = new RealTimeVisitor()
            {
                ContactId = this._hubContextService.Resolve(hubContext).ContactId,
                CreatedDate = DateTime.UtcNow,
                RealTimeConnections = new List<RealTimeConnection>()
                {
                    new RealTimeConnection()
                    {
                        ConnectionId = this._hubContextService.Resolve(hubContext).ConnectionId,
                        ConnectedAt = DateTime.UtcNow,
                        UserAgent = this._hubContextService.Resolve(hubContext).Headers,
                        GeoCoordinates = new GeoJson() { Type = "point", Coordinates = new []{0d,0d} }
                    }
                },
                RealTimeMetaData = new RealTimeMetaData()
                {
                    MetaData = keysAndValuesContainer.Container,
                    GeoCoordinates = new GeoJson() { Type = "point", Coordinates = new[] { 0d, 0d } },
                    LastUpdateDate = DateTime.UtcNow
                }
            };
    
            await this._realTimeVisitorCollection.InsertOneAsync(realTimeUser);
    
            return await Task.Run(() => realTimeUser);
        }
    
        private async Task<bool> Update(FilterDefinition<RealTimeVisitor> filter, UpdateDefinition<RealTimeVisitor> update)
        {
            UpdateResult result = await this._realTimeVisitorCollection.UpdateOneAsync(filter, update);
    
            return result.IsAcknowledged;
        }
    
        private FilterDefinition<RealTimeVisitor> GetContactIdFilter(string contactId)
        {
            return Builders<RealTimeVisitor>.Filter.Eq(c => c.ContactId, contactId);
        }
    
    
        private IMongoCollection<T> GetCollection<T>(string connectionString, string collectionName) where T : class
        {
            var url = new MongoUrl(connectionString);
    
            return new MongoClient(url).GetDatabase(url.DatabaseName).GetCollection<T>(collectionName);
        }
    
    }
    

    I know there are a lot of methods but let me go through some of them.
    The actual connection to the MongoDB take place in method GetCollection.

    To retrieve a RealTimeVisitor you can do it by the Object Id(Generated by MongoDB) or the Contact Id(Code wise it’s so much easier then previous versions).

    await _realTimeVisitorCollection.Find(visitor => visitor.Id== id).FirstOrDefaultAsync();
    

    You can also use FilterDefinition.
    Read more here – Find or Query Data with C# Driver

    In the update methods I used the FilterDefinition(query) and the UpdateDefinitionBuilder:

    FilterDefinition<RealTimeVisitor> filter = Builders<RealTimeVisitor>.Filter.Eq(c => c.Id, id)
    
    UpdateDefinition<RealTimeVisitor> update = Builders<RealTimeVisitor>.Update.Set("RealTimeMetaData.loc.Coordinates",
            new BsonArray { geoCoordinate.Value.Longitude, geoCoordinate.Value.Latitude });
    
    UpdateResult result = await this._realTimeVisitorCollection.UpdateOneAsync(filter, update);
    

    For the update I used method UpdateOneAsyncbut there are also UpdateManyAsync and ReplaceOneAsync.
    Read more here – Update Data with C# Driver

    Send data back

    After the data has been stored we make a new call to the client to get his/her current location.

    //Get geodata
    this.Clients.Caller.onWhereIs();
    

    Here is the client method.

    realTimeConnector.on("onWhereIs", function () {
        RealTime.ClientTracker.TrackPerRequest(function (position) {
    
            if (!position.coords)
                return;
    
            var locationUserObject = {};
            locationUserObject["Coordinates"] = RealTime.ClientTracker.StringFormat("{0},{1}", position.coords.latitude, position.coords.longitude, "1");
    
            var jsonObject = {};
            jsonObject["Container"] = locationUserObject;
    
            realTimeConnector.invoke("sendClientLocationData", jsonObject).done();
        });
    });
    

    The interesting part is when we got the coordinates we call the hub and send the geocoordinates.

    The coordinates will be stored in MongoDB and then reversed geocoded(by Geocoding.net).

    public async Task SendClientLocationData(Object jsonData)
    {
       KeysAndValuesContainer visitorMetaDataContainer = JsonConvert.DeserializeObject<KeysAndValuesContainer>(jsonData.ToString());
    
        if (!visitorMetaDataContainer.ContainsParamkey(KeysAndValuesContainerKeys.Coordinates))
            return;
    
        GeoCoordinate? geoCoordinate =
            this._geoCoordinateRepository.Get(visitorMetaDataContainer.GetValueByKey(KeysAndValuesContainerKeys.Coordinates));
    
        bool success = await this._realTimeUserRepository.UpdateGeoLocation(Context, geoCoordinate);
    
        if (!success)
            return;
    
        string geoData = GetGeoLocationData(geoCoordinate);
    
        if (String.IsNullOrWhiteSpace(geoData))
            geoData = "No address";
             
        this.Clients.Caller.onSetClientData(geoData);
    
    }
    
    private string GetGeoLocationData(GeoCoordinate? geoCoordinate)
    {
        if (!geoCoordinate.HasValue)
            return string.Empty;
    
        IGeocoder geocoder = new GoogleGeocoder();
        IEnumerable<Address> geoData = geocoder.ReverseGeocode(geoCoordinate.Value.Latitude, geoCoordinate.Value.Longitude);
    
        return geoData.First().FormattedAddress;
    }
    

    Finally we send the geodata back by calling client method onSetClientData.

    Where the client function will present the data.

    realTimeConnector.on("onSetClientData", function (text) {
        $("#geoData").text(text);
    });
    

    Client leaves website

    Last scenario will be when the visitor leaves the website. It will trigger OnDisconnected on the hub. The current connection will be removed from the visitor object and if no more connections, the visitor will be removed from the database.

    public override async Task OnDisconnected(bool stopCalled)
    {
        await this._realTimeUserRepository.RemoveConnection(Context);
    
        RealTimeVisitor realTimeVisitor = await this._realTimeUserRepository.Get(Context);
    
        if (!realTimeVisitor.RealTimeConnections.Any())
            await this._realTimeUserRepository.Delete(realTimeVisitor.Id);
    
        await base.OnDisconnected(stopCalled);
    }
    

    I really like the idea of working with data in real-time. The cool thing with this technique is that you can also interact or even communicate with the visitors in real-time. I will try to do some more posts around the real-time subject.

    That’s all for now folks 🙂