Dockers Dockers Dockers with Sitecore for developers

Dear Sitecore friends, I hope you are all good 🙂
Sitecore recently released another successful version, Sitecore 9.2. You should download it and play with it at once!

Pieter Brinkman (Senior Director Technical Marketing for leading .NET Experience Platform Sitecore) has some really good posts, describing all cool features:
Sitecore 9.2 – Time to Market Highlights
Sitecore 9.2 – Innovation Highlights.
How Sitecore’s Investment strategy impacts the developer’s job

Another cool thing, Sitecore has a new facelift! Check out their new website, https://www.sitecore.com. Notice the “Personalization View” toggler in the top bar. Very nice Sitecore 🙂

Developers, developers, developers

Today I want to share with you how great and helpful dockers can be in your developer life 🙂
Martin Miles great post – Starting with Docker and Sitecore, was a huge eye-opener for me. I’m really thankful for this.

There are two great “Sitecore docker” repositories (that I know of) at the github:

Sitecore/docker-images
Created by Per Manniche Bering, which is now a “Sitecore” repository. Great job Per! You should also read his blogpost, invokecommand.net. Check ot the post – Sitecore on Docker Swarm in production. Very interesting.

The repository has A LOT OF Sitecore versions, from 8 to 9.2 – All images.
The only down-side is that they don’t have Sitecore Identity Server, but I have a feeling it will come 🙂

avivasolutionsnl/sitecore-docker
Created by Aviva Solutions B.V. – Great job, thank you for this!
This repository focuses on the latest versions. They also have Sitecore Commerce – how cool is that!
And they have Sitecore Identity Server.

They have some great tips regarding Docker. I really like the use of whales-names:
To set the Docker container service names as DNS names on your host edit your hosts file. A convenient tool to automatically do this is whales-names

Let the two great repositories be our influencers 🙂

The goal is to have a working/running Sitecore website on dockers on your local dev machine.
By the way, have a running Sitecore website… It sounds so simple, are you aware of how many sites/services/servers it takes to have a “running Sitecore website”?
This is for Sitecore 9.2:
CM website
CD website
Sitecore Identity
xConnect
xConnect AutomationEngine
xConnect Indexworker
SQL server
Solr server

Modules:
Sitecore JavaScript Services
Sitecore PowerShell Extensions
Sitecore Experience Accelerator

That is the “simple setup” 🙂

And there is more, like:
Sitecore Universal Tracker
Sitecore Experience Commerce 9.1

Setting up a Sitecore instance is not easy, so how can we make it simpler? By using dockers, of cource 😉

Ok, first of all, we need to install Docker. Go to https://hub.docker.com, sign up and install Docker. Read all about it here – Install Docker Desktop for Windows

Docker needs the Hyper-V enabled, or does it?

Windows 10–1809 (“October 2018 Update”) + Docker engine 18.09.1 + the Windows 1809 base images from Dockerhub are the first combinations that allows you to run “real” Windows containers on Windows 10, without Hyper-V virtualization.

But hey, how about the poor sods who are using VirtualBox? VirtualBox does not play well when Hyper-V is enabled. Fear not good people, here is the solution.
*We will NOT do the Add/Remove Features to turn the Hyper-V support off, it will give you tons of headache.
Let’s follow Scott Hanselman’s advice – Switch easily between VirtualBox and Hyper-V with a BCDEdit boot Entry in Windows 8.1:
*It also works on Windows 10

From the administrative command prompt, copy the boot menu with a “No Hyper-V” entry:
*Note the first command gives you a GUID and you then copy it and use it with the second command.

C:\>bcdedit /copy {current} /d "No Hyper-V" 
The entry was successfully copied to {ff-23-113-824e-5c5144ea}. 

C:\>bcdedit /set {ff-23-113-824e-5c5144ea} hypervisorlaunchtype off 
The operation completed successfully.

In order to access the new boot menu, we need to do a Restart but hold down shift on the keyboard while clicking Restart with the mouse.

You will get a Blue Screen. Select “Other Operating Systems” and your “No Hyper-V” option is in there.

Now, you can run Virtual Box nicely but still choose Hyper-V when you want. 🙂

Ok, let’s move on. You now have Docker installed and all is well.
Time to generate some Sitecore Images. How about we try out the Sitecore/docker-images?
According to How to use in the readme doc, they suggest we setup a private Docker repository(It’s important it is private we don’t want any Sitecore images laying around in the open/public). Here is Azure Container Registry suggested and I have to agree, it’s easy to setup.

So let’s do it, we will create an Azure Container Registry!
First we need to login/sign up to Azure Portal – https://azure.microsoft.com/en-us/services/container-registry/
After successful login/sign in, search for “ContainerRegistry”.
Select it and hit the Create container registry button

Give it a proper name, select a resource and select Basic or Standard. Standard will give you 100 GB and Basic will give you 10 GB.

We now have a working Azure Container Registry 🙂

One last thing, let’s have admin enabled, we will use it for the powershell script…

Next thing is to fork the GitHub repository – Sitecore/docker-images
Open it in VSCode but before that, please install the Powershell extension. We will need it when we debug/run the powershell scripts.

In the readme doc there is an example of a powershell script, which will generate images and push them to a container registry. Let’s create a powershell script file, Run.ps1, and put it in the root of our forked github repository.

Here is the Run.ps1 script(with some changes). Notice the username and password…
The script will download all Sitecore packages that are needed in order to generate Sitecore images. The cool thing here is that you can filter/choose what version of Sitecore images you want to build. Let’s build some Sitecore 9.2 images 🙂

$azureRepository = "sitecoreimages.azurecr.io"
$userName = "SitecoreImages"
$password = "SECRET PASSWORD FROM YOUR ENABLED ADMIN IN AZURE"

# Login
docker login $azureRepository -u $userName -p $password

# Load module
Import-Module (Join-Path $PSScriptRoot "\modules\SitecoreImageBuilder") -Force

# Settings
$installSourcePath = (Join-Path $PSScriptRoot "\packages") # PATH TO WHERE YOU KEEP ALL SITECORE ZIP FILES AND LICENSE.XML, can be on local machine or a file share.
$registry = $azureRepository ` # On Docker Hub it's your username or organization, else it's the hostname of your own registry.
$sitecoreUsername = "YOUR SITECORE USERNAME"
$sitecorePassword = "SECRET PASSWORD HERE"

$baseTags = "sitecore-*:9.2.0*1903" # optional (default "*"), set to for example "sitecore-*:9.1.1*ltsc2019" to only build 9.1.1 images on ltsc2019/1809.

# Restore packages needed for base images, only files missing in $installSourcePath will be downloaded
SitecoreImageBuilder\Invoke-PackageRestore `
    -Path (Join-Path $PSScriptRoot "\images") `
    -Destination $installSourcePath `
    -Tags $baseTags `
    -SitecoreUsername $sitecoreUsername `
    -SitecorePassword $sitecorePassword

# Build and push base images
SitecoreImageBuilder\Invoke-Build `
    -Path (Join-Path $PSScriptRoot "\images") `
    -InstallSourcePath $installSourcePath `
    -Registry $registry `
    -Tags $baseTags `
    -PushMode "WhenChanged" # optional (default "WhenChanged"), can also be "Never" or "Always".

$variantTags = "*:9.2.0*1903" # optional (default "*"), set to for example "sitecore-xm1-sxa-*:9.1.1*ltsc2019" to only build 9.1.1 images on ltsc2019/1809.

# Restore packages needed for variant images, only files missing in $installSourcePath will be downloaded
SitecoreImageBuilder\Invoke-PackageRestore `
    -Path (Join-Path $PSScriptRoot "\variants") `
    -Destination $installSourcePath `
    -Tags $variantTags `
    -SitecoreUsername $sitecoreUsername `
    -SitecorePassword $sitecorePassword

# Build and push variant images
SitecoreImageBuilder\Invoke-Build `
    -Path (Join-Path $PSScriptRoot "\variants") `
    -InstallSourcePath $installSourcePath `
    -Registry $registry `
    -Tags $variantTags `
    -PushMode "WhenChanged" # optional (default "WhenChanged"), can also be "Never" or "Always".

If you dont want to add the images to a container registry, just comment out the “docker login” part and set -PushMode “Never”

The PowerShell script needs a folder for the Sitecore packages and your Sitecore license. We will put the folder in the root of our forked Github repository.

$installSourcePath = (Join-Path $PSScriptRoot "\packages") # PATH TO WHERE YOU KEEP ALL SITECORE ZIP FILES AND LICENSE.XML, can be on local machine or a file share.

Time to run the script. Go to Debug and select Start Debugging or just hit F5 🙂

Voila, here we have the images in Azure:

Let’s see if they are on our computer? Just enter:

docker images

Ok, our Sitecore 9.2 images are now generated. We have them in Azure but also on our computer.

Next thing will be to create Docker containers in order to run our Sitecore website 🙂

Wait a minute, what is the difference between Docker images and Docker containers?

Docker Image is a set of files which has no state, whereas Docker Container is the instantiation of Docker Image. In other words, Docker Container is the run time instance of images.

Let’s dive into our forked GitHub repository. Locate tests folder and select 9.2.0 rev. 002893. Open up windowsservercore, now you will see a bunch of docker-compose files. So much fun stuff to play with 🙂 How about we try out SXA running on CM and CD? Let’s select the docker-compose.xp.sxa.yml file.

Here is the docker-compose.xp.sxa.yml.

 
version: '2.4'

services:

  sql:
    image: sitecoreimages.azurecr.io/sitecore-xp-sxa-1.9.0-sqldev:9.2.0-windowsservercore-${WINDOWSSERVERCORE_CHANNEL}
    volumes:
      - .\data\sql:C:\Data
    mem_limit: 2GB
    ports:
      - "44010:1433"

  solr:
    image: sitecoreimages.azurecr.io/sitecore-xp-sxa-1.9.0-solr:9.2.0-nanoserver-${NANOSERVER_CHANNEL}
    volumes:
      - .\data\solr:C:\Data
    mem_limit: 1GB
    ports:
      - "44011:8983"

  xconnect:
    image: sitecoreimages.azurecr.io/sitecore-xp-xconnect:9.2.0-windowsservercore-${WINDOWSSERVERCORE_CHANNEL}
    volumes:
      - .\data\xconnect:C:\inetpub\xconnect\App_Data\logs
    mem_limit: 1GB
    links:
      - sql
      - solr

  xconnect-automationengine:
    image: sitecoreimages.azurecr.io/sitecore-xp-xconnect-automationengine:9.2.0-windowsservercore-${WINDOWSSERVERCORE_CHANNEL}
    volumes:
      - .\data\xconnect-automationengine:C:\AutomationEngine\App_Data\logs
    mem_limit: 500MB
    links:
      - sql
      - xconnect

  xconnect-indexworker:
    image: sitecoreimages.azurecr.io/sitecore-xp-xconnect-indexworker:9.2.0-windowsservercore-${WINDOWSSERVERCORE_CHANNEL}
    volumes:
      - .\data\xconnect-indexworker:C:\IndexWorker\App_Data\logs
    mem_limit: 500MB
    links:
      - sql
      - solr

  cd:
    image: sitecoreimages.azurecr.io/sitecore-xp-sxa-1.9.0-cd:9.2.0-windowsservercore-${WINDOWSSERVERCORE_CHANNEL}
    volumes:
      - .\data\cd\sc:C:\inetpub\sc
    ports:
      - "44002:80"
    links:
      - sql
      - solr
      - xconnect

  cm:
    image: sitecoreimages.azurecr.io/sitecore-xp-sxa-1.9.0-standalone:9.2.0-windowsservercore-${WINDOWSSERVERCORE_CHANNEL}
    volumes:
      - .\data\cm\sc:C:\inetpub\sc
      - .\data\creativeexchange:C:\inetpub\sc\App_Data\packages\CreativeExchange\FileStorage
    ports:
      - "44001:80"
    links:
      - sql
      - solr
      - xconnect

Here are some changes, notice the volumes for CM and CD. We will make the website folder(for CM and CD) available. We want more than just the logs, we want to look and feel the files 🙂
Let me come back to this when the docker-compose has run.

Time to run the docker-compose.xp.sxa.yml. file. We can do this from VSCode in the terminal window with the following command:

 
docker-compose -f docker-compose.xp.sxa.yml up

Just to be sure, we want to verify that the Docker containers have been created successfully. Start/open up a PowerShell window and enter the following command:

 
docker ps

It will list all current containers:

Yes we have them all 🙂

Ok, next will be to copy all files from www folder(C:\inetpub\sc) in docker container’s CM and CD, to data/cm/sc and data/cd/sc.

The preferred way is to NOT copy files for CM and CD. This is how you should do it:

In each Sitecore image there is a script embedded for this purpose: https://github.com/Sitecore/docker-images/blob/master/images/9.2.0%20rev.%20002893/windowsservercore/sitecore-xm1-cm/Sitecore/Scripts/Watch-Directory.ps1. It will continuously copy content from a volume mount into /inetpub/sc, see https://github.com/sitecoreops/sitecore-health-extensions/blob/master/docker-compose.yml#L27 and how to configure ENTRYPOINT: https://github.com/sitecoreops/sitecore-health-extensions/blob/master/docker-compose.yml#L22. In general, you can use https://github.com/sitecoreops/sitecore-health-extensions as a reference on how to wire up a solution.

We will create the following script. It will locate the docker containers windowsservercore_cm_1 and windowsservercore_cd_1. In order to copy files, we need to temporarily stop the containers. Now we can copy all the files from inetpub/sc(wwwroot) and put them in data/cm/sc and data/cd/sc. When we are done we start the containers again 🙂

function CopySitecoreFiles($sitecoreContainerName, $outputDir){
    
    $dockerId = docker ps -q --filter name=${sitecoreContainerName}

    docker stop $dockerId

    $dockerIdAndPath = "${dockerId}:/inetpub/sc"

    docker cp $dockerIdAndPath "${outputDir}"

    docker start $dockerId
	
	Write-Output "Sitecore files from container, " $sitecoreContainerName  ", have been copied to " $outputDir
}


CopySitecoreFiles "windowsservercore_cm_1" "data/cm/sc"

CopySitecoreFiles "windowsservercore_cd_1" "data/cd/sc"

Success! Just look at all the lovely files 🙂

We are almost finished, we need to do some whaaaaling
We want to set the Docker container service names as DNS names in our hosts file. For that, we will use whales-names.
Open up your command window and enter the following command:

npm install -g whales-names

Let’s synchronize docker container hostnames in our hosts file

$ whales-names

And in your hosts file you should have something like this:

# whales-names begin
172.26.26.116	468c43ff56fc cm windowsservercore_cm_1
172.26.23.66	6da57c9a621c cd windowsservercore_cd_1
172.26.31.9	feaaaee365bf windowsservercore_xconnect-automationengine_1 xconnect-automationengine
172.26.24.226	678703d5421f windowsservercore_xconnect-indexworker_1 xconnect-indexworker
172.26.21.170	66fabe03d491 windowsservercore_xconnect_1 xconnect
172.26.17.199	fc547c9e5c4e sql windowsservercore_sql_1
172.26.28.252	d142262322b9 solr windowsservercore_solr_1
# whales-names end

Ok let’s browse and test our CM site, http://windowsservercore_cm_1/sitecore/login:

Test our CD site, http://windowsservercore_cd_1/:

Tracking is working fine 🙂

Let me know if you have any issues and I will try to help/guide you.

That’s all for now folks 🙂

Let’s enjoy – Bill Gates and Steve Ballmer Playday