こんにちは。

今回は、ARM ベースの Microsoft Azure (Azure IaaS v2) における負荷分散や Auto Scale の構成、考え方、周辺情報などを記載します。

ここでは ARM (Azure Resource Manager) Template を使ったサンプルを紹介しますが、Azure CLI, PowerShell などのコマンドを使用してマルチ インスタンス構成をおこなう際も、ここで解説するリソース構成を理解して、「どのリソース」、「どのプロパティ」などの判断 (勘所) の参考にしてください。(以前投稿した「Azure Resource Manager の template の How-to」、「Azure Resource (ARM) の REST API による管理」の事前知識が必要です。)

なお、ここでは Azure が標準で提供するリソース (Load Balancer, Auto-Scaling 等) に絞って紹介しますが、Azure では 3rd Party が提供するインフラ (Barracuda, HAProxy など) も Marketplace から快適に使っていただけますので (これらは Azure 上で充分な動作確認がおこなわれています)、構成や要件によってこれらもうまく使ってください。

負荷分散 – Load Balancer, Application Gateway

「Azure Resource Manager の template の How-to」で解説した Load Balancer を使わない Virtual Machine では、Virtual Machine 用の Public IP (publicIPAddresses) と NIC (networkInterfaces) を準備して Virtual Machine に割り当てていました。

Azure Load Balancer を使った構成では、Load Balancer (loadBalancers) の Frontend (一般利用者からの入口) に この Public IP (publicIPAddress) を構成し、Backend (Virtual Machine 側への出口) を各 Virtual Machine の NIC (networkInterfaces) と接続します。下図のような構成です。(下図では、単一の Load Balancer に対して 1 組の Frondend と Backend を記載していますが、単一の Load Balancer リソースで複数の組も登録できます。)

下記の Resource template では、2 つの NIC (testnic01, testnic1), 2 つの Virtual Machine (testvm0, testvm1) を作成し、これらの NIC に Load Balancer (testlb) の Backend Pool を割り当てています。

「Azure Resource Manager の template の How-to」で解説したように、既定で {domainNameLabel}.{region}.cloudapp.azure.com のフォーマット、つまり、今回の場合、matsuendpoint.eastus.cloudapp.azure.com でこの LB の Endpoint に接続できます。(もちろん、割り振られた IP アドレスを使って接続しても OK です。)

なお、今回は外向けの Load Balancer ですが、Internal Load Balancer の場合は、Load Balancer の frontendIPConfigurations として、publicIpAddress ではなく、サブネット内の privateIpAddress を設定すれば OK です。

{  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",  "contentVersion": "1.0.0.0",  "parameters": {  },  "variables": {  },  "resources": [{  "name": "matsustorage",  "type": "Microsoft.Storage/storageAccounts",  "location": "East US",  "apiVersion": "2015-06-15",  "properties": {"accountType": "Standard_LRS"  }},{  "name": "testvnet",  "type": "Microsoft.Network/virtualNetworks",  "location": "East US",  "apiVersion": "2016-03-30",  "properties": {"addressSpace": {  "addressPrefixes": ["10.0.0.0/16"  ]},"subnets": [  {"name": "testsubnet0","properties": {  "addressPrefix": "10.0.0.0/24"}  }]  }},{  "apiVersion": "2016-03-30",  "type": "Microsoft.Network/publicIPAddresses",  "name": "testip",  "location": "East US",  "properties": {"publicIPAllocationMethod": "Dynamic","dnsSettings": {  "domainNameLabel": "matsuendpoint"}  }},{  "name": "[concat('testnic', copyIndex())]",  "type": "Microsoft.Network/networkInterfaces",  "location": "East US",  "apiVersion": "2016-03-30",  "dependsOn": ["Microsoft.Network/virtualNetworks/testvnet","Microsoft.Network/loadBalancers/testlb"  ],  "properties": {"ipConfigurations": [  {"name": "[concat('ipconfig', copyIndex())]","properties": {  "privateIPAllocationMethod": "Dynamic",  "subnet": {"id": "[concat(resourceId('Microsoft.Network/virtualNetworks', 'testvnet'), '/subnets/testsubnet0')]"  },  "loadBalancerBackendAddressPools": [{  "id": "[concat(resourceId('Microsoft.Network/loadBalancers','testlb'), '/backendAddressPools/testbackend')]"}  ]}  }]  },  "copy": {"name": "testniccopy","count": 2  }},{  "name": "testlb",  "type": "Microsoft.Network/loadBalancers",  "location": "East US",  "apiVersion": "2016-03-30",  "dependsOn": ["Microsoft.Network/publicIPAddresses/testip"  ],  "properties": {"frontendIPConfigurations": [  {"name": "testfrontend","properties": {  "publicIPAddress": {"id": "[resourceId('Microsoft.Network/publicIPAddresses','testip')]"  }}  }],"backendAddressPools": [  {"name": "testbackend"  }],"loadBalancingRules": [  {"Name": "rule01","properties": {  "frontendIPConfiguration": {"id": "[concat(resourceId('Microsoft.Network/loadBalancers', 'testlb'), '/frontendIpConfigurations/testfrontend')]"  },  "backendAddressPool": {"id": "[concat(resourceId('Microsoft.Network/loadBalancers', 'testlb'), '/backendAddressPools/testbackend')]"  },  "probe": {"id": "[concat(resourceId('Microsoft.Network/loadBalancers', 'testlb'), '/probes/testprobe')]"  },  "protocol": "Tcp",  "frontendPort": 80,  "backendPort": 80}  }],"probes": [  {"name": "testprobe","properties": {  "protocol": "Tcp",  "port": 80,  "intervalInSeconds": 15,  "numberOfProbes": 2}  }]  }},{  "type": "Microsoft.Compute/availabilitySets",  "name": "testas",  "apiVersion": "2016-03-30",  "location": "East US",  "properties": {"platformFaultDomainCount": 3,"platformUpdateDomainCount": 3  }},{  "name": "[concat('testvm', copyIndex())]",  "type": "Microsoft.Compute/virtualMachines",  "location": "East US",  "apiVersion": "2016-03-30",  "dependsOn": ["Microsoft.Storage/storageAccounts/matsustorage","Microsoft.Compute/availabilitySets/testas","[concat('Microsoft.Network/networkInterfaces/testnic', copyIndex())]"  ],  "properties": {"hardwareProfile": {  "vmSize": "Standard_D1"},"osProfile": {  "computername": "[concat('mycomputer', copyIndex())]",  "adminUsername": "demouser",  "adminPassword": "P@ssw0rd01"},"storageProfile": {  "imageReference": {"publisher": "MicrosoftWindowsServer","offer": "WindowsServer","sku": "2012-R2-Datacenter","version": "latest"  },  "osDisk": {"name": "[concat('testdisk', copyIndex())]","vhd": {  "uri": "[concat('http://matsustorage.blob.core.windows.net/vhds/test',copyIndex(),'.vhd')]"},"caching": "ReadWrite","createOption": "FromImage"  }},"networkProfile": {  "networkInterfaces": [{  "id": "[resourceId('Microsoft.Network/networkInterfaces', concat('testnic',copyIndex()))]"}  ]},"availabilitySet": {  "id": "[resourceId('Microsoft.Compute/availabilitySets','testas')]"}  },  "copy": {"name": "testvmcopy","count": 2  }}  ],  "outputs": {  }}

上記の通り、Availability Set (availabilitySets リソース) は Load Balancer と一緒に構成しておいてください。

また、以降も同様ですが、このままでは RDP (Windows の場合) や SSH (Linux の場合) を使った各 VM への接続はできませんので、実際の構成では、「Azure Resource Manager の template の How-to」で解説したように各 Virtual Machine のインスタンスに直接アクセス可能な Public IP (および、その Public IP を持つ NIC) も同時に構成して、各 Virtual Machine のリモート管理をおこなえるようにします。(上述の Load Balancer の loadBalancingRules に RDP や SSH の Port も追加できますが、この場合、これらの接続も分散されることになります。)

上述の Load Balancer が Hash ベースの Layer-4 Load Balancer (TCP/IP レベル) であるのに対して、Azure では Layer-7 Load Balancer である Azure Application Gateway も使えます。Application Gateway を使用すると、Cookie based affinity, SSL off-load, path based のルーティングなどが可能です。

L4 Load Balancer を構成する際は、上述の通り、接続する Backend の NIC を Load Balancer の Backend Address Pool に割り当てましたが、
Application Gateway (L7 Load Balancer) を構成する際は、単純に IP Address のマッチングで Backend と接続します。

下記のサンプル (Resource Template) では、2 つの Virtual Machine を作成し、Application Gateway (L7 Load Balancer, ARR) からその Private IP にアクセスするように構成しています。(前半部分は同じなので、一番最後の applicationGateways リソースのみ注目してください。)

いくつか注意点があります。まず、Application Gateway が使用する Subnet は、必ず、他のリソースが使用していない専用のものを使います。また、L4 Load Balancer (上述) と異なり、Frontend の Public IP は DNS (domainNameLabel) を割り当てず、自動で割り振るようにしてください。(dnsNameLabel を指定するとエラーになります。) この場合、FQDN として、a0d831a6-c95f-4de7-b7ef-ac92176edbd5.cloudapp.net などの形式の DNS 名が割り当てられて、下記の outputs に記載している方法で取得・参照できます。

{  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",  "contentVersion": "1.0.0.0",  "parameters": {  },  "variables": {  },  "resources": [{  "name": "matsustorage",  "type": "Microsoft.Storage/storageAccounts",  "location": "East US",  "apiVersion": "2015-06-15",  "properties": {"accountType": "Standard_LRS"  }},{  "name": "testvnet",  "type": "Microsoft.Network/virtualNetworks",  "location": "East US",  "apiVersion": "2016-03-30",  "properties": {"addressSpace": {  "addressPrefixes": ["10.0.0.0/16"  ]},"subnets": [  {"name": "testsubnet0","properties": {  "addressPrefix": "10.0.0.0/24"}  },  {"name": "testsubnet1","properties": {  "addressPrefix": "10.0.1.0/24"}  }]  }},{  "name": "testnic0",  "type": "Microsoft.Network/networkInterfaces",  "location": "East US",  "apiVersion": "2016-03-30",  "dependsOn": ["Microsoft.Network/virtualNetworks/testvnet"  ],  "properties": {"ipConfigurations": [  {"name": "ipconfig0","properties": {  "privateIPAllocationMethod": "Static",  "privateIPAddress": "10.0.0.30",  "subnet": {"id": "[concat(resourceId('Microsoft.Network/virtualNetworks', 'testvnet'), '/subnets/testsubnet0')]"  }}  }]  }},{  "name": "testnic1",  "type": "Microsoft.Network/networkInterfaces",  "location": "East US",  "apiVersion": "2016-03-30",  "dependsOn": ["Microsoft.Network/virtualNetworks/testvnet"  ],  "properties": {"ipConfigurations": [  {"name": "ipconfig1","properties": {  "privateIPAllocationMethod": "Static",  "privateIPAddress": "10.0.0.31",  "subnet": {"id": "[concat(resourceId('Microsoft.Network/virtualNetworks', 'testvnet'), '/subnets/testsubnet0')]"  }}  }]  }},{  "name": "testvm0",  "type": "Microsoft.Compute/virtualMachines",  "location": "East US",  "apiVersion": "2016-03-30",  "dependsOn": ["Microsoft.Storage/storageAccounts/matsustorage","Microsoft.Network/networkInterfaces/testnic0"  ],  "properties": {"hardwareProfile": {  "vmSize": "Standard_D1"},"osProfile": {  "computername": "mycomputer0",  "adminUsername": "demouser",  "adminPassword": "P@ssw0rd01"},"storageProfile": {  "imageReference": {"publisher": "MicrosoftWindowsServer","offer": "WindowsServer","sku": "2012-R2-Datacenter","version": "latest"  },  "osDisk": {"name": "testdisk0","vhd": {  "uri": "http://matsustorage.blob.core.windows.net/vhds/test0.vhd"},"caching": "ReadWrite","createOption": "FromImage"  }},"networkProfile": {  "networkInterfaces": [{  "id": "[resourceId('Microsoft.Network/networkInterfaces', 'testnic0')]"}  ]}  }},{  "name": "testvm1",  "type": "Microsoft.Compute/virtualMachines",  "location": "East US",  "apiVersion": "2016-03-30",  "dependsOn": ["Microsoft.Storage/storageAccounts/matsustorage","Microsoft.Network/networkInterfaces/testnic1"  ],  "properties": {"hardwareProfile": {  "vmSize": "Standard_D1"},"osProfile": {  "computername": "mycomputer1",  "adminUsername": "demouser",  "adminPassword": "P@ssw0rd01"},"storageProfile": {  "imageReference": {"publisher": "MicrosoftWindowsServer","offer": "WindowsServer","sku": "2012-R2-Datacenter","version": "latest"  },  "osDisk": {"name": "testdisk1","vhd": {  "uri": "http://matsustorage.blob.core.windows.net/vhds/test1.vhd"},"caching": "ReadWrite","createOption": "FromImage"  }},"networkProfile": {  "networkInterfaces": [{  "id": "[resourceId('Microsoft.Network/networkInterfaces', 'testnic1')]"}  ]}  }},{  "apiVersion": "2016-03-30",  "type": "Microsoft.Network/publicIPAddresses",  "name": "testip",  "location": "East US",  "properties": {"publicIPAllocationMethod": "Dynamic"  }},{  "name": "testgw",  "type": "Microsoft.Network/applicationGateways",  "location": "East US",  "apiVersion": "2016-03-30",  "dependsOn": ["Microsoft.Network/virtualNetworks/testvnet","Microsoft.Network/publicIPAddresses/testip","Microsoft.Compute/virtualMachines/testvm0","Microsoft.Compute/virtualMachines/testvm1"  ],  "properties": {"sku": {  "name": "Standard_Medium",  "tier": "Standard",  "capacity": "2"},"gatewayIPConfigurations": [  {"name": "gwipconf0","properties": {  "subnet": {"id": "[concat(resourceId('Microsoft.Network/virtualNetworks','testvnet'),'/subnets/testsubnet1')]"  }}  }],"frontendIPConfigurations": [  {"name": "gwfrontip0","properties": {  "PublicIPAddress": {"id": "[resourceId('Microsoft.Network/publicIPAddresses', 'testip')]"  }}  }],"frontendPorts": [  {"name": "gwfrontport0","properties": {  "Port": 80}  }],"backendAddressPools": [  {"name": "gwbackpool0","properties": {  "BackendAddresses": [{  "IpAddress": "10.0.0.30"},{  "IpAddress": "10.0.0.31"}  ]}  }],"backendHttpSettingsCollection": [  {"name": "gwbackhttpset0","properties": {  "Port": 80,  "Protocol": "Http",  "CookieBasedAffinity": "Disabled"}  }],"httpListeners": [  {"name": "gwhttplis0","properties": {  "FrontendIPConfiguration": {"Id": "[concat(resourceId('Microsoft.Network/applicationGateways','testgw'), '/frontendIPConfigurations/gwfrontip0')]"  },  "FrontendPort": {"Id": "[concat(resourceId('Microsoft.Network/applicationGateways','testgw'), '/frontendPorts/gwfrontport0')]"  },  "Protocol": "Http",  "SslCertificate": null}  }],"requestRoutingRules": [  {"Name": "gwrule0","properties": {  "RuleType": "Basic",  "httpListener": {"id": "[concat(resourceId('Microsoft.Network/applicationGateways','testgw'), '/httpListeners/gwhttplis0')]"  },  "backendAddressPool": {"id": "[concat(resourceId('Microsoft.Network/applicationGateways','testgw'), '/backendAddressPools/gwbackpool0')]"  },  "backendHttpSettings": {"id": "[concat(resourceId('Microsoft.Network/applicationGateways','testgw'), '/backendHttpSettingsCollection/gwbackhttpset0')]"  }}  }]  }}  ],  "outputs": {"applicationGatewayDns": {  "type": "string",  "value": "[reference(resourceId('Microsoft.Network/publicIPAddresses', 'testip')).dnsSettings.fqdn]"}  }}

2017/07 追記 : Azure Container Service (ACS) で Virtual Network (VNet) がサポートされました。Container 上の Guest OS も対象に Azure Load Balancer などのネットワーク構成が可能です。

Azure Virtual Machine (VM) Scale Set

上記では Azure Virtual Machine リソースを 1 台ずつ (台数分) 準備し、そこに対して Load Balancer (または Application Gateway) リソースを関連付けましたが、Azure ではこうした複数台の同一構成を単一のリソースとして定義できる Azure Virtual Machine Scale Sets が提供されています。

例えば、下記は、VM Scale Set を使って 5 台の同一構成の VM を設定し、ここに Load Balancer を設定して HTTP (Port 80) の負荷分散を構成しています。(VM Scale Set は、Load Balancer や Application Gateway を使わず単独で定義することも可能です。)
なお、注意点として、Scaling Set で使用するマシン群は、それぞれ異なる Storage Account が必要ですので、下記の通り、Virtual Machine の数だけ Storage Account を作成します。

{  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",  "contentVersion": "1.0.0.0",  "parameters": {  },  "variables": {  },  "resources": [{  "name": "[concat('matsustorage', copyIndex())]",  "type": "Microsoft.Storage/storageAccounts",  "location": "East US",  "apiVersion": "2015-06-15",  "properties": {"accountType": "Standard_LRS"  },  "copy": {"name": "teststoragecopy","count": 5  }},{  "name": "testvnet",  "type": "Microsoft.Network/virtualNetworks",  "location": "East US",  "apiVersion": "2016-03-30",  "properties": {"addressSpace": {  "addressPrefixes": ["10.0.0.0/16"  ]},"subnets": [  {"name": "testsubnet0","properties": {  "addressPrefix": "10.0.0.0/24"}  }]  }},{  "apiVersion": "2016-03-30",  "type": "Microsoft.Network/publicIPAddresses",  "name": "testip",  "location": "East US",  "properties": {"publicIPAllocationMethod": "Dynamic","dnsSettings": {  "domainNameLabel": "matsuendpoint"}  }},{  "name": "testlb",  "type": "Microsoft.Network/loadBalancers",  "location": "East US",  "apiVersion": "2016-03-30",  "dependsOn": ["Microsoft.Network/publicIPAddresses/testip"  ],  "properties": {"frontendIPConfigurations": [  {"name": "testfrontend","properties": {  "publicIPAddress": {"id": "[resourceId('Microsoft.Network/publicIPAddresses','testip')]"  }}  }],"backendAddressPools": [  {"name": "testbackend"  }],"loadBalancingRules": [  {"Name": "rule01","properties": {  "frontendIPConfiguration": {"id": "[concat(resourceId('Microsoft.Network/loadBalancers', 'testlb'), '/frontendIpConfigurations/testfrontend')]"  },  "backendAddressPool": {"id": "[concat(resourceId('Microsoft.Network/loadBalancers', 'testlb'), '/backendAddressPools/testbackend')]"  },  "probe": {"id": "[concat(resourceId('Microsoft.Network/loadBalancers', 'testlb'), '/probes/testprobe')]"  },  "protocol": "Tcp",  "frontendPort": 80,  "backendPort": 80}  }],"probes": [  {"name": "testprobe","properties": {  "protocol": "Tcp",  "port": 80,  "intervalInSeconds": 15,  "numberOfProbes": 2}  }]  }},{  "name": "testss",  "type": "Microsoft.Compute/virtualMachineScaleSets",  "location": "East US",  "apiVersion": "2016-03-30",  "dependsOn": ["teststoragecopy","Microsoft.Network/virtualNetworks/testvnet","Microsoft.Network/loadBalancers/testlb"  ],  "sku": {"name": "Standard_D1","tier": "Standard","capacity": "5"  },  "properties": {"upgradePolicy": {  "mode": "Manual"},"virtualMachineProfile": {  "storageProfile": {"osDisk": {  "vhdContainers": ["http://matsustorage0.blob.core.windows.net/vhds","http://matsustorage1.blob.core.windows.net/vhds","http://matsustorage2.blob.core.windows.net/vhds","http://matsustorage3.blob.core.windows.net/vhds","http://matsustorage4.blob.core.windows.net/vhds"  ],  "name": "testdisk",  "caching": "ReadOnly",  "createOption": "FromImage"},"imageReference": {  "publisher": "MicrosoftWindowsServer",  "offer": "WindowsServer",  "sku": "2012-R2-Datacenter",  "version": "latest"}  },  "osProfile": {"computerNamePrefix": "mycomputer","adminUsername": "demouser","adminPassword": "P@ssw0rd01"  },  "networkProfile": {"networkInterfaceConfigurations": [  {"name": "testnic0","properties": {  "primary": "true",  "ipConfigurations": [{  "name": "ipconfig0",  "properties": {"subnet": {  "id": "[concat(resourceId('Microsoft.Network/virtualNetworks', 'testvnet'), '/subnets/testsubnet0')]"},"loadBalancerBackendAddressPools": [  {"id": "[concat(resourceId('Microsoft.Network/loadBalancers','testlb'), '/backendAddressPools/testbackend')]"  }]  }}  ]}  }]  }}  }}  ],  "outputs": {  }}

作成された VM Instance (5 台のインスタンス) を確認するには REST API を発行する必要があるので、下図の通り、Azure Resource Explorer で見ていただくと良いでしょう。(Azure Portal 上では、単一の VM Scale Set として表示されてしまい、各 Instance の確認はできません。)
なお、computerName は、例えば上記構成の場合、mycomputer-0 のように、インスタンスごとにハイフンと数字が付与された個別の名前が設定されます。

なお、上記の構成のままでは、各 Virtual Machine に Web Server のセットアップなどがおこなわれていないため、Port 80 に接続してもエラーになりますが、VM Scale Set の構成に、「Azure Resource Manager の template の How-to」で解説した Extension も指定できるので、マシンをセットアップされた状態で展開することも もちろん可能です。(azure-quickstart-templates / 201-vmss-windows-autoscale などを参照してみてください。)

補足 : VM Scale Sets の個別の Virtual Machine インスタンスに (RDP や SSH を使って) アクセスする方法については、「Azure Virtual Machine Scale Sets (VMSS) の構成方法」に記載しましたので参照してください。(2016/03 追記)

Auto-Scaling (自動スケール)

CPU 利用率に応じた Virtual Machine の拡張・縮退 (増加・減少) などの、いわゆる Auto-Scaling を設定するには、上述の VM Scale Set リソースと Auto Scale Setting (autoscaleSettings) リソースを使用します。
例えば、上述の VM Scale Set に CPU 負荷に応じた Auto-Scaling を設定するには、下記の通り Resource Template に設定します。

metricName は Performance Counter の値で、下記の通り、Azure の Diagnostics が収集する内容を Auto Scale Settings で参照するため、VM Scale Set には Diagnostics Extension を必ず構成します。(Linux の場合は LinuxDiagnostics Extension を使用します。Linux の場合の構成例は azure-quickstart-templates / 201-vmss-ubuntu-autoscale を参照してください。なお、下記では xmlCfg の値を省略していますが、ここには「Azure VM (v2) の診断ログの構成 (Windows 編)」で解説している診断構成 (XML) の Base64 エンコード文字列を指定してください。)

なお、今回は Azure Load Balancer (L4 Load Balancer) と共に構成していますが、ここで Application Gateway (L7 Load Balancer) を使用しても構いません。

{  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",  "contentVersion": "1.0.0.0",  "parameters": {  },  "variables": {  },  "resources": [{  "name": "[concat('matsustorage', copyIndex())]",  "type": "Microsoft.Storage/storageAccounts",  "location": "East US",  "apiVersion": "2015-06-15",  "properties": {"accountType": "Standard_LRS"  },  "copy": {"name": "teststoragecopy","count": 5  }},{  "name": "matsudiagstorage",  "type": "Microsoft.Storage/storageAccounts",  "location": "East US",  "apiVersion": "2015-06-15",  "properties": {"accountType": "Standard_LRS"  }},{  "name": "testvnet",  "type": "Microsoft.Network/virtualNetworks",  "location": "East US",  "apiVersion": "2016-03-30",  "properties": {"addressSpace": {  "addressPrefixes": ["10.0.0.0/16"  ]},"subnets": [  {"name": "testsubnet0","properties": {  "addressPrefix": "10.0.0.0/24"}  }]  }},{  "apiVersion": "2016-03-30",  "type": "Microsoft.Network/publicIPAddresses",  "name": "testip",  "location": "East US",  "properties": {"publicIPAllocationMethod": "Dynamic","dnsSettings": {  "domainNameLabel": "matsuendpoint"}  }},{  "name": "testlb",  "type": "Microsoft.Network/loadBalancers",  "location": "East US",  "apiVersion": "2016-03-30",  "dependsOn": ["Microsoft.Network/publicIPAddresses/testip"  ],  "properties": {"frontendIPConfigurations": [  {"name": "testfrontend","properties": {  "publicIPAddress": {"id": "[resourceId('Microsoft.Network/publicIPAddresses','testip')]"  }}  }],"backendAddressPools": [  {"name": "testbackend"  }],"loadBalancingRules": [  {"Name": "rule01","properties": {  "frontendIPConfiguration": {"id": "[concat(resourceId('Microsoft.Network/loadBalancers', 'testlb'), '/frontendIpConfigurations/testfrontend')]"  },  "backendAddressPool": {"id": "[concat(resourceId('Microsoft.Network/loadBalancers', 'testlb'), '/backendAddressPools/testbackend')]"  },  "probe": {"id": "[concat(resourceId('Microsoft.Network/loadBalancers', 'testlb'), '/probes/testprobe')]"  },  "protocol": "Tcp",  "frontendPort": 80,  "backendPort": 80}  }],"probes": [  {"name": "testprobe","properties": {  "protocol": "Tcp",  "port": 80,  "intervalInSeconds": 15,  "numberOfProbes": 2}  }]  }},{  "name": "testss",  "type": "Microsoft.Compute/virtualMachineScaleSets",  "location": "East US",  "apiVersion": "2016-03-30",  "dependsOn": ["teststoragecopy","Microsoft.Network/virtualNetworks/testvnet","Microsoft.Network/loadBalancers/testlb","Microsoft.Storage/storageAccounts/matsudiagstorage"  ],  "sku": {"name": "Standard_D1","tier": "Standard","capacity": "5"  },  "properties": {"upgradePolicy": {  "mode": "Manual"},"virtualMachineProfile": {  "storageProfile": {"osDisk": {  "vhdContainers": ["http://matsustorage0.blob.core.windows.net/vhds","http://matsustorage1.blob.core.windows.net/vhds","http://matsustorage2.blob.core.windows.net/vhds","http://matsustorage3.blob.core.windows.net/vhds","http://matsustorage4.blob.core.windows.net/vhds"  ],  "name": "testdisk",  "caching": "ReadOnly",  "createOption": "FromImage"},"imageReference": {  "publisher": "MicrosoftWindowsServer",  "offer": "WindowsServer",  "sku": "2012-R2-Datacenter",  "version": "latest"}  },  "osProfile": {"computerNamePrefix": "mycomputer","adminUsername": "demouser","adminPassword": "P@ssw0rd01"  },  "networkProfile": {"networkInterfaceConfigurations": [  {"name": "testnic0","properties": {  "primary": "true",  "ipConfigurations": [{  "name": "ipconfig0",  "properties": {"subnet": {  "id": "[concat(resourceId('Microsoft.Network/virtualNetworks', 'testvnet'), '/subnets/testsubnet0')]"},"loadBalancerBackendAddressPools": [  {"id": "[concat(resourceId('Microsoft.Network/loadBalancers','testlb'), '/backendAddressPools/testbackend')]"  }]  }}  ]}  }]  },  "extensionProfile": {"extensions": [  {"name": "Microsoft.Insights.VMDiagnosticsSettings","properties": {  "publisher": "Microsoft.Azure.Diagnostics",  "type": "IaaSDiagnostics",  "typeHandlerVersion": "1.5",  "autoUpgradeMinorVersion": true,  "settings": {"xmlCfg": "PFdhZENmZz4gPERpYWdu...","storageAccount": "matsudiagstorage"  },  "protectedSettings": {"storageAccountName": "matsudiagstorage","storageAccountKey": "[listkeys(resourceId('Microsoft.Storage/storageAccounts', 'matsudiagstorage'), '2015-06-15').key1]","storageAccountEndPoint": "https://core.windows.net"  }}  }]  }}  }},{  "name": "testas",  "type": "Microsoft.Insights/autoscaleSettings",  "location": "East US",  "apiVersion": "2015-04-01",  "dependsOn": ["Microsoft.Compute/virtualMachineScaleSets/testss"  ],  "properties": {"name": "testas","targetResourceUri": "[resourceId('Microsoft.Compute/virtualMachineScaleSets', 'testss')]","enabled": true,"profiles": [  {"name": "testprofile0","capacity": {  "minimum": "1",  "maximum": "5",  "default": "1"},"rules": [  {"metricTrigger": {  "metricName": "\Processor(_Total)\% Processor Time",  "metricNamespace": "",  "metricResourceUri": "[resourceId('Microsoft.Compute/virtualMachineScaleSets', 'testss')]",  "timeGrain": "PT1M",  "statistic": "Average",  "timeWindow": "PT5M",  "timeAggregation": "Average",  "operator": "GreaterThan",  "threshold": 60.0},"scaleAction": {  "direction": "Increase",  "type": "ChangeCount",  "value": "1",  "cooldown": "PT5M"}  },  {"metricTrigger": {  "metricName": "\Processor(_Total)\% Processor Time",  "metricNamespace": "",  "metricResourceUri": "[resourceId('Microsoft.Compute/virtualMachineScaleSets', 'testss')]",  "timeGrain": "PT1M",  "statistic": "Average",  "timeWindow": "PT5M",  "timeAggregation": "Average",  "operator": "LessThan",  "threshold": 50.0},"scaleAction": {  "direction": "Decrease",  "type": "ChangeCount",  "value": "1",  "cooldown": "PT5M"}  }]  }]  }}  ],  "outputs": {  }}

上記の autoscaleSettings リソースの rules (metricTrigger と scaleAction) に注目してください。この設定では、CPU 負荷 (\Processor(_Total)\% Processor Time) をベースとした Metrics で、1 分ごと (PT1M) に確認をおこない、CPU 負荷が 60 % を超えた状態 (threshold 60.0, GreaterThan) が 5 分 (PT5M) 続いた場合に 1 インスタンス Scale-out (Increase) し、逆に、50 % 未満 (LessThan) が 5 分続いた場合には 1 インスタンス Scale-in (Decrease) します。

前述の VM Scale Set 単体のサンプルでは配置と共にすべての Virtual Machine が起動 (running) しますが、今回の Auto Scale Setting (autoscaleSettings) を設定した場合は、初期配置後、時間経過と共に何台か Scale-in されていくはずです。(REST API で確認できます。下図は、実際に Azure Resource Explorer で確認した例で、3 台に縮退されています。)

Scale-out や Scale-in の状況 (History) は、VM Scale Set の Audit Log (監査ログ) で確認できます。

※ 参考情報

azure-quickstart-templates/101-application-gateway-public-ip
https://github.com/Azure/azure-quickstart-templates/tree/master/101-application-gateway-public-ip

azure-quickstart-templates/201-vmss-windows-autoscale
https://github.com/Azure/azure-quickstart-templates/tree/master/201-vmss-windows-autoscale

※ 変更履歴

2016/04  Network, Compute 関連の apiVersion を 2015-06-15 から 2016-03-30 に変更 (overprovisioning などに対応)