Unfortunately, you can't move an Azure VM from a non-availability zone deployment to an availability zone. You also can't switch the zone after deploying the VM into one zone. The script sample in this post, help you address both of these scenarios in an automated manner.

Word of Caution

The script works by deleting the VM and then recreating it within an availability zone. If the script errors out in the middle, then you will end up with no VM. The script exports the VM configurations into a JSON file for this situation as a fallback option. But you need to know how to use that to recreate your VM. You should do below two things to ensure you don't run into issues:

• Try the script first for non-production and non-critical VMs. I recommend creating a dummy/temporary VM and then trying the script.
• Ensure that there is a backup for any production/critical VMs before moving these into Availability Zones to ensure you have something to fail back to.

Script location in GitHub

You can view and download the latest version of the script, directly from GitHub here: Move-AzVMToAvailabilityZone.ps1

Script Working

The script has a custom function called Export-VMConfig. It uses this function to export the VM configurations into a JSON file. This file serves as a backup (only for the configurations) in case anything goes wrong during the script execution. You can use this to recreate the VM.

The script then stops the VM using the Stop-AzVM command. It then creates a snapshot of the OS disk and then, creates an Azure Disk with Zone information from that snapshot. Then the script performs the same thing for all of the data disks. Then it deletes the Azure VM using the Remove-AzVM cmdlet. Then it starts building the configurations for the new VM using the New-AzVMConfig cmdlet.

The script reapplies the tags and diagnostic profile information. Then it sets the OS disk as per the OS of the original VM. Then it adds the data disks. Then it adds NIC(s) and keeps the same NIC as primary. If there is a Public IP from the Basic SKU then the script removes it because it doesn't support zones.

Finally, the script recreates the VM using the New-AzVM cmdlet.

Manual action after the script execution

After the new VM has been validated, I recommend waiting anywhere from 24hrs to a week before cleaning up the residual non-availability zone resources. Once you are ready, delete the older OS and data disks and related snapshots. These should be not attached to any VMs.

I hope that the script helps you in moving your existing infrastructure to avail the redundancy options provided by the Availability Zones in Azure.

Auto Calculate and Apply tags

As we discussed in the last post, whenever you can you should try to auto calculate and apply the tags in an automated fashion. Some examples include:

• This could be based on the name of the resource group if you have a naming convention defined for a resource group (by using substring on the resource group name).
• Another use case could be calculating date time. You can leverage utcNow() and substring() functions to find date time as per your formatting standards.

Let's look at the second use case as that is more generic and applicable to all scenarios. You can use this to apply the below tags:

• BuildDate - date stamp when the resource is first deployed. Right now Azure does not store this metadata and without this tag, it is hard to find this down the line. So it is a good practice to tag each resource at the creation time.
• UpdateDate - date and time stamp whenever the resource is updated.

Policy sample

Let's look at the BuildDate tag and how to apply this automatically for each and every resource.

Note:

• The policy applies the tag in the MM/DD/YYYY format. This ensures that the BuildDate tag everywhere adheres to a standard.
• The policy applies only when the tag does not exist on the resource. Once the tag is there, the policy will not apply to those resources (based on the if conditions specified below within the policy)

Let's jump into the policy sample now. A complete sample can be found at my GitHub repository (link provided at the bottom of this post).

"mode": "All",
"policyRule": {
  "if": {
    "anyOf": [
      {
        "field": "tags['BuildDate']",
        "exists": "false"
      },
      {
        "field": "tags['BuildDate']",
        "notMatch": "##/##/####"
      }
    ]
  },
  "then": {
    "effect": "modify",
    "details": {
      "roleDefinitionIds": [
        "/providers/microsoft.authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c"
      ],
      "operations": [
        {
          "operation": "addOrReplace",
          "field": "tags['BuildDate']",
          "value": "[concat(substring(utcNow(),5,2),'/', substring(utcNow(),8,2),'/',substring(utcNow(),0,4))]"
        }
      ]
    }
  }
}

You can also find the policy for the UpdateDate tag in GitHub with a timestamp along with the date stamp.

Complete Policy Samples on GitHub

You can find the complete policy samples on the GitHub in my policy samples repository here: AzurePolicySamples - Tagging.

Combining strategies

In the previous few posts, we looked at individual tagging strategies. In this post, we will combine those strategies to build a custom tagging strategy for your environment. The combination looks at defining the tags at a higher level and then inheriting these tags at the resources under that level. This means that this combination tagging strategy will employ the following:

1. Applying highest level tags at the Subscription level and then inheriting those at the lower levels
2. Applying the group level tags at the Resource Group level and then inherit those at the individual resources
3. Applying specific tags at the individual resource level
4. Enforcing other tagging related standards
5. Auto calculate and apply tags (as many as you can)

Let's look at these in detail.

1. Planning tags at the subcription level

You want to apply the tags at the subscription level that are generic to the whole subscription. One example could be a tag for the Environment i.e. Dev, Test, Prod, etc. If you deploy different subscriptions for different environments then it is an easy one that you can apply in your environment.

2. Planning tags at the resource group level

Generally, you will want to group your resources by the department and a specific application within that department. Whatever is your grouping choice based on, you can have those tags planned for the resource group level and then inherit these automatically at the resource level. E.g. DepartmentName, ApplicationName, ApplicationOwner, etc. can be applied at the resource group level and then inherited below.

3. Planning tags at the resources level

There will be some information that needs to be tagged at the individual resource level. A couple of examples of this are:

• BusinessCriticality - criticality of the resource based on the business requirements
• UpdateDate - Timestamp when a resource is updated

4. Enforce the tagging standards

If you have any standards you want to enforce for the tagging you can do that directly within the Azure policy as well. E.g. for the ApplicationOwner Tag i.e. the owner of the application related to the resource deployed should be an email id. You can enforce this easily via an Azure policy.

You can also have naming conventions that you can enforce. Or you can have a set of values that you can have the tag value adhere to.

We will look at these in upcoming blogs.

5. Auto calculate and apply tags

Whenever you can you should try to auto calculate and apply the tags in an automated fashion. Some examples include:

• This could be based on the name of the resource group if you have a naming convention defined for a resource group (by using substring on the resource group name).
• Another use case could be calculating date time. You can leverage utcNow() and substring() functions to find date time as per your formatting standards.

We will look at these in detail in upcoming posts.

Complete Policy Samples on GitHub

You can find the complete policy samples on the GitHub in my policy samples repository here: AzurePolicySamples - Tagging.

In the last post, we looked at enforcing the tags at the subscription level and inheriting those at the resource level. In this post, we are looking at policies and various nuances in implementing the third strategy i.e. enforcing tags at the Resource Group level and inheriting at each resource.

Require Tags on a Resource Group

You can require tags on a Resource Group by using the below Azure policy. This is an extension of the in-built policies. This extends the policy to leverage multiple tags in a single policy.

The below policy is enforcing the ApplicationOwner and DepartmentName tags on the Resource Groups. It denies the creation of Resource Groups if any one of the tag is not present.

Note: You can parameterize the tag name and its value as well very easily. You can check in-built policies for reference. Or let me know if this is something you want to see and I can create a parameterized sample.

"policyRule": {
  "if": {
    "allOf": [
      {
        "field": "type",
        "equals": "Microsoft.Resources/subscriptions/resourceGroups"
      },
      {
        "anyOf": [
          {
            "field": "tags['ApplicationOwner']",
            "exists": "false"
          },
          {
            "field": "tags['DepartmentName']",
            "exists": "false"
          }
        ]
      }
    ]
  },
  "then": {
    "effect": "deny"
  }
}

You will get an error similar to below when trying to create a resource group after you have defined and assigned this policy in your subscription.

When you get the above error after trying to create the Resource Group without tags (if you have the policy assigned already to enforce tags on RGs), then click on line #1 in the screenshot. In the pop-up blade that opens up, click on "Raw Error" as shown above as #2. Then scroll down to see the details of the policy that stopped the resource group creation.

Inheriting Tags from Resource Groups

You can easily inherit the tags from a Resource Group down to its underlying resources by using an Azure policy specifically for this purpose. You have three approaches for inheriting the tags from the resource group level:

1. Inherit the tags and their value only if this tag is missing from the resource
2. Inherit and replace the tag if it doesn't match the value of the same tag at the resource group level
3. Inheriting the tags regardless of what is specified at the resource level

Which approach works for you depends on your requirements. The first two approaches are identical to what we discussed in the last post for the subscription level. You can view those at this link and just replace subscription with resource group level functions: Enforcing Tags at the Subscription level and Inheriting on underlying resources.

We have seen approach 3 being the most common and we will discuss that next.

Inheriting the tags regardless of what is specified at the resource level

In the below policy, you check that all of the below conditions should be met before the action can be taken:

• The ApplicationOwner and DepartmentName tags exist at the resource group level
• Both of these tags does not have an empty value at the resource group level

If both of these conditions are true then the "modify" effect is applied. Within this effect, the tags from the resource group level are applied at the resource level.

Note: You can parameterize any values within this sample policy. Also, you can extend this policy to multiple tags (without creating additional policies).

"policyRule": {
  "if": {
    "allOf": [
      {
        "value": "[resourceGroup().tags['ApplicationOwner']]",
        "exists": "true"
      },
      {
        "value": "[resourceGroup().tags['ApplicationOwner']]",
        "notEquals": ""
      },
      {
        "value": "[resourceGroup().tags['DepartmentName']]",
        "exists": "true"
      },
      {
        "value": "[resourceGroup().tags['DepartmentName']]",
        "notEquals": ""
      }
    ]
  },
  "then": {
    "effect": "modify",
    "details": {
      "roleDefinitionIds": [
        "/providers/microsoft.authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c"
      ],
      "operations": [
        {
          "operation": "addOrReplace",
          "field": "tags['ApplicationOwner']",
          "value": "[resourceGroup().tags['ApplicationOwner']]"
        },
        {
          "operation": "addOrReplace",
          "field": "tags['DepartmentName']",
          "value": "[resourceGroup().tags['DepartmentName']]"
        }
      ]
    }
  }
}

Complete Policy Samples on GitHub

You can find the complete policy samples on the GitHub in my policy samples repository here: AzurePolicySamples - Tagging.

In the last post, we looked at enforcing the tags at the resource level. In this post, we are looking at policies and various nuances in implementing the second strategy i.e. enforcing tags at the Subscription level and inheriting at each resource.

Add or Replace Tags on a Subscription

As the subscription will already exist, it doesn't make sense to "require tags" on the subscription. Instead, you will "add or replace a tag" on the subscription level.

You can add or replace a tag on a subscription by using the below Azure policy. This is an extension of the in-built policies. This combines multiple policies into a single one to simplify the management of policies.

The below policy is enforcing the Environment tag on the Subscription. It adds the tag if it is not present. If it is present and the value is not "Dev" then it updates the the tag's value to Dev.

Note: You can parameterize the tag name and its value as well very easily. You can check in-built policies for reference. Or let me know if this is something you want to see and I can create a parameterized sample.

"policyRule": {
  "if": {
    "allOf": [
      {
        "field": "type",
        "equals": "Microsoft.Resources/subscriptions"
      },
      {
        "anyOf": [
          {
            "field": "tags['Environment']",
            "exists": "false"
          },
          {
            "field": "tags['Environment']",
            "notEquals": "Dev"
          }
        ]
      }
    ]
  },
  "then": {
    "effect": "modify",
    "details": {
      "roleDefinitionIds": [
        "/providers/microsoft.authorization/roleDefinitions/4a9ae827-6dc8-4573-8ac7-8239d42aa03f"
      ],
      "operations": [
        {
          "operation": "addOrReplace",
          "field": "tags['Environment']",
          "value": "Dev"
        }
      ]
    }
  }
}

Inheriting Tags from Subscription

You can easily inherit the tags from a subscription down to its underlying resources by using an Azure policy specifically for this purpose. You have two approaches for inheriting the tags from the subscription level:

1. Inherit the tags and their value only if this tag is missing from the resource
2. Inherit and replace the tag if it doesn't match the value of the same tag at the subscription level

Which approach works for you depends on your requirements. For both these scenarios, there are in-built Azure policies provided by Microsoft. Let's look at both of these.

1. Inheriting the tag - only if it is missing

In the below policy, you check that all of the below conditions should be met before the action can be taken:

• The Environment tag does not exist on the resource
• The Environment tag at the subscription level is not equal to an empty value

If both of these conditions are true i.e. the Environment tag is missing and has a valid value at the subscription level, then the "modify" effect is applied. Within this effect, the tag from the subscription level is applied at the resource level.

Note: You can parameterize any values within this sample policy. Also, you can extend this policy to multiple tags (without creating additional policies).

"policyRule": {
  "if": {
    "allOf": [
      {
        "field": "tags['Environment']]",
        "exists": "false"
      },
      {
        "value": "[subscription().tags['Environment']]",
        "notEquals": ""
      }
    ]
  },
  "then": {
    "effect": "modify",
    "details": {
      "roleDefinitionIds": [
        "/providers/microsoft.authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c"
      ],
      "operations": [
        {
          "operation": "addOrReplace",
          "field": "tags['Environment']",
          "value": "[subscription().tags['Environment']]"
        }
      ]
    }
  }
}

2. Inherit and replace the tag if it doesn't match the value of the same tag at the subscription level

In the below policy, you check that all of the below conditions should be met before the action can be taken:

• The Environment tag at the resource level doesn't have the same value as the one at the subscription level
• The Environment tag at the subscription level is not equal to an empty value

If both of these conditions are true i.e. the Environment tag doesn't have the same value as the one at the subscription level and it has a valid value at the subscription level, then the "modify" effect is applied. Within this effect, the tag from the subscription level is applied at the resource level.

Note: You can parameterize any values within this sample policy. Also, you can extend this policy to multiple tags (without creating additional policies).

"policyRule": {
  "if": {
    "allOf": [
      {
        "field": "tags['Environment']]",
        "notEquals": "[subscription().tags['Environment']]"
      },
      {
        "value": "[subscription().tags['Environment']]",
        "notEquals": ""
      }
    ]
  },
  "then": {
    "effect": "modify",
    "details": {
      "roleDefinitionIds": [
        "/providers/microsoft.authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c"
      ],
      "operations": [
        {
          "operation": "addOrReplace",
          "field": "tags['Environment']",
          "value": "[subscription().tags['Environment']]"
        }
      ]
    }
  }
}

Complete Policy Samples on GitHub

You can find the complete policy samples on the GitHub in my policy samples repository here: AzurePolicySamples - Tagging/.

In the last post, we looked at the importance of tagging in Azure and various tagging strategies. You can view that post here: Designing Tagging Strategy for Microsoft Azure - Part 1 - Basics. In this post, we are looking at policies and various nuances in implementing the first strategy i.e. enforcing tags at the resource level.

Requiring Tags on a resource

You can require a tag on a resource by using the below Azure policy. This is an extension of the in-built policy that requires one tag. This extends it to two tags and shows you how you can add even more tags using just one policy.

"policyRule": {
  "if": {
        "anyOf": [
          {
            "field": "tags['ApplicationOwner']",
            "exists": "false"
          },
          {
            "field": "tags['DepartmentName']",
            "exists": "false"
          }
        ]
  },
  "then": {
    "effect": "deny"
  }
}

The above policy requires two tags:

1. ApplicationOwner - Person responsible for the resource
2. DepartmentName - Name of the department to which the resource belongs

Requiring a Tag and also enforcing a format

Let's assume that you require the ApplicationOwner tag. But you also want to ensure that the user trying to deploy it are using an email id. E.g. if the domain is HarvestingClouds.com then the email id will look something like xxxx@harvestingclouds.com, where "xxxx" could be anything. In policy you will match this by using the "" wildcard character. The match should be then with "@harvestingclouds.com". The policy will look something like below:

"policyRule": {
  "if": {
    "not": {
      "allOf": [
        {
          "field": "tags['ApplicationOwner']",
          "exists": "true"
        },
        {
          "field": "tags['ApplicationOwner']",
          "like": "*@harvestingclouds.com"
        }
      ]
    }
  },
  "then": {
    "effect": "deny"
  }
}

Complete Policy Samples on GitHub

You can find the complete policy samples on the GitHub in my policy samples repository here: AzurePolicySamples - Tagging/.

One of the most useful yet most ignored features within Microsoft Azure is Tagging. Tags provide a way to add additional data to your resources. Having a strong tagging strategy from the beginning can provide more insights into your environment. They are especially helpful in understanding how the cost is distributed in your environment. In this post, we are going to look at a few strategies to implement tagging in your environment.

Where and What Tags to consider

Think of Tags as additional metadata that is added to the resources within Azure. Everything is a resource and therefore can have tags. At a bare minimum, any resource that generates billing should have a tag applied.

The first thing you need to define when it comes to tagging is what tags you should consider enforcing/applying in your environment. Here are some that I recommend:

1. ApplicationOwner - Owner of the application to which the resource belongs.
2. ContactDL - Distribution List (DL) that can be contacted by the cloud operations team in case there is any action that needs to be taken related to the resource.
3. DepartmentName - Department to which the resource belongs.
4. ApplicationName - Application name for which the resource has been deployed
5. CostCenter - Cost center that should be billed (within the organization).
6. ResourceBuildDate - Date when the resource was first deployed
7. ResourceUpdateDate - Date when the resource was updated in any way.
8. BusinessCriticality - Business criticality of the application. The highest criticality applications should be deployed in a redundant fashion.

Managing shared resources - There will be many resources (e.g. virtual networks) that will be used by various departments. These should be tied to the cloud operations team. The expense for these resources should either be shared equally by each department or a weighted distribution based on the overall cloud resources consumption.

How to enforce Tags

Once you have defined the tagging strategy, you will need to ensure that the tags are applied for each and every resource that should have tags on them. The easiest way to enforce this is by using Azure Policy. We will be looking at this in the next blog.

Strategy 1 - Applying Tags for each resource

The easiest and most basic strategy is to apply the tags for each and every resource. You simply define Azure policies to enforce the tagging for all resources. Also, you define the policy for each and every tag you want to enforce.

The downside to this policy are:

1. Some resources don't have the ability to apply tags when creating these from the Azure portal. So you will need to apply these resources programmatically via ARM Templates, Bicep configurations, Azure CLI, PowerShell, etc.
2. You will need to manage the tags at the resource level. Even if the basic tags like BusinessCriticality etc. are not specified, the resource creation will fail.

To enhance this strategy you can try to calculate some tags and auto-apply those when each resource is created or updated. E.g. Build Date can be auto-calculated, validated and applied to each resource during its creation time.

Strategy 2 - Applying Tags at the Subscription and inheriting at each resource

The next strategy is to specify the key tags at the subscription level and then apply these automatically at the lower levels. You can use Azure policies to auto-apply this. The biggest advantage of this strategy is that you don't need to apply these tags for each and every resource. You just apply the tags at the subscription level and then auto-apply these to each resource within the subscription when those resources are deployed. An example would be a tag for the environment. If you have a subscription for each environment (i.e. one for dev, another for test/QA, and another one for staging and a separate one for production), then you can have an environment tag defined at the subscription level and auto-applied to each resource deployed within that subscription.

The downside to this strategy is that you can use this only for the most generic resources. E.g. within your subscription, you will have different resource groups for different departments, so you can't have the tag for DepartmentName applied at the subscription level (unless you have a subscription corresponding to one department only).

Strategy 3 - Applying Tags at the Resource Groups and inheriting at each resource

Similar to the above strategy, in this third strategy, you enforce the tags at the resource group level and then automatically apply these to each resource that is deployed within the resource group.

Similar to the above strategy you get the advantage of auto applying tags and not think about those when planning to deploy individual resources. As a best practice, you should still plan for the tags for each resource, but these will be auto inherited if you miss any. E.g. If you are creating a Resource Group for a specific department and a specific application within that department then you can easily define the DepartmentName and ApplicationName at the Resource Group level and then auto inherits those for each resource that is deployed within that resource group.

The only disadvantage is that any resource-specific tags still need to be applied at the resource level. E.g. Business Criticality may vary for each resource even within a resource group.

Strategy 4 - A mix of the above strategies

The best strategy is a mix of all the above strategies as below:

1. The most generic tags should only be deployed at the subscription level and then auto inherited for all resources under that subscription
2. The Resource Group should enforce tags that are specific to that group and then auto inherit for all resources under that resource group
3. The resource-specific tags should be either auto-calculated and applied or enforced for each resource. You should also auto-calculate, validate, and auto-apply as many resources as you can.

E.g. you can have an environment tag at the subscription level, a department and application tag at the resource group level, a build data and update date at the resource level auto calculated and applied, and other tags enforced at the resource level.

In the next posts, we will look at ways to apply these strategies with policy samples.

Note that this Index is updated regularly as more posts are added around this topic.

1. Set Storage Redundancy for Recovery Services Vault
2. Cross Region Restore (CRR) for Azure Virtual Machines using Azure Backup
3. Soft Delete for Recovery Services Vault
4. Bring your own keys (BYOK) for recovery services vaults - 1 Overview
5. Bring your own keys (BYOK) for recovery services vaults - 2 Enable managed identity for your Recovery Services vault
6. Bring your own keys (BYOK) for recovery services vaults - 3 Assign permissions to the vault to access the encryption key in the Azure Key Vault
7. Bring your own keys (BYOK) for recovery services vaults - 4 Enable Soft Delete and purge protection on the Azure Key Vault
8. Bring your own keys (BYOK) for recovery services vaults - 5 Assign the encryption key to the Recovery Services vault
9. Backup Center in Microsoft Azure
10. Understanding the backup policy for SQL Server in Azure VM
11. Understanding the backup policy for SAP HANA in Azure VM (Database)
12. Troubleshooting the backup issues for SQL Server on Azure VM or SAP HANA on Azure VM (Database)
13. Configuring Azure Backup for Azure File Shares
14. Cross Region Restore (CRR) for Azure Virtual Machines is now generally available
15. Managing the backup-related alerts via the Azure Monitor
16. Simplifying the Azure Backup Center - Part 1
17. Simplifying the Azure Backup Center - Part 2
18. Azure Recovery Services Vaults vs Backup Vaults

The resources they protect

The key difference between the two vault types is the type of datasource that can be protected by that vault. Let's take a look at each vault's supported datasource types.

Azure Recovery Services vaults can protect the following types of datasources:

• Azure Virtual machines
• SQL in Azure VM
• Azure Files (Azure Storage)
• SAP HANA in Azure VM
• Azure Backup Server
• Azure Backup Agent
• DPM

Azure Backup vaults can protect the following types of datasources:

• Azure Database for PostgreSQL servers
• Azure Blobs (Azure Storage)
• Azure Disks
• Kubernetes Service
• AVS Virtual machines

Backup Center - a single pane of glass to backup management

Azure Backup Center is the place that unifies the management of the backup of all the datasources across different vaults. We explored this in detail in the below two blog posts. Check these out to learn more about the backup center:

• Simplifying the Azure Backup Center - Part 1
• Simplifying the Azure Backup Center - Part 2

ZRS for Managed Disks

ZRS as an option for the managed disk is the new feature that has just been introduced.

Note that this option is available for:

• Azure Premium SSDs
• Standard SSDs

At the time of writing of this post, this option is only available for the below regions:

• West Europe
• North Europe
• West US 2
• France Central

When you will try to create a new managed disk, while selecting the disk size, you can modify its redundancy option. If you are using one of the region mentioned above, you will view "Zone-redundant storage" as the option for the disk SKU. You will have the option to select either premium SSD or Standard SSD only.

ZRS and GZRS options for Storage Accounts

For the Storage Accounts, this feature is more widely available across various regions. This feature has been available for some time now for the storage accounts.

When creating new storage accounts, you can select ZRS and GZRS for the storage account data redundancy. These stand for:

• ZRS - Zone-redundant storage - data is replicated across different zones.
• GZRS - Geo-zone-redundant storage - this is a mix of GRS and ZRS. In addition to data being replicated across zones, it is also replicated across different geo regions.

Biggest Advantages

There are lots of advantages to leveraging ZRS storage redundancy in your solutions. The big ones include:

• If a virtual machine becomes unavailable in an affected zone, you can continue to work with the disk by mounting it to a virtual machine in a different zone.
• You can also use the ZRS option with shared disks to provide improved availability for clustered or distributed applications E.g. SQL FCI, SAP ASCS/SCS, or GFS2.

References:

• Zone redundant storage (ZRS) for Azure Disk Storage
• Azure Storage redundancy in the primary region

In this post, we will look closely at various sections and actions within it. When you launch Backup Center, as we saw in the previous post, you are presented with the Overview screen. This screen is powerful in itself in presenting you with a unified view of backup of various datasource types across your environment. Other parts of the Backup Center are clubbed into the following three main categories:

1. Management
2. Monitoring & Reporting
3. Policy and Compliance

Let's look at these three areas in detail.

1. Management

The management section lets you manage various aspects of your backup. The first aspect is the backup instances. Here you can view and manage various backup instances filtered by your datasource type. The key thing to note is that the view and various columns that are shown automatically update based on the datasource type selected. You can also set up new backup and /or trigger restore from the existing backup by using the buttons at the top of the page.

Next section "Backup Policies" let's you view and manage backup policies across your environment. You can make updates as required. You can also create new backup policies as well.

The third and the last section in this category is the ability to view Vaults (both backup and recovery services vaults) across your environment. You can navigate to those vaults by clicking on the corresponding row. You can also create a new vault from here.

2. Monitoring and Reporting

The monitoring and reporting section helps you with supporting your backup infrastructure. It has the following three sections:

1. Backup jobs - shows you all the backup-related jobs, across your environment.
2. Alerts - shows you all alerts related to your backup data across different vaults.
3. Metrics - provide monitoring of various metrics on the backup data. The main two metrics available are "Backup health events" and "Restore health events".
4. Backup reports - detailed reporting into the backup data linked to log analytics.

3. Policy and Compliance

The policy and compliance section is all about the governance of the backup data. The main sub-sections in this section are:

1. Backup compliance - it shows you the compliance of various resources related to backup based on the policies and initiatives applied in your environment. It looks for the backup category for the policies and initiatives.
2. Azure policies for backup - All the Azure policies, filtered by the "backup" category. You can also view the initiatives here by updating the drop down based filters.
3. Protectable datasources - These are all the resources that can be protected.

Protectable datasources is very important section as it helps you easily find all the resources for which you haven't configured the protection. Filter by datasource type to view the resources that can be protected for that type. E.g. in the image below it shows you all the VMs that can be protected via the backup services.

That is all there is to it. Explore all these sections one by one and play around with different options available in each. Backup Center is very powerful in unifying the backup experience and providing a single pane of glass to monitor and manage your backups.

References:

• Overview of Backup center
• Support matrix for Backup center

Navigating to the Backup Center

To navigate to the Backup Center, just go to all Services and just search for it. Or better approach is to find it by searching for it at the top search bar. Once launched you get the view shown in the below image. Note at the middle top section that you can filter the data being shown by following categories:

• Subscription (across your tenant that you have access to)
• Resource Group
• Location
• Type (type of the backup data)
• Vault (where it is being protected)

The Datasource types that you can manage directly from the Backup Center include:

• Azure Virtual machines
• SQL in Azure VM
• Azure Files (Azure Storage)
• SAP HANA in Azure VM
• Azure Disks
• Azure Blobs (Azure Storage)
• Azure Datacenter for PostgreSQL servers

Allowed Actions

Let's check out what all you can do with the Backup Center.

Unsupported Scenario

Currently, the only unsupported scenario is updating the vault settings at scale. You have to navigate to each individual vault to update its settings.

In the next post, we will look more closely at various sections and actions within the Backup Center. You can read the second part here: Simplifying the Azure Backup Center - Part 2

References:

• Overview of Backup center
• Support matrix for Backup center

The script I present in this post automates the triggering of this on-demand backup for multiple SAP HANA databases. The script takes the name of VMs and related SAP HANA database information in a csv file as input.

NOTE: As the feature to trigger the SAP HANA database is available only with the CLI cmdlet, we are using the CLI command. Please ensure that you have AZ CLI also installed in your environment by following the instructions here: How to install the Azure CLI.

Script location

The latest version of the script can be found in GitHub here: Trigger-AzSAPHANAOnDemandBackup

The input CSV file sample is also located at the same location. Use this file to provide the VM names and related SAP HANA database information as an input to the script. Update the path to this file in the script input variables. Update this file as per your environment.

Script Working

The script starts by checking if the input config file (with VM names) exists or not. If not, then the script prints the message to check the path and exit. It then imports the CSV file. Then it sets the context to the right recovery services vault and starts iterating through the VMs. Then it sets the item and container variables as below.

$itemName = "SAPHanaDatabase;$instanceName;$databaseName"
$containerName = "VMAppContainer;Compute;$VMResourceGroup;$vmName"

Finally, it triggers the on-demand backup using the below az cli command.

az backup protection backup-now --resource-group $rgOfVault --item-name $itemName --vault-name $vaultName --container-name $containerName --backup-type Full --output table

I hope this script helps you save lots of time during on-demand backup creation. If you have any improvements you would like to see in the script feel free to provide that as a comment here or a pull request on GitHub.

The script I present in this post automates the triggering of this on-demand backup for multiple VMs and also lets you specify a date for the expiry of this backup. The script takes the name of VMs in a csv file as input.

Script location

The latest version of the script can be found in GitHub here: Trigger-AzVMOnDemandBackup

The input CSV file is also located at the same location. Use this file to provide the VM names as an input to the script. Update the path to this file in the script input variables.

Script Working

The script starts by checking if the input config file (with VM names) exists or not. If not, then the script prints the message to check the path and exit. It then imports the CSV file. Then it sets the context to the right recovery services vault and starts iterating through the VMs. Then it fetches the backup container for the current VM using the below command.

$backupcontainer = Get-AzRecoveryServicesBackupContainer `
            -ContainerType "AzureVM" `
            -FriendlyName $vmName

It uses the container to fetch the backup item corresponding to the current VM.

$item = Get-AzRecoveryServicesBackupItem `
            -Container $backupcontainer `
            -WorkloadType "AzureVM"

Finally, it triggers the on-demand backup using the Backup-AzRecoveryServicesBackupItem cmdlet.

Backup-AzRecoveryServicesBackupItem -Item $item -ExpiryDateTimeUTC $dateTillExpiry

I hope this script helps you save lots of time during on-demand backup creation. If you have any improvements you would like to see in the script feel free to provide that as a comment here or a pull request on GitHub.

NOTE: You should enable Network Watcher in your environment for this script sample to work. Also, note that you need to enable the Network watcher only in the region where your resources exist.

Script location

The latest version of the script can be found in GitHub here: Test-NetworkWatcherRouteNextHop.

A sample input CSV file is also located at the same location.

Script Working

For the script to work, make sure that you have filled the input CSV file as per your environment. In every test, there will be a source and a destination for the communication pair. The script primary looks for the below information from the csv file:

1. SourceSubscription - The subscription for the source.
2. SourceRG - The resource group of the source VM
3. SourceVMName - Name of the source VM
4. Source IP - Indicative - Actual IP address of the source VM. Please ignore the "indicative" text
5. DestinationIP-Actual - Actual IP address of the destination VM
6. SourceLocation - Azure region location of the source VM

NOTE: Within the script update the Network Watcher names as per your primary and secondary regions in Azure where "Get-AzNetworkWatcher" cmdlet is used. E.g. to fetch the Network Watcher in the East US 2 region the command used is as below.

$nw = Get-AzNetworkWatcher -Name NetworkWatcher_eastus2 -ResourceGroupName NetworkWatcherRG

The script then fetches the VM using the Get-AzVM cmdlet.

$vm = Get-AzVM -Name $SourceVMName -ResourceGroupName $SourceVMResourceGroupName

Finally, it executes the test by using the Get-AzNetworkWatcherNextHop command.

$nextHop = Get-AzNetworkWatcherNextHop -NetworkWatcher $nw -TargetVirtualMachineId $vm.Id -SourceIPAddress $SourceVMIPAddress -DestinationIPAddress $DestinationIPAddress

To get the results it uses the output of the above command.

$eachVM.NextHopType = $nextHop.NextHopType
$eachVM.IPAddressResult = $nextHop.NextHopIpAddress
$eachVM.RouteTableID = $nextHop.RouteTableId

You can then compare the output with the expected output to check if the test case passed or failed.

If you have any improvements you would like to see in the script feel free to provide that as a comment here or a pull request on GitHub.

You have two key options when it comes to any Azure policy. You can either use it only to audit which resources are not compliant. Or you can enforce the required settings. We ideally want the latter effect, so that if someone changes the settings the settings are auto-corrected.

Finding the required resources within the policy

You first need to find the relevant resources. The two conditions you need to apply within the policy are:

1. You need to filter for all Storage Account resources
2. Then you need to filter for Storage Accounts for which "allowBlobPublicAccess" is not equal to "false" i.e. for which the allow blob public access is enabled.

For the first condition i.e. to filter by Storage account resource you use the below condition within the policy:

{
          "field": "type",
          "equals": "Microsoft.Storage/storageAccounts"
 }

For the second condition, i.e. to filter for Storage Accounts for which "allowBlobPublicAccess" is not equal to "false" you can use the below condition:

{
          "not": {
            "field": "Microsoft.Storage/storageAccounts/allowBlobPublicAccess",
            "equals": "false"
          }
}

Option 1 - Auditing for compliance

If you want to only Audit these storage accounts for which the allow blob public access is enabled, for the effect you can specify to only "audit" as shown below:

  "then": {
    "effect": "audit"
  }

This will only show the non-compliant storage accounts under the compliance report.

Option 2 - Enforcing the setting (Recommended)

To go one step further you want to enforce the setting. You have two ways to do this:

1. Deny the operation
2. Reapply the settings.

To deny the operation simply specify the effect as "deny" as shown below:

  "then": {
    "effect": "deny"
  }

To reapply the settings automatically you can use the below "modify" effect in the policy. You need to specify the guid for the role with which the effect will be applied. And then you specify the "addOrReplace" operation to update the setting as shown below:

"then": {
      "effect": "modify",
      "details": {
        "roleDefinitionIds": [
          "/providers/microsoft.authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c"
        ],
        "operations": [
          {
            "operation": "addOrReplace",
            "field": "Microsoft.Storage/storageAccounts/allowBlobPublicAccess",
            "value": false
          }
        ]
      }
    }

NOTE: You can create Exemptions in the policy if you have a genuine case where the access needs to be opened. E.g. a public static website hosted on the Storage Account.

Complete Policy Sample GitHub Location

To find the complete policy sample can be found in the GitHub at this location: DisableBlobPublicAccess.json

Reference: Prevent anonymous public read access to containers and blobs

Finding the setting - Allow blob public access

To find this setting, navigate to the storage account for which you want to enable this setting. Then navigate to Configuration under Settings. Find the configuration for "Allow Blob public access" and set it to "Disabled" to disable any public blob access.

More ways to secure the storage account

There are various ways to further lockdown and secure your storage account. Let's look at a few of these.

Locking the Networking to only selected networks

One of the most important settings is to update the Storage Account's networking settings to not allow access from "All Networks". You should lock it down to specific networks only or to specified Public IP addresses only.

Other security related settings

Another security-related configuration is to require secure transfer. This means that when the data travels to and from the storage account HTTPS should be used instead of HTTP. In the case of file shares, the encrypted connection should be used as well.

Next Steps

In the next post, we will look at automatically enforcing blocking of the public access to storage accounts across your subscription with Azure policies. We will have this policy apply for any existing and new Storage Accounts. You can view that post here: Block all public access to Azure Storage Accounts - via Azure policy - with complete sample.

Reference: Prevent anonymous public read access to containers and blobs

One thing that is not available is a report of VMs and their related OS and Data disks. You may need this sometime as per your requirements. In this post, I am sharing one such script that pulls a report around all VMs and their all related disks.

Script location

The latest version of the script can be found in GitHub here: Get-AzResourceInfo-VMsAndAllRelatedDisks

A sample output CSV file and a sample excel report (based on the output csv) are also located at the same location.

Script Working

The script selects all subscriptions and iterates over those. Then within each subscription, it fetches all Resource Groups and iterates over those. The script also indicates a way for filtering over Resource Groups (while fetching these) by using wildcards.

For each Resource Group, it fetches all resources and iterates over these. Then it filters out the resources of type virtual machines. Specifically, it uses the filter to match the type to "Microsoft.Compute/virtualMachines".

$resource.Type -eq "Microsoft.Compute/virtualMachines"

Then for every VM, it fetches three different types of information and adds each as a separate row to the output CSV:

1. The VM itself
2. OS disk details
3. Data disk details - one-row entry for each data disk

The script uses a value/column for "attachedToVM" for all types of rows to indicate which VM the entry belongs to. The script also outputs "ResourceType" column that you can further use to filter data.

Now that you have the data, you can create custom reports as per your requirements. One such report is provided in the same location for your easy reference.

If you have any improvements you would like to see in the script feel free to provide that as a comment here or a pull request on GitHub.

Resource Types to restrict

One such example is Route Tables. If someone in your organization with contributor rights creates a Route Table that is applied on a critical subnet then that user can divert all traffic from that subnet.

Public IP Addresses are another big one that you want to restrict. Any public IP in your environment needs to have a valid business justification. These are the connection to the outside world and therefore you need to ensure that if one exists in your environment then that has proper safeguards (like Firewall) etc. in place.

Classic resources should also be limited. You want to conform to the latest standards (ARM-based) and stay away from classic compute and storage resources within Azure.

We recommend the below resource types to consider restricting in your environment as a starting place. You can then extend these and add/remove any types as per your requirements:

• Route Tables
• Public IP addresses
• DNS Zones
• Network Watchers
• Search Services
• Bastion Hosts
• Elastic Pools
• Classic Compute
• Classic Storage

Policy Sample

The below policy sample implements the restrictions and limits the resource types (that we discussed above) from being created at all. If any of the mentioned resource type is created then the policy denies that resource creation operation.

{
    "mode": "All",
    "policyRule": {
      "if": {
        "anyOf": [
          {
            "field": "type",
            "like": "Microsoft.Network/dnszones*"
          },
          {
            "field": "type",
            "like": "Microsoft.Network/networkWatchers*"
          },
          {
            "field": "type",
            "like": "Microsoft.Search/searchServices*"
          },
          {
            "field": "type",
            "like": "Microsoft.Network/publicIPAddresses*"
          },
          {
            "field": "type",
            "like": "Microsoft.Network/bastionHosts*"
          },
          {
            "field": "type",
            "like": "Microsoft.Sql/servers/elasticpools*"
          },
          {
            "field": "type",
            "like": "Microsoft.Network/routeTables*"
          },
          {
            "field": "type",
            "like": "Microsoft.ClassicCompute/*"
          },
          {
            "field": "type",
            "like": "Microsoft.ClassicStorage/*"
          }
        ]
      },
      "then": {
        "effect": "deny"
      }
    },
    "parameters": {}
  }

Additional Considerations - restricting VM SKUs

In addition to restricting the resource types, you should also restrict VM SKUs. A SKU from the high performance computing series like HBv3-series can eat up your budget very fast. So as a good practice you should be restricting the SKUs to only the ones that you allow. You will need to maintain the list and update it as different projects require additional SKUs or want to upgrade to the latest series. But that is easily done and hardly takes any time. The benefits far outweigh this one small downside.

Complete Policy Samples on GitHub

You can find the complete policy samples on the GitHub in my policy samples repository here: AzurePolicySamples - Enforcing Standards.

Sample naming convention

Let's assume that the naming convention for the Resource Groups has the below features/restrictions:

• The name should start with "RG-"
• The name should end with the environment i.e. dev, test, or prod
• The name should also end with the location i.e. US East, US West, etc.

E.g. the name should be something like RG-AccoutingRecordsApp-Dev-USE to denote that the resource group is for the accounting department for some "records app". This resource group is in a dev environment and within the US East location.

Enforcing the naming convention

To enforce the naming convention we will have negative conditions so that later we can apply the deny operation to stop the resource group from even creating. Also note that all of the conditions should be met. If any of the conditions is not met then the resource group creation will fail.

Policy sample:

{
    "mode": "All",
    "policyRule": {
      "if": {
        "allOf": [
          {
            "field": "type",
            "equals": "Microsoft.Resources/subscriptions/resourceGroups"
          },
          {
            "anyOf": [
              {
                "field": "name",
                "notLike": "RG-*"
              },
              {
                "allOf": [
                  {
                    "field": "name",
                    "notLike": "*-DEV-USE"
                  },
                  {
                    "field": "name",
                    "notLike": "*-DEV-USW"
                  },
                  {
                    "field": "name",
                    "notLike": "*-TEST-USE"
                  },
                  {
                    "field": "name",
                    "notLike": "*-TEST-USW"
                  },
                  {
                    "field": "name",
                    "notLike": "*-PROD-USE"
                  },
                  {
                    "field": "name",
                    "notLike": "*-PROD-USW"
                  }
                ]
              }
            ]
          }
        ]
      },
      "then": {
        "effect": "deny"
      }
    },
    "parameters": {}
  }

Complete Policy Samples on GitHub

You can find the complete policy samples on the GitHub in my policy samples repository here: AzurePolicySamples - Enforcing Standards.

Implementing restrictions via Azure Policy

While trying to implement this restriction we need to consider the fact that the resource groups can only be in a geographical location while the resources have a special location called "global". So the restrictions will have:

• The location field should be equal to the resource group's location. You can compare by using the function: resourceGroup().location
• The location field can be equal to "global"

Policy sample:

{
    "mode": "Indexed",
    "policyRule": {
      "if": {
        "allOf": [
          {
            "field": "location",
            "notEquals": "[resourceGroup().location]"
          },
          {
            "field": "location",
            "notEquals": "global"
          }
        ]
      },
      "then": {
        "effect": "deny"
      }
    },
    "parameters": {}
  }

Note that the policy has negative conditions as the effect is denying the resource creation/update operation.

Complete Policy Samples on GitHub

You can find the complete policy samples on the GitHub in my policy samples repository here: AzurePolicySamples - Location Restrictions.

• The primary location within Azure that is closest to you
• Optional - The location that provides the best cost for the Express Route, closest to you and based on your preferred provider
• The secondary location that is usually a paired region for the primary location
• Additional locations based on your extended offices or different geographical branches

Once you have defined the regions where your resources will be deployed, you want to lock this down via Azure policies so that even by accident, no one deploys resources to any other region. This helps with governance and supportability in the longer run. Let's look at how to apply this next. For this post, let's assume that the locations selected are East US and West US i.e. the resources can only be deployed within these two regions.

Location restrictions at the Resource Group level

For the resource group level you want to have the following conditions and then deny the operations if true:

1. The type of the resource is the Resource Group
2. The field for "location" is not in the allowed list of locations.

The "allOf" operator before all these conditions ensure that all the conditions should be met at the same time.

Policy sample:

{
    "mode": "All",
    "policyRule": {
      "if": {
        "allOf": [
          {
            "field": "type",
            "equals": "Microsoft.Resources/subscriptions/resourceGroups"
          },
          {
            "field": "location",
            "notIn": [
              "eastus",
              "westus"
            ]
          }
        ]
      },
      "then": {
        "effect": "deny"
      }
    },
    "parameters": {}
  }

Location restrictions at the Resource level

The one main caveat with some resources is that they are not deployed in a specific geographic region. They have a special region named "global" where such resources are deployed. We will factor this into the list of allowed locations when defining the policy.

Also we don't need to specify the type, which ensures that this policy is applied to all resource types.

Policy sample:

{
    "mode": "Indexed",
    "policyRule": {
      "if": {
        "not": {
          "field": "location",
          "in": [
            "eastus",
            "westus",
            "global"
          ]
        }
      },
      "then": {
        "effect": "deny"
      }
    },
    "parameters": {}
  }

Additional considerations

You also want to sometimes restrict the resources to the location of the resource group where they are deployed. We will be looking at this in our next blog post.

Complete Policy Samples on GitHub

You can find the complete policy samples on the GitHub in my policy samples repository here: AzurePolicySamples - Location Restrictions.

With this update, Azure Key Vault guarantees that valid Key Vault requests will have availability 99.99% of the time.

Let's understand what that change translates to. The permissible downtime or unavailability time has been reduced as follows.

• Daily: From 1m 26s to 8s
• Weekly: From 10m 4s to 1m
• Monthly: From 43m 49s to 4m 22s
• Quarterly: From 2h 11m 29s to 13m 8s
• Yearly: From 8h 45m 56s to 52m 35s

As you can see from the above numbers, even if the SLA doesn't look much bigger change, it actually is a very big difference in the allowed downtime.

Reference: Azure Key Vault SLA raised to 99.99%

Azure Site Recovery now supports cross-continental disaster recovery giving a new meaning to failover. This feature is available for 3 region pairs. These pairs are:

1. Southeast Asia and Australia East
2. Southeast Asia and Australia Southeast
3. West Europe and South Central US

This means that a workload running in Southeast Asia can be replicated to Australia East and failed over across continents in case of a disaster.

Official announcement link: Azure Site Recovery now supports cross-continental disaster recovery for 3 region pairs.

This provides a lot more flexibility in providing whatever names you want to assign to the log analytics workspace and following your naming standards without worrying that someone else across the globe could be using the same name.

Workspace uniqueness updates

Workspace uniqueness is maintained as follow:

• Workspace ID – global uniqueness remained unchanged.
• Workspace resource ID – also needs to be globally unique.
• Workspace name – needs to be unique per resource group.

Cross workspace queries

Cross workspace queries should now reference workspaces by either:

• Qualified name, or
• Workspace ID, or
• Azure Resource ID

Cross workspace queries referenced by resource name (i.e. the workspace name) will fail due to ambiguous names when you have multiple workspaces with that name.

Official update link: Log analytics workspace name uniqueness is now per resource group.

Note that this Index is updated regularly as more posts are added around this topic.

1. What is Azure Bastion
2. Creating an Azure Bastion Host
3. Connecting to Windows VMs using Azure Bastion
4. Connecting to Linux VMs using Azure Bastion
5. Connecting to VM Scale Sets (VMSS) using Azure Bastion
6. Clipboard access and going full screen
7. NSG and Firewall configurations
8. Managing Azure Bastion - Session Management, Monitoring and Alerting

In the previous few posts, we looked at various aspects of the Azure Bastion service. In this post, we will look at various monitoring and alerting capabilities related to Azure Bastion.

Monitoring remote sessions

You can monitor all the remote sessions being facilitated by Azure Bastion. To do this navigate to the Azure Bastion host. Click on the "Sessions" under settings. On the right side, you will be able to view all the active sessions. Click on the Refresh button to update the list with any added or removed session.

Note that in the list you will see many pieces of valuable information. Few important ones are:

1. Target Host Name - Hostname of the VM that has an active session.
2. UserName - User name that was used to connect to the VM.
3. Protocol - RDP or SSH protocol, that is used to connect to the target VMs (by the Bastion host). It is ssh for Linux and RDP for Windows VMs.

Managing remote sessions - Deleting the sessions

If required, you can delete any session from the list of active sessions. To do this simply right-click on the session name in the list and click on the Delete in the pop-up menu.

The person connected to the VM via Azure Bastion will see a prompt similar to the below. Even after a reconnection attempt from this prompt the connection won't be successful. If the end-user wants to connect again then they will have to initiate the connection again from the Azure portal.

Monitoring Azure Bastion

The monitoring capabilities are integrated with Azure Monitor. Navigate to the Monitor service in the Azure portal and then click on the "Metrics" option in the left settings pane. On the right side, set the Scope to the Azure Bastion host, that you want to monitor. Next, select one of the available metrics from the list of Metric dropdown.

The options available for Metric are:

1. Availability - Bastion communication status
2. Saturation - Total memory
3. Saturation - Used CPU
4. Saturation - Used memory
5. Traffic - Session count

The one shown in the screenshot below is for "Traffic - Session count".

Creating Alerts on Azure Bastions

From the Monitor, you can also create Alerts related to Azure Bastion. This experience is also similar to creating an alert for any other service in Azure. To create a new alert, click on the "+ New alert rule" button under the Alert section.

In the "Create alert rule", select the scope and set it to the Azure Bastion service. Then click on the "Add condition" button. This will bring a pop-up blade to "Configure signal logic". This is the condition on which the alert will be triggered. Check all the signals available here.

Additionally, you can select or create an Action Group to send notifications and take actions whenever an alert is triggered.

In the earlier posts, we saw how to set up the Azure Bastion service and how to connect to VMs using this service. In this post, we will see what ports are involved and on which side to allow this communication. That is which ports are involved in the outbound communication between the target VMs and Bastion host and which ports are involved in the inbound communication. You will need this information when dealing with any NSGs or Firewalls.

Key Points

Azure Bastion is a fully managed platform PaaS service from Azure that is hardened internally to provide you with secure RDP/SSH connectivity. You don't need to apply any NSGs on the Azure Bastion subnet. Because Azure Bastion connects to your virtual machines over private IP, you can configure your NSGs to allow RDP/SSH from Azure Bastion only. This removes the hassle of managing NSGs each time you need to securely connect to your virtual machines.

Note: UDR is not supported on an Azure Bastion subnet. You don’t need to force traffic from an Azure Bastion subnet to Azure Firewall because the communication between Azure Bastion and your VMs is private. Therefore unless you have a complex deployment you will need to define the rules only at the NSG levels.

Inbound rules for the Azure Bastion Subnet

Note: These are the rules for the Inbound or Ingress communication for the Azure Bastion host. These are applied to the NSG for the Bastion subnet.

Navigate to the "Inbound security rules" in the NSG and click on the "+ Add" button to add individual rules. As a best practice, leave a range of 10 or higher when providing priority to the rules. Ensure that the order is correct and in the required sequence.

Add the rules for the following:

1. Inbound on 443 from the Internet - Allow HTTPS inbound on port 443 for TCP protocol. Use the service tag of "Internet" for the Source and "Any" for the destination. This rule is required for you to be able to make HTTPS connections to the Azure Bastion host from the Azure portal.
2. Azure Bastion control plane - Allow 443 for TCP protocol. Use the service tag of "GatewayManager" for the Source and "Any" for the destination. This rule is required for the Azure Bastion control plane, i.e. Gateway Manager to be able to talk to Azure Bastion
3. Azure Bastion data plane communication - Allow 443 for TCP protocol. Use the service tag of "AzureLoadBalancer" for the Source and "Any" for the destination. This rule allows communication between the underlying components of Azure Bastion.
4. Azure Load Balancer Health probes - Allow 5701 and 8080 for "Any" protocol. Use the service tag of "VirtualNetwork" for the Source and also "VirtualNetwork" for the destination. This rule enables Azure Load Balancer to detect connectivity.

Outbound rules for the Azure Bastion Subnet

Note: These are the rules for the Outbound or Egress communication for the Azure Bastion host. These are applied to the NSG for the Bastion subnet.

Navigate to the "Outbound security rules" in the NSG and click on the "+ Add" button to add individual rules. As a best practice, leave a range of 10 or higher when providing priority to the rules. Ensure that the order is correct and in the required sequence.

Add the rules for the following:

1. Traffic to target VMs - Allow SSH and RDP outbound on ports 22 and 3389 for Any protocol. Use the "Any" for the Source and the service tag of "VirtualNetwork" for the destination. This rule allows Bastion to be able to connect to target VMs for SSH and RDP connectivity.
2. Azure Bastion data plane communication - Allow outbound on ports 443 for TCP protocol. Use the "Any" for the Source and the service tag of "AzureCloud" for the destination. This rule allows outbound communication for the components of Azure Bastion to talk to each other.
3. Azure Cloud communication - Allow outbound on ports 5701 and 8080 for Any protocol. Use the "VirtualNetwork" for the Source and also the service tag of "VirtualNetwork" for the destination. This rules is required for Azure Bastion to send diagnostics logs, metering logs, and other information to various public endpoints within Azure cloud.
4. Internet communication - Allow outbound on port 80 for Any protocol. Use the "Any" for the Source and the service tag of "Internet" for the destination. This rule is required for session and certificate validation.

Inbound rules for the Target VM's NSG

Note: These are the rules for the Inbound or Ingress communication for the Target VM that you will RDP to via the Bastion host. These are applied to the NSG for the subnet of the Target VM or directly to the network interface card of the VM.

You will need to allow ports 3389 (RDP) and 22 (SSH) inbound on the target VM. This needs to be allowed for the Bastion host to be able to make the RDP and SSH connections respectively to the target VM.

Note

1. You don't need both ports i.e. 3389 (RDP) and 22 (SSH). You should limit it to either one based on the operating system within the VM. For Windows VMs you will select the 3389 port and for the Linux VMs, you will select port number 22. This is shown as number 4 in the below screenshot.
2. You should update the Source IP address range to the address range of the Bastion subnet i.e. "AzureBastionSubnet" subnet only. This is shown as number 5 in the below screenshot.

You can read more about these in the official documentation here: Working with NSG access and Azure Bastion

When working with Azure Bastion you connect to various VMs via the Bastion host. For Windows VMs, you connect via RDP and for Linux VMs you connect via SSH. In both scenarios, you will need to copy and paste the text from the local machine to the remote VM. In Bastion this is managed via the Clipboard manager on the Bastion host.

Before you start make sure that you know how to connect to the VMs and various caveats linked to that as we discussed in the earlier posts.

Clipboard Access

When trying to connect for the first time or trying to connect on a new browser, the Bastion service will request you to allow the Text and Image related clipboard access. I highly recommend allowing this. If you opt to not allow this access then later, you won't be able to share any clipboard with the session.

Note: Only text copy/paste is supported.

When you are in the remote session, launch the Bastion clipboard access tool palette by selecting the two arrows on the left-center of the session.

In the pop-out window, any text you have copied on the local machine will automatically appear in the text box in this window. You can review or modify the text in this window and the clipboard in the remote VM will be updated automatically.

Full Screen

Another very useful option that is available in the pop-out option is to go full screen. Just click on the button as indicated below to enter the full-screen mode. This is recommended when you have to work on the remote VM for a long period of time.

Press the Escape button to exit the Full Screen.

In the previous two posts, we saw how to RDP to a Windows VM and how to SSH to a Linux VM. In this post, we will see how to connect to the instances of a Virtual Machine Scale Set (VMSS). Note that in the case of VMSS, Bastion will automatically decide whether to use RDP or SSH based on the operating system of the instance.

To start, navigate to your Virtual Machine Scale Set (VMSS) and click on the Instances. Select the instance to which you want to connect.

To connect to the VM via Bastion, first, navigate to the Linux VM in Azure. Click on the Connect button. In the pop-out menu, click on the Bastion option.

Next, click on the Use Bastion button on the next screen.

In the "Connect using Bastion" screen, note the Bastion host being used to connect. This is only for information. You may need this information to troubleshoot if you run into any issues.

Next, I recommend leaving the check box for "Open in new window" checked. This ensures that the connection opens in a new tab or window and uses the maximum area. If you uncheck this then the connection will open in a new blade to the right and the view will be limited.

Provide the credentials of the instance that you are trying to connect.

Hit the Connect button to connect to the instance.

A new window/tab will open in the browser with the VM connected.

In the previous blog, we saw how to RDP to a Windows VM using Bastion. In this post, we will look at how to SSH to a Linux VM using the Bastion service.

To connect to the VM via Bastion, first, navigate to the Linux VM in Azure. Click on the Connect button. In the pop-out menu, click on the Bastion option.

Next, click on the Use Bastion button on the next screen.

In the next screen to "Connect using Azure Bastion" make note of the following:

1. At the top you will see the Bastion host being used for the connection. This is for your information only. You do not need to take any action here. You may need this if you run into any issues and have to troubleshoot.
2. I recommend leaving the check box for "Open in new window" checked. This ensures that the connection opens in a new tab or window and uses the maximum area. If you uncheck this then the connection will open in a new blade to the right and the view will be limited.
3. Next provide the Username that will be used to connect via SSH to the Linux VM
4. Authentication type is important. Select the one that you want to leverage. The next entry will update as per the selection here.
5. Provide the additional information for the selection you made in the previous point.
6. The advanced settings are optional and can be skipped. This does not relate to all authentication types.
7. Finally click on the Connect button to connect to the VM

The options for authentication types are as follows:

1. Password - This is to use the normal username and password-based authentication
2. SSH Private Key - This option is used when you want to provide the key phrase manually
3. SSH Private Key from Local File - This is to upload the private key file from the local computer. You can optionally provide a passphrase for this.
4. SSH Private Key from Azure Key Vault - This is to use a private key file that you have previously uploaded to the Azure Key Vault.

Once you click on the Connect button, the SSH session to the VM opens up in a new tab.

Note: Almost all the Caveats from the previous blog for RDP to Windows VM also apply to the SSH to Linux VMs.

In the last post, we deployed the Azure Bastion service. In this post, we will look at how to connect to a Windows VM via RDP using the Bastion service.

Connecting to the Windows VM via RDP

To start an RDP connection to a Windows host, navigate to the virtual machine that you want to connect to. Click on the Connect button. In the pop-out option, select the Bastion option. Click on the "Use Bastion" button on the next screen.

In the next screen to "Connect using Azure Bastion", enter the credentials for the VM to which you are trying to connect. Keep the check box for "Open in new window" checked. If you don't select this checkbox then the RDP session will open in another blade to the right. I prefer to open the RDP session in a new window for maximum screen space.

Your VM will open in a new tab. You will be able to perform many actions that you can do via direct RDP (with exception of a few).

Caveats during the connectivity

There are some caveats when connecting via the Bastion service. Here are a few that I have encountered or was able to generate.

The first one is the pop-up blocked alert. When you are trying to connect using the option to open in the new window, then if you have a pop-up blocker enabled and the Azure portal is not whitelisted then you will see the below alert in the portal when trying to connect. The connection will not succeed.

Solution: click on the small icon for blocked pop-up in the address bar of the browser, as shown below, and select to Allow the pop-ups from the Azure portal.

When trying to connect for the first time or trying to connect on a new browser, the Bastion service will request you to allow the Text and Image related clipboard access. I highly recommend allowing this. If you opt to not allow this access then later, you won't be able to share any clipboard with the session.

If for any reason the connection is interrupted then you will see a dialog saying "Disconnected". You will have the option to Reconnect or exit by clicking on the Close button.

Connection Error may come e.g. when restarting the VM. It will automatically attempt to reconnect in few seconds. You can also force reconnect by clicking on the "Reconnect" button.

As you can see that using the Bastion service to connect to the Windows VMs is a very easy and straightforward process. In the next post, we will look at how to connect to a Linux VM via SSH using the Bastion service.

In the previous post, we looked at what the Azure Bastion service is. We discuss its architecture, how it works, key benefits, pricing, etc. In this post, we will look at how to create the Azure Bastion service in the Azure portal.

As we noted in the previous post, the Azure Bastion host is linked to a particular virtual network. In this post, when we will perform the deployment, you will notice that the deployment will be done related to a particular virtual network that we will need to select. This should be the virtual network into which your virtual machines are also deployed which you want to RDP or SSH into via the Azure Bastion service.

Pre-requisites

Before you start setting up the Azure Bastion host, you need to have a subnet in your Virtual network where you are deploying it.

1. The subnet must be named AzureBastionSubnet.
2. The subnet must be at least /26 or larger.

In order to make a connection, the following roles are required:

1. Reader role on the virtual machine
2. Reader role on the NIC with private IP of the virtual machine
3. Reader role on the Azure Bastion resource

Creating the Azure Bastion host

To create a Bastion host, you can navigate to the Bastions section in the Azure portal. Click on the "+ Add" button to add a new Bastion host.

A new blade for "Create a Bastion" will open up. In this new blade, provide the details for:

• Subscription and the Resource Group where you want to deploy the Bastion service.
• Provide a name for the Bastion host and the region where it should be deployed. Note that the region should be the same as where the Virtual Network and the VMs are located.
• Next, select the target Virtual network. It will automatically select the subnet with the name as AzureBastionSubnet. Note that this subnet should have the size as /27 or larger. If there is no such subnet then you will get an error.
• Next, either create a new public IP address or use an existing public IP for the Bastion host.

In the next screen provide the Tags. Review all the settings and then create the Bastion host.

That's all there is to it. Once the deployment is complete we will be ready to leverage this to connect to the other VMs in the virtual network.

Note that we will not be able to connect to the Bastion host itself. It is a fully managed service and is used under the hood to connect to the VMs via RDP or SSH internally. We will be able to connect to other VMs in the network, over port 443 directly from the Azure portal, directly in the browser.

In the next post, we will look at how to connect to VMs using the Azure Bastion host.

1. Details about Azure Bastion

Azure Bastion is a fully platform-managed PaaS service that provides RDP/SSH over TLS i.e. port 443 to all the VMs in the network. Think of this as a managed Jump Box or Jump Server service provided by Microsoft. The service is deployed via a managed host into your virtual network. This host is like a fully managed Jump box and is kept up to date by Microsoft.

Note that you will need to deploy one host for each of your virtual networks. Using one host, you will be able to connect to all the VMs in the same virtual network. The Bastion host needs a Public IP address. All the rest of the VMs don't need any public IP address.

Here are key points about Azure Bastion:

• Azure Bastion provides an integrated platform alternative to manually deploying and managing jump servers or Jump boxes to shield your virtual machines.
• It is a fully platform-managed PaaS service
• Bastion host is provisioned inside your virtual network
• Secure and seamless RDP/SSH connectivity to your virtual machines directly from the Azure portal over TLS
• Virtual machines do not need a public IP address, agent, or special client software
• One Bastion host is needed per virtual network
• The public IP of the Bastion resource on which RDP/SSH will be accessed (over port 443)
• You don't need an RDP or SSH client. Use the Azure portal to let you get RDP/SSH access to your virtual machine directly in the browser.
• Azure Bastion doesn't move or store customer data out of the region it is deployed in.

2. Shortcomings or Areas of Improvement

The below features are currently either not available or are not supported:

• IPv6 is not supported. Azure Bastion supports IPv4 only.
• Only text copy/paste is supported. Features, such as file copy, are not supported yet.
• It doesn't work with AADJ VM extension-joined machines using Azure AD users.
• Azure Bastion currently supports only en-us-qwerty keyboard layout inside the VM.

3. Architecture - How Azure Bastion works

Bastion Hosts are Jump servers that are deployed with a public IP address. These reside at the perimeter of your network. All the other VMs do not need any public IP address. You will be able to connect to the VMs via this Bastion host. The VMs can be deployed in one or many different subnets in the same virtual network.

You need to have a dedicated subnet with the name as AzureBastionSubnet. This subnet must be at least /27 or larger. This subnet should be deployed before creating the Bastion host.

You connect to the Bastion server directly from within the browser on port 443. The Bastion host in turn connects to the VM at port 3389 for RDP and 22 for SSH.

The subnet for the Azure Bastion host needs to have connectivity to the rest of the subnets. This is available by default. If you have NSGs then Bastion-related communication should be allowed. We will look at these rules in detail in a later post.

Note that UDR is not supported on an Azure Bastion subnet. You don’t need to force traffic from an Azure Bastion subnet to Azure Firewall because the communication between Azure Bastion and your VMs is private.

Microsoft keeps Azure Bastion hardened and always up to date for you to ensure that it can withstand attacks from outside.

4. Benefits of using Azure Bastion

There are various benefits of using Azure Bastion. The main one being able to RDP/SSH to your VMs without exposing them via any public IP address. Below is the list of various benefits of having Azure Bastion in your virtual networks:

• Secure and seamless RDP and SSH access to your virtual machines
• No Public IP exposure on the VM
• Help limit threats such as port scanning and other types of malware targeting your VMs
• Fully managed, autoscaling, and hardened PaaS service
• Uses a modern HTML5-based web client and standard SSL ports. This makes Firewall and other security rules very easy to manage.
• Existing authentication works i.e. Existing credentials and SSH keys will still be used for connecting to the VMs
• Same single pane of glass experience to connect to all the VMs
• Bastion host servers are designed and configured to withstand attacks. The Azure platform protects against zero-day exploits by keeping the Azure Bastion hardened and always up to date for you.

5. Concurrency Considerations

Both RDP and SSH are a usage-based protocol. High usage of sessions will cause the bastion host to support a lower total number of sessions. The numbers below assume normal day-to-day workflows.

These workload types are defined here: Remote Desktop workloads

6. Pricing

There are two components of the pricing related to Azure Bastion:

1. Fixed charge for the service. This is the charge billed hourly for deploying the service. E.g. in an East US location, this charge is around $0.19 per hour.
2. Outbound data transfer charges. This is the charge based on the total outbound data transfer. This is further tiered into various categories based on the total consumption.

The pricing details can be found here: Azure Bastion pricing

You can read more information about Azure Bastion in the official documentation here: Azure Bastion Overview.

This current process will involve around 504 routes corresponding to each IP address space published by Microsoft for Azure Storage service alone. To make matters even more complex, you have a limit on the total number of routes allowed in a subscription. You can have a max of 200 Route Tables and only 400 routes allowed per Route Tables. You can check all the service limits here: Azure subscription and service limits, quotas, and constraints

Solution

Microsoft has published the solution for this problem. You can now leverage the Service Tags when defining User Defined Routes (or UDRs) in a Route Table. I believe this addresses a long-requested feature and makes the task to route any Microsoft service-related communication easy.

This service is currently in Public Preview (at the time of writing this blog post). Also, this service is only available via APIs i.e. via REST, PowerShell, CLI, and can also be used in ARM templates. This feature is not currently available through the Azure Portal.

You can now specify a Service Tag for the address prefix parameter in a user-defined route for your route table. You can choose from tags representing over 60 Microsoft and Azure services to simplify route creation and maintenance.

• You no longer need to manually update routes when services change or add to their list of endpoints. Routes with Service Tags will update automatically to include new changes.
• This also eliminates the need for regularly updating routes based on the IP data in the weekly JSON file downloads that Microsoft provides.
• This also helps reduce the likelihood of running into the routes per route table limit (400) which is common when configuring routing for multiple Microsoft and Azure services. By using Service Tags, you can avoid this, since the tag condenses all ranges for that service into one group.
• For example, Microsoft lists more than 4,500 prefixes that collectively represent the Azure address space. You can now use one route with the AzureCloud Service Tag which will include all of these.

Learn more about this capability here: Public preview: Service Tags for User Defined Routing

So where will we get the IP addresses or URLs for the SUSE server?

Solution

The SUSE team publishes the required IP addresses in the below 2 XML files. These are dynamic links that the SUSE team keeps updating and should be referred if any communication is broken.

The URLs for the IP addresses are:

• https://susepubliccloudinfo.suse.com/v1/microsoft/servers/smt.xml
• https://susepubliccloudinfo.suse.com/v1/microsoft/servers/regionserver.xml

It is recommended to open the IP addresses for the regions where our VMs are deployed. The required communication occurs on Port 443 i.e. HTTPS.

Long term solution

Long term solution is to deploy a SUSE RMT (or Repository Mirroring Tool) server in the environment. For older versions of SLES, you may require SMT instead. It is highly recommended to upgrade and use RMT instead.

"RMT server establishes a proxy system for SUSE Customer Center with repositories and registration targets. This helps you to centrally manage software updates within a firewall on a per-system basis while maintaining your corporate security policies and regulatory compliance."

Here is a guide for configuring the RMT: Repository Mirroring Tool Guide.

General syntax to copy is as shown below:

azcopy copy 'https://storage.cloud.google.com/<bucket-name>/<object-name>' 'https://<storage-account-name>.blob.core.windows.net/<container-name>/<blob-name>'

An example is as below:

azcopy copy 'https://storage.cloud.google.com/mybucket/myobject' 'https://mystorageaccount.blob.core.windows.net/mycontainer/myblob'

This feature is currently in preview at the time of writing of this blog. You can read more about how to practically use AzCopy to copy form the GCP Storage to Microsoft Azure Storage block blob here: Copy data from Google Cloud Storage to Azure Storage by using AzCopy.

Step by step process

Here is how the step by step process looks like:

1. Select an approved base image. This is the Windows or Linux Operating System image that Microsoft has approved to be leveraged as a base for the OS image.
2. Create a VM from the approved base image.
3. Configure the VM. In this step, you install the binaries and perform configurations as per your requirements. You also install all the required updates and patches on the VM and ensure that the VM meets the security standards.
4. Generalize the VM and Capture the image for the VM.
5. Publish the VM from the Shared Image Gallery (SIG) to the Marketplace

The option to Capture the image is from the VM blade click on the Capture button at the top.

During the Capture process ensure to use the option to "Share image to Shared image gallery"

To check the detailed walkthrough check this official documentation: How to create a virtual machine using an approved base

• Azure SQL Database,
• Azure Synapse Analytics, and
• Azure SQL Managed Instance

This feature allows you to grant and deny UNMASK permission at the schema-level, the table-level, and the column-level. This enhancement provides a more granular way to control and limit unauthorized access to SQL assets on Azure and improve data security management.

You can read more about this feature towards the end of the doucmentation here: Dynamic data masking - Permissions section

Using this you can monitor:

1. Azure SQL Databases
2. Azure SQL Managed Instance
3. SQL Server on Azure VM

Now you can monitor things like:

• CPU Utilization percentage
• Memory Utilization percentage
• Data IO percentage
• Number of User Connections
• Active Temp Tables
• Memory Grants Pending
• Processes blocked
• Deadlocks/sec
• Total data file size
• Total log file size
• Total log file used size
• Free space in tempdb
• Memory broker clerk size, etc.

Where are the SQL Insights

You can access the SQL insights by navigating to the Monitor in the Azure portal. And then in the Insights section, you can navigate to the "SQL (preview)" option.

Pricing

There is no separate pricing for the SQL Insights. All costs are incurred by the virtual machines that gather the data, the Log Analytics workspaces that store the data, and any alert rules configured on the data.

You can learn more about the SQL insights from the official documentation here: Monitor your SQL deployments with SQL insights

You can customize notifications and also take action if the alert is triggered by leveraging the Action Groups in Azure. You can check how to create Action Groups in this post: Creating the Action Groups for alerts in Microsoft Azure.

How to configure Alerts based on Forecasts

Navigate to "Cost Management + Billing" and select the "Cost Management" option.

In the Cost Management blade, select the option for "Budgets" and then click on "+ Add".

In the next screen under "Create a budget" select the appropriate values for the following:

• Budget scope
• Budget name
• Reset period - under most circumstances this should be Billing month
• Creation date
• Expiration date
• Provide an amount for the Budget

In the next screen for "Set alerts", under the Alert conditions, select the type as "Forecasted". Provide the percentage of budget. E.g. if you provided the budget as $150 and selected the percentage of budget as 80% then the alert would be triggered at $120.

Provide the Action Group to alert via SMS or emails. You can alert via email by providing individual email ids or distribution list's id for "Alert recipients (email)" entry.

You can read more about this feature at the official blog here: Prevent exceeding Azure budget with forecasted cost alerts

• Understanding the Azure Route Server
• Creating the Azure Route Server

In this post, we will look at configuring the Azure Route Server. The below steps are involved in configuring the Route Server:

1. Set up the peering by adding a Peer. This could be a Network Virtual Appliance or NVA.
2. Establish BGP session from the NVA
3. Optionally you can configure route exchange if you are trying to connect to ExpressRoute gateway or a VPN gateway.

Let's look at these steps in detail.

Set up the peering

Navigate to the Route server by click on this link: Azure Route Server.

Click on the "Peers" and then on the right side click on the "+ Add" button to add the peering. A pop-up window will open on the side to "Add Peer". In here enter the values for the NVA. Specifically, enter a name you want to give to this peering. And enter the ASN or Autonomous Systems Number for your NVA. Also, enter the IP address of the NVA the Route Server will communicate with to establish BGP.

Note: The virtual network where you deployed the Route Server should have connectivity to the IPv4 IP address of the NVA.

Establish BGP session from the NVA

The connection with the NVA will not be established until you finish the configurations on the NVA. These configurations will vary depending on the type of NVA (e.g. Check Point, Palo Alto, to name a few). To complete the connection you will need to provide the NVA team with 2 pieces of information regarding the Route Server. These are:

1. Peer IPs for the Route Server
2. ASN number of the Route Server

You can get this information from the Overview page of the Route Server as shown below.

Configure route exchange - for ExpressRoute gateway or a VPN gateway

If you have an ExpressRoute gateway or VPN gateway or both and you want them to exchange routes with the Route Server, you can enable route exchange. It is simply a toggle setting in the Route server. You can access this from the Configurations section under the settings. Select "Enabled" for the "Branch-to-branch" setting and click on the Save button to save the settings.

You can find all the Route Server related official documentation here: Azure Route Server documentation

In this blog post, we will be setting up an Azure Route Server via the Azure Portal.

Pre-requisite

If you are setting up Azure Route Server for your Network Virtual Appliance or NVA, the two key pre-requisites are:

1. For you to be able to set up an Azure Route Server, you need to have an empty subnet in your virtual network with the name "RouteServerSubnet". This subnet should have an address space with a prefix of at least "/27" or higher.
2. The NVA should allow the BGP protocol to leverage the benefits of having the Azure Route Server.

Finding the Route Server section

To find the section for Route Server navigate to this direct link: Azure Route Server

To create a new Route Server, simply click on the "+ Create new route server" button.

Configurations

In the Basics section, provide the values for Subscription, Resource Group, Instance name of the route server, and Region for the deployment. Next you will need to select the Virtual network in the same region where you want to deploy the route server. The wizard will automatically select the subnet with the name "RouteServerSubnet". As noted earlier, this subnet should have at least a "/27" prefix for the address space.

Next, provide any Tags, and then click on "Review + create" and finally click on the "Create" button to create the route server.

Behind the scene it creates two resources of following types:

• Microsoft.Network/virtualHubs
• Microsoft.Network/virtualHubs/ipConfigurations

If you check the Resource Group that you used for deployment, it will look empty. But when you click on the check box for "Show hidden types" you will be able to view the Virtual Hub that was deployed for the Route Server.

In the next post, we will look at configuring the Route Server. You can view the post here: Configuring the Azure Route Server

This service works with:

1. Network Virtual Appliance or NVA
2. ExpressRoute
3. Azure VPN Gateway

You can enable or disable the route exchange on the Azure Route Server with a simple command. Also, you don't need to manually configure or maintain route tables.

With NVAs, it allows you to exchange routing information directly through Border Gateway Protocol (BGP) routing protocol between any NVA that supports the BGP routing protocol and the Azure Virtual Network.

Pre-requisite

1. For you to be able to set up an Azure Route Server, you need to have an empty subnet in your virtual network with the name "RouteServerSubnet". This subnet should have an address space with a prefix of at least "/27" or higher.
2. The NVA should allow the BGP protocol to leverage the benefits of having the Azure Route Server.

Benefits

Azure Route Server simplifies the configuration, management, and deployment of your NVA in your virtual network.

1. You no longer need to manually update the routing table on your NVA whenever your virtual network addresses are updated.
2. You no longer need to update User-Defined Routes manually whenever your NVA announces new routes or withdraw old ones.
3. You can peer multiple instances of your NVA with Azure Route Server.
4. The interface between NVA and Azure Route Server is based on a common standard protocol i.e. BGP.
5. You can deploy Azure Route Server in any of your new or existing virtual networks.

In the next blog post, we will create an Azure Route Server in the Azure portal. You can view that post here: Creating the Azure Route Server

This also means that you can leverage Action Groups to send notifications to email, SMS, Azure app push notifications, and voice calls. You can also leverage Action Groups to take actions when an alert is triggered.

All you need to do is navigate to the Alerts section in the Monitor and you will be able to monitor and manage the alerts directly from this pane.

Currently, this feature is supported for the following workloads: Azure Databases for PostgreSQL server, Azure Blobs, and Azure Managed Disks. Support for other workloads will be added in the near future. Alerts are currently generated for security-related scenarios and job failure scenarios, with more enhancements planned in the short term.

You can read more about this feature here: Azure Monitor alerts for Azure Backup

To configure this, navigate to the Azure SQL Database and click on the "Auditing" option under the security settings. In the right pane, select the "View server settings".

In the server level auditing settings, toggle the setting for "Enable Auditing of Microsoft support operations" to audit all the activities done by the Microsoft support. You can select the target of the audit logs as either one or many selections from the below:

1. Storage
2. Log Analytics
3. Event Hub

This strengthens the security posture even further for your SQL Databases.

I have created a list of the prominent ones that Microsoft has announced will retire on 29th Feb, 2024 are:

• Update your scripts to use Az PowerShell modules
• Switch to Azure Data Lake Storage Gen2
• Classic Application Insights
• AKS legacy Azure AD integration
• Jenkins plug-ins for Azure
• Network Performance Monitor
• Azure Network Watcher Connection Monitor (classic)
• Azure Batch ‘CloudServiceConfiguration’ pools
• Azure Application Gateway analytics
• Switch Azure Media Services REST API and SDKs to v3
• Classic Azure Migrate
• Update the Azure Cosmos DB Java SDK
• Azure Stack Edge Pro FPGA
• Azure Batch rendering VM images & licensing
• The standard version of Custom Voice
• Azure Cognitive Services Text Analytics v2.x
• Upgrade your Azure AD Connect sync to a newer version
• Azure Batch Transcription and Customization Rest API v2

There are few other services that Microsoft has announced will retire a bit earlier.

At least the first one in the above list can be easily taken care of. You can automatically migrate your scripts at least to the newer Az PowerShell modules by leveraging the migration tool and steps from here: Automatically migrate PowerShell scripts from AzureRM to the Az PowerShell module.

Now, this feature for Virtual Machines is generally available.

Note that:

• Azure zonal pinned VMs that are backed up with RS vault enabled with this feature will now be provided with options to restore in any zone of customer choice.
• Azure Backup supports all managed and unmanaged VMs for Cross Region Restore. Classic VMs remains to be unsupported.
• Microsoft is starting with the replication RPO of up to 12 hours in the secondary region even though the storage SLA of geo-replication is 15 mins.
• The pricing is unchanged and stays the same. This is really good of Microsoft to not increase the pricing of the feature.

While Microsoft has moved the preview of CRR for Azure VMs to GA, CRR for SQL/SAP HANA databases running in Azure VMs will continue to be in preview.

Microsoft has added the support for soft delete. You can enable this while creating the storage account under the "Data protection". You have 4 key options that you can enable. These options are:

1. Turn on point-in-time restore for container
2. Turn on soft delete for blobs
3. Turn on soft delete for containers
4. Turn on soft delete for file shares

These options are shown below.

Accessing the Soft Delete capabilities for a blob

If you have enabled the soft delete for blobs, then you will view a toggle to "Show deleted blobs". Any deleted blobs which was deleted within the configured number of days during the creation of the storage account will show up with the status as "Deleted".

You can then either right-click or click on the three dots at the end of the line for the blob with Status as "Deleted". You will see the option for "View previous versions". Click on this option to see any deleted versions of this blob.

From the deleted versions of the blob, you can select any of the version and then you can either download that blob (by clicking on the Download version button) or restore it (by clicking on the Make current version button).

Alternatively, you can click on the deleted blob's name and a pane will open up. In this pane, you can click on the "Undelete" button. If you have multiple versions then to undelete you will have to select the version first.

After restoring the blob, it will show up with the status as Active.

Note:

1. The soft delete feature retains the data for only a fixed number of days. You define this number based on your recovery strategy during the creation of the storage account.
2. If you delete the Storage Account then you can't restore the blobs.

You can read more about this feature in the official documentation here: Soft delete for blobs

1. Microsoft Network
2. Internet

Selecting Microsoft global network delivers traffic via Microsoft global network closest to the user. Internet route option uses transit ISP network. Egress data transfer price varies based on the routing selection. The Microsoft global network has lower latency and is very fast. It is also very secure and is the default choice. You should select this option for any enterprise customers. Internet option on the other hand provides a cost-optimized alternative.

How to set Routing Preferences for a VM

You can set the Route Preferences for a VM via its public IP address. The public IP address can be associated with resources such as virtual machines, virtual machine scale sets, internet-facing load balancers, etc.

While creating the public IP address, you can provide the preference as shown below.

How to set Routing Preferences for a Storage Account

You can also set the routing preference for Azure storage resources such as blobs, files, web, and Azure DataLake. This is done when creating the Azure Storage Account. The setting is under the Networking part of the creation wizard as shown below.

Note:

1. The routing preference option of a Public IP can’t be changed once created.
2. By default, traffic is routed via the Microsoft global network for all Azure services.

You can read more about Routing preferences here: Routing Preference - Microsoft documentation

Note: This blog post assumes that you already have a recovery service vault. If you don't you can create one easily by navigating to the Recovery service vault section in the Azure portal.

Step by step backup configurations

To backup the file share, navigate to the backup vault and go to the "Protected items" and click on the "Backup items". On the right-hand side, click on the "Azure Storage (Azure Files)".

From the Backup items pane, you can select the "+ Add" button to add new backup items to the vault.

In the Backup Goal, select the "Azure File Share" for the question "What do you want to backup?". Click on the Backup button to start the configurations.

The next step is to select the Storage Account on which the File share exists. Once the storage account is selected, you can select the File share by clicking on the "Add" button, selecting the file share, and then clicking Ok in the popup.

The next step is to configure the backup policy. By default, a policy is created for you. It is highly recommended to customize this policy as per your business requirements. At a bare minimum, you should provide a unique and descriptive name for this policy. Remember that this policy can be reused multiple times for other File share backups as well.

Also, provide a backup schedule and the retention of daily backup points. Optionally, you can configure weekly, monthly, and yearly backup retention values.

Once the backup policy is created, click on the "Enable backup" button to submit the deployment to configure the backup.

Note that when you configure the backup it creates a lock on the Storage Account. You should not delete this lock otherwise the backup will not work as desired.

For the RDP license to work, it needs to validate with the RDP Licensing Server. For this, the Firewall needs to allow communication. To be able to allow this communication, you need to know the following:

1. The IP address of the VM where the RDP license is configured
2. The IP address of the RDP Licensing Server
3. The port numbers and the protocol on which to allow the communication

The port numbers on which the communication occurs are as below:

1. TCP on port number 135. This is the main port where communication occurs.
2. TCP on 49152–65535 i.e. RPC dynamic address range. A dynamic port is assigned from this range for validation-related communication.

Using this information ensure that the Firewall in your environment is configured appropriately to allow communication.

1. First thing you should do for SAP HANA on Azure VM (Database).

Before you go down the path of troubleshooting, make sure that you have re-run the pre-registration script. You will either resolve the issues or will get more understanding of why the issues are coming. You can download the script form this link: Pre-registration Script

Note that you may need to re-discover the databases after running the script again. If you look in the official troubleshooting guide you will notice that most of the issues are resolved by this one thing i.e. re-running the pre-registration script.

2. Unhealthy status

Possible cause # 1:

• The unhealthy status could simply mean that you have just configured the backup on a database and there is no backup for the database present in the Azure recovery services vault. In this scenario, you can either wait for the backup schedule to trigger and take the initial backup. Or you can trigger a manual backup.

Possible cause # 2:

• Another reason could be that the Backup service is not able to take the backups of the SQL databases on the Azure VM. You should navigate to the Monitoring section in the recovery services vault and click on the Backup jobs. Filter the jobs by the name of the database or the Azure VM name. Look for any failed jobs and check the error details. These failed jobs will also contain the details about the reason why the job failed.

In one instance, my jobs were failing because the disk drive for Log was completely full. The resolution was to shut down the VM, expand the disk space via the Azure portal and then start the VM. Then expand the disk from the disk manager from within the VM. We manually triggered the full backup and it was successful. This in turn changed the status from Unhealthy to Healthy.

3. Unreachable or Not reachable status

The most common cause for this issue is that the Azure Backup service is not able to reach the VM. Chances are that the backup-related communication is being blocked somewhere in the network. The backup agent indicates to the VM that it should send the backup to Azure. The communication happens mostly Outbound from the VM to the below services in Azure:

1. Azure Backup
2. Azure Storage
3. Azure Active Directory

If any of the services is not allowed then the status changes to Unreachable.

You may have two or both scenarios in your environment:

1. Your VM has an NSG linked to the VM's network interface card or an NSG linked to the subnet in which the VM exists.
2. Your VM is in an environment where all communication is gated via a Network Virtual Appliance (NVA) or Firewall.

If you have NSGs then skip to the last section to see how to configure the NSGs for the backup service to work. If you have Firewall then continue here on how to troubleshoot.

You should look for outbound communication from the VM to these services in the Firewall if you have any. You will need to validate against the public IP addresses for these public services. But how will you find the IP addresses for these. We will look at that next.

4. How to find the IP addresses for Azure services

Microsoft publishes the public IP addresses for all of its services in a JSON file. This is categorized by Service tags. These service tags correspond to each of the Azure services.

You can download this JSON file by navigating to this link: Azure IP Ranges and Service Tags – Public Cloud

After downloading this JSON, search for "Storage", "Active Directory" and "Backup" services to find the section which defines a list of public IP addresses for that service. Note that there will be multiple IP address ranges for each service. Look for the region-specific ranges where you have deployed the VM and the backup vault.

If you find an IP address from the list being blocked for any of the 3 services mentioned above, then you should follow the next section and work with your Firewall team to open these required rules.

5. Configuration of the Firewall to allow Azure backup

Note this only applies to the backup of SQL Server on Azure VM or SAP HANA on Azure VM (Database). Allow access to the following services in the Firewall for the network infrastructure where the Azure VM exists.

6. NSG configurations to allow Azure backup

Note this only applies to the backup of SQL Server on Azure VM or SAP HANA on Azure VM (Database).

Go to the NSG i.e. Network Security Group of the Azure VM which is either linked via subnet or via the network interface of the VM. Navigate to the Outbound rules and add three rules to allow communication to the below destination Service Tags on all ports:

1. Storage
2. Azure Backup
3. Azure Active Directory

The rules will look like below:

7. Permissions issue or UserErrorSQLNoSysadminMembership

If you are getting the error for "UserErrorSQLNoSysadminMembership" then that means the permissions for SQL Server on Azure VM have not been set appropriately. This usually happens if you built the SQL server on the Azure VM yourself. That is you didn't use any marketplace image for SQL on Azure VM.

You need to set the permissions as defined here: Set VM permissions.

More Information

If you are still facing issues, do refer the below links for official documentation:

• Back up SAP HANA databases in an Azure VM
• Back up a SQL Server database in an Azure VM
• Troubleshoot SQL Server database backup by using Azure Backup
• Troubleshoot backup of SAP HANA databases on Azure

Policy Creation

Navigate to the recovery services key vault and then under the Manage, click on the Backup Policies. To create a new policy click on the "+ Add" button.

For the policy type, select "SAP HANA in Azure VM (Database)". This will open the blade to Create the policy for SAP HANA databases which are deployed in the Azure VMs.

In the blade to create the policy, you need to provide the below information:

1. Policy name - provide a descriptive name
2. Full Backup - this is the schedule and retention of the full backup of the databases
3. Differential Backup - this controls the differential backup of the databases
4. Incremental Backup - this controls the incremental backup of the databases
5. Log Backup - this defines the log backup for the databases

For the full backup in the policy, if you enable it then at minimum you need to define:

1. Frequency - daily or weekly
2. Time and Time zone - when the backup will occur
3. Retention for the daily backup. It can be a minimum of 7 days and up to a maximum of 9999 days.

Optionally you can also configure the monthly, weekly, and yearly backup retention policies.

The log backup policy determines the behavior of the log backups. At minimum you need to define:

1. The frequency of the log backups - it can range from a minimum of 15 minutes to a maximum of once every 24 hours
2. Retention of the log backups - it can range from a minimum of 7 days to a maximum of 14 days

Practical Backup Considerations

Below are some general backup considerations based on my experience in different projects. These are only generic considerations and should be evaluated against your organization's compliance standards and specific requirements.

• The time for backup should be when the systems are not being used by any of the teams. If you work in global teams i.e. working around the clock, then ensure the timing when the system is under the least load.
• Use smaller retention windows in the non-production environments than the production environments
• If the data is not critical then usually Full backup is taken Daily and retained for around 14 days in non-prod environments. Weekly, Monthly, and Yearly retention periods are not required.
• In the prod environment, the Full backup is still taken Daily but the retention period is increased to more than 30 days. Max you can select is 9999 days. Also, based on your SLAs and compliance standards you should enable the weekly, monthly, and yearly backup retention periods as well.
• Differential backup can be skipped in the non-production environments based on the criticality of the data in the lower environments
• Log backup can be reduced to once every 2 or 4 hours with a retention period of around 7 days in the non-prod environments.
• In the prod environments the Log backup can be set to 1 hour or less. The retention should be around 14 days based on the SLAs. Note that 14 days is the max you can go.

Policy Creation

Navigate to the recovery services key vault and then under the Manage, click on the Backup Policies. To create a new policy click on the "+ Add" button.

For the policy type, select "SQL Server in Azure VM". This will open the blade to Create the policy for "SQL Server in Azure VM".

In the blade to create the policy, you need to provide the below information:

1. Policy name - provide a descriptive name
2. Full Backup - this is the schedule and retention of the full backup of the databases on the SQL Server in Azure VM.
3. Differential Backup - this controls the differential backup of the databases on the SQL Server in Azure VM
4. Log Backup - this defines the log backup for the databases
5. SQL Backup Compression - you can enable or disable the backup compression. The default value is disabled.

For the full backup in the policy, if you enable it then at minimum you need to define:

1. Frequency - daily or weekly
2. Time and Time zone - when the backup will occur
3. Retention for the daily backup

Optionally you can also configure the monthly, weekly, and yearly backup retention policies.

The log backup policy determines the behavior of the log backups. At minimum you need to define:

1. The frequency of the log backups - it can range from a minimum of 15 minutes to a maximum of once every 24 hours
2. Retention of the log backups - it can range from minimum of 7 days to a maximum of 35 days

Practical Backup Considerations

Below are some general backup considerations based on my experience in different projects. These are only generic considerations and should be evaluated against your organization's compliance standards and specific requirements.

• The time for backup should be when the systems are not being used by any of the teams. If you work in global teams i.e. working around the clock, then ensure the timing when the system is under the least load.
• Use smaller retention windows in the non-production environments than the production environments
• If the data is not critical then usually Full backup is taken Daily and retained for around 14 days in non-prod environments. Weekly, Monthly, and Yearly retention periods are not required.
• In the prod environment, the Full backup is still taken Daily but the retention period is increased to more than 30 days. Max you can select is 9999 days. Also, based on your SLAs and compliance standards you should enable the weekly, monthly, and yearly backup retention periods as well.
• Differential backup can be skipped in the non-production environments based on the criticality of the data in the lower environments
• Log backup can be reduced to once every 2 or 4 hours with a retention period of 7-14 days in the non-prod environments.
• In the prod environments the Log backup can be set to 1 hour or less. The retention should be 14-35 days based on the SLAs. Note that 35 is the max you can go.

To begin navigate to the Azure SQL database and click on the "Sync to other databases" section. Click on the Tables section as highlighted below.

Click on the ellipse in the tables section. It will open up the blade and select the database to which we want to sync and refresh the schema.

As shown below, select the database and select the tables to sync along with columns selection, and click save.

To trigger the sync process navigate to the Database sync group page and click on sync as shown below. This will ensure that the databases are synced as per the direction you have specified.

To stop the syncing process, click on the Stop button as shown below.

After a successful syncing, you will be able to see the synced tables and related schema in the db_test01 database from the db_test02 database.

To read more about the sync groups, check the official documentation here: Set up SQL Data Sync between databases in Azure SQL Database and SQL Server

To begin, navigate to the existing sync group and click on it to open the new blade as shown below.

Click on the Databases section to add the databases as members of the sync group.

Next, click on “Add an Azure Database”. Provide the following details about the target database to add it as a member for syncing.

Once you click Ok, you will see the databases added under the "Member Database" section as shown below.

You can add more azure SQL databases by clicking on the “Add an Azure Database”. If you want to add the on-premises SQL database into the member database then go with the “Add an On-Premises Database” option.

To start navigate to the Azure SQL database on which you want to create the sync group. In this click on the "Sync to other databases" option. Click on the “New Sync Group” as shown below. It will open the “Create Data Sync Group” blade.

This will open the new blade for "Create Data Sync Group".

Following are the details you need to provide while creating the sync group:-

1. Provide the meaningful name to the sync group name.
2. Sync Metadata Database – Microsoft recommends creating a New database for this property. But for the sake of the demo, I am using the existing database.
3. Automatic Sync – If you select On, then enter a number and select Seconds, Minutes, Hours, or Days in the Sync Frequency section. The first sync begins after the selected interval period elapses from the time the configuration is saved.
4. Conflict Resolution – It has two options: Hub win and Member win. Hub win means when conflicts occur, data in the hub database overwrite conflicting data in the member database. Member win means when conflicts occur, data in the member database overwrite conflicting data in the hub database.
5. Private Link – This option is in preview mode as of now and will be managed by Microsoft. Once you click on Ok, It will create the Sync group as shown below.

You can add more azure SQL databases by clicking on the “Add an Azure Database”. If you want to add the on-premises SQL database into the member database then go with the “Add an On-Premises Database” option.

That's all there is to it. In the next posts, we will delve deeper into sync group and related options.

The concept is around Hub and members databases. All the databases in the member group can sync up with the Hub. The member databases can be on-premises databases or on cloud databases.

Where to access this feature

Navigate to the SQL databases and click on the “Sync to other databases” in the settings as shown below.

In the upcoming posts, we will look into this feature in detail.

With the Backup Center you can manage:

1. Backup instances - all instances across different vaults
2. Backup policies - all policies across different vaults
3. Vaults - all Recovery Services Vaults

You can also monitor:

1. All backup jobs and filter for Failed jobs
2. Fetch backup reports

You can also control the policy compliance for Backup directly from the backup center.

The backup center is currently supported for Azure VM backup, SQL in Azure VM backup, SAP HANA in Azure VM backup, Azure Files backup, Azure Blobs backup, Azure Managed Disks backup, and Azure Database for PostgreSQL Server backup.

You can learn more about the Backup Center here: Overview of Backup center

First, make sure that you have a key in the key vault. Also, the key should be at least RSA 2048. The key should be in the Enabled state in the key vault. If you don't have one then you can generate one using the key vault. In a practical scenario, you should use your own custom keys.

You can generate a custom key in the Key Vault by navigating to the "Keys" under settings. Next, click on the "+ Generate/Import" button. In the next blade to "Create a key" you can provide the values similar to below to generate a key. Click Create button to create a key.

Now that you have a key in the Key vault, we can use this to encrypt the backup vault (i.e. the recovery services vault). Navigate to the recovery services vault and click on Properties. Here, click on the "Update" link under the Encryption Settings.

To use your own key, select the check box for "Use your own key". You can then either provide a URI for the key or select a key from a key vault. In this post, we will select the second option. Next, click on the "Select key from Key Vault" link to open a new blade.

In the new blade for "Select key from Azure Key Vault", select the subscription, Key vault, and the Key that you want to use to encrypt. Click on the Select button and then the Save button to save the settings.

When you click Save the setting is not final. It submits a job that you can monitor in the Backup Jobs section under Monitoring in the recovery services vault. Once this job completes successfully, as shown below, only then the configurations are complete.

Note that once you have enabled this setting you can't revert back to using the platform managed keys. The check box for "Use your own key" in the Encryption settings becomes disabled. You can however update the custom key being used to encrypt everything.

• Soft Delete - This is to prevent any accidental deletion of the key vault and its related objects like Keys, Secrets, and Certificates, etc. You can recover any deleted items up to a period that you define as a retention period (in a number of days). You can still purge the deleted resources and therefore permanently delete the resources.
• Purge Protection - If you have purge protection, then the soft-deleted items can not be purged till a particular time (specified by the retention period for the Soft Delete). This along with the Soft delete setting ensures that the key vault-related resources that are deleted can not be purged before the time period is expired.

Note that both Soft Delete and Purge protection settings are required to use the key vault with the recovery services vault to provide your custom key for encryption.

These settings are in the Properties section of the Key Vault. When you enable the Soft delete option, you also provide the retention period in days for the deleted vaults.

You can set up the Soft delete and purge protection during the creation of the key vault as well.

When the key vault is deleted, which has the soft delete enabled, then:

• You can not create any other vault with the same name as the deleted vault
• You may list all of the key vaults and key vault objects in the soft-deleted state for your subscription as well as access deletion and recovery information about them. Users should be granted appropriate permissions to be able to list the keys of the deleted vaults.
• Only a specifically privileged user may restore a key vault or key vault object by issuing a recover command on the corresponding proxy resource.
• Only a specifically privileged user may forcibly delete a key vault or key vault object by issuing a delete command on the corresponding proxy resource.

Note:

• Unless a key vault or key vault object is recovered, at the end of the retention interval the service performs a purge of the soft-deleted key vault or key vault object and its content. Resource deletion may not be rescheduled.
• Once soft delete has been enabled, it cannot be disabled.

Navigate to your Azure Key Vault that you want to use with the Backup vault. Click on the Access policies under Settings. Click on the "+ Add Access Policy" link.

In the new blade to "Add access policy" select the "Key permissions" by clicking on the drop down and selecting the below permissions:

• Get and List under the Key Management Operations
• Under the Cryptographic Operations select Unwrap Key and Wrap Key

Next, under the Select principal, click on the "None selected" link to bring up the blade to "Select a principal". Here provide the Object ID of the managed identity of the recovery services vault. This was created in the previous step. Once you enter the Id, the name of the Vault along with the id will appear. Click on the name and it will be added to the "Selected items". Click on the Select button at the bottom of the blade. Then click on the Add button to add the access policy.

Note that the policy has not been added yet. To finally add the policy ensure to click on the Save button as shown below. This is a common mistake that I have seen people making and I am guilty of this myself as well. If you navigate away from this screen the policy will not be saved.

To enable the managed identity for the Recovery Services vault, navigate to the vault and select "Identity" under Settings. Toggle the Status to On. Click the save button to save the settings.

You will get a warning explaining that you are setting up the managed identity for the vault and that the vault will be accessible via Active Directory managed identity. Hit Ok to proceed. Once the settings are saved, you will see an Object Id. Take a note of this Id. You will also see a button for "Azure role assignments" where you can manage the permissions for this managed identity.

Note:

• A system assigned managed identity is restricted to one per resource.
• It is tied to the lifecycle of this resource. This means that when the resource is deleted its managed identity will also be deleted.
• You can assign permissions to this identity in RBAC similar to how you assign permissions to individual users
• The managed identity is authenticated with Azure AD, so you don’t have to store any credentials in code.

This provides you with more granular control over the encryption process. This also ensures that only you as a customer control the data encryption and decryption. It is highly recommended for any sensitive data that you are protecting in the cloud.

Please note that:

• You should configure the keys before protecting any items in the vault
• Once you have configured the keys these can not be changed afterward.
• The Recovery Services vault can be encrypted only with keys stored in an Azure Key Vault, located in the same region.
• Also, keys must be RSA 2048 keys only and should be in the enabled state.

To configure your vault to encrypt with customer-managed keys, the below steps must be followed. These steps should be followed in this order only:

1. Enable managed identity for your Recovery Services vault
2. Assign permissions to the vault to access the encryption key in the Azure Key Vault
3. Enable soft-delete and purge protection on the Azure Key Vault
4. Assign the encryption key to the Recovery Services vault

We will look at these steps in detail in the following posts.

Configurations

To configure Soft Delete for a recovery services vault, navigate to its Properties. In here, click on the "Update" link under the Security Settings. This will open a pop-up blade where you can enable or disable the soft delete option.

Note:

• This option is enabled by default for new Recovery Services vaults. This section gives you the option to disable the setting if you require it.
• The additional 14 days of retention for backup data in the "soft delete" state don't incur any cost to you.

Soft delete protection is available for these services:

• Soft delete for Azure virtual machines
• Soft delete for SQL Database in Azure VM
• Soft delete for SAP HANA in Azure VM workloads

Soft Delete Behavior

Once you have the soft delete enabled, go ahead and delete a VM from the Backup items under the protected items (non-critical and demo only). After you have deleted the VM, you will see that it is marked as Deleted, as shown below. The VM will continue to appear in the protected items list for the next 14 days. Click on the VM item for more options.

In the next blade, you can see that there is an option for "Undelete" to recover the data for the VM's backups. There is also a message that says that when the protected item was deleted and for how long can you recover the data. After this period the data will be permanently deleted.

You can learn more in the official documentation here: Soft delete for Azure Backup

In this post, we will look at how to take control of the replicated data.

By default, the data that is replicated to the secondary region is available to restore in the secondary region only if Azure declares a disaster in the primary region. But now you can also control the cross-region restore (CRR) yourself. You can trigger a cross-region restore regardless of whether the primary region is available or not. Azure Backup now supports restoring Azure Virtual Machines as well as disks from a secondary region.

Behind the scene, Azure Backup leverages storage accounts’ read-access geo-redundant storage (RA-GRS) capability to support restores from a secondary region. If you enable cross-region-restore, Microsoft upgrades your backup storage from GRS to read-access geo-redundant storage (RA-GRS). Charges for storage are separate from the cost of Azure Backup.

Note that due to delays in storage replication from primary to secondary, there will be latency in the backed-up data being available for a restore in the secondary region.

Cross-Region Restore (CRR) supports the following data sources:

• Azure VMs
• SQL databases on Azure VMs
• SAP HANA databases on Azure VMs

Configurations

To update the Cross-Region Restore (CRR) settings:

1. Navigate to the recovery services vault and click on the Properties under Settings
2. Then click on the "Update" link under the Backup Configuration
3. In the Backup Configurations blade, make sure that the setting for redundancy is set to "Geo-Redundant". Then select "Enable" for the "Cross Region Restore" setting.

Finally, click on the Save button to save the settings.

Note: Cross Region Restore is currently non-reversible storage property. This means that you won't be able to disable the property after enabling it.

Once you have the option enabled, when you will navigate to the Protected Items -> Backup Items -> Azure Virtual Machine -> select a VM that is protected and has a valid backup. You will view a new option to "Restore to Secondary Region".

To be able to use this option, you don't just need there to be a valid backup but also need that backup to be replicated to the secondary region. If your primary region is "East US" then you need to have at least one copy in the "West US" region. Also, note that you will only be able to restore to a secondary region that is a paired region of the region where you have deployed the resources.

You can find which is the secondary paired region based on the region where you have deployed the resources by referring the table in this link: Business continuity and disaster recovery (BCDR): Azure Paired Regions

You can find which is the secondary paired region based on the region where you have deployed the resources by referring the table in this link: Business continuity and disaster recovery (BCDR): Azure Paired Regions

Where to update configurations

To update the redundancy settings:

1. Navigate to the recovery services vault and click on the Properties under Settings
2. Then click on the "Update" link under the Backup Configuration
3. In the Backup Configurations blade, the setting for redundancy is the first one and is available under the "Storage replication type"

Note: This setting can not be changed if you have any item being protected in the vault. You will only be able to modify if you don't have anything protected.

By default, the data that is replicated to the secondary region is available to restore in the secondary region only if Azure declares a disaster in the primary region. But now you can also control the cross-region restore (CRR) yourself. We will look at that feature in the next blog post that you can access here: Cross Region Restore (CRR) for Azure Virtual Machines using Azure Backup

You take action if the alert is triggered by leveraging the Action Groups in Azure. You can check how to create Action Groups in this post: Creating the Action Groups for alerts in Microsoft Azure.

How to configure the Alerts

Navigate to "Cost Management + Billing" and select the "Cost Management" option.

In the Cost Management blade, select the option for "Budgets" and then click on "+ Add".

In the next screen under "Create a budget" select the appropriate values for the following:

• Budget scope
• Budget name
• Reset period - under most circumstances this should be Billing month
• Creation date
• Expiration date
• Provide an amount for the Budget

In the next screen for "Set alerts", under the Alert conditions, provide the percentage of the budget. E.g. if you provided the budget as $150 and selected the percentage of budget as 80% then the alert would be triggered at $120.

Provide the Action Group to alert via SMS or emails. You can alert via email by providing individual email ids or distribution list's id for "Alert recipients (email)" entry.

That's it! Now the alert is up and running. If you want to test the alert then update the budget and the alert percentage to close the spending and the alert should be triggered in short while.

This is a very helpful feature that lets you take the control of your spending and stay on top of the overall usage.

Action Groups Location

The action groups are located under the Azure Monitor, in the Alerts section. Here, click on the "Manage actions" button to modify and create action groups.

Creating Action Groups

In the Manage actions blade, click on the "+ Add action group" button.

Enter the basics information in the first blade to Create an action group.

In the notifications tab, select the type of notifications. Also add additional information for Email, SMS, Azure app Push Notifications, and Voice, etc.

Under the Actions tab, select if you want to perform any action. The possible values for action are:

1. Automation Runbook
2. Azure Function
3. ITSM
4. Logic App
5. Secure Webhook
6. Webhook

Next, add the Tags and review the final information. Click on the Create button once done.

Now that the Action Group is ready, you can use this in any alerts that you will create. The notifications and actions inside the Action Group will be triggered whenever the alert is triggered.

For further details, you can refer the official documentation here: Create and manage action groups in the Azure portal

Using the Action Groups you can send notifications via the following methods when an action group triggers:

1. Group emails,
2. SMS,
3. Azure app notifications, and
4. Voice calls.
5. You can also send emails to an Azure Resource Manager role using Action Groups.

Optionally, you can also take actions using the following features of Azure, when an action group is triggered:

1. Automation Runbook
2. Azure Function
3. ITSM
4. Logic App
5. Secure Webhook
6. Webhook

Action Groups are located in the Azure Monitor under the "Alerts" section. Navigate to "Manage actions" to create or modify action groups.

In the next blog post, we will look at how to create Action Groups in the Azure Portal. You can check the next blog here: Creating the Action Groups for alerts in Microsoft Azure

If you are well versed with one platform this series is to cross-train you on the competitor platform. The intention is to know the pros and cons of each platform and begin with the basic services. You need to know the competition well before you can advocate any platform.

Disclaimer: I am a Microsoft Azure MVP and a bit biased towards Azure. I make every attempt to view both platforms impartially and list pros and cons as they are without any personal opinions.

Note that this Index is updated regularly as more posts are added around this topic.

1. Virtual Machines vs EC2 instances

   a. Combined

   1. Finding Resources
   2. Differences in IOPS

   b. Azure

   1. Creating Azure Virtual Machines
   2. Monitoring of Azure VMs
   3. Boot Diagnostics of Azure VMs
   4. Azure VM Insights
   5. Backing up Azure VMs
   6. Protecting Azure VMs

   c. AWS

   1. Creating EC2 Instances
   2. Monitoring of EC2 instances
   3. Backing up EC2 instances Part 1
   4. Backing up EC2 instances Part 2
   5. Backing up EC2 instances Part 3

2. Batch Services

   a. Combined

   1. Batch Services Comparision

   b. Azure

   1. Creating Batch account
   2. Creating Batch Pool in an Account
   3. Creating Job in the Batch Pool
   4. Creating Tasks in a Batch Job

   c. AWS

   1. Working with AWS Batch Service

3. Auto Scaling

   a. Combined

   1. Azure VMSS vs AWS Auto Scaling

   b. Azure

   1. Creating Virtual Machine Scale Sets (VMSS) - Part 1
   2. Creating Virtual Machine Scale Sets (VMSS) - Part 2
   3. View Running Instances in VMSS
   4. Updating the Scaling policies of VMSS
   5. Viewing the Run History of VMSS

   c. AWS

   1. All Related Resources for Auto Scaling
   2. Creating Launch Configurations
   3. Creating Launch Templates
   4. Creating Auto Scaling Groups - Part 1
   5. Creating Auto Scaling Groups - Part 2
   6. AWS Auto Scaling service

4. Disks

   a. Combined

   1. Azure Managed Disks vs AWS Elastic Block Store (EBS)

5. Storage

   a. Combined

   1. Microsoft Azure Storage Accounts vs AWS Simple Storage Service (S3)

   b. Azure

   1. Creating Azure Storage Account
   2. Data Transfer Options
   3. Event driven automation of Azure Storage accounts
   4. Securely Accessing Storage Accounts
   5. Configuring Firewall and vNet access on Storage accounts
   6. Geo Replication on Storage accounts

   c. AWS

   1. Creating Simple Storage Service or S3 buckets
   2. Working with S3 bucket and uploading files
   3. Configuring S3 Bucket Properties
   4. Setting Permissions on S3 Bucket
   5. Adding Lifecycle Rules for S3 buckets
   6. Configuring replication on S3 buckets
   7. Setting up Access Points on S3 bucket

6. Networking

   a. Combined

   1. Azure Virtual Networks vs AWS Virtual Private Cloud (VPC)

   b. Azure

   1. Creating Virtual Network
   2. Working with Virtual Networks (vNets) and Subnets
   3. Service Endpoints
   4. Creating Network Security Groups (NSGs)
   5. Creating rules in NSGs and assigning NSGs
   6. Using Service Tags to define rules in NSGs

   c. AWS

   1. Creating Virtual Private Cloud (VPC)
   2. Working with VPCs and Subnets
   3. Route Tables
   4. Network ACLs
   5. Security Groups
   6. Network ACLs vs Security Groups

Solution

Microsoft provides the capability to write NSG rules using Service Tag as a destination. When you leverage a Service tag the system ensures that it accounts for all the IP addresses for that service. Not only that, if any IP address changes or a new IP address is added then the Service Tag will account for that automatically. You can target whole services across different regions or services in a particular region. E.g. You can select Storage as a service or select "Storage.EastUS" to target the Storage service but only in the East US region.

Where is the Option

In the Inbound security rules, this option is available for the Source. In the Outbound security rules, this option is available for the Destination. Once you select the service tag as the option, then you are presented with a drop-down for the Service Tag selection. Internet is the default option for the Service Tag. You can type and search for the particular service tag e.g. Azure Storage, Azure Backup, etc.

You can read more about Service Tags here: Virtual network service tags.

You can download the eBook by clicking on this link: 6 Step Migration Strategy – Systematic Approach to Migrating your Workloads to Microsoft Azure

Below is the index for this series which is updated periodically.

Azure Data Factory related posts

1. Creating a Data Factory
2. Exploring the Azure Data Factory layout
3. Creation of Azure SSIS Integration Runtime (IR) into Azure Data Factory

Azure SQL related posts

1. Finding Azure SQL Databases in the Azure Portal
2. Creating Azure SQL Databases
3. Connecting to Azure SQL DB using SSMS
4. Enabling Geo-Replication for Disaster Recovery
5. Data Masking of sensitive information
6. Exploring the new Query Editor in the Azure Portal
7. Advanced Data Security
8. Query Performance Insights
9. Automatic Tuning and Performance recommendation
10. Creating Elastic Pool in Azure SQL Server
11. Azure Active Directory Admin - Manage Azure SQL via AD Users
12. Managing Backups for Azure SQL Database
13. Restoring Azure SQL Database from the backup
14. Setting up Private Endpoints on an Azure SQL Server

A Private Endpoint is a fundamental block for a private link in Azure. It enables Azure resources, like Virtual Machines (VMs), to communicate privately with linked resources. Under the hood, it creates a network interface card (NIC) on the Azure SQL Server and attaches that to your Virtual network. That way you get a private IP address assigned to your Azure SQL Server. Now from any other resource on this network, you can securely access the Azure SQL Server without the communication leaving the network at all.

Microsoft also goes one step further by creating a Private DNS integration. It leverages a private DNS zone to provide a DNS name for your Azure SQL Database Server which is mapped to the Private IP address (instead of the usual Public).

In this post, you will be setting up the private endpoints to the SQL server through Azure Portal. Then, you can securely access the Azure SQL Database Server from the VM.

Create a private endpoint

In this section, you will create a private endpoint to it.

On the upper-left side of the screen in the Azure portal, select Create a resource > Private Link Center (Preview).

In Private Link Center - Overview, on the option select Private Endpoints and click Add.

Enter or select this information:

• Subscription - Select your subscription.
• Resource group - Select myResourceGroup. You created this in the previous section.

Under instance details provide:

• Name - Enter myPrivateEndpoint. If this name is taken, create a unique name.
• Region - Select the geo region where you want to deploy the underlying resources

In the next screen for "Create a private endpoint - Resource", enter or select this information:

• Connection method - Select the radio button for "Connect to an Azure resource in my directory"
• Subscription - Select your subscription.
• Resource type - Select Microsoft.Sql/servers.
• Resource - Select your server
• Target sub-resource - Select sqlServer

Click next once done.

In Create a private endpoint (Preview) - Configuration, enter or select this information:

Under the networking section, provide:

• Virtual network - Select MyVirtualNetwork.
• Subnet - Select mySubnet.

Under the Private DNS integration, provide:

• Integrate with private DNS zone - Select Yes.
• Private DNS Zone - Select (New)privatelink.database.windows.net

Select Review + create. You're taken to the Review + create page where Azure validates your configuration. When you see the "Validation passed" message, select Create.

Once the private endpoint is created your Azure SQL Server is ready to be connected via it's new Private IP address. This address will be in the Virtual network you connected it to. There will also be a network interface card (NIC) resource that actually links the Azure SQL Server to the Virtual Network.

Enabling Network Watcher

First of all, you need to enable network watcher. You should do so for every region where you have a virtual network deployed or will deploy in the near future. To do so simply navigate to the Network Watcher service (by searching for it). In the Overview screen, expand the regions area as shown below. Right-click on the region for which you want to enable this service and select "Enable network watcher".

The first time you do this, it will create a new resource group called "NetworkWatcherRG". If you check this resource group after enabling network watcher for any region, it will look empty, but it actually contains hidden resources of type "microsoft.network/networkwatchers".

Once the network watcher is enabled, you are ready for consuming various mini tools (or different sections) that give you troubleshooting features.

IP Flow Verify

This is to check the IP Flow. If you want to check if the traffic is able to flow or is being blocked then this is the section you want to check. Select the VM and its network interface for which you want to check the IP Flow. The Local IP address will be auto-populated. Fill in the details for the Remote IP address and Remote port and click on the Check button. It will simulate the traffic and will let you know the results.

Next Hop

If you have various Route tables and you want to verify that the traffic follows a particular route or not then "Next Hop" is the section you want to check. You can check if traffic from a source to destination will go via network virtual appliance or not based on a route defined in a route table or not.

Simply select the source and destination. The Source IP address will populate based on the network interface selected. And the destination IP address will be input by you. Click on the "Next hop" button once you are ready. The output will show you what the next hop will be and the route along with the route table that is directing that traffic to that next hop.

Effective Security Rules

You may have various Network Security Groups (NSGs) applied to a VM i.e. one NSG applied directly on its network interface and another one applied at the subnet level. If you want to check what is the result set of these NSGs and which NSGs are effectively applied on your VM, then this is the section that you can use. Simply select your Virtual Machine (VM) and the effective security rules will be displayed in the bottom section.

For more information please check this link: Azure Network Watcher

To start first navigate to the Log Analytics workspaces. Click on the workspace which is the target for NSG Flow Logs in your Network Security Groups (NSGs). Within this workspace, click on the Logs section. If you are opening this for the first time, you will see a "Getting Started" button. Click on that and go through the tutorial if you want.

Once the Logs section is opened up (as shown below), you can type the queries we will discuss next in the main area (marked number 3 below). The results will appear in the bottom section. You can use the time filters (also shown below as number 2) to filter the data to a specific date and time range.

All the logs from NSG Flow Logs are sent to the "AzureNetworkAnalytics_CL" table in the Log Analytics. You can start querying this table for the data. Below are various queries that have helped me a lot in troubleshooting NSGs.

NOTE: In the below examples, "10.20." uniquely identify my virtual network. You will need to tweak this value as per the IP address space of your virtual network. You can even specify multiple address spaces for both sources and destinations.

Checking all Denied Traffic

AzureNetworkAnalytics_CL| extend NSGRuleAction=split(NSGRules_s,'|',3)[0]| extend NSGRuleName=tostring(split(NSGRules_s,'|',1)[0])| where NSGRuleAction == "D" | summarize count() by VM_s,VMIP_s,SrcIP_s,DestIP_s,DestPort_d

To extend the above query, we can also check the denied Traffic with Time Generated, NSG Name and Rule Name, Subnet Names and more info:

AzureNetworkAnalytics_CL| extend NSGRuleAction=split(NSGRules_s,'|',3)[0]| extend NSGRuleName=tostring(split(NSGRules_s,'|',1)[0])| extend NSGName=tostring(split(NSGList_s,'/',2)[0])| where NSGRuleAction == "D" | summarize count() by SourceIP=SrcIP_s, DestinationIP=DestIP_s, DestinationPort=DestPort_d, TimeGenerated, NSGName, NSGRuleName, SourceSubnet=Subnet1_s, DestinationSubnet=Subnet2_s

Intra vNet allowed traffic for a specific rule

AzureNetworkAnalytics_CL| extend NSGRuleAction=split(NSGRules_s,'|',3)[0]| extend NSGRuleName=tostring(split(NSGRules_s,'|',1)[0])
| extend NSGName=tostring(split(NSGList_s,'/',2)[0])
| where NSGRuleAction == "A"
| where NSGRuleName == "privateipinbound-allow" 
| where SrcIP_s contains "10.20."
| where DestIP_s contains "10.24."
| summarize count() by SourceIP=SrcIP_s, DestinationIP=DestIP_s, DestinationPort=DestPort_d, TimeGenerated, NSGName, NSGRuleName, SourceSubnet=Subnet1_s,     DestinationSubnet=Subnet2_s, L4Protocol_s

Query to review specific rule for specific IP and only Allowed traffic

AzureNetworkAnalytics_CL| extend NSGRuleAction=split(NSGRules_s,'|',3)[0]| extend NSGRuleName=tostring(split(NSGRules_s,'|',1)[0])| where NSGRuleAction == "A"
| where NSGRuleName == "temp-allowall-inbound" 
| where SrcIP_s contains "10.20." 

Intra vNet allowed traffic for a specific rule

AzureNetworkAnalytics_CL| extend NSGRuleAction=split(NSGRules_s,'|',3)[0]| extend NSGRuleName=tostring(split(NSGRules_s,'|',1)[0])
| extend NSGName=tostring(split(NSGList_s,'/',2)[0])
| where NSGRuleAction == "A"
| where NSGRuleName == "privateipinbound-allow" 
| where SrcIP_s contains "10.20."
| where DestIP_s contains "10.20."
| summarize count() by SourceIP=SrcIP_s, DestinationIP=DestIP_s, DestinationPort=DestPort_d, TimeGenerated, NSGName, NSGRuleName, SourceSubnet=Subnet1_s, DestinationSubnet=Subnet2_s, L4Protocol_s

Intra vNet allowed traffic for a specific NSG

AzureNetworkAnalytics_CL| extend NSGRuleAction=split(NSGRules_s,'|',3)[0]| extend NSGRuleName=tostring(split(NSGRules_s,'|',1)[0])
| extend NSGName=tostring(split(NSGList_s,'/',2)[0])
| where NSGRuleAction == "A"
| where NSGName contains "systemcenter"
| where SrcIP_s contains "10.20."
| where DestIP_s contains "10.20."
| summarize count() by DestinationPort=DestPort_d, NSGName, NSGRuleName, SourceSubnet=Subnet1_s, DestinationSubnet=Subnet2_s, L4Protocol_s, TimeGenerated

Intra vNet allowed traffic for a specific NSG for Inbound Traffic Only

AzureNetworkAnalytics_CL| extend NSGRuleAction=split(NSGRules_s,'|',3)[0]| extend NSGRuleName=tostring(split(NSGRules_s,'|',1)[0])
| extend NSGName=tostring(split(NSGList_s,'/',2)[0])
| where NSGRuleAction == "A"
| where NSGName contains "systemcenter"
| where SrcIP_s contains "10.20."
| where DestIP_s contains "10.20."
| where FlowDirection_s == "I"
| summarize count() by DestinationPort=DestPort_d, NSGName, NSGRuleName, SourceSubnet=Subnet1_s, DestinationSubnet=Subnet2_s, L4Protocol_s, TimeGenerated

Please note that the above queries are only to help you kick start the troubleshooting. These should be used as a reference. Note how the properties are expanded and details extracted (using split and extend functions). If you want me to explain any of these queries in detail, please mention so in the comments below and I can provide details.

Note that the Network Watcher is a pre-requisite for this. It will be auto-enabled for the region of the NSG when the Flow Logs is set up.

To start, navigate to the Network Security Groups in the Azure Portal. Select the NSG for which you want to enable the Flow Logs. Scroll all the way down in the settings and select the "NSG Flow Logs" setting. Click on the Flow Logs in the middle area to open up the settings for it.

Flow logs status should be turned On to be able to log all the flow logs. Version 2 has more information for throughput etc. Click on the Storage option to configure the storage account to be used to export the flow logs.

Scroll down further to check the Log Analytics workspace settings. Turn on the traffic analysis status. For better and detailed logging set the "Traffic Analysis processing interval" to "Every 10 mins" instead of every 1 hour. Finally, select the Log Analytics workspace where you want the logs to be sent. Try to be uniform and select the same workspace for all NSG flow logs.

Now the flow logs will automatically capture and you are ready to start troubleshooting the traffic. We will check how to do this and common queries that you can reuse in the next few blog posts.

Based on the replication strategy selected on your storage accounts, you can leverage the geo-replication option to restore your storage accounts to a secondary location. Azure Storage replication copies your data so that it is protected from transient hardware failures, network or power outages, and natural disasters. If an outage renders the primary endpoint unavailable, then you can initiate a failover to the secondary endpoint to rapidly restore write access to your data.

You can access this option by navigating to your storage account and clicking on the "Geo-replication" option.

This option shows you the replication you selected for your storage account while creating it. By clicking on the Storage endpoints option, you can see what the endpoints will look like if you failover to the secondary region. You copy and can keep these handy to update your application using these endpoints.

Scroll down to view the primary and secondary regions and the button to prepare for the failover.

When you click on this button, a new popup will open and will give you the option to failover. The secondary region will become primary when you do.

This option is very useful to perform any disaster recovery if there are any issues in your primary region.

For more information click here: Disaster recovery and account failover

E.g. you can specify a complete Virtual Network or Microsoft Azure Backup as a source for Inbound rules very easily by simply leveraging Service tags. Even if the address space of that virtual network changes in the future, you will not need to alter its related Network Security Group (NSG) rules.

Simply navigate to any NSG and then go to Inbound rules (or Outbound rules). Edit one of the existing rules or create a new one. For Souce in case of inbound rules (and destination in case of outbound rules), select "Service Tag" from the drop-down. From the drop-down for the "Source service tag" select the service tag for the service, for which you need the rule.

Note from the arrow in the above screenshot that the vertical scrollbar is huge. Since this feature was launched, so many services have been configured as Service Tags and have made administrators' life easier in terms of configurations required.

For more information please check this link: Virtual network service tags

An Azure-SSIS IR provisions:

• Running packages deployed into the SSIS catalog (SSISDB) hosted by an Azure SQL Database server or a managed instance (Project Deployment Model).
• Running packages deployed into file systems, file shares, or Azure Files (Package Deployment Model).

You can create the self-hosted Azure-SSIS IR with your desired configuration or you can use the run time integration used by the Azure.

Let’s create the Azure-SSIS IR as per below steps:

1.) Go to data factory created in Azure Portal and Select the Author & Monitor tile to open its Let's get started a page on a separate tab. There, you can continue to create your Azure-SSIS IR.

2.) After clicking on Configure SSIS Integration, it will open the integration runtime setup window. It requires as following parameters:

1. Name – Provide the appropriate Name to your integration runtime
2. Description – it’s not a mandatory parameter to provide
3. Type is the default setting. You can’t change this one.
4. Set the location
5. Node size – Select the node size for your integration runtime cluster. Choose the higher number if you want to run packages that require more memory/computing power.
6. Node Number – Choose the number from the list. Higher number if you want to run the many packages in parallel.
7. Edition/license – Select the SQL server edition of your integration runtime. Select the enterprise if you want to choose the advanced features of integration runtime.
8. Save money - select the Azure Hybrid Benefit option for your integration runtime: Yes or No. Select Yes if you want to bring your SQL Server license with Software Assurance to benefit from cost savings with hybrid use.

Next is the Integration runtime setup general settings.

Next up we have the Integration runtime setup SQL settings.

Next up we have the Integration runtime setup Advanced settings.

Finally, we have a summary for the IR setup.

<img src="/images/15851225805e7b0d1475046.png" alt="IR Setup 4" - Summary>

As you can see below in the screenshot, the self-hosted Azure-SSIS IR has been created.

Alternatively, to create the Azure-SSIS IR on Azure Data Factory UI portal go to Edit tab -> Connections -> Integration runtimes and click New.

In the Integration Runtime Setup panel, select the Lift-and-shift existing SSIS packages to execute in Azure tile, and then select Next and proceed to create IR setup from here, similar to what we did earlier.

Now once we have the Integration Runtime we can start importing and running SSIS packages.

To manage the data factory pipelines, go to All resources -> Data factories. You will see all the data factories have been added or created. Select the data factory you want to manage.

Once you select the data factory, it will open the new window (known as a blade in Azure) where you can manage your Data Factory.

Let’s discuss a few sections of the Data Factory page.

Section 1: General properties

1. Overview: The first page when you land on this window is the overview page.
2. Activity log: Platform logs provide detailed diagnostic and auditing information for Azure resources and the Azure platform they depend on. They are automatically generated although you need to configure certain platform logs to be forwarded to one or more destinations to be retained.
3. Access Control (IAM): Review the level of access a user, group, service principal, or managed identity has to this resource.
4. Tags: This is used to provide some keywords to your resource.
5. Diagnostic and solve problems: This provides you the list of troubleshooting links for the known issues a user can come across.

Section 2 Getting Started

Quick Start: This provide the basic tutorial/videos link on how to work with the Data factory. They have been provided by Microsoft and you can see the documentation about the subject. It is really helpful as it gives a quick understanding of the topic.

Section 3 Monitoring

1. Alerts: Configure alert rules and attend to fired alerts to efficiently monitor your Azure resources.
2. Metrics: Configure alert rules and attend to fired alerts to efficiently monitor yourAzure resources. For more details, please refer to the Microsoft official documentation for Metrics at https://docs.microsoft.com/en-us/azure/azure-monitor/platform/metrics-getting-started
3. Diagnostic settings: They are used to collect platform logs and metrics in Azure. To get more information on how to create the diagnostic settings using Azure Portal/CLI/Powershell, please refer to the official Microsoft documentation at https://docs.microsoft.com/en-us/azure/azure-monitor/platform/diagnostic-settings

Section 4 Author & Monitor

Author and Monitor link is the most important link which takes you to the authoring console. This console opens up the new window. This is where you can create Azure-SSIS IR, pipelines, linked services, data sets, and data flow, etc. I will be explaining this more in the upcoming posts.

Section 5 Monitoring

This section is the summarization of the activities happening in the Azure Data Factory. Pipelines status whether it was successful or failed. TriggerRun information that when the pipelines ran. You can also check the CPU and Memory utilization of the IR. You can view or set up all these monitoring status on the Metrics mentioned in section 3.

To start, navigate to your Azure Function and click on the "Platform Features" tab. Here you will see the option for "API Management" as shown below.

Once you click on API Management, it takes you to a new blade. Here you can simply link your Function app to an existing API Management or even create a new one and then link the app.

Now that the app is linked, you can now manage API operations, apply policies, edit and download OpenAPI specification files, or navigate to API Management instance for a full-featured experience.

For more information please check this link: Create an OpenAPI definition for a serverless API using Azure API Management

Azure Data Factory is a cloud-based ETL service to run the SSIS packages. You can lift and shift the existing SSIS packages in Azure and run them fully in ADF. It is code-free UI to run, monitor and manage the packages.

Using the Azure Data Factory, you can create and schedule data-driven workflows (called pipelines) that can ingest data from disparate data stores.

To start, navigate to All services - > Analytics -> Data Factories.

Click Add or Create Data Factory as highlighted below.

Provide the following details in the below screen:

1. Appropriate name to your data factory
2. Select the v2 version. This is the latest version which includes the latest features of Azure data factory
3. The next one is to select the subscription and resource group. Here you can create a new resource group in case you don’t want to use the existing resource group.
4. Select the Location
5. Enable GIT: If you have checked this option, you will need to provide the additional details about the GIT URL, repo URL, Branch Name and Root URL.

Click Create. The deployment will be done in a couple of mins or so to get this done.

You can see we have created the Data Factory. Now you are ready to create the workflow pipelines for this Azure data factory.

I will be explaining about its monitoring section and how to create pipelines in my next blogs. So keep tuning in!

Start by navigating to the "Cost Management + Billing" section. Click on the "Cost Management" management option. In the new blade, click on the "Connectors for AWS (Preview)" option under the settings menu. You will see the option to configure the sync from AWS to Azure here by clicking on the "Add connector" button.

Under basic settings, just provide a display name, management group and a subscription for the connector.

In the "AWS properties" section provide the value for:

• Role ARN
• External ID
• Report Name

You need to take some steps in AWS to get these values. These steps are described in detail here: Create a role and policy in AWS for Billing Connector in Azure

Finally, review all the settings and create the connector. Now you can view all the cost and billing data for both Microsoft Azure and Amazon's AWS in one place.

For more information please check this link: For more information please check this link:

From time to time, you will need to restore your database from an earlier backup. In this post, we will be discussing the restoring the Azure SQL database from the backup file.

To restore the database, go to the Azure SQL database you want to restore. Once you select the SQL database, you can click on Restore as marked in the screenshot.

It will open the options to select the backfile for restore as shown below.

The first dropdown is to select the type of backup. You can either select Point-in-time or the LTR (Long Term backup retention) to select the file. When you select the Point-in-time, it shows the earliest date when the restore is taken. It is the first time the Azure has taken the back up of your SQL database. Database Name would be the name by which the database will be restored at the server. You can change the name as per your requirement.

The next one is the Restore point selection, it will show the dates highlighted for which backup is available.

Then it will ask on which server you want to restore the backup. Once selected, then you can set up the tiered pricing for the restored SQL database.

That's it! You are all set now. Hit that button and restore your database.

In this post, we will discuss managing the backup for the Azure SQL database. Azure takes the back up at the most convenient time when the database is not much in use. We can’t control at what time the back up took but we can set up the days/week/years for how long we want to keep the backup files.

To manage the backup setting, go to the Azure SQL Server under which our Azure SQL Database has been created. Navigate to the "Manage Backups" option under the Settings. Select the Azure SQL database from the list. Once you select the Database, it will enable the "Configure retention" button.

The configurations have mainly four types of retentions configurations settings.

1. PITR – Point in time restore configuration – This allows you to set the number of days you want to keep the backup. All Azure SQL databases (single, pooled, and managed instance databases) have a default backup retention period of seven days. You can change it to 14/21/28/35 days.

2. Long-Term Retention Configurations

   • Weekly LTR backups – You can specify the number between 1 and 520. This is for how many weeks you want to retains the backup.
   • Monthly LTR backups – You must specify the number between 1 and 120 month(s) and select the Months from the dropdown.

Azure active directory admin allows managing the other active directory user to manage the Azure SQL database. SSMS provides an option to connect with the server using the "Azure active directory authentication". To leverage this functionality, the Azure active directory admin has to provide access to the user.

In this post, we are looking at this option and checking the way to add the Azure Active directory admin through Azure Portal.

Go to All resources-> SQL server -> Settings-> Active Directory Admin-> Set Admin

Add the user from the select dropdown. Once selected, you will be able to see the added user on the home page of active directory admin.

Now you are ready to have this AD user manage the database. Now this user can add other AD users to the database (instead of simple SQL users)

To get more details about this, please refer to the Microsoft official link Use Azure Active Directory Authentication for authentication with SQL

Azure SQL Database Elastic Pools are a simple, cost-effective solution for managing and scaling multiple databases that have varying and unpredictable usage demands. The databases in an elastic pool are on a single Azure SQL Database server and share a set number of resources at a set price.

Elastic Pools exist only at the SQL Server level in Azure. These help to pool resources when you are building multiple databases on the same server. In this post, we will be checking how to create the elastic pool in the Azure SQL Server.

To Navigate to the creation of an elastic pool, go to the server under which you want to create the elastic pool.

To create the elastic pool, mention the valid name in the Elastic pool name section. To select the compute and storage, click on the configure elastic pool and it will show below the blade.

Here you assign the total number of DTU’s and max database size of all the databases in the elastic pool.

In the Databases section, you can add the databases which you want to add into this elastic pool that we are creating.

In the ‘Per database settings’ section, you can provide the min and max DTU assigned to each database added in the elastic pool.

You can also apply Tags during the creation (which is highly recommended). Finally, you review and create the resources. The deployment will be submitted and the pool created for you.

In this post, we will be discussing the performance recommendation and automatic tuning provided by Azure. To make your SQL database perform better we need some insights to get better recommendations and it is provided by default by Azure.

Performance Recommendations

Performance overview provides a summary of your database performance and helps you with performance tuning and troubleshooting.

(Per Microsoft documentation) the performance recommendation options available for single and pooled databases in Azure SQL Database are:

• Create index recommendations - Recommends the creation of indexes that may improve the performance of your workload.
• Drop index recommendations - Recommends removal of redundant and duplicate indexes daily, except for unique indexes, and indexes that were not used for a long time (>90 days). Please note that this option is not compatible with applications using partition switching and index hints. Dropping unused indexes is not supported for Premium and Business Critical service tiers.
• Parameterize queries recommendations (preview) - Recommends forced parameterization in cases when you have one or more queries that are constantly being recompiled but end up with the same query execution plan.
• Fix schema issues recommendations (preview) - Recommendations for schema correction appear when the SQL Database service notices an anomaly in the number of scheme-related SQL errors that are happening on your SQL database. Microsoft is currently deprecating the "Fix schema issue" recommendations.

Automatic Tuning

Azure SQL Database automatic tuning provides peak performance and stable workloads through continuous performance tuning based on AI and machine learning. Automatic tuning is a fully managed intelligent performance service that uses built-in intelligence to continuously monitor queries executed on a database, and it automatically improves their performance. This is achieved by dynamically adapting the database to the changing workloads and applying tuning recommendations. Automatic tuning learns horizontally from all databases on Azure through AI and it dynamically improves its tuning actions. The longer a database runs with automatic tuning on, the better it performs. Azure SQL Database automatic tuning might be one of the most important features that you can enable to provide stable and peak performing database workloads.

What can automatic tuning do for you:

• Automated performance tuning of Azure SQL databases
• Automated verification of performance gains
• Automated rollback and self-correction
• Tuning history
• Tuning action T-SQL scripts for manual deployments
• Proactive workload performance monitoring
• Scale-out capability on hundreds of thousands of databases
• The positive impact to DevOps resources and the total cost of ownership

In this post, we will be discussing the Azure SQL Database query performance insights which provide intelligent query analysis for single and pooled databases. It helps identify the top resource consuming and long-running queries in your workload. This helps you find the queries to optimize to improve overall workload performance and efficiently use the resource that you are paying for.

To Navigate to the Query Performance Insight, navigate to All resources-> SQL Databases-> Select the database. Under the menu on the left hand side go to the Intelligent Performance category -> select Query Performance Insights

• The panel describes the Deeper insight into your databases resource (DTU) consumption
• Details on top database queries by CPU, duration, and execution count
• The ability to drill down into details of a query, to view the query text and history of resource utilization
• Annotations that show performance recommendations from database advisors

Optionally, you can select the Custom tab to customize the view for:

• Metric (CPU, duration, execution count).
• Time interval (last 24 hours, past week, or past month).
• The number of queries.
• Aggregation function.

In this post, we will be discussing the Advanced Data Security feature on the Azure SQL Servers. It is enabled at the server level and will be automatically enabled at the database level. It is a charged service. Once you enabled it, you will be charged per month for this service. To access the advanced data security, go to all resources and select the server for which you want to enable the ADS to feature and go to the Security section, and click on advanced data security.

Below are the settings for ADS:

1. Select the ‘On’ to enable the ADS.
2. Select the subscription from the subscripton dropdown. You need to have the storage account for this. If you don’t have a storage account previously created, it will ask you to create the new one.
3. Provide the appropriate email address to get any activity notification.
4. You can select the type of protection from the list as highlighted in the screenshot.

Different types of Advanced Thread Protection types include:

1. SQL Injection – to protect you from such attacks
2. SQL injection vulnerability – to check if there is any such vulnerabilities in the database
3. Data exfiltration
4. Unsafe action
5. Brute Force
6. Anomalous client logins

You can also view and update the Advanced Data Security settings from the Azure SQL Database as shown below.

Although, as the settings are applied and billed at the server level, you get more control to view the settings from the server. The graphs and analytics of data security are shown at the Database level.

In this post, we will see how to use a query editor in the Azure portal. Query Editor helps us to query the tables, view and stored procedure on the selected SQL databases.

To navigate to the Query editor, go to the All Resources-> SQL database -> select the database.

On the left-hand side, you can select the "query editor(preview)" menu option. It will give you the option for providing the login credentials to connect to the Database within your SQL Server.

Once you log in, you can view all the tables, views and stored procedure resides in the selected SQL database. In the query panel, you can write down the query and see the results immediately at the bottom of the screen.

In this post, we will be managing the security of data within Azure SQL Databases by implementing the data masking on data. Via this option, you can mask the sensitive information from your database like credit card details, email addresses, etc.

Select the Azure SQL Database on which you want to implement the data masking. As shown below, it is under the security section. Once you click on it, it will open the blade on which you can set up the mask for different schema under your database.

SQL Users excluded from masking (administrators are always excluded): This option means the data will be visible without any masking to the admin users. You can add other users to whom you want to make all the data visible.

Click ‘Add Mask’, Select the schema, table, and column on which you want to do masking.

We have different Masking field formats. These include, but not limited to Email addresses, Credit card information, etc.

In this post, we will be checking how to enable Geo-replication for the Azure SQL database. Through this feature, you set up active replication to a different Geo location. This enables you to recover your database in case there is a disaster level event in your primary region.

To begin, select the Azure SQL database for which you want to enable the geo-replication as shown below.

Once you click on the Azure SQL database, it will open up the properties of that SQL database. Click on the geo-replication as highlighted in the screenshot and it will open up the geo-replication window. #1 shows you the primary region we selected while creating the SQL database. #2 will provide the list of regions you can select for the replication. Once you select the region, the new blade will open to set up the secondary region for the SQL database.

Create the Target server and select the pricing tier for the new region.

It will take some time to set up the resources, but once it's done, you can switch between the primary and the secondary region in case of failover.

Connecting Azure SQL DB using SSMS

In this post, we will be showing how to connect SSMS (SQL Server Management Studio) to Azure SQL databases. Launch SSMS and the "Connect to Server" popup should open up automatically. Azure doesn’t support Windows Authentication. So you will need to log in using SQL server authentication with the username and password created in the Azure portal for the server or via AAD options.

Before login, you will need the connectivity details and to get that follow the below steps.

1. Get the server details to which you will connect. Make sure you get the server name instead of the database name. Sometimes people get confused. The server will have the naming convention as servername.Database.windows.net
2. Set up the server firewall in the Azure portal. Otherwise, you will not be able to connect with the server. It will throw an error.

To set up the Server Firewall: Go to all resources and select SQL Databases, it will show up all the databases you have created in your subscription.

Select the database to which you want the connectivity. Let’s take an example of the selected database below. Once you click on your database (test01 in the example below), it will open up the blade for that specific database. Go to the "Set server Firewall" option as selected.

Here you can add the IP ranges and/or specific IP addresses from where you want the access allowed to this Database. In the below example, we are adding the Client IP (i.e. the IP address of the client from where you are accessing the Azure portal.

By default, there will be no IP added under the Client IP address section. Once you click on step 1, then it will populate the IP address of your machine. In the large organization, you can set up the start and end IP.

Note that there is a radio button to allow Azure services and resources to access this server, this will ensure the azure services can monitor your database performance to do analytics for the diagnostic operations. Once you click Save, you should be able to connect the Azure SQL databases using SSMS.

In my previous blog, I showed you how to search for Azure SQL databases. In this post we will see how to create the databases. To create the Azure SQL database, you need to know the subscription under you want to create the Azure SQL database.

Below is the first screen to start with the creation of the SQL database. Here you do the following:

1. Select the Subscription under which you want to create the SQL database.
2. You can select the Resource group from the dropdown of already created resource groups or you can create a new one as highlighted below. The next section defines the properties of the new SQL database.
3. Give the appropriate name of the SQL database
4. Select the already existing server under which you want to create the SQL database or you can create the new server.
5. Select ‘No’ under the SQL Elastic pool option. Select ‘Yes’ if you want your resources to be shared between multiple databases.
6. The next one is compute and storage, Azure automatically selected the basic one for me. In case you want more powerful resources, you can change this configuration by clicking on the Configure database link.

The next screen is for Networking. Leave be the default setting as shown in the below screenshot.

Next Screen is Additional settings. This screen is used to set the Collation of the SQL database, data source, and advanced data security. Data source by default is None. If you want to import the backup file of some existing database then select the Backup. If you select the ‘Sample’ option then AdventureWorksLT will be created as the sample database.

Database Collation keeps the default one until unless you have another collation to select based upon different regions.

Advanced Data Security: Azure charges extra if you select this option. For the time being, I am selecting as ‘Not now’. We will revisit this setting in detail in a later blog post.

The next screen is about Tags. Tags are used to categorize your resources. These can later be used in billing and automation etc.

The final screen is ‘Review + Create’. You can review all the options selected in previous screens and it will provide you the estimated cost per month based upon your selection. If you are ok with it then click on the Create button at the bottom of the screen.

You can see the progress at the top right-hand side.

Once the deployment is done, You can see the newly created databases under All services -> SQL databases.

Azure SQL Database is Microsoft's Platform as a Service (PaaS) offering for the Microsoft SQL Server Database. In the next few weeks, we will be going back to the basics and delving into this service from basic to intermediate. We will visit all it's nitty gritty's and understanding in detail what this service has to offer.

Let's start with where to find Azure SQL Databases with the Azure Portal.

Start by going to http://portal.azure.com and clicking on the “All services” as shown below.

It will open up the featured services on the screen as shown below. You can select the SQL Databases from the featured services and if it's not there then you can search in the search box as highlighted in the image.

The next screen will give the option to create SQL databases and you can see previously created SQL databases in the current subscription or selected subscription.

That's all there is to it.

In the last post, we created a new Network Security Group or NSG. Now in this post, we will work with these NSGs and create some rules. We will also check how to assign these to a subnet (or network interface) in Azure.

There are two types of rules that you define in an NSG:

1. Inbound Rules - these rules dictate what incoming traffic will be allowed.
2. Outbound Rules - these rules determine what outgoing traffic will be allowed.

You define the source and destination for each rule along with the protocol and port information. For inbound traffic, the destination will be the associated subnet or network interface. Similarly, for the outbound traffic, the source will be the associate subnet or network interface.

The rules in an NSG has a priority number. Smaller the priority, the higher in the order it is and it will be executed first. When inbound or outbound traffic hits the NSG then the rules are evaluated in the order based on their priority. If a matching rule is found, then the subsequent rules are NOT evaluated. The traffic is either allowed or denied, as per what is defined in the matching rule.

Navigate to your Network Security Groups (NSGs) and check the overview screen. You will notice that there are default rules for both Inbound and Outbound rules. These allow connectivity between subnets within the same virtual network. You can update these as well, but the best practice is to overwrite these by writing another rule with higher priority. Note that a lower number means a higher priority.

Adding Inbound and Outbound Rules

You can add/update the Inbound and Outbound Rules from the settings. Since these two are very similar, we will only be looking at the Inbound security rules. These are more important as these dictate who can talk to the VM from the outside world( i.e. out of your virtual network).

Click on the "Inbound security rules" option under the Settings menu. Check all the default rules. To create a new rule, click on the "+Add" button.

One Inbound rule consists of:

1. Source (IP address in CIDR format) and it's port ranges
2. Destination (IP address in CIDR format) and it's port ranges
3. Protocol for the communication
4. Action - this is either allow or deny and based upon this select the traffic will be either allowed or dropped
5. Priority - a number depicting the priority of a rule in NSG. Lower this number, higher the priority of a rule
6. Provide a name and description for the rule as well.

For defining Source or Destination you have various template options that you can choose from. These include:

1. Any
2. IP Addresses
3. Virtual Network
4. Application security group

That's it! Hit create and the NSG rule is ready.

Assigning the NSGs

These NSGs can be assigned at two levels:

1. Subnet level - At this level, all the inbound and outbound rules in the NSG are applied to all the Network Interfaces (i.e. VMs) inside the subnet.
2. Network Interface level - At this level, the rules only apply to that particular network interface and other interfaces are not affected.

Simply navigate to either one of those as per your requirements. In the below screenshot, we are at the subnet level. Click on the "+Associate" button. In the popup select the virtual network and the subnet in it, with which you want to link this NSG. Hit Save to save these configurations.

That's it! You are all set. Now you can control the traffic via NSGs and allow only the traffic that adheres to the standard at your organizations with utmost importance to security aspects.

Network Security Groups or NSGs are kind of a firewall, where you define what traffic will be allowed and what traffic will be blocked.

Note that whereas AWS has Network ACLs at the subnet level and Security Groups at the EC2 instance level, Microsoft Azure only has NSGs. These can be applied both at the subnet level as well as at the network interface level (i.e. VM level). The way these are designed, they resemble Network ACLs a lot.

You start by navigating to all services -> Networking category -> Network security groups.

All your existing NSGs are listed here. To create a new one click on the "+Add" button.

Provide the subscription and resource group for the NSG. Also, provide a name and the region where the NSG will be deployed.

Provide the tagging information to better categorize the resources.

Review all the settings and hit create.

Now that the NSG is created for us we will see how we can define rules in this and assign it to subnets or network interfaces, in the next post.

Service Endpoints in Azure Virtual Networks provide the ability to connect to various Microsoft public services (like Azure SQL and Azure storage) securely. From official documentation: "The endpoints also extend the identity of your VNet to the Azure services over a direct connection. Endpoints allow you to secure your critical Azure service resources to only your virtual networks. Traffic from your VNet to the Azure service always remains on the Microsoft Azure backbone network.".

When you enable Service Endpoints, the IP address for a Microsoft service, switches from a public IP address to a private IP address. If you have any firewall rules in place then those will need to be updated. E.g. a VM connecting to a Microsoft Azure SQL database.

Note that a service endpoint is always enabled on a Subnet level.

To begin, navigate to the Virtual Networks and select the virtual network you require. Under settings menu, click on the "Service endpoints". To create a new service endpoint click on the "+Add" button.

You are prompted to enter a service. You can pick from one of the service available. Microsoft.Storage and Microsoft.Sql are the most common use cases from the list. Microsoft keeps updating this list over a period of time.

Once the service is selected, next select the subnet for which you want to enable the service endpoint. Hit Ok and you are done. It will take some time for the configurations to finish at the backend, but now you will have connectivity from your network to the service leveraging private IP addresses internally (instead of public IP addresses).

For more information check this link: Virtual Network service endpoints

In this post, we will start working with the Virtual Networks (or vNets) and Subnets. To start, navigate to all services and then go to the Networking category. Find the option for Virtual network (the very first option) and click on this. All your vNets will be listed here. Find the one on which you want to work and click on the same.

Under the Settings category, you will find the option for "Address space". This is where you can view the complete address space for your virtual network. Also, this is where you can expand your virtual network by adding more address spaces.

Next up, you have:

1. Connected Devices - These are all the network interfaces (i.e. NIC cards on the virtual machines or VMs) that are connected to this virtual network. You can also filter through the data by name or even IP address.
2. Subnets - All the subnets on the virtual network are listed here.
3. Adding Subnets - You can add more subnets to the vNet by clicking on the "+ Subnet" button. E.g. you can have a subnet hosting an internet-facing application. And then you can have another subnet, hosting a database, that will only be accessible from the application.
4. IPv4 available addresses - These are the number of IP addresses available in a subnet. This gives you an indicator of how many devices can connect to a subnet and when your subnet is about to fill up.

There is much more that can be done on a virtual network and we will keep on exploring the key areas that will be most useful to you.

Microsoft Azure's virtual network capabilities as part of the Infrastructure as a Service (IaaS) offering is called Virtual Network or unofficially known as vNet. It provides connectivity between various virtual machines (or VMs) in that network. It can also extend the connectivity to other networks. You can also extend the virtual networks to your on-premises networks.

In this post, we will be creating a new Virtual Network. To start creating the virtual networks, click on the 3 lines on the top left and then click on the "+New" button. Here click on the Virtual Network option under the Networking category.

Alternatively, you can navigate to all services and then go to the Networking category. Find the option for Virtual network (the very first option) and click on this. All your vNets will be listed here (as shown below). You can click on the "+Add" button to create a new virtual network.

To start with, provide the subscription and the Resource Group details. Provide a name for the virtual network and also the region where you want this to be deployed.

The second screen is the most important in the wizard. This is where you define the address spaces for your vNet in CIDR notation. You also define your subnets here. Every virtual network should have at least one subnet.

In the next screen for security, you can enable the DDoS protection (i.e. protection from the DDoS attacks on your network) and Firewall (i.e. Azure Firewall).

Next, provide the tags to categorize your network resource.

Finally, review all the settings and create the virtual network.

Now that the virtual network is created for us, in the next few posts, we will start working with this and perform more tweaks and configurations.

Network ACLs (or Access Control Lists) and Security Groups are very similar in their functionality. Both are acting as a firewall and restricting the traffic. There are few key differences between the two that we need to understand so that we can create and apply this appropriately to any situation at hand.

Scope

Network ACLs are applied at the Subnet level. These restrict the traffic, coming in or out of the subnet. ACLs are therefore automatically applied to all resources (e.g. EC2 instances) in the subnet.

Whereas the Security Groups are applied at the EC2 instances level.

Network ACLs act as a secondary layer of defense. Also, if anyone forgets to apply Security Groups at the EC2 instances then Network ACLs assist in setting up standards in terms of allowed and blocked traffic in your environment.

(Image source: https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Security.html)

Rule Types

In a Security Group, all the traffic is automatically denied. You can only define "Allow" rules. And the traffic is allowed as per these rules.

Whereas in Network ACLs you decide and configure for each rule if the traffic is Allowed or Denied

Rules Evaluation

In a Security Group, all rules are evaluated, regardless of their order, to determine what traffic is allowed. As all traffic is denied by default and you can only specify allow rules (with no priority or rule number), all the rules need to be evaluated to determine if the traffic is allowed or not.

In contrast, in Network ACLs, the rules are evaluated in order. If a matching rule is found for the traffic, then subsequent rules are NOT evaluated and the traffic is either Allowed or Denied as mentioned in the first matching rule.

Statefulness

A Security Group is Stateful. Return traffic is automatically allowed, regardless of any rules. In contrast, the Network ACLs are Stateless. Return traffic must be explicitly allowed by the rules.

For more information please check the official information here: Internetwork Traffic Privacy in Amazon VPC

Security Groups are another security feature in AWS Virtual Private Clouds (VPCs). It is like a firewall that restricts the traffic at the EC2 instance level. These are applied to the EC2 instances. Once the traffic is filtered by the Network ACLs at the subnet level, it is then filtered further by the security group.

NOTE: All the traffic is automatically denied in a Security Group. Only the traffic defined in a security group is allowed to EC2 instances.

To access the security groups, navigate to the VPC section in AWS and then click on the "Security Groups" option in the Security menu on the left. You can view all your security groups here. Select one and view its properties in the bottom panel. These properties include Description, Inbound rules, Outbound rules, and Tags. You can either right-click on the security group or click on the Actions button to view the action that you can take on this security group. The key actions we are interested in includes:

• Edit Inbound rules
• Edit Outbound rules

"Edit Inbound rules" option is to edit the inbound rules. These govern what traffic will be allowed coming into the EC2 instances. Each rule contains:

1. Type - this determines the protocol or type of traffic. You have loads of pre-defined types, or you can add a custom option from the list
2. Protocol - this is determined by the type
3. Port Range - this is also dependent on the type selected. For pre-defined types, this is auto-populated. For custom type, you can define a custom port range.
4. Source - this is the place from where the communication will originate (that is coming into the associated subnet). You can have source as Anywhere or can define a custom IP address range via a CIDR
5. Description - description of the rule.

"Edit Outbound rules" option is to edit the outbound rules. These govern what traffic will be allowed coming out of the EC2 instance. These rules are very similar to the inbound rules. The only difference is that now the traffic is originating from the EC2 instance and is going outbound. Hence we need to specify a Destination (instead of a Source) while defining the outbound rules.

Please Note that just like for Network ACLs, in a Security Group the address "0.0.0.0/0" means all IP addresses. This can be used for Source in inbound rules or Destination for outbound rules if the exact IP address or IP address range is unknown. Although this should be used with caution. As a best practice, this should be avoided at all costs and should be replaced with as small IP address ranges as possible.

Network ACLs or Access Control Lists is like a firewall for your subnets. There are inbound and outbound rules defined in these ACLs that determine what traffic is allowed in a subnet and what traffic will be blocked. These are great for the security of your network.

To access the Network ACLs, navigate to the VPC section in AWS and then click on the "Network ACLs" option in the Security menu on the left.

You can view all your ACLs here. Select one and view its properties in the bottom panel. These properties include Details, Inbound rules, Outbound rules, Subnet associations and Tags. You can either right-click on the ACL or click on the Actions button to view the action that you can take on this ACL. The key actions we are interested in includes:

• Edit Subnet associations
• Edit Inbound rules
• Edit Outbound rules

"Edit Subnet associations" option is to edit the linking of ACL to a subnet. You can associate an ACL to one or multiple subnets.

"Edit Inbound rules" option is to edit the inbound rules. These govern what traffic will be allowed coming into the subnet. Each rule contains:

1. Rule number or Rule # - this is the order in which the rules are executed.
2. Type - this determines the protocol or type of traffic. You have loads of pre-defined types, or you can add a custom option from the list
3. Protocol - this is determined by the type
4. Port Range - this is also dependent on the type selected. For pre-defined types, this is auto-populated. For custom type, you can define a custom port range.
5. Source - this is the place from where the communication will originate (that is coming into the associated subnet)
6. Allow/Deny - this determines if the inbound communication defined by your rule will be allowed or denied.

"Edit Outbound rules" option is to edit the outbound rules. These govern what traffic will be allowed coming out of the subnet. These rules are very similar to the inbound rules. The only difference is that now the traffic is originating from within this subnet and is going outbound. Hence we need to specify a Destination (instead of a Source) while defining the outbound rules.

Please Note that the address "0.0.0.0/0" means all IP addresses. This can be used for Source in inbound rules or Destination for outbound rules, if the exact IP address or IP address range is unknown. Although this should be used with caution. As a best practice, this should be avoided at all costs and should be replaced with as small IP address ranges as possible.

When you go to create a new resource, search for VMWare and you will see the following 3 solutions:

• VMware Solution by CloudSimple - Service - enables you to consume dedicated VMware Cloud environments on Azure.
• VMware Solution by CloudSimple - Node - dedicated, isolated Azure bare metal infrastructure to create native VMware Cloud environments on Azure.
• VMware Solution by CloudSimple - Virtual Machine - unified, consistent management of Virtual Machines deployed on your VMware Cloud.

For more information please check this link: Azure VMware Solutions

A Route Table is just as the name suggests, it is a table consisting of the routes. This determines how the traffic is routed between the subnets and even out of the virtual network. Each entry in this table is called a "route". When you create a VPC, one main route table is automatically created for your VPC and is associated with the subnet of the VPC. You can create custom route tables. E.g. you may want to send all traffic to a network virtual appliance i.e. a firewall. You may have Palo Alto or CheckPoint or any other firewall in your environment and you want the traffic to route via that firewall. You can achieve this via route tables.

To start, navigate to your VPC section in AWS. Then from the left menu, click on the "Route Tables" option. You can create a new table here as well by clicking on the "Create route table" button. For now, we will work with an existing route table on our VPC. Select the route table that you want to view the details or edit.

At the bottom panel, you will be able to view the following settings for the selected route table:

• Summary
• Routes
• Subnet Associations
• Edge Associations
• Route Propagation
• Tags

You can also either right-click on your route table or click on the Actions button to view the options related to this route table. Two options that we are interested in are:

• Edit subnet associations
• Edit routes

Under the "Edit subnet associations" section, you can view the currently associated subnets. You can also link other subnets to the selected route table.

Under the Edit routes option, you can edit the current routes and can also add new routes to the route table. A single route simply defines a Destination, Target, Status and Propagated. The destination is where the traffic (that originated in your subnet) is headed. A destination of "0.0.0.0/0" means all traffic. Note that all zeros i.e. all traffic-related route should be the last one. These routes are evaluated in order.

If the traffic is destined for an IP address that is defined by 1st rule, then it will be sent to the device defined by the Target value of the route. The value "local" for the Target means to allow the traffic and keep it local, instead of sending anywhere else. You can also send the traffic to virtual appliances and gateways here.

Routes tables are very powerful in determining how the traffic will flow in your environment and should be designed appropriately and tested thoroughly.

Once we have a Virtual Private Cloud or VPC in AWS, we can now start getting familiar with the AWS console options and how to work with VPC and the subnets in it.

We will first need to navigate to VPC from All services and then the "Network & Content Delivery" category.

From the Dashboard, you can navigate to the "Your VPCs" option. Next, you select your VPC from the list of all VPCs. As a best practice, you should provide names for all the VPCs. As you can see from the screenshot below, the first VPC has a name but the second one does not. To provide a name you can simply hover over that VPC in the name column and it will give you an option to edit and provide a new name.

Once you have selected your VPC, you can view it's Description, CIDR blocks, Flow logs and Tags related details at the bottom part of the screen.

Next, you have your VPC selected, either right click on your VPC or click on the Actions menu at the top to view list of actions that you can perform on your VPC. One of this action is very useful. This is to "Edit CIDRs".

While editing CIDRs, you can add a new CIDR by clicking on the "Add IPv4 CIDR" button. You will do this (for example) if the number of available IP addresses are enough and for increasing demand, you need to expand your VPC.

For every VPC you have multiple Subnets under that VPC. You can view these under the Subnets menu from the left. As a best practice, here too you should have a name for each of the subnets. As you can see from the below screenshot, only one subnet has a name. When you select a subnet you can view it's details in the bottom part of the screen. These details include:

• Description
• Flow Logs
• Route Table
• Network ACL
• Tags
• Sharing

You can also right-click on the selected subnet or click on the Actions button at the top to view the list of actions that you can perform on this subnet. Most important ones are for:

• Edit network ACL association
• Edit route table association

We will look at other services related to VPCs and Subnets in the next few posts.

Virtual Private Cloud or VPC is Amazon's virtual network capabilities as part of the Infrastructure as a Service (IaaS) offerings. It provides connectivity between various EC2 instances in that network. It can also extend the connectivity to other networks or even on-premises.

Accessing VPCs and Launching Create Wizard

Let's start by creating one in AWS. We will first need to navigate to VPC from All services and then the "Network & Content Delivery" category.

Here in the Dashboard (or even the VPC tab), click on the "Launch VPC Wizard" button to launch the wizard to create a new VPC.

Different Pre-packaged Networking Scenarios

When the wizard launches to create a VPC, there are 4 pre-packaged scenarios to choose from. Note that these scenarios can be built by performing the settings manually and building VPCs separately. These are just like the starter packs (or pre-baked templates) to get you started.

The first one is to build a VPC with a Single Public Subnet. Here is the official description: "Your instances run in a private, isolated section of the AWS cloud with direct access to the Internet. Network access control lists and security groups can be used to provide strict control over inbound and outbound network traffic to your instances."

When you select this option it creates: A /16 network with a /24 subnet. Public subnet instances use Elastic IPs or Public IPs to access the Internet.

The next one is to build a VPC with Public and Private subnets. Here is the official description: "In addition to containing a public subnet, this configuration adds a private subnet whose instances are not addressable from the Internet. Instances in the private subnet can establish outbound connections to the Internet via the public subnet using Network Address Translation (NAT)."

When you select this option it creates: A /16 network with two /24 subnets. Public subnet instances use Elastic IPs to access the Internet. Private subnet instances access the Internet via Network Address Translation (NAT). (Hourly charges for NAT devices apply.)

The 3rd one is to build a VPC with Public and Private subnets and Hardware VPN access. Here is the official description: "This configuration adds an IPsec Virtual Private Network (VPN) connection between your Amazon VPC and your data center - effectively extending your data center to the cloud while also providing direct access to the Internet for public subnet instances in your Amazon VPC."

When you select this option it creates: A /16 network with two /24 subnets. One subnet is directly connected to the Internet while the other subnet is connected to your corporate network via IPsec VPN tunnel. (VPN charges apply.)

The last one is to build a VPC with a Private subnet only and Hardware VPN access. Here is the official description: "Your instances run in a private, isolated section of the AWS cloud with a private subnet whose instances are not addressable from the Internet. You can connect this private subnet to your corporate data center via an IPsec Virtual Private Network (VPN) tunnel."

When you select this option it creates: A /16 network with a /24 subnet and provisions an IPsec VPN tunnel between your Amazon VPC and your corporate network. (VPN charges apply.)

For this post, I have selected the first scenario and clicked Next.

Providing Key Details for VPC

In the next screen, you provide all the essential details to create your VPC along with the dependent resources like it's subnets etc. These settings are as follows (numbering follows the numbers in the screenshot below):

1. IPv4 address space for the whole VPC. This is provided in the CIDR notation.
2. Once you enter the IP address space for the VPC, this area shows you the number of available IP addresses. These are the maximum number of network interfaces that can join this virtual network (or EC2 instances that have these network interfaces). Note that this number is a little less than total number of IP addresses in the CIDR notation. This is because some IP addresses are reserved by AWS for it's own functionality.
3. A name for the VPC to identify it
4. Subnet's IPv4 CIDR address space. This should be contained inside the CIDR for the VPC and should be smaller than that. A subnet is a sub-part of the VPC and therefore will have lesser number of IP addresses than the complete VPC.
5. A name for the subnet
6. Service Endpoint, that allows your VPC to connect to a service like S3. You can set this up at a later point if you need.
7. Enable DNS hostnames to allow DNS hostnames for the instances that can be connected using these hostnames instead of just IP addresses.

Once you are done configuring, hit create to create the VPC. Now that we have a VPC we will start working with it next.

Microsoft Azure and Amazon's AWS cloud platforms provide networking as part of the Infrastructure as a Service (IaaS). Both provide similar services with some differences in the implementation of these services. Microsoft has named it's service as "Virtual Networks", whereas Amazon calls its service as "Virtual Private Cloud" or most commonly called as "VPC".

Both Microsoft Azure Virtual Networks and AWS VPC have "Subnets" inside a network. The VMs in Azure or EC2 instances in AWS has network interfaces that connects to the subnets within the Virtual Networks (or VPCs) and receive an IP address from that subnet. The traffic route inside these networks is governed by "Route Tables".

Traffic restrictions are implemented slightly differently in the two platforms. Microsoft Azure has "Network Security Groups". These can be applied at either Subnet or Virtual Machine level. Whereas AWS has Network Access Control Lists or ACLs that are applied at the subnet level. Also, AWS has Security Groups that are applied to the EC2 instances.

Accessing Microsoft Azure Virtual Networks

To view Virtual Networks in Azure, navigate to All services and then "Virtual Networks" under the Networking category.

Accessing AWS Virtual Private Clouds or VPCs

To view the VPCs, navigate to All Services and then click on VPC under the Networking category.

We will be looking at how to work with these services in detail in the next few posts.

For more information, click on the below official links: Azure Virtual Network Amazon Virtual Private Cloud

Access points can be used to provide access to your bucket securely from a network (VPC) or from the internet. To access bucket resources from a VPC access point, you’ll need to use the AWS CLI, AWS SDK, or Amazon S3 REST API.

To create a new access point, navigate to your S3 bucket and navigate to the "Access points" tab. Click on the "+Create access point" button.

Provide a name for the access point. Select if you want to create this access point for a virtual network (VPC) or the internet. If you select the former then provide an ID for the VPC.

That's all there is to it. Now you are ready to access your S3 bucket securely from a VPC.

For more information click here: Creating Access Points Restricted to a Virtual Private Cloud

For any S3 bucket with critical objects, you should always set up the data to replicate to a different bucket in a different region. You can do both cross-region replication or same-region replication using the replication rules.

To begin, navigate to your S3 bucket and click on the "Management" tab. Then click on the "Replication" settings. Click on the "+ Add rule" to create a new rule.

Pre-requisite: Versioning is a pre-requisite on the S3 bucket for you to be able to set up replication. If you haven't done so the wizard will give you an option to enable this setting here.

Next, while setting sources, you can either set to replicate all the objects in the bucket or filter objects to be replicated based on the prefix or tags that they have.

Next, you have to select a destination bucket. This can be in the same region or a different region. This can even reside in a different account altogether.

If you don't have any bucket in the destination then you can create one right here while setting the destination.

While creating a new destination bucket, you can set its options like Storage class, object ownership, replication time control, etc.

Next up we have IAM role using which the data will be replicated. This should have appropriate access in both source and destination.

Finally, you review and create the replication rule.

Once the rule is set up the data will start getting copied over and protected.

When you have object i.e. files in your S3 bucket, you are essentially paying for the space that these objects take up. After a certain period of time, you will either no longer need these objects or you may want to archive these. If we keep them lying and take actions manually then there are chances that we forget about these and end up paying for the extra space. Lifecycle within S3 buckets is a way to set an expiration date to the files and take automated actions at the expiration.

To start you navigate to your S3 bucket and go to the Management tab. Click on the Lifecycle here. To create a new lifecycle rule click on the "+Add lifecycle rule" button. A new popup launches to set up a new rule. You can have as many rules as you want.

To set up a lifecycle rule, set up a name for this rule. Also, set up a scope for this rule. You can either apply this rule to all objects in the S3 bucket or reduce the scope by using prefixes for the names or leveraging tags.

Transitions is the setting for moving the storage objects from one storage class to another. In the example below, the transition is set from Standard storage to Standard-IA after 90 days. You can set more transitions here. E.g. you can move the data to Glacier storage after 180 days.

Under Expiration settings, you can set the objects to delete after a certain period of time.

Finally, review the settings and create the lifecycle rules.

Setting up lifecycle rules is a great way to optimize costs in the longer run. It is a best practice to set up a policy at your organization and adhere to that for every bucket. It may seem like an extra but can help save costs big time.

You can manage the access to the S3 bucket at a much granular level through its permission settings. You can access its settings by navigating to the S3 bucket and clicking on the "Permissions" tab.

Here you have the option to modify:

• "Block public access" related policies
• Access Control Lists - modifying and creating them
• Bucket policies
• CORS configurations - i.e. Cross-Origin Resource Sharing for HTTP access

The "Block public access" policies are those that you set up while creating the S3 bucket. You can modify these settings here by clicking on the Edit button and can set this up at a granular level.

You can set up much more granular access for the below using Access Control Lists:

1. Bucket Owner
2. Public access
3. S3 log delivery group

You can set up or revoke the access for below operations:

• List objects
• Write objects
• Read bucket permissions
• Write bucket permissions

There are various settings on the S3 bucket that you can either toggle or configure. These properties can make your S3 bucket more secure or optimize performance or can help you to manage it better.

To access these properties, simply navigate to your S3 bucket and click on the Properties tab.

When you click on anyone, it expands to show the details. Many of the properties are simply toggle switches. Others require a bit more configuration.

Below you can view the settings for;

• Versioning - allows you to keep a history of the changes. This is required as a pre-requisite for some features (that we will discuss in upcoming posts)
• Server access logging - to log all events at the Server level
• Static website hosting - This is one of the most useful feature of S3 buckets that is not available in Microsoft Azure. If you have a static website, i.e. simple HTML pages with no database etc. you can provide this as a website by simply copying all your files to a bucket and enabling this option. It provides you with a public URL that you can use to browse your website, starting with a startup page.
• Object-level logging - to log all events at the object level i.e. at the level of each file. It uses CloudTrail service which should be deployed in the same geo region as the bucket.

Below is the setting for:

• Default encryption - You can select to encrypt your objects i.e. all files in the bucket using either AES-256 or AWS-KMS.

You can scroll further down to access more advanced settings like:

• Object lock - prevents objects from accidental deletion
• Tags - to organize resources and get better-categorized cost reporting
• Transfer acceleration - enables optimizations to transfer data at fast speeds
• Events - to set up notifications for specific events on the bucket
• Requester pays - using this option, the requester pays for the data access instead of the owner of the data

Properties tab allows you loads of customizations on your S3 bucket in terms of how your data is stored and how it is accessed.

Once you have an S3 bucket in AWS, you can start uploading files to it and working with it. In this post, we will start working with the S3 bucket we created in an earlier post and will start uploading files to it.

To start, simply navigate to the S3 service and click on your bucket. It will take you to the Overview screen.

The first thing you should do is to create a folder. As a best practice, try to organize your files systematically as much possible. While creating the folder, you can set it's encryption settings.

Once inside the folder (or even from the root level), you can create more folders or directly upload files by clicking on the Upload button. This launches the upload files wizard. Simply drag and drop your files and folders here to start the upload.

You not only just upload files but set some properties to it that we will visit next.

Next, you set the permissions for the uploaded files. This defines who gets access to the files.

You can set the storage class for the files uploaded. Standard is the default option. There are various other options that you can choose from based on your performance and cost requirements. Glacier storage tier is for the archival data, that is not available immediately but you can keep it archived for a long period of time. Note that you are billed based on your selection here so make sure that you select the appropriate Storage class.

Scroll down to view more properties for Encryption, Metadata, and Tagging.

Finally, review all the settings and hit the Upload button to upload the files.

Once the files are uploaded, you can select on each of these files and access a lot of actions from the "Actions" menu. These include downloading the files, changing it's storage class, adding tags, deleting it, etc.

Now that we have started consuming the S3 bucket and it has some files in it, in the next few posts, we will delve a little deeper into the settings of an S3 bucket and it's functionality.

AWS Simple Storage Service or S3 is the managed service provided by the AWS environment for storing your blobs like your photos, videos, and any other files. In this service, you create S3 buckets as resources. These are very similar to Microsoft Azure storage accounts.

To start, navigate to S3 service from All services.

Here you will see the option to create a new bucket. Click on the button to get started.

Provide a name and region for the bucket. Note that the name can only contain small letters and numbers and has to be unique within AWS space. The region should be closer to where you or your customer is located.

When you scroll down you have settings to control the access to the S3 bucket. Default is to "Block all public access". You can tweak the settings and select more granular access by unchecking the main checkbox and then selecting the ones that applies to your situation.

Also, to help prevent accidental deletion of your data you can enable the object locks on the S3 bucket.

Once you hit the create button, the bucket is created for you. Next, we will start working with this S3 bucket.

To secure access to your Microsoft Azure Storage accounts, you can configure a firewall on these. You can allow only particular IP addresses or an address range (specified in CIDR format). Also, instead of specifying IP addresses you can configure access to only specific virtual networks (vNets) in your environment and therefore restrict access to other vNets.

To access these settings, navigate to your storage account and under the settings, click on the "Firewall and virtual networks" option.

Here you have multiple options and ways to configure the firewall. Let's look at these in a systematic way:

1. First, you can either leave the access open to all networks or limit it to a specific virtual network. If you select the latter option, you should also select your virtual network for which you want to provide access. You do so by clicking on the "+Add existing virtual network" option. It will open up a new blade and here you can find your existing vNet and add the same.
2. Next, you configure the Firewall and add IP addresses or a range of IP addresses (in CIDR format) for which the access needs to be opened up
3. Finally, you have the exceptions. Unless the data is very critical and confidential, I recommend that you should always allow access to the Microsoft services by selecting the first checkbox. It should be automatically checked by default. You can also allow access to storage account logging and metrics from any network here.

Firewall and vNet access is generally overlooked option. But you should always configure this in your environment to secure access to your storage accounts.

When working with Microsoft Azure Storage accounts, you will need to access these accounts either from a program or programmatically from a script or code. There are two primary ways to securely access your storage accounts. In this post, we will review these and find out how to access their settings from the portal.

Access Keys

The first is to use the Access keys. Linked to every storage account, there are two access keys. These are referred to as primary and secondary keys or simply key1 and key2. To access these keys, navigate to your storage account and then click on the "Access keys" under settings. Note that you only need one key (anyone) to be able to connect to the storage account. Now when you are using a tool like Storage Explorer or PowerShell scripts to connect to your storage account, you can use this key along with the name of the storage account to connect and perform operations.

Note that these keys are like keys to the kingdom for your storage accounts. You can't tweak the control and restrict it in any way. These keys are meant only for the administrators and should not be used by anyone else. You also can't set any kind of expiry on these keys. So once somebody has these keys they can keep on using these to access the storage accounts. The only way to change is to access the keys and click on the refresh arrow symbol next to the key1 or key2 text to regenerate these keys. As a best practice, you should always use SAS tokens i.e. Shared Access Signature keys to perform any operations.

Shared Access Signature (SAS tokens)

SAS tokens are the way you should be accessing the storage accounts, programmatically or otherwise. These are what you should be using (with an expiry date) to give temporary access to anybody in your organization or even outside if there is any sensitive data on your storage accounts.

From official page: "A shared access signature (SAS) is a URI that grants restricted access rights to Azure Storage resources. You can provide a shared access signature to clients who should not be trusted with your storage account key but whom you wish to delegate access to certain storage account resources. By distributing a shared access signature URI to these clients, you grant them access to a resource for a specified period of time. An account-level SAS can delegate access to multiple storage services (i.e. blob, file, queue, table). Note that stored access policies are currently not supported for an account-level SAS."

Once you tweak all the settings for what kind of access you want to provide, then click on "Generate SAS and connection string" it will generate a Shared Access Signature (SAS token), and a connection string and various URIs using that SAS token. These URIs for storage account's blob, file share, tables and queue services.

Now you know the best way to connect to your storage accounts securely. Depending upon your situation ensure to use the correct approach. If in confusion, always go with Shared Access Signatures.

Microsoft Azure Storage accounts generate various events. You can automate these events and can take action when these events occur.

Build reactive, event-driven apps with a fully managed event routing service that is built into Azure. Event Grid helps you build automation into your cloud infrastructure, create serverless apps, and integrate across services and clouds.

Few examples could be:

• When a user uploads a large video file, a time-to-live value is assigned to that file and that file can either be archived or deleted after that time.
• If some image is uploaded then immediately you can run Optical Character Recognition (OCR) on the image and can capture the data into a database.
• If someone uploads a document file then automatically that is processed and the details captured can be input into a database if the document adheres to a certain format
• Generating transcripts for video uploads etc.

Note that this feature is available out of the box in V2 storage accounts. To access, simply navigate to your storage account resource and click on the Events option.

The solution actually leverages other Azure resources to provide this automation. Some of these include:

1. Azure Event Grid - it has built-in support for events coming from Azure Storage blobs
2. Azure Logic Apps (serverless workflow and integration)
3. Azure Functions (serverless code)

You create a subscription to Event Grid. It receives the event notification and passes the information to the handlers to take some actions. Logic Apps and Azure Functions are two such handlers where you can write the automation. We will delve deeper with an example in a future post.

When you have a Storage account in Microsoft Azure, you want to transfer data to the blob storage to consume the storage. The initial data transfer can be huge and you want to optimize this data transfer.

Navigate to your Storage account and click on "Data transfer" setting to view all the options that you have. Use this section to define your estimated data size for transfer, approximate estimated network bandwidth and transfer frequency. Based on your selection this wizard will show you relevant options that will best suit you to transfer your data.

Let look at these options closely. The options you have are discussed below.

1. AzCopy

This is a command-line utility that allows the scripted or programmatic transfer.

• A command-line data transfer utility
• Copy data to and from Azure blobs, files, tables
• Best use: Resilient bulk data transfer at high throughput

You can learn more about this service here: AzCopy

2. Azure PowerShell or Azure CLI

These are actually two different options for scripted or programmatic transfer.

• A command-line interface to manage Azure Resources
• Best use: Build scripts to manage Azure resources and small data sets
• Azure PowerShell installs on Windows or use in browser with Azure Cloud Shell
• Azure CLI installs on macOS, Linux, Windows or use in browser with Azure Cloud Shell

You can learn more about Azure PowerShell here: Azure PowerShell You can learn more about Azure CLI here: Azure CLI

3. Directly from the Azure Portal

This is the easiest option from all the options as using this you are directly uploading the data from the Azure portal.

• A web-based interface
• Explore files and upload new files one at a time
• Best use: If you don’t want to install tools or issue commands

You can learn more about this service here: Data transfer for small datasets with low to moderate network bandwidth

4. Azure Data Factory

Here you are using managed data pipelines and have the most control over your data coming into the Azure storage account.

• A hybrid data integration service with enterprise-grade security
• Create, schedule, manage data integration at scale
• Best use: Build recurring data movement pipelines

You can learn more about this service here: Azure Data Factory

5. Azure Storage Explorer

Another Graphical interface (GUI) option provided through a standalone software utility that you can install on your laptop or desktop.

• A GUI-based cross-platform client
• Upload or download from Azure blobs, files, tables, queues, and Azure Cosmos DB entities
• Best use: Easy file management

You can learn more about this service here: Azure Storage Explorer

6. Azure Storage REST API/SDK

This option is directly leveraging the underlying APIs/SDK to transfer the data. This is a highly customizable option. You can even build your own utilities using this option. This option also allows for scripted or programmatic transfer by writing your own custom code to transfer the data.

• Programmatic access to Blob, Queue, Table, and File services in Azure
• Best use: Build your custom applications

You can learn more about this service here: Azure Storage REST API Reference

7. Azure File Sync

This is the continuous sync and site-local caching option.

• Move files from a server to a cloud-native Azure file share with zero downtime
• Up to 100 TiB capacity per Azure file share
• Multi-site sync to multiple servers

You can learn more about this service here: Planning for an Azure File Sync deployment

8. Azure Data Box Edge and Data Box Gateway

Azure Data Box Edge is an on-premises device.

• On-premises Microsoft physical network device. It supports SMB/NFS
• Edge compute processes data in local cache before fast, low bandwidth usage transfer to Azure
• Best use: Preprocess data, inference Azure ML, continuous ingestion, incremental transfer

Whereas, Azure Data Box Gateway is a virtual device, also sitting on-premises.

• On-premises virtual network device in your hypervisor
• Local cache-based fast, low bandwidth usage transfer to Azure over SMB/NFS
• Best use: Continuous ingestion, cloud archival, incremental transfer

You can learn more about this service here: Azure Stack Edge

9. Azure Data Box and Data Box Disk

Azure Data Box is the offline transfer via a device.

Time to transfer: Typically 80 TB/order in 10-20 days

• A 100 TB (80 TB usable) rugged, encrypted device shipped by Microsoft
• Offline transfer to Azure Files, blobs, managed disks, over SMB/NFS/REST
• Best use: Initial/recurring bulk data transfer for medium to large data sets

Azure Data Box Disk is very similar to Data Box. it is also offline transfer via a device (in this case a disk).

Time to transfer: Typically 35 TB/order in 5-10 days

• A 40 TB (35 TB usable) set of up to five, encrypted, 8 TB SSDs shipped by Microsoft
• Mount USB 3.0/SATA disks as drives for transfer to Azure Files, blobs, managed disks
• Best use: Initial/recurring bulk transfer for small to medium data sets

You can learn more about Azure Data box service here: Azure Data Box

10. Azure Import/Export

This is another offline transfer device option. Key features are:

• Ship up to 10 of your own disks to transfer data to and from Azure
• Mount disks as drives for offline transfer to Azure Files, Blobs
• Best use: Initial bulk transfer for small to medium data sets

You can learn more about this service here: Azure Import/Export

Note: This blog has been updated with Private endpoints feature and latest screenshots.

As we saw earlier, Microsoft Azure's Storage account is a managed service for storing your blobs like your photos, videos, and any other files. Microsoft has bundled below services within Storage Accounts:

• Blob storage - to securely store your blobs like photos, videos, and other files, etc.
• File shares - SMB file shares
• Table storage - Tabular storage to store non-relational data
• Queue storage - to scale apps

These are very similar to the AWS S3 buckets.

To access Microsoft Azure Storage Accounts, navigate to All services -> Storage category -> Storage accounts.

Here you can view all your existing storage accounts (if you have any). Click on the "+Add" button to create a new Storage account.

For the basic settings, start with selecting the right subscription and creating or using an existing Resource Group.

Provide a name for the storage account. The name can only contain small letters and have to be unique in Microsoft DNS space. Select the geo location for the deployment. Next, you have the performance setting between standard and premium.

You also select between the two account kinds. There is version 1 and version 2. For all new storage accounts select Version 2.

You decide the replication strategy and select the type of storage account based on the replication. You have options like Zone redundant storage (ZRS), Locally redundant storage (LRS), Globally redundant storage (GRS) and Read Accessible - Globally redundant storage (RA-GRS). For maximum replication, select RA-GRS.

Next, decide if you want Hot or Cool access tier. The default is Hot. Cool is for archival storage.

Under networking you have 3 options for connectivity:

• Public endpoint (all networks)
• Public endpoint (selected networks)
• Private endpoint

The private endpoint is the latest option that assigns a network interface (and therefore a private IP address) to your storage account. With this you can access your storage account leveraging the nic card, securely from your virtual network, as if it was just another resource on that network.

Under advanced settings, you select the security, large file shares, data protection, data lake storage etc.

You can apply tags next to categorize the resource. It is optional but is highly recommended as a best practice.

Finally, review and create the storage account.

For storing your blobs like your photos, videos, and any other files, both the cloud platform have great services to offer. AWS calls its service as Simple Storage Service or more popularly known as S3. Microsoft Azure calls its service as Storage Accounts. Microsoft has bundled below services within Storage Accounts:

• Blob storage - to securely store your blobs like photos, videos, and other files, etc.
• File shares - SMB file shares
• Table storage - Tabular storage to store non-relational data
• Queue storage - to scale apps

From all these services, Azure blob storage within Storage accounts is the key comparable service to S3.

To access AWS S3, you can do so by navigating to all Services and navigating to S3 under the Storage category as shown below.

To access Microsoft Azure Storage Accounts, navigate to All services -> Storage category -> Storage accounts.

We will review how to get started with each of these services in the next couple of blogs.

Both Microsoft Azure and AWS provides a managed service for managing the disks of your virtual machine (or EC2 instance). It is the storage optimized for I/O intensive read/write operations. For use as high-performance Azure virtual machine storage. Microsoft Azure offering is called "Managed disk" whereas AWS calls it "Elastic Block Store (EBS) Volume".

AWS - Elastic Block Store (EBS) Volume

Let's look at AWS's offering first. When you create EC2 instances, you automatically get at least one volume (i.e. disk) for the OS disk. You can additionally add multiple volumes for the data disk. You can view all this data by navigating to EC2 and then under the settings go to the "Elastic Block Store" category. You will find your Volumes listed here.

There is also an option to view the Snapshots which shows you any snapshots taken on the volume, taken manually or from AWS Backup service.

Microsoft Azure Managed disks

Within Microsoft Azure, the disks of Virtual Machines or VMs used to be VHD files sitting on a Storage account (similar to S3 bucket). Microsoft upgraded its offering to Managed Disks with loads of granular controls and optimizations. These disks can be switched from standard HDD to premium SDD by just switching a value from a dropdown. You can also control role-based access control of individual disks (a feature that was not available before in Microsoft Azure).

You can find the managed disks under All services -> Compute category. Click on Disks to view all the disks in your environment.

If you want to view a specific disk on your VM then you can also navigate to your virtual machine and then find the Disks setting. The OS and any data disks will also be listed there.

When you click on any one of the managed disk, it takes you to the overview page and you can see various details about the disk including the "Owner VM" to which this disk is connected. You can directly take a snapshot of the disk from here.

Under settings of the disk, one notable setting is the Configuration of the disk. You can quickly select the Storage type and Size of the disk.

For more information, please use these links:

• Azure managed disks
• Elastic Block Store (EBS)

AWS Auto Scaling is the latest offering from AWS that enables you to quickly discover all scalable resources in your environment and then set up application scaling for you. You can either choose one of the AWS recommended settings to optimize performance or cost or both. Or you can also define your own custom scaling conditions.

You start by navigating to the AWS Auto Scaling by going to All Services and then navigating to the Management and Governance section.

You can create multiple scaling plans here. Start creating your scaling plan by clicking on the "Get started" button.

Next, you find the scalable resources in the AWS environment. You can do so in various ways. You can search for resources on the AWS CloudFormation stack. Or you can search by Tags applied to the resources. Or you can directly select EC2 Auto Scaling groups.

As shown below, we have selected our auto scaling group, that we created in one of the previous blog posts.

In the next step, you have the option to specify the scaling strategy. You start by providing a name and then scroll down for more details.

You can either select one of the predefined auto scaling strategies optimized for availability or cost or both. You can even define your own Custom strategy and have more granular control over these settings.

Expand the Configuration details to view the applied settings.

Next up you have various advanced settings like dynamic scaling and predictive scaling. To configure, first, select your auto scaling group and then expand the category you want to tweak.

Finally, you review all the settings and create your scaling plan.

Once the scaling plan is ready, it will do most of the heavy lifting for you by automating the scaling in and scaling out of the instances based on the scaling strategies defined in your plan.

This is part 2 of the blog where we are creating the Auto Scaling Group in AWS. Let's dive right in, where we left off in the last blog.

Now we are at Step 3, here we define the optional Load balancing details and Health checks. Later is something you should always have configured as a best practice.

In Step 4 we will configure the group size and scaling policies. You provide a minimum and maximum no. of instances along with a desired capacity which becomes the default no. of instances.

Scrolling down you have the scaling policies. I recommend having a "Target tracking scaling policy". There are different criteria that you can configure. In the screenshot below, you have the policy based on the average CPU utilization.

You can configure various notifications on your auto scaling group next.

Finally, you have the option to configure tags on the auto scaling group.

Review all the settings and create the auto scaling group. That's all there is to it. Simulate some load on your application and monitor how the instances scale out and in compliance with your scaling policies.

Auto Scale Groups is how you can have multiple instances hosting the same application running at the same time in AWS. This is what provides you with high availability for your applications deployed on EC2 instances in AWS. These scale in and out based on your scaling conditions. You can create auto scale groups in AWS using any one of the Launch Template or Launch configurations. Templates are the recommended way to go.

You start by navigating to the EC2 in AWS.

Scroll all the way down and navigate to Auto Scaling Groups. Click on the "Create Auto Scaling group" to start the creation wizard.

Right on the first screen, you provide a name for the Auto Scaling group. You also select if you want to use Launch Template or Launch Configurations. Whichever you decide you select the appropriate value from the drop-down.

Once you select the template it shows you detailed settings that are configured inside the template.

Under Step2, you configure the purchase options and instance types. You have the option of adhering to the template configurations as shown below.

Or you have the option of combining purchase options and instance types along with the settings from the template. You can provide cost optimization and instance type details here.

We are only halfway here. We will continue creating the Auto Scaling Group in part 2 of this blog.

Launch Templates is the information about how to create an EC2 instance. It has all the relevant information to launch a new EC2 instance. It contains other instance-level settings, such as the Amazon Machine Image (AMI), instance type, key pair, and security groups, etc. This helps you to provide a predictable and repeatable way of deploying multiple EC2 instances in your environment.

Please note that these provide similar functionality as Launch Configurations (which we will saw in the previous blog). Launch Templates is a new way of doing things. You can have multiple versions of a launch template but not for a launch configuration. Any new features are being added to the Launch Templates and that is what AWS recommends to use.

Launch Template (or Launch Configuration) is also required for creating Auto Scaling Groups in AWS.

Start by navigating to EC2 in AWS.

Navigate to the "Launch Templates" under the Instances category. If you will not have any templates here then you will be presented with the general information. Click on the "Create launch template" to start creating a new template.

You start by providing basic information like template name, version description,etc. Note that the version is one of the features that sets a launch template apart from a launch configuration in AWS. The later can not have any version.

If you want to use this template with the Auto Scaling then make sure to check the box to provide guidance as shown below.

Next provide the AWS Machine Image, Instance type details and Key pair information for login.

Next, you provide the networking details under Virtual Private Cloud to define the connectivity along with a security group. You also provide the Storage information.

Finally, you can add tags to your launch template. You also should configure at least one network interface for your EC2 instances. Click on "Create launch template" to create the launch template with all the selected settings.

Now that we have the Launch Template, we are ready to use it to create Auto Scaling Groups. We will check this in the upcoming blog.

Launch Configuration is the configuration information about how to create an EC2 instance. It has all the relevant information to launch a new EC2 instance. It also contains other instance-level settings, such as the Amazon Machine Image (AMI), instance type, key pair, and security groups, etc. This helps you to provide a predictable and repeatable way of deploying multiple EC2 instances in your environment.

Please note that these provide similar functionality as Launch Templates (which we will check in the next blog). This is the older way of doing things. You can have multiple versions of a launch template but not for a launch configuration. Any new features are being added to the Launch Templates and that is what AWS recommends to use.

This (or Launch Template) is also required for creating Auto Scaling Groups in AWS.

You start by navigating to EC2 in AWS and then clicking on Launch Configurations under "Auto Scaling" category as shown below. Click on "Create launch configuration" to launch the wizard.

The first thing you do, similar to creating a new EC2 instance, is to select an AWS Machine Image or AMI.

Next, you select the Instance type. This defines your vCPUs and Memory for the underlying EC2 instance (that will be built using these configurations).

Next, you provide a name for your launch configurations. You can also configure to use Spot instances here to save on cost (for any non-critical workloads). You can also configure CloudWatch detailed monitoring.

Next up are the Storage settings. You update the settings for the OS disk and add more volumes if you need.

Next, you provide the security group configurations. You can create a new one or reuse one of the existing ones.

Finally, you review the settings and create your launch configurations.

Before the launch configurations can be completed, you are also prompted to provide a key pair to be used to authenticate.

That's it! Now you have a reusable configuration that you can use to build multiple EC2 instances in a predictable fashion.

AWS has various features that provide autoscaling features. Each resource type has it's unique functionality and these resources can work with each other.

These services are:

• Launch Templates - the latest addition - this is a template for the EC2 instances. It has all the relevant information to launch a new EC2 instance. You can configure the compute, storage, networking, etc. inside your launch template. It also contains other instance-level settings, such as the Amazon Machine Image (AMI), instance type, key pair, and security groups, etc. This helps you to provide a predictable and repeatable way of deploying multiple EC2 instances in your environment.
• Launch Configurations - this is very similar to launch templates. This is the older way of doing things. You can have multiple versions of a launch template but not for a launch configuration.
• Auto Scale Groups - This is how you can have multiple instances hosting the same application running at the same time in AWS. This is what provides you with high availability. These scale in and out based on your scaling conditions. You can create auto scale groups in AWS using any one of the Launch Template or Launch configurations. Templates are the recommended way to go.
• AWS Auto Scaling - This is the latest offering from AWS that enables you to quickly discover all scalable resources in your environment and then set up application scaling for you. You can either choose one of the AWS recommended settings to optimize performance or cost or both. Or you can also define your own custom scaling conditions.

In the next few blog posts, we will take a practical look at these resources and how to work with each of these.

One side benefit is that these logs can now be sent to any 3rd party SIEM tools as well.

To get started with this service, simply navigate to your Azure Active Directory and then scroll all the way down to Diagnostic settings. Click on "+Add Diagnostic Setting" to get started.

Select the log type for "Audit logs" and for destination select "Send to Log Analytics" and configure your workspace. Save your settings.

Now the Azure AD logs will start appearing in the Log Analytics workspace. You can query this data and can analyze the data in various ways as well.

For more information check this official blog on this functionality in detail:

• Azure Active Directory Activity logs in Azure Log Analytics
• How to set up the integration

The key points are:

• The underlying compute resources (i.e. CPU and memory) are auto-scaled based on the demand
• The billing is done on a per-second basis based on the usage of vCores

When you go to Configure pricing on your database (either during the creation or afterward from the settings), under the vCore based pricing model you now select the Serverless option.

For more information please check this link: Azure SQL Database serverless

When you have an active Microsoft Azure Virtual Machine Scale Set or VMSS it is imperative that you monitor how your VMSS has been running over the period of time. This helps you tweak the autoscaling settings of your VMSS. You should monitor this continuously for any critical applications in your environment. Ideally, to begin with, this should be done in a test or QA environment with a simulated load on your application running for a long period of time. And then you should keep doing this in upper environments as a best practice.

To access this setting, navigate to your VM scale set and navigate to the "Scaling" option under the Settings menu. Here click on the "Run history" tab. All the history for how the instances were scaled up and down will be visible here. You can filter the data based on time to reduce the window. At the bottom you also have Operations list view what operations occurred on the VMSS and at what times.

This option should be used in conjunction with the performance data for the app or workloads running on your VMSS for a complete picture. If you view any performance drops, you should tweak the settings to scale out earlier than what it is configured. E.g. If the scale out operation is configured if the Average CPU percentage goes above 75% and you still see performance hits, then you should tweak that setting from 75% to a lower value like 65%.

There may also be some times during the day or week when there will be more load on your application. To ease this, you can add a schedule for scaling based on time. Note that there is no single best combination. The best settings for you depends upon your application.

In Microsoft Azure Virtual Machine Scale Sets or VMSS, understanding and setting up appropriate scaling settings is paramount. You have the option to either set the scaling settings manually or set them to auto-scale based on different criteria.

To access these settings, navigate to your VMSS and go to the Scaling option under Settings. Ensure that "Custom autoscale" is selected. Then click on the "Configure" button from the top.

In the next screen, you can "scale based on a metric" to scale based on different metrics on the VM. Examples of such metrics include CPU percentage, Memory consumption, etc. Here you define:

• Scale out - This is the setting for how the instance count should increase. E.g. if the Average CPU percentage increases beyond 75% then increase the count by 1.
• Scale in - This is the setting for how the instance count should decrease. E.g. if the Average CPU percentage decreases below 25% then decrease the count by 1.

You also have the option to scale to a specific instance count based on a time schedule.

Another setting you have control over here is to set the Instance Limits. You define a minimum and a maximum number of instances with what the default or an initial number of instances is going to be. Note that the default value can only be between a minimum and a maximum number of instances.

Another key point to note is that you can add multiple scaling conditions to ensure autoscaling occurs as you require. Now that you have the autoscaling set up, next you should simulate load on your application running on VMSS and validate that you are not facing any performance issues. If you do then you should tweak the autoscaling numbers.

When you have a Microsoft Azure Virtual Machine Scale Set or VMSS you want to view how many and what instances are running in your VMSS. You can do so very easily by navigating to your VMSS.

From All Service -> go to Virtual machine scale sets. Click on your VMSS to open up in it's own blade. Then from the left hand menu, navigate to the Settings category and click on Instances. This will show you all the instances in your VMSS along with their status.

E.g. in the screen below, you can view 3 instances, 1 of which is in Running state and 2 are in Creating state.

This is part 2 of the post for creating Microsoft Azure Virtual Machine Scale Sets or VMSS. Let's dive right in.

In continuation of the creation of VMSS wizard, next up you have the Scaling options. You provide the initial instance count. If you want to set up autoscaling, I recommend you to set up Custom policy rather than manual. You can automate based on various conditions like CPU exceeding certain thresholds etc. You can also tweak these settings once the VMSS has been set up and can add multiple criteria.

Next, you have Management settings. Here you can provide Upgrade policies, set up monitoring for diagnostics, Identity, Automatic OS upgrades, Instance termination notifications etc.

To monitor health, you can set up rules here. E.g. Let's say you have a web app running on the VMs in VMSS. You can set to monitor health by monitoring port HTTP 80 at a particular path of the app running on your workloads.

Various advanced settings can be set up on the next screen. If you are running Linux VMs in your VMSS then you can leverage "Cloud init" to automate the configurations on the start-up of every new instance in VMSS. Proximity placement groups ensure that the different instances in your VMSS are deployed closer to each other physically in a datacenter.

You can also decide between the VM generation etc.

You can provide Tags next to better categorize the resources.

Finally, you review all the settings and hit Create to create the VMSS.

Microsoft Azure Virtual Machine Scale Sets or VMSS lets you deploy your workload into identical Virtual machines. These VMs are load-balanced internally and the load is also managed automatically for you. You can define various autoscaling schedules to scale your resources up and down based on the demand. E.g. if Average CPU percentage increases beyond a number then you can automate to increase the number of running instances i.e. scale up. Similarly, you can configure to reduce the instances i.e. scale down if the average CPU percentage goes down a certain number. This auto-scaling is provided out of the box with VMSS.

Let's start working with VMSS by creating one.

You will find VMSS under the Compute category.

When in the blade for VMSS, click on the "+Add" button

Provide the subscription and Resource Group just like for any other resource creation in Azure. Provide a name for the VMSS, a region where it will be deployed and Availability set if you want any.

Next, you provide the Instance details. This is very important. This is the underlying Image and it's compute size. This is the consumption that you will be having in your VMSS based on the number of instances running.

You also provide authentication information for connectivity.

Next, we need to provide the disk details (just like for a single VM). Here you can decide the type of disk to use for OS disk. You can also create and attach data disks here.

Next up we have networking details. You provide information regarding which virtual network the underlying VMs will be connected to in this VMSS. You also provide the Network Interface that VMSS will be using for connectivity.

In part 2 we will continue with the wizard and setting up of VMSS.

Autoscaling is a key aspect of Infrastructure as a Service (IaaS) in the cloud. The demand for the service and therefore the load on your applications will vary based on various factors. It will vary as per:

• The time of the day
• The day of the month
• Any holidays
• Any special events etc.

Your application needs to scale out or scale in based on the dynamic demands. You can NOT find a perfect mix and pre-determine the exact number of instances (of Virtual machines or EC2) that you need to run. If you will define the lower number then the performance will suffer. On the other hand, if you overestimate then you will be looking at costly bills. You always want to optimize between Cost and Performance.

This is where Autoscaling comes into the picture. With this, you can have rules in place that will automatically scale out or scale in your number of instances running the application.

As a pre-requisite, you need to have the application which can be deployed on multiple instances behind a load-balancer.

Both Azure and AWS provide out of the box services for this functionality. In both platforms, you can define rules as to how the scaling should occur in an automated fashion. E.g. you can have the number of instances increase by 1 if the average CPU percentage usage increases more than 75%. And also another rule for decreasing the number of instances by 1 if the average CPU percentage usage reduces below 25%.

Microsoft Azure has package everything into one service called Virtual Machine Scale Set or more popularly known as VMSS. Even the automation for autoscaling is also built directly into this service and is as easy as setting up a few configurations.

Amazon's AWS also has this capability, although this is scattered over different services. These include:

• Launch Configurations - a pre-packaged set of configurations that define how to set up new EC2 instances
• Launch Templates - newer way to do the same thing as Launch Configurations. These have more features like versioning, etc.
• Auto Scaling Groups - a load-balanced group of EC2 instances
• AWS Auto Scaling - a newer service providing the automation for scaling out and scaling in of EC2 instances

We will be working on these services in the next few posts and taking a closer look at each of these.

AWS Batch Service

AWS Batch Service has below main concepts:

• Job - This is what is sent for execution to the Job Queue. This is the main work that you are trying to accomplish e.g. processing an image.
• Job Queue - When you submit a job, it is sent to a job queue. This has linked compute resources. AWS Batch Scheduler places the job on a compute resource within a compute resource environment
• Compute Environment - This is the logical grouping of compute resources i.e. EC2 instances that can receive jobs from a job queue

AWS Batch automatically scales out your compute resources when there are jobs to work upon. It later scales it back when the job queue is empty.

Working with the Service

AWS Batch service can be found under all Services and then navigating to the Compute category. Click on Batch to launch the AWS Batch service.

Here you are presented with the default "Getting Started" screen. It gives you an overview of how the service works.

Clicking on the Getting Started button takes you to the wizard to set up the service.

First, you define the Job and its details. You start with where you want to run your job and defining a run time for the job.

Next, you provide the container properties for the job. The command you define is what is executed when this job is run. This is where you define what compute power will be required to run the job. E.g. 2 vCPUs and 2000 MiB of Memory. You also define the number of attempts to be made to run the job successfully. Job Execution timeout is another thing you define here in the number of seconds. If the job does not finish up in this time then it is stopped and moved to the Failed queue.

You can optionally define parameters and environment variables as well.

Under Step 2, you define the compute environment and set up the Job queue. Compute environment is where the job is run. You define its behavior in this section. You can choose between:

• On-Demand - for any critical and business impacting jobs. This is most predictable
• Spot - very cost-effective option. The cost savings are up to 90% of the On-demand option. It uses the EC2 Spot instances to do so.

You can also define a minimum, desired and a maximum number of vCPUs here.

You set up your networking next. Also, this is where you define the Job Queue and link it to the Compute environment.

Hit Create and you should be all set. AWS creates the underlying resources and shows you the status.

Viewing the Resources and Logs

You can now navigate to the Dashboard and view the newly created resources.

You can navigate to the Jobs section and find your job that was executed. It will look like a guid. Click on this job. In the popup, click on "View logs". This will take you to the CloudWatch window and will show you the logs.

That's all there is to it. You can define simple jobs like execute some applications to very complex jobs like image and video processing.

For more information check these link:

• AWS Batch service

So far we have seen that to consume Azure Batch service we need a Batch account. Within this Batch account, we create a Pool. Next, we create a Job that runs on this Pool. Finally, every Job has multiple Tasks that are actually executed within the Job (and on the Pool)

In this post, we will create on such Task in a Job.

To create a Task, you begin by navigating to your Batch account (if not there already). Next click on the Jobs option under the Features category. Find your job and click on the same.

Within the job, find the Tasks menu and click on that. You will see "+Add" option here to add a new Task.

Next, a wizard will open up to add a new Task. Here primarily you provide a task Id, display name and Command line. Note, that this command line is where you specify what to run.

A simple command line may look like below. This displays the Batch environment variable and then waits for 180 seconds. This is used in the screenshots and later output.

cmd /c "set AZ_BATCH & timeout /t 180 > NUL"

A more complex command line, to execute an app package may look like:

cmd /c %AZ_BATCH_APP_PACKAGE_app01#1.0%\\myapp.exe

You can configure more settings like environment settings, task dependencies, and Application packages etc. Hit Submit when ready to create the Task.

To check the output of the task execution we will check the output stream. Navigate to your task. And then under the "Task Files" section click on the "Files on node" option. Here you will find the "stdout.txt" file. Click on this file to view the output.

The output stream (for the simple command line from above) will look like below. You can download this for offline usage if you require it.

This concludes a brief overview of how to consume Azure Batch service.

For more information check this link: Azure Batch service

Now when you will create a new App Service in Microsoft Azure, you can view different Python versions that are currently supported.

You can also view various supported Java versions.

Under the Operating System settings you can see the Linux as an option. Note that based on the Runtime stack selected, this option maybe pre-selected for you and you won't be able to change this setting. If you will select a Runtime Stack that is supported on both OS then you will have the option to switch between the two platforms. E.g. for .Net core you can select any one of the two platforms.

For more information please check this link:

Once you have a Batch account set up with a Pool (of compute nodes) running, you will need a job running in this pool to consume all that computing power.

To create a Job in the Batch Pool, you first navigate to your Batch account (if not there already). Then scroll down and select Jobs. Click on "+Add" to add new jobs.

In the blade to Add Job, provide a unique Job Id. Add this job to the appropriate pool by clicking on the "Select a pool to run the job on". This will bring up another blade where all your pools will be listed. Selected the one you want and hit Select button.

Now that the job is also ready, you can start adding Tasks to this job. You can add multiple tasks to one job. We will check that next.

Once we have a Batch account, we will need a pool of compute nodes. You can do that by creating a Pool in the Batch account.

First, we will need to navigate to the Batch account. To do this first go to All Services in the Azure portal menu.

Next, find Batch accounts under the Compute category.

Under Batch accounts, all your accounts will be listed. Click on the account in which you want to create the pool. Under the account settings, scroll down to the Features category and click on Pools. You will not have any pools. Click on "+Add" to create a new one.

"Add Pool" blade will open up. Provide a pool ID and an optional display name. Provide the Operating system details that you want to target. You can use custom images or one of the marketplace image. E.g. in the screenshot below, the marketplace image for windows server 2016 Datacenter with small disk is selected.

You can provide the node size for each compute node in the pool. You will be charged based on the no. of nodes and the size of each node. You can provide Scale settings. It can be pre-configured as shown below or can be set to auto-scale.

You can next configure a task to start on setting up the pool. You also have various other settings to customize the pool setup.

If you scroll down further, you will see the options to configure virtual network connectivity to the pool. E.g. if you want to run an app that has some network dependency, that you can also run that by configuring your pool to connect to your virtual network.

If you already own Windows Server licenses then you can configure Hybrid Use Benefit or HUB licensing. Note that this option will only show up for Windows server related compute nodes.

Click Ok to create the pool.

In the next post, we will create a Job in this pool.

For more information check this link: Azure Batch service

To consume the Azure Batch service, you need a Batch account.

To create a Batch account navigate to Create a new service (from the 3 lines on the top left corner), then in the menu under the Compute category, find the "Batch Services" option.

Just like any other resource, you provide a Subscription and a Resource Group. You also provide a name for the account. This name should only contain lowercase letters and should be unique in Azure's DNS space. This is used to uniquely identify your Batch account. You also select the location for your Batch service account.

Next, you have advanced settings. Here you select the Pool allocation mode. You have a Batch Service or User subscription as two options. This setting specifies whether to provision compute node pools in a subscription managed by the Batch service, or in the subscription in which you are creating the new Batch account. In most cases, this should be set to default i.e. managed by Batch service.

Next, you can apply tags to categorize the resource and get better control over automation, reporting, and billing, etc.

Finally, you review the setting and create the Batch account. Note that you have the option to download the template here so that you can automate the deployment next time you need to deploy this or deploy it uniformly across different environments.

Azure immediately shows you the status of your deployment. This screen refreshes automatically and will update to show when the deployment is complete.

In the next post, we will look at creating a Pool within the Batch account.

For more information check this link: Azure Batch service

Both Microsoft Azure and Amazon Web Services provide Batch service. These services are very similar in nature as they are trying to solve the same issue. Both are trying to run large-scale parallel and high-performance computing applications efficiently in the cloud.

Both the services leverage underlying computing infrastructure to execute the Jobs. Both have different mechanisms to save you cost. Where Microsoft leverages Hybrid Use Benefit (or HUB) licensing to reduce the cost of having Windows Server, AWS has integrated the usage of Spot instances directly to the AWS Batch service to reduce the cost of the underlying compute.

We will explore each platform's service in detail in the upcoming posts.

For more information check these link:

• Azure Batch service
• AWS Batch service

For your mission-critical workloads, you can configure active replication of the VM to secondary geolocation. You can do so right from the settings of the VM. Under the hood it uses Azure Site Recovery to set up this disaster recovery.

Navigate to your VM and under the Operations category, click on the "Disaster recovery". Here you can select the Target region for the replication. This region becomes your secondary location and should be factored in your Disaster Recovery strategy. E.g. There should be appropriate infrastructure in place to allow for connectivity to this secondary location.

Next, you have Advanced Settings for the Disaster Recovery. You can select the settings for the Target, i.e. what the Resource Group, Virtual Network etc. is going to be in the target region, if we failover the VM to that region. You also have Storage settings, where you define the cache storage account (used to replicate the managed disks) and the source to target managed disk mappings.

Replication and Extension settings can also be updated here if you require. I recommend that under Replication settings you ensure that the vault selected is appropriate one if you have more than one vaults in your environment.

Finally, review all the settings and start the replication by click on the button as shown below. It triggers an initial replication.

You always have an active copy of the VM in the secondary location (delay of up to 15 mins, depending on your replication policy). You can trigger Failover if anything goes wrong in the primary region. Once the DR even is over, you can also Failback to the original location.

For more information check this link: Set up disaster recovery to a secondary Azure region for an Azure VM

Backing up a VM is the best practice from an operations perspective. The VM should be backed up on periodic intervals and these backups should be retained for some time as well.

You can backup a VM, right from its settings. After you navigate to Azure Virtual Machines, click on the VM for which you want to configure backup. Then under the Operations category, click on the Backup option. Here to configure the backup, you select the Backup vault, Resource Group and the backup policy.

You can also create a new backup policy on the fly. To do so, you need to click on the "Create (or edit) a new policy" link.

Under the create or edit Backup policy, you can configure how often and when the backup should be taken. You also configure for how long the backup should be retained.

For more information check this link: Back up an Azure VM from the VM settings

In this post, we will see how to create an on-demand backup. This type of backup is typically needed when you want to perform a critical operation and this can impact the functionality of the instance. Therefore, prior to performing such an operation, you want to ensure that you have the latest backup of the instance.

You access the option to create the on-demand backup by navigating to the AWS Backup service and then selecting the "Protected resources". You will see the button for creating the on-demand backup here.

In the wizard to create the on-demand backup, you provide the resource for which you want to create the on-demand backup. You can trigger the backup rightaway or delay the start by specifying a backup window. You can also set the expiration of the backup under Lifecycle. You can select an existing vault or create a new one.

Finally, you select the IAM role that backup service will use to impersonate while creating the recovery point. Optionally you can also create and apply Tags.

For more information please check this link: AWS Backup

In the previous post, we configured the backup plans in AWS Backup. Now we will use those backup plans and assign some resources to that plan. Specifically, we will assign the EC2 instance to configure the backup for that instance

Assigning Resources

Once the Backup Plan has been created or there is one already in the system, then when you navigate to AWS Backup service, your view will be similar to below. From the left menu, click on Backup Plans (if not already selected). Click on "Assign Resources" button.

In the Assign Resources wizard, provide a name to the resource assignment. Also, provide an IAM role that the backup service will use to create the recovery points. Next, you select the resources you want to assign. Below is the combination of selections to select the EC2 instance.

You have two options for assigning resources. The first is to assign multiple resources by Tags. Second is to provide a Resource Id (which is unique for every resource).

You have different types of resources that you can backup using AWS Backup service. These include:

1. EC2 instances
2. Elastic Block Store or EBS
3. Relational Database Service or RDS
4. Elastic File System or EFS
5. Storage Gateway
6. Dynamo DB

Once you click on Assign resources, the ec2 instance will be assigned to the backup plan. It will not show up under the protected resources until the next schedule of the backup plan creates a recovery point for the resource.

For more information please check this link: AWS Backup

To backup EC2 instances, you use the AWS Backup service. This is a fully managed, centralized backup service simplifying the management of your backups for your Amazon Elastic Block Store (EBS) volumes, your databases (Amazon Relational Database Service (RDS) or Amazon DynamoDB), AWS Storage Gateway and your Amazon Elastic File System (EFS) filesystems.

Configuring Backup Plans

To be able to use AWS Backup you need Backup Plans configured and created.

Let's first navigate to the service by going to Services and then "AWS Backup" service under the Storage category.

When you launch this service for the very first time, you are provided with loads of information regarding the service. The first thing that you need to do is to create a Backup Plan. Click on the button to "Create Backup plan".

You can either start from an existing plan (to copy all it's settings) or build a new plan. If you don't have anything configured then you will have to select the option to Build a new plan. Provide a name for the plan and scroll down.

Next, provide the Backup rule configurations. With every backup plan, you can have multiple backup rules. Basically, these define when a backup is taken, how often it is taken, when does this backup expires and so on. You start by specifying a name for the rule. Then you specify a Schedule for the backup. The options include Daily, Weekly, Monthly etc. You can also customize the backup window, i.e. at what hour of the day the backup is taken.

Under Lifecycle, you define when the backup should transition to Cold storage and when it should expire. You also select a Vault for the backup. This is the vault resource where the recovery points created by the Backup service are organized.

Next, you can copy the backup to multiple regions. In this way you get redundancy. If the backup primary region goes down, you can use the copy from a different region to restore.

Here you also specify the Tags to add to a backup plan to organize and categorize this in a better way.

Click on the "Create plan" button once done. As you can see from the two notes above, once the plan is created, you can assign resources to this backup plan. We want to add our EC2 instance to this plan. We will do this in the next blog post.

For more information please check this link: AWS Backup

Azure Monitor for VMs or Insights is the enhanced version of the analytics on the VM. It includes the performance and dependencies related data for the virtual machines.

This is very similar to "Enable Detailed Monitoring" using CloudWatch in AWS.

Setting Up

To set this up navigate to your virtual machine and then click on the Insights section under the Monitoring category. Click on the Enable button to start. Note that the VM should be in the running state for you to be able to enable this setting.

If you get any error while clicking on the Enable button, ensure that the VM is running. If it is already running then wait and retry after few minutes. Sometimes the VM bootup takes time to finish (depending on the Compute capacity of the VM).

For the VM Insights to work, it needs to connect to a "Log Analytics Workspace". When you will click Enable button, it will ask to connect to a workspace. When you click Enable, a Microsoft Monitoring Agent (MMA) extension is installed on the VM and is connected to the workspace selected.

Analytics Data

Once successfully enabled and the extension deployed, the Insights data will update to show actual data from the VM. If you don't see it or can't refresh the view, then navigate to any other section and come back to this section to force the refresh.

The first tab is for the Performance data. This shows you interactive and filterable data. Some of the data includes Logical Disk Performance, CPU Utilization, Available Memory, Logical Disk analytics, Network data etc.

Next tab is for Map or Service Map. This shows you the dependencies data. This shows you a graphical and interactive view of how the VM interacts with other VMs and outside the network, what ports it is communicating on, etc.

For more information check this link:

Azure Monitor for VMs

As the configurations are stored separately from the application code, the developers and designers now have the flexibility to modify the app behavior without them requiring to alter the code multiple time and performing multiple deployments of the application.

When you navigate to the new service creation wizard in Microsoft Azure, search for App Configurations. This should bring up this new service as shown below.

For more information please check this link: Azure App Configuration

Azure VM Boot diagnostics lets you diagnose what is happening under the hood during the boot-up. If you face any issues with the boot-up of the VM then this is where you will go to find the diagnostics data.

Where is this setting

You can find this setting when you navigate to the Azure Virtual Machines section and click on your VM. Under the settings menu, scroll all the way down to the "Support + troubleshooting" category. You will find the Boot diagnostics option here.

Setting up Boot Diagnostics

If this is not set up for your VM then you can do so easily. All you need is a storage account. You can also change this anytime. Click on the settings button at the top. Click On and select the storage account. You can also create a new one if you require it.

Checking the Boot Diagnostics data

You can view the screenshot to check where the VM is at with the boot-up process under the hood. If you are not able to log into the VM then this is the main area you want to check to see if the VM is ready (or if there are any updates being applied to the VM).

You can download the Serial logs data for the VM as well if you need to do advanced troubleshooting. This is especially helpful in case of the Linux VMs.

For more information click here: How to use boot diagnostics to troubleshoot virtual machines in Azure

Within Azure, when you click on the Virtual Machine, you have a lot of control of various aspects of the VM right from one screen. When you navigate to the Azure Virtual Machines section, you are presented with all the virtual machines in your environment. Here when you click on any one VM, a new screen pops up. Microsoft calls this as new Blade. In this blade, the Overview section is the first one and is selected by default. It shows you lots of information for the VM like it's a subscription, resource group, public and private IP address, a button to connect to the VM, etc.

When you scroll a little bit down you get the out of the box metrics for the VM. These metrics include CPU data, Network data, Disk IOPS, etc. You also have time filter at the top.

From the left-hand side menu of the selected Virtual Machine, scroll down to the Monitoring section. This has loads of information and options. The key option you want to explore is the "Metrics" option. Here you can build custom graphs to monitor different metrics by adding each one. Click on Add Metrics and select the same from the drop-down list. The graph will update to show all the selected matrices.

We will explore more options in the subsequent posts as well.

For more information, refer to the below links: Microsoft Azure Virtual Machines

• Standard v2
• WAF v2

These tiers are backed by Microsoft's SLA of 99.95% availability. These tiers have various optimizations in terms of Autoscaling, Zone redundancy, faster provisioning, improved performance, etc.

You can see these new tiers when creating a new Application Gateway. For the Tier settings, in the drop-down, now you will see these two new SKUs also listed and available as well.

For more information please check this link: Application Gateway

Monitoring is centralized in AWS via Cloud Watch. Few out of the box metrics are integrated into the EC2 instances screen. Various in-depth and detailed metrics are available in the Cloud Watch product.

To view the monitoring data, navigate to the EC2 section and then go to the Instances section. Click on the EC2 instance for which you want to view the metrics. Then in the below section, you are showed the data for the instance. Here select the Monitoring tab. You have different settings here that can help you with monitoring your EC2 instance. The key settings include:

• CloudWatch alarms - this is a way to set up the alarms or alerts
• Enable Detailed Monitoring - this is to enable advanced diagnostics data capturing on the instance. With this, you get detailed monitoring of various metrics that you can also control
• View all CloudWatch metrics - when you click on this option, it will take you to CloudWatch resource. There you can pick and choose various metrics for your EC2 instance and visualize the same.
• The bottom pane is where by default the key metrics for the instance are displayed. These metrics include CPU Utilization, Disk Read and Writes, Network analytics etc.

For more information, refer to the below links: Microsoft Azure Virtual Machines

Amazon Elastic Compute Cloud (EC2)

When you click on this link, it takes you to the GitHub website to authenticate you and then bring you back to the Azure portal once authenticated.

For more information please check this link: Azure now supports GitHub identity single sign-on

Deciding the IOPS (Input/Output Operations Per Second, pronounced i-ops) is one of the most critical decisions when building your virtual machines. But this is also one of the most ignored aspects of the deployment. This performance measurement of your OS and Data Disks can alter your experience significantly. Let's see how it differs in the two platforms.

In AWS

• For Provisioned IOPS (SSD) volumes, you can provision up to 50 IOPS per GiB. Provisioned IOPs (SSD) volumes can deliver up to 64000 IOPS and are best for EBS-optimized instances.
• For General Purpose (SSD) volumes, baseline performance is 3 IOPS per GiB, with a minimum of 100 IOPS and a maximum of 16000 IOPS. General Purpose (SSD) volumes under 1000 GiB can burst up to 3000 IOPS.
• Magnetic volumes, previously called standard volumes, deliver 100 IOPS on average and can burst into hundreds of IOPS.

If you remember when we were building the EC2 instances, you have the option to add/tweak the storage in Step 4. Here you can see that you can select between the different type of Storage disks that we discussed above.

When you select the "Provisioned IOPS SSD (io1)" option, the column for IOPS turns into a text box and you can tweak the IOPS to your hearts' content. Please note that you can not go beyond 50 IOPS per GiB. E.g. If the disk you selected is 8 GiB in Size then you can't go beyond a maximum of 400 IOPS (i.e. 8x50).

In Azure

• Premium SSD disks offer high-performance, low-latency disk support for I/O-intensive applications and production workloads.
• Standard SSD Disks is a cost-effective storage option optimized for workloads that need consistent performance at lower IOPS levels.
• Use Standard HDD disks for Dev/Test scenarios and less critical workloads at the lowest cost.

You can directly update the type of disk for OS under the second screen of the Virtual Machine creation wizard i.e. under the Disks category. You can select between the three types. The OS Disk size is determined by the VM Size selected. This can be increased later. You can click on "Create and attach a new disk" link and add a Data Disk to the VM.

In the "Create a new disk" wizard, you can change the size of the disk. Now here you are shown based on the size, how much IOPS you will get. Microsoft defines different tiers and provides IOPS based on the tier selected i.e. based on the disk size selected.

Below you see the IOPS for Premium SSD tiers.

Below you see the IOPS for Standard SSD tiers.

Below you see the IOPS for Standard HDD tiers.

For more information, refer to the below links: Microsoft Azure managed disks

Amazon EBS Volume Types

To get started, create a new Function App in Azure. While you are creating a new one, now for the "Runtime stack" you can select "PowerShell Core" as the option.

For more information please check this link: Serverless automation using PowerShell preview in Azure Functions

In the previous post, we covered how to create EC2 instances in AWS. You can revisit that post here: Creating EC2 Instances in AWS

In this post, we will check how to create a Virtual Machine in Azure. If you know one, it is very easy to translate the information to another platform. The essential part is to look out for the quirks of each platform and note how these differ from each other. Let's dive in.

You can create the VM resource in one of the two ways. Either by navigating to "Create a resource" and then finding the VM you want to build.

Or you can navigate to the VMs section and then click on the Add button to build the VM.

You are presented with the "Create a virtual machine" wizard. Here you ensure that the subscription and the resource group is correct. Subscription is how to buy and use Azure. You have different plans like "Pay as you go" when subscribing to Azure. Resource Groups are nothing but a logical grouping of resources in Azure.

You also provide the Instance details like the name of the VM, the region where the VM will be deployed, any redundancy options etc.

You select the VM image that you want to build e.g. Windows Server 2016 Datacenter. This is similar to selecting the AMI or Amazon machine image in AWS. The Size decides how many vCPUs and Memory in GiB is allocated to the VM. This is also how you will be billed per month for the VM. Note that you only pay when the VM is running. If the VM is in Stopped and Deallocated state then you are not charged for the Compute consumption of the VM. Then you are only charged for the storage component of the VM.

When you scroll down, this is also where you provide the credentials to be used with the VM. You also configure allowed ports for the VM. E.g. RDP is opened here. You can tweak this further in the Networking section ahead.

The biggest difference here is that, if you own a Windows license on-premises then you can leverage that to save up to 49% in Azure via Azure Hybrid Benefit.

Next, you select the VM disks. Here based on your selection you get faster and slower IOPS. You can also add new or existing disks as data disks.

In the networking section, you can select an existing Virtual network or create a new one. You can select subnets within that network or create a new one as well. Same goes for the Public IP. This is also where you can tweak the network security group.

Under the management section, you can configure Monitoring options. You can also configure Auto-shutdown configurations on the VM as well.

Under Advanced settings, you can install Microsoft or Third party extensions like Chef, Puppet etc. You can also automate one time installations in your linux VMs via Cloud init.

Next, you can assign Tags to categorize the resources. These come in very handy when creating billing reports. Note that not just the VM but underlying resources like OS and Data disks, network interface cards, etc. will also be tagged.

Finally, you can review the settings. Azure also gives you the option to "Download a template for automation". This is your ARM Template including the JSON file, that you can leverage to automate or replicate the deployment of the instance multiple times across different environments.

Then you are prompted with a screen where you can check the deployment status. Once succeeded you are provided with a link to directly navigate to the virtual machine resource.

You can now view the provisioned VM under the Virtual Machines section.

For more information, refer to the below links: Microsoft Azure Virtual Machines

In this post, we will look at how to create EC2 instances in Amazon Web Service (AWS).

You start by selecting the option to create or launch the instances.

In the Instance creation wizard, the breadcrumb at the top show you where in the wizard you are currently at. The first screen is to choose an image for the VM or EC2 instance. The image is called an Amazon machine image or AMI. You can also search for an AMI. Under the image name, you can view the details of the image. Make sure you select the right image. Click on the Select button once decided.

Next, select the instance type. Here you are selecting the actual compute details i.e. the number of vCPUs and the Memory in GiB. You are billed as per the instance type you select.

Next, you select various Instance details. This screen is very vast and makes your deployment very configurable. In this post we will not go into each of these settings. Here you can decide to create an Auto Scaling Group, request spot instances, add placement group, select domain join directory, decide shutdown behavior etc. One key setting that you select here is the Networking details. You select your VPC (Virtual Private Cloud) network and the subnet for the instance.

Next, you configure the Storage details. You can tweak the default storage assigned and can also add more storage. Based on the IOPS requirements you select the Volume type. You can encrypt the storage as well.

Next, you add tags to your instance and underlying volumes. Tags are a great way to categorize resources. Below is just an example of a couple of tags assigned to the instance and underlying volumes.

Now you configure the firewall on the Instance via Security Groups. You can create a new one or select an existing one. This is where you decide what can connect to your instance and what can not. In the screenshot below the RDP connectivity on port 3389 is open. Please note that the Source of "0.0.0.0/0" means that it is open for all internet. It is highly recommended that you change this setting to only your IP addresses from where you will connect.

Finally, you review the details. If you need to view the details in detail just expand that category. Click the Edit buttons on the right to edit that setting. Hit Launch to trigger the instance provisioning.

Before the instance can be provisioned, you are prompted with a popup to select a key pair or create a new one. This is a Private and Public key pair that is used to authenticate. You download and keep the Private key. Note that you can only download the Private key at the time of creation. You will not be able to access it later.

You can check the launch status on the next screen.

Finally, you can view and check your instance. It will be listed in the list under the Instances section under EC2 service.

For more information, refer to the below links: Amazon Elastic Compute Cloud (EC2)

It works at the application layer, i.e. layer 7. It monitors the global routing and automatically routes to the best available host to optimize the performance. The routing is based on the URL paths. It also provides instant failover at the global level. It uses smart health probes to check if your application is available or not. It can also perform SSL offloading to encrypt all the traffic.

To configure a Front Door, you define:

• Frontend host
• Backend pools
• Routing rules

For more information check this official links:

• Azure Front Door
• Front Door Overview

ExpressRoute Direct gives you the ability to connect directly into Microsoft’s global network at peering locations strategically distributed across the world. This is different from simple ExpressRoute connectivity in the way that now you are not just connecting to one region but getting access to the complete Microsoft global network. It helps you with massive data ingestion to storage, physical isolation, dedicated capacity, and burst capacity to use Microsoft’s global backbone to access Azure regions at tremendous scale.

ExpressRoute Global Reach is the service where if you have two datacenters, which are located at different geo-locations and both are connected to Microsoft Azure via Express Route then these two datacenters can also connect to each other securely via Microsoft's backbone.

As shown in the below example, the two locations i.e. San Francisco and London are connected to Azure via ExpressRoute. Due to the Global Reach feature, now the traffic between these two datacenters can also traverse on the Microsoft backbone securely (as shown in the green color).

(Image source: Microsoft)

For more information check here:

• ExpressRoute Direct
• ExpressRoute Global Reach

Virtual Machines is one of the key services for Infrastructure as a Service or IaaS. It is the virtualization of the OS and allows you to run your workloads in the cloud. Microsoft and Amazon have different branding and implementation of these services. Where Microsoft calls this as Azure Virtual Machines, Amazon refers to a similar offering as Elastic Compute Cloud or EC2.

Let's look at where you find the EC2 services. You navigate to the All Services section in AWS and then under the Compute category, you find the EC2 option.

EC2 category has loads of information and your virtual machines or Instances are not shown right away. You have to click on Instances under Instances category as shown below. Then you will be able to view all your existing instances. Here you will also have the option to create a new Instance by click on "Launch Instance".

Similarly, in Azure, you navigate to the 3 lines on the top left. Then you can either click on "All Services" (4th option) or you will see "Virtual machines" as an option directly. You can also drag and reorder these options as per your preference.

In the next screen, you can directly access all your Virtual machines or VMs. You can click on your VM or create a new one right from here. You just need to click on the Add button as shown below.

Next, we will dig deeper and view how the creation of these resources differ.

For more information, refer to the below links: Microsoft Azure Virtual Machines

Amazon Elastic Compute Cloud (EC2)

In nutshell, this feature provides you with the ability to enforce or audit users who set passwords to common and weak passwords. When set in enforce mode this feature stops them from being able to set a weak or common password. E.g. you could stop the end users from using a password like "test123" or "password1" etc. You can add as many general passwords to ban. As a good practice, you should add your organization's name or any common internal password to this list with any variants you believe end users may use. The end goal is to try and make the password as much difficult and secure as possible.

This feature also allows you to specify the thresholds to lock a user in case of failed sign-in attempts. It allows you to set the time for which the user will be locked out before they can try signing in again.

This feature can be accessed by navigating to the Azure Active Directory and then selecting "Authentication methods" under the Security section. You will see the "Password Protection" option opened up in its own blade.

Note: Preview customers MUST update the agents to the latest version (1.2.116.0 or higher) immediately. The current agents will stop working after July 1, 2019.

For more information and the announcement blog from Microsoft, read here: Azure AD Password Protection is now generally available!

You can check the video here:

Pre-requisites

You need the following to be able to use this service:

1. Azure AD (Active Directory) - If you have an Azure subscription, then you already have this.
2. Windows Server Active Directory in sync with Azure Active Directory
3. Azure subscription, containing a virtual network that either contains or is connected to the Windows Server Active Directory - this is where the virtualization will be created and configured

Additionally, the Azure VM you create for Windows Virtual Desktop must be:

1. Standard domain-joined or Hybrid AD-joined. Virtual machines can't be Azure AD-joined.
2. Running one of the following supported OS images: Windows 10 Enterprise multi-session or Windows Server 2016

Working with the Service

At a very high level the steps involve the following:

1. Grant Azure Active Directory permissions to the Windows Virtual Desktop Preview service
2. Assign the TenantCreator application role to a user in your Azure Active Directory tenant
3. Create a Windows Virtual Desktop Preview tenant
4. Create a Windows Virtual Desktop Host Pool - available as an offering from the Azure Marketplace right now

You can learn more about this preview and detailed step by step getting started guide here: Windows Virtual Desktop Preview

You can check the video recording here:

The major difference between the two is that Azure Functions provide you with lot more control over the code that gets executed but you have to author and manage that code. Whereas with Azure Logic Apps you are working only in a designer and configuring the activities. You get less control but at the same time, you do not have to manage the underlying code as well. You only need to manage the properties of your activities (or logic blocks) in your workflow.

Additionally, here are more differences between these two services:

You can check the video recording here:

In this session we talked about:

1. Cloud Computing and Azure - to understand the place of Microsoft Azure in the Cloud Computing world and how we fit in
2. Basic Concepts in Azure - to lay the right foundation on which all concepts will be built
3. General Services in Azure - to provide you with the most essential services in Microsoft Azure
4. Tips & Tricks - to give you that edge and make you more efficient in usage of Microsoft Azure

This blog is an Index of various blogs in the series "Azure Tips & Tricks":

1. Quickly run commands or script on an Azure Virtual Machine (VM) without logging into the VM
2. Bootstrap Automation for New resources for repeated deployments in multiple environments
3. Bootstrap Automation for Existing resources for repeated deployments in multiple environments
4. Find solutions to most common problems for any resource
5. Quickly navigate Azure Portal and search documentation with ease
6. Quickly assign and check the Azure Policy and Initiative Compliance status of your resources with new Azure feature

To access this information from the Resource Group level, just navigate to your resource group and search for "Policies" option under Settings of the blade.

To access this from a resource e.g. a Virtual Machine, navigate to the VM and search for "Policies" setting. This is under "Operations" section of the VM.

In the above example, the policy to apply tag is not compliant. You can click on individual policy names and investigate further.

As soon as I apply the tags and the policies compliance is checked next time, it will change the status to Compliant (as shown below). As a best practice, you should be reviewing the compliance status of your environment at periodic intervals.

• Based on Trusted Execution Environments: Intel SGX, Virtualization Based Security (VBS)
• Comm application patterns: Protect data confidentiality, integrity, and sensitive IP
• Protect data and code in use: Isolated portion of processor and memory, code and data cannot be viewed/modified
• Cloud offering: TEE-enabled compute platform, cloud attestation, first-party‒enabled services
• Centrally combine data sources, Communicate with secure endpoints, licensing and DRM

At the time of writing of this blog, this feature is limited preview access under NDA (announced via blog) of a specialized Virtual Machine Series (DC-series) that will become part of the  Azure Compute portfolio and the first installment of the broader Azure Confidential Computing (ACC) initiative. Put simply, confidential computing offers protection that to date has been missing from public clouds: encryption of data while in use.

You can read more about this here: Azure confidential computing

1. Pin/Favorite most common Services and re-ordering them

For a recent project, I was working with Azure Route Table a lot. I was navigating to these multiple times and making a lot of changes. Instead of going to "All Services" and searching for Route Table service every time, you search for it only once and Star it to mark it as Favorite. It will start showing up in the Services list to the left under Favorites.

Next, you want to re-order the Services under Favorites, so you don't have to scroll to navigate to most common services. You can do so by hovering the cursor on the service name under Favorites, and then click and drag on the ellipse (i.e. 3 dots to the right of the service name) as shown below.

Now you will have the most commonly visited service at the tip of your hands.

2. Pin most common Resources to your dashboard

Another scenario I faced in one of the projects was that every time I logged in, I had to make some changes to a VM and connect to the same using it's dynamically assigned Public IP. That meant navigating to this VM, every day and starting the VM and connecting to it. I had to start the VM as auto-shutdown was configured on the VM to save cost. Instead of navigating to the Virtual Machines and then searching for the VM and then navigating to it, you can simply Pin the VM to your dashboard. Next time you open the portal, the VM will be right there on the Dashboard.

You can similarly Pin other resources like SQL Databases, etc.

3. Searching for Resources, Resource Groups, Services and Documentation using the new Search bar

One of the cool features I stumbled upon is the new search bar in the Azure portal. It had made navigating Azure portal so much easier. You just start searching for the resource or Resource Group or Service name in the text box at the top of the portal. You don't even have to type the whole thing. Just start typing and the results will start popping up. Look for different sections in the suggestions box the pops up under the text box. Click on your desired resource or Resource Group or Service.

One advantage of this search box is that you can not just search for resources or resource groups or services, you can also search for Marketplace solutions and Documentation. I have leveraged this feature to search for documentation a lot. You can't remember everything and you have to reference documentation every now and then. Instead of googling the documentation, you can quickly search for it, without leaving the Azure portal and access it. You don't have to worry about navigating away from the Azure portal. The documentation will open up in a separate tab automatically.

I hope that these tips will help you increase your efficiency. Let us know which tip helped you and which you already knew about, in the comments below.

This tip is generic in nature that shows you where to look for various solutions to the most common problems related to a resource. You can find this option under any resource as "Diagnose and solve problems"

Troubleshooting Workflow

Using this a common troubleshooting workflow will look like this:

1. Click on the "Diagnose and solve problems" option under your resource's blade.
2. Look for Resource Health. I have had a scenario where after a lot of troubleshooting around Azure Automation, we came to know that the service was experiencing issues in our intended region (US South Central). A quick check from this will rule out the obvious.
3. Next, you want to check the "Activity logs". You can check those right from this screen. You can find if anybody changed the configurations which might have resulted in errors you are observing.
4. Next section shows you some of the most common issues and how to resolve these. Scroll through these. Sometimes these include step by step guides to help you. Follow these and ensure you did not miss a step.
5. If nothing works, you can finally open a Microsoft Support request right from this screen. Note: If you can't find your issue under common problems, I highly recommend that you search for it in forums like StackOverflow etc. before you open the Microsoft Support request. If it is a business critical issue, then I would recommend that you open the support ticket first and then search for the issue in the interest of time.

You want to automate this because you want to deploy these resources multiple times at different times. You want to deploy these in a predictive fashion without any errors due to manual mistakes creeping in. The number of resources can be fairly large and you will actually end up saving time on the creation of these automation templates instead of manually creating everything.

Generating Automation for an existing set of resources

To generate automation for an existing set of resources, either go to the resource group or to any of the resource. Under Settings look for the option for "Automation script". Depending upon number of resources and type of resources, it may take some time to generate the template.

Note: Even though the option says "Automation script" it still generates an ARM Template for the automation.

Once the template is ready you can do the following:

1. You can inspect the template right there on the screen.
2. You can download the template or directly add it to your library. Once in the library, you can modify it and share it with your team. Or you can directly Deploy it by clicking on the deploy button.
3. Look out for the errors regarding the resources that can't be exported. E.g. Key vault policies can't be exported by this wizard.
4. Next, the Template and Parameters are shown. You can also view, Azure CLI, PowerShell, .Net or Ruby code samples to deploy the template.
5. Make sure that you inspect the template thoroughly. Remove the resources that you don't need.
6. Also, there will be a lot of hard-coded values in this auto-generated template. It will still be better than starting everything from scratch and doing everything manually.

Reuse Existing

You can first search for your resources or scenarios in the Microsoft Quick Start templates. Note that these are sample templates from Microsoft and community members. These templates can be found here: Azure Quickstart Templates.

When you click on a template name you will see multiple files as shown below. Actual template file will always be named as "azuredeploy.json". The related parameters file will be named as "azuredeploy.parameters.json". All other files are optional and are the supporting files for the template.

You can download these files and edit them. You can fork them directly in GitHub and start editing in a source control environment, but your changes will be public (unless you change your repository to Private). If you do not want to make changes and want to directly deploy one of the template, simply click on the "Deploy to Azure" button. This will open Azure portal and you will need to sign into your subscription. Then the template deployment blade will open with the template information already populated. You will just need to select right Subscription, Resource Group and provide various parameters.

It is really simple and saves you lot of time instead of authoring the template from scratch. Why re-invent the wheel. Reuse instead!

Generating template for new resource creation in Azure

If you are creating new resource from the Azure portal and you have to perform this same action multiple times, you should consider using an ARM Template. If you want to create an ARM Template for a resource for which you can't find a template, even then you can use Azure portal to bootstrap the template creation, rather than starting from scratch.

Simple go ahead and start creating your resource from Azure portal. At the last screen, instead of actually creating the resource, look for a link with the name as "Download a template for automation". This will open another blade which will show you the template. You can download the template from here. It will also include the related parameters file and a script to perform the deployment of your template. The below screenshot shows you the option for Storage Account creation.

Note: This option will not be available for all resource types. Microsoft is adding this option to more and more resources everyday.

Note 1: This feature uses the Azure VM Agent on the VM and will not work if the agent is missing.

To leverage this option navigate to your VM in the portal and select the option to "Run Command" under "Operations" section of the VM.

As shown above, you can Run Complete PowerShell script right from this screen. There are various options for common scenarios like checking IP Config which simply executes "ipconfig /all". You can check these common scenarios and the underlying script by clicking on them. If you want to execute then simply click "Run" in the popup blade.

Note 2: You will need to wait for the Output to appear. It takes some time and you just have to be patient. Once you click run, it gets disabled and the operation is running. Once the operation is complete you will see the output in the window. Do not navigate away from this popup blade/window.

Example of Running PowerShell Script

Let us say that you want to install IIS on the VM for quick testing. This can be done easily by running the below PowerShell:

Install-WindowsFeature -name Web-Server -IncludeManagementTools

Click on the "RunPowerShellScript" option. In the popup blade, type the script and hit Run. Wait for the output to appear. Do not navigate away from the window. You can check the status as indicated below.

That's all there is to it. For more information you can check the official documentation that can be found here: Run PowerShell scripts in your Windows VM with Run Command

Note: Use these scripts with caution as these will affect the networking and communication between resources. Go through the scripts and ensure that you know what the scripts are doing before executing these in your environment. Also, always test these in a dev environment before running in any other environment.

Script samples and how they work

The scripts in this sample are:

1. Disassociate-SubnetsFromUDRs.ps1 - This script disassociates the subnets from the Route Tables
2. Associate-SubnetsToUDRs.ps1 - This script reassociates the subnets back to the Route Tables. This uses a CSV file as an input. A sample of the CSV file is also provided along with the script.

The disassociation script fetches the Route Tables and then gets the associated Subnet details from there. It then removes the association by setting the route table property on the subnet to $null as shown below:

Set-AzureRmVirtualNetworkSubnetConfig `
-Name $subnetName `
-VirtualNetwork $virtualNetwork `
-AddressPrefix $subnetAddressPrefix `
-RouteTable $null |
Set-AzureRmVirtualNetwork

The association script works in an exactly opposite way. The major difference is that this script fetches the information from a CSV file as provided along with the sample. This can also be autogenerated (prior to disassociation). The script sample for generating this CSV can be found here: Generate Report for Route Tables with associated Subnets and related information

The command for creating the association of Subnet to the Route Table looks like below.

Set-AzureRmVirtualNetworkSubnetConfig `
-Name $subnetName `
-VirtualNetwork $virtualNetwork `
-AddressPrefix $subnetAddressPrefix `
-RouteTable $routeTable |
Set-AzureRmVirtualNetwork 

Location of the Scripts

You can find this script in GitHub at this location: User Defined Routes (UDRs) or Route Tables Related Scripts

One use case is to use this script to export the associations so that if any association gets deleted then you can recreate later by reference to this report. Another script sample uses the output of this script to perform the association of the subnets to the Route Tables. That sample can be found here: Disassociate and Associate Subnets to Route Tables

Script Requirements and Workings

The script only requires you to update the path for the output csv report.

The script connects to the Azure and fetches all Route Tables by using the below command.

$routeTables = Get-AzureRmRouteTable

It then finds all the subnet linked on those Route Tables by using the Subnets property on the route table object as shown below.

$routeSubnets = $routeTable.Subnets

The script then iterates over all the subnets one by one. It then finds the related subnet information like Subscription id, the virtual network of the subnet and the Resource Group of the Virtual Network. It saves all this information to an array object named $results.

Finally, the script outputs all the results using the Export-Csv cmdlet:

$results | Export-Csv -Path $PathToOutputCSVReport -NoTypeInformation

Location of the Script

You can find this script in GitHub at this location: Report-UDRsWithSubnetInfo.ps1

Now once you have locks, you won't be able to remove the resources or remove configurations from them. To be able to remove configurations or to remove the resources itself you will need to remove the locks. You can do so in an automated fashion by using this script sample.

Script Requirements and Workings

The script does not have any specific requirements.

The script sample removes lock from all Azure Route Tables, but the same concept can be applied to any type of resource. It first fetches the Route Tables by using the below command:

$routeTables = Get-AzureRmRouteTable

Then it simply removes the lock by using the below command:

Remove-AzureRmResourceLock -LockName DoNotDelete -ResourceGroupName $routeTable.ResourceGroupName -ResourceName $routeTable.Name -ResourceType $routeTable.Type -Force

Location of the Script

You can find this script in GitHub at this location: Remove-Locks.ps1

The two scripts

The two scripts provided are:

1. Copy-SnapshotToStorage.ps1 - This copies the Snapshot to a Storage account. This storage account can be in different subscription or even different region. The script uses the "Start-AzureStorageBlobCopy" cmdlet and also shows the progress of the copy operation.
2. Create-VMFromSnapshot.ps1 - This creates a VM from a Snapshot in a Storage account. It creates a Managed Disks VM leveraging the snapshot.

Scenario 1 - Moving a Managed VM or a Disk across subscriptions or regions

You will need to follow the below steps to move a VM built using Managed disks across subscriptions or regions:

1. First, create a snapshot on the VM
2. Then move the snapshot to a Storage account in the target area. Use the first script here.
3. Then create a VM using the snapshot by leveraging the second script provided in this post.

Scenario 2 - Moving the Image across subscriptions or regions

You will need to follow the below steps to move an Image built using Managed disks across subscriptions or regions:

1. Create a new VM using the image
2. Create a snapshot on the VM
3. Move the snapshot across to a Storage Account in the target area. Use the first script here.
4. Create a VM using the snapshot in the target area using the second script.
5. Finally, capture the VM to an image. Delete the image and snapshots from both source and target once image is created in the target area.

Location of the Script

You can find this script in GitHub at this location: Working with Azure VM Snapshots

With the Script sample in this post, you can take the export of multiple VMs by providing the same in a csv configuration file. This file is very simple and the creation of this file itself can also be automated.

The script can also be altered to take an export of all the VMs in the environment.

Script Requirements

The script requires you to populate the configurations CSV file. The sample CSV file is also provided along with the script. This configurations file should have the following columns:

1. Computer - Name of the VM in Azure
2. ResourceGroupName - Resource Group of the VM

Script Working

This is a very simple but one of the most reusable script. It simply takes the export in two steps.

First it fetches the VM as shown below:

$currentVm = Get-AzureRmVM -ResourceGroupName $ResourceGroupName -Name $virtualMachineName -ErrorAction Stop

Then, it takes the export by using the below command.

$currentVm | ConvertTo-Json -Depth 100 | Out-File -FilePath $fileName

The Trick: The most important thing is the Depth parameter when using ConvertTo-Json cmdlet. If you do not specify this then nested properties may not export properly. Specifying a large enough depth ensures that all nested properties are also exported.

The script encapsulates all this logic into a reusable function. It also provides the best practices template to invoke this function. Note: Check the comments marked with "ToDo" where you should be making the changes.

Location of the Script

You can find this script in GitHub at this location: Export-VMConfig.ps1

NOTE:

• Before you apply the HUB licensing, please make sure whether you are eligible for HUB licensing benefits or not. Refer the official documentation here: Azure Hybrid Benefit

Script Requirements

The script requires you to populate the configurations CSV file. The sample CSV file is also provided along with the script. This configurations file should have the following columns:

1. Computer - Name of the VM in Azure
2. OSType - Windows or Linux
3. ResourceGroupName - Resource Group of the VM
4. HUBLicensingNeeded - yes or no. The HUB Licensing is set for only the VMs for which this value is set to yes.

Script Working

In its entirety, the HUB licensing is just a switch for "LicenseType" on the VM. For Windows servers, if it's value is set to "Windows_Server" then this means that you own a Windows Server license and are claiming the HUB Licensing benefits on this VM.

The script applied the HUB licensing in 3 steps. First, it fetches the VM.

$currentVm = Get-AzureRmVM -ResourceGroupName $ResourceGroupName -Name $virtualMachineName -ErrorAction Stop

Then it applies the HUB licensing as shown below.

$currentVm.LicenseType = "Windows_Server"

Lastly, it updates the VM object.

Update-AzureRmVM -VM $currentVm -ResourceGroupName $ResourceGroupName

Note: As a best practice, the script also checks whether the HUB Licensing is already applied or not. If the HUB licensing is already there on the VM then the Script does applies it again. There may be small downtime when the VM is updating and therefore a short downtime should be planned when doing this operation.

The script encapsulates all this logic into a reusable function. It also provides the best practices template to invoke this function. Note: Check the comments marked with "ToDo" where you should be making the changes.

You can find more related details in official documentation here: Azure Hybrid Benefit for Windows Server

Location of the Script

You can find this script in GitHub at this location: Apply-HubLicensing.ps1

Script Requirements

The script requires you to populate the configurations CSV file. The sample CSV file is also provided along with the script. This configurations file should have the following columns:

1. Computer - Name of the VM in Azure
2. OSType - Windows or Linux
3. ResourceGroupName - Resource Group of the VM
4. SizeConversionNeeded - yes or no
5. NewVMComputeSize - new size of the VM. e.g. Standard_DS4_v2. This should conform to the VM size names. For Windows VMs these names can be found by navigating to one of the sizes related link here: Sizes for Windows virtual machines in Azure

Script Working

The script works in 3 simple steps. In the first step the script fetches the Azure VM using Get-AzureRmVM cmdlet and ensures that it exists.

$currentVm = Get-AzureRmVM -ResourceGroupName $ResourceGroupName -Name $virtualMachineName -ErrorAction Stop

The second step is to update the size property for the Hardware profile of the vm object.

$currentVm.HardwareProfile.VmSize = $vmSize 

Lastly, the script updates the VM using the below command.

Update-AzureRmVM -VM $currentVm -ResourceGroupName $ResourceGroupName

Note that there will be a restart on the VM during the size conversion. Do plan for an outage although very minimal. Do keep some buffer in the outage window to account for any unforeseen issues that may occur during the conversion. I have converted various VMs and have never encountered any issues. Still, when I plan for an outage, as a best practice I do plan for a larger window.

The script encapsulates all this logic into a reusable function. It also provides the best practices template to invoke this function. Note: Check the comments marked with "ToDo" where you should be making the changes.

Location of the Script

You can find this script in GitHub at this location: Change-AzureVMSize.ps1

Script Requirements

The script requires you to populate the configurations CSV file. The sample CSV file is also provided along with the script. This configurations file should have the following columns:

1. Computer - Name of the VM in Azure
2. OSType - Windows or Linux
3. ResourceGroupName - Resource Group of the VM
4. ManagedDiskConversion - yes or no. Set this to yes for the VMs for which you want to perform the conversion from older disk type to Managed Disk VMs

Script Working

The process of Managed Disk conversion is different for a VM with Availability Set and one without it. For this reason the script first checks for the Availability Set of the VM.

$avSet = Get-AzureRmAvailabilitySet -ResourceGroupName $rgName -Name $asName -ErrorAction Stop

It then checks and sets the Managed flag on the Availability Set by using the following command.

Update-AzureRmAvailabilitySet -AvailabilitySet $avSet -Managed -ErrorAction Stop

It then stops the VM, converts it to Managed Disk Vm and then waits for 600 seconds. The conversion is performed by following command.

ConvertTo-AzureRmVMManagedDisk -ResourceGroupName $rgName -VMName $vm.Name

Note: The script does not start the VM to avoid unnecessary costs. E.g. if the VM was stopped to begin with and you didn't want to start it. Unnecessary starting of a VM will incur you charges that you do not want. You can still update the script to start the VMs by simply adding the cmdlet for Start-AzureRmVM if you want.

The script encapsulates all this logic into a reusable function. It also provides the best practices template to invoke this function. Note: Check the comments marked with "ToDo" where you should be making the changes.

Location of the Script

You can find this script in GitHub at this location: Convert-VMToManagedDisk.ps1

Script Requirements

The script requires you to populate the configurations CSV file. The sample CSV file is also provided along with the script. This configurations file should have the following columns:

1. Computer - Name of the VM in Azure
2. OSType - Windows or Linux
3. ResourceGroupName - Resource Group of the VM
4. EnableBackup - yes or no
5. RecoveryServicesVaultName - Recovery Services Vault Name for the Backup
6. BackupProtectionPolicy - Backup Protection Policy name inside the Recovery Services Vault. E.g. Daily-30--Weekly-8--Monthly-6

Script Working

The script first fetches the Azure Recovery Services vault using the below command.

$recoveryServicesVault = Get-AzureRmRecoveryServicesVault -Name $RecoveryServicesVaultName

It then uses this information to set the Recovery Services Vault Context as shown below.

$recoveryServicesVault | Set-AzureRmRecoveryServicesVaultContext -ErrorAction Stop

It then fetches the container for the VM Backup using below command. Through this, it checks if the Backup is already configured on the VM or not. If it is already configured then no further action is taken.

$namedContainerCheck = Get-AzureRmRecoveryServicesBackupContainer -ContainerType "AzureVM" -Status "Registered" -FriendlyName $virtualMachineName

The Backup policy is fetched next:

$policy = Get-AzureRmRecoveryServicesBackupProtectionPolicy -WorkloadType "AzureVM" | where {$_.Name -eq $BackupProtectionPolicy}

Finally, the Backup is configured on the VM using the below command.

Enable-AzureRmRecoveryServicesBackupProtection -Policy $policy -Name $virtualMachineName -ResourceGroupName $ResourceGroupName -ErrorAction Stop

Once the Backup is successfully configured, as a best practice, we need to trigger an initial Backup of the VM. The same is performed by the below commands.

Write-Host "Fetching the Recovery Services Backup Container"
$namedContainer = Get-AzureRmRecoveryServicesBackupContainer -ContainerType "AzureVM" -Status "Registered" -FriendlyName $virtualMachineName
Write-Host "Fetching the Recovery Services Backup Item"
$item = Get-AzureRmRecoveryServicesBackupItem -Container $namedContainer -WorkloadType "AzureVM"
Write-Host "Triggering a Backup on the VM"
$job = Backup-AzureRmRecoveryServicesBackupItem -Item $item

The script encapsulates all this logic into a reusable function. It also provides the best practices template to invoke this function. Note: Check the comments marked with "ToDo" where you should be making the changes.

Location of the Script

You can find this script in GitHub at this location: Enable-VMBackup.ps1

This script sample can be reused in multiple scenarios and in multiple scripts. It will ensure that the script is executed only via an elevated prompt or else it will throw error with relevant details for the end user to take corrective action.

Script workings

The script first fetches the current Windows security principal by using the below commands:

$WindowsID = [System.Security.Principal.WindowsIdentity]::GetCurrent()
$WindowsPrincipal = New-Object System.Security.Principal.WindowsPrincipal($WindowsID)

It then fetches the Administrator Role related details using below commands:

$adminRole = [System.Security.Principal.WindowsBuiltInRole]::Administrator

Now that it have both the required information, the script checks if the current Windows Principal is part of the Administrator role or not by using below condition:

if ($WindowsPrincipal.IsInRole($adminRole))
{
    return $True
}
else
{
    return $False
}

The complete script sample turns this into a reusable logic and provides a template for using this with other code.

Location of the Script

You can find this script in GitHub at this location: Get-IsElevated.ps1

How the Script works

The script takes the names of the Resource groups to be created as an array. You can modify the script to take this input from a CSV file along with other parameters like the location of the Resource Group and the related role assignments.

The script uses the below command to create a Resource Group.

New-AzureRmResourceGroup -Name $rg -Location eastus

Note that the location is hard-coded in the above command. This can also be parameterized.

It then uses the below command to make the role assignments on this Resource Group.

New-AzureRmRoleAssignment -ObjectId $ADGroup01.Id -RoleDefinitionName "Contributor" -ResourceGroupName $rg

Note that you can use the same command to make the assignments for a group, user or an app registration. You just need to provide the appropriate Id for the ObjectId parameter.

Location of the Script

You can find this script in GitHub at this location: Create-ResourceGroupsWithRBACAssignments.ps1

The output CSV generate by that script can be updated. It can be corrected for missing tag values or incorrect tag values, etc. The updated csv then can become an Input to the script sample discussed in this blog. This script will read from the updated CSV file and will apply the tags back to the Azure.

Note: This script takes lot of time because the underlying cmdlet takes lot of time to set the tags (i.e. "Set-AzureRmResource" cmdlet). So it is advised that you filter to only the resources for which you want to update the tags.

Script Workings

As mentioned earlier, this script inputs the CSV file generated earlier. The schema for this input file can be checked from the output of Get script from the link mentioned above.

The script then connects to Azure and updates the Tags. If the Tags were not present already then it adds the tagging information. It uses the below command to update the tags.

Set-AzureRmResource -Tag $r.Tags -ResourceId $r.ResourceId -Force

Location of the Script

You can find this script in GitHub at this location: Set-AzureRmTags.ps1

What is updated in this version: This version now can parse the Tags. It even factors for the spacing between the tag names. You will need to tweak the script to parse your own tags. Sample is provided in the script. You will need to modify the lines between 87 and 145 in the lined script.

You can still refer the older script here along with the details mentioned in that post: Script Sample - Generate Azure Resources Report by Tags

Location of the Script

You can find this script in GitHub at this location: Get-AzureRmTagsReport - v3.0.ps1

Next Steps - Setting the updated Tags

Now, what do you do witht he output CSV report that you got from running this script. You will update this csv file by making the corrections. If any resources will be missing any tags then you will add the same. After that this CSV file will become an input to another script, which will Set the changes for the Tags back to Azure resources. This script can be found here: Script Sample - Set Tags on Azure Resources.

1. Get-AzureRmAllResourceReport - which fetches the report for all Azure resources in all subscriptions
2. Get-AzureRmResourceReportForOneResourceGroup - which fetches the report for Azure resources in a particular Resource Group

Script Inputs

All you need to do is to provide the path for the output csv file. In the second script you also need to provide the name of the Resource Group.

The script will iterate all Azure resources and will provide an output csv file. For Azure VM and SQL the script will also provide additional information.

Location of the Script

You can find this script in GitHub at this location: Scripts to Fetch Azure Resource Reports

Currently, the script factors in Virtual Machines as individual resources, but these can be replaced by any resource type.

Inputs and Script Requirements

The script takes in the following inputs:

1. csvLocation - this is the CSV containing the name of the resources on which you want to provide the role-based access. An example CSV is also provided with the script. It is simply two column CSV. The first column being the VM Name (or resource name if you want to generalize) and the second column is the Resource Group containing that resource.
2. role - This is the role that you want to assign. E.g. "Virtual Machine Operator"
3. Scope - this can be one of the 3 values viz. "VirtualMachine", "Subscription" or "ResourceGroup"
4. Usernames - this can be an array of the user names who will be assigned the access on the resources
5. Groupnames - if groups need to be assigned access then they can be mentioned here
6. RBACOperatorFlag - a boolean value with the default value of true. You set this to false when you are making changes and do not want the script to perform any actions. You do not need to worry about this parameter for most scenarios and can ignore this.

Example Execution

Example execution of the script will look like below.

Apply-RBACRoleToResources -csvLocation "C:\Users\aman\Documents\AzureVM.csv" -role "Virtual Machine Operator" -Scope "VirtualMachine" -UserNames "user1,user2" -GroupNames "abac" 

The above command will read the VM names and their related REsource Groups form the AzureVM.csv file. This will assign "Virtual machine operator" role at the Virtual Macine level. This role will be assigned to the user1 and user2 users along with abac Group name.

Location of the Script

You can find this script in GitHub at this location: Apply-RBACRoleToResources.ps1

The script sample in this post is about checking multiple Vms in your environment and check if there is any Pending Reboot on any of the VMs. The script uses WMI queries to check if there is any pending reboot on the machine.

This script was not authored by me. I received it from one of my colleagues and I couldn't track the original author of this script. Nevertheless, this is a very useful script for system administrators.

Script Usage

The script can be used as shown below:

C:\Users\aman\Downloads> import-module .\Get-PendingReboot.ps1
C:\Users\aman\Downloads> $Servers = Get-Content C:\Users\aman\Downloads\Servers.txt
C:\Users\aman\Downloads> Get-PendingReboot -Computer $Servers | Export-Csv C:\Users\aman\Downloads\PendingRebootReport.csv -NoTypeInformation

In the 2nd command above, the Server.txt file should simply have the list of names of the computers in your environment with one computer name per line.

Location of the Script

You can find this script in GitHub at this location: Get-PendingReboot.ps1

You have two types of locks in Azure:

1. ReadOnly - You won't' be able to alter any configuration of the resource
2. DoNotDelete - You will be able to add configurations but will not be able to remove configurations or even delete the resource

Do Not Delete is the lock, that as a best practice, you should apply on all critical resources in the environment. Once this lock is there on the resources, even the global administrator will not be able to delete the resources. The only way to delete the resources will be to delete the lock first and then delete the resources.

The Script Sample Details

This script sample leverages this concept of locks and uses the below cmdlet to apply the locks on various critical resources in the environment.

New-AzureRmResourceLock -LockLevel CanNotDelete -LockName DoNotDelete -ResourceName $vNet.Name -ResourceType $vNet.Type -ResourceGroupName $vNet.ResourceGroupName -LockNotes "Do Not Delete Lock" -Confirm -Force

The above command uses New-AzureRmResourceLock cmdlet to create the Do Not Delete lock on a Virtual Network.

This script iterates through all subscriptions that your account has access to and then applies the lock to all resources of type:

1. Virtual Network
2. Route Tables
3. Express Routes
4. Virtual Network Gateways
5. Virtual Network Gateway Connections
6. Recovery Services Vaults (i.e. ASR Vaults)

Location of the Script

You can find this script in GitHub at this location: Apply-LocksOnVariousAzureResources.ps1

Also, there are some pieces of information that you can get only from inside the VM. E.g. the domain to which a VM is connected. This scripts ensures that the script is available. If it is available then it can also fetch such pieces of information for you.

What does the Script check

The script makes multiple checks which include:

1. Ping check using the Test-Connection cmdlet
2. PS Remoting checking by trying to open a PS Session using New-PSSession cmdlet
3. The script also tries to connect to the WMI and list various object using Get-Wmiobject cmdlet
4. Similar to the PS Remoting check it also checks for WS Man by using the Test-WSMan cmdlet

It can then fetch VM related information by either remoting into it or by using WMI. E.g. it can fetch domain of the VM by using below WMI cmdlet.

$Domain = $wmi.GetStringValue($HKLM, $MachineKey, "NV Domain").sValue

Script Inputs and Requirements

The script only takes below two input parameters:

1. NameOfTheVM
2. IPOfVM

The script requires a credential object with the name as "AutomationCredentialName". These credential should have login and remoting access into the VM.

Location of the Script

You can find this script in GitHub at this location: Check-VMAvailability.ps1

This Runbook sample connects to Azure using Service Principal and fetches various VM properties. The Runbook can be tweaked to fetch more or less information as per the requirements.

Script Input and Requirements

The script takes the below inputs:

1. VM Name - name of the VM
2. ResourceGroupNameofVM - name fo the Resource Group in which the VM exists
3. Azure Tenant Id - Id of the Tenant. This is the Azure Active Directory's Id which can be found in Azure AD properties in the portal
4. Subscription Id - Id of the Subscription in which the VM exists

The VM also requires a credential Object i.e. a Service Pricipal, which will have access to the Virtual machine.

Working of the Script

The script uses the below command to log into the Azure using the Service Principal.

$AzureRMConn = Login-AzureRmAccount -ServicePrincipal -Credential $Cred -TenantId $AzureTenantId -ErrorAction SilentlyContinue -ErrorVariable LoginError

If the connection is successful then the Subscription is selected using below cmdlet.

$ConnSubs = Select-AzureRmSubscription -SubscriptionId $SubscriptionID

Then the actual VM information is fetched using below cmdlet.

$VM = Get-AzureRmVM -ResourceGroupName $VMResourceGroup -Name $VMName

This fetches basic VM information. To fetch detailed information regarding the network interface of the VM, below cmdlets are used.

$nicId = $VM.NetworkProfile.NetworkInterfaces[0].Id
$VMNetowrkInterface = Get-AzureRmNetworkInterface -ResourceGroupName $VMResourceGroup -Name $nicId.Split('/')[8]
$VMMainIPConfig = $VMNetowrkInterface | Get-AzureRmNetworkInterfaceIpConfig
$strVMMainNicName = $VMMainIPConfig.Name
$strVMMainNicIPAddress = $VMMainIPConfig.PrivateIpAddress

The first cmdlet above fetches the Id of the first Network Interface on the VM. Then the second cmdlet uses this Id to fetch the actual network interface. The third cmdlet uses Get-AzureRmNetwrokInterfaceIpConfig to fetch the IP configuration on this network interface. Then last two lines use this Ip configuration to fetch the Private IP address on the VM.

Location of the Script

You can find this script in GitHub at this location: Get-VMInfoFromAzure.ps1

This script sample helps you bootstrap the Runbook for you.

Script Inputs and Requirements

The script takes the following input parameters:

1. SMTP Server URL
2. SMTP Server Port
3. User Credentials - Boolean value to check if user credentials are required or not
4. Credential Name - Credential name for the credential asset in your Azure Automation Account
5. EnableSsl - Boolean value to select if you want to enable SSL or not
6. Email From - From address for the email
7. Email To - Receipient of the email
8. Email Subject - Subject of the email 9 Message Body - This can be html. So you can compose your email to make it look more professional

The script also requires you to have a Credential asset in your Azure Automation Account. It's name is specified in the 4rth parameter above. This credential is used to connect to the SMTP server. In the sample this smtp server is an O365 email account.

Script Working

The script uses a SMTP Client to send the email. This client is created using the below cmdlet.

$SmtpClient = New-Object System.Net.Mail.SmtpClient $SMTPServerUrl, $SMTPServerPort

where $SMTPServerUrl is the URL of the SMTP server and $SMTPServerPort is the port number used by the SMTP server.

Later, this client is used to send the email by using it's send function as below.

$SmtpClient.Send($Message)

Location of the Script

You can find this script in GitHub at this location: Send-EmailNotificationFromAzureAutomationRunbook.ps1

1. Recovery Plan Name
2. Failover Type
3. Failover DIrection
4. Subscription Name
5. Resource Group Name
6. VM Name

Sample Recovery Plan Context

A sample Recovery Plan Context looks like below. Please note that this is formatted appropriately here. In the Runbook it will look flattened i.e. instead of multiple lines, the context will be contiguous and will be presented in a single long line. Its data type is Object when coming from ASR Recovery Plan.

{
   "RecoveryPlanName":"RecoveryPlan-Test",
   "FailoverType":"Test",
   "FailoverDirection":"PrimaryToSecondary",
   "GroupId":"Group1",
   "VmMap":{
      "aaaaaa-1111-1111-1111-aaaaaaaaaaa":{
         "SubscriptionId":"bbbbbbbb-2222-2222-2222-bbbbbbbbbbb",
         "ResourceGroupName":"rg-dr-test-asr",
         "CloudServiceName":null,
         "RoleName":"VMName-test",
         "RecoveryPointId":"cccccccccc-3333-3333-3333-cccccccccccc",
         "RecoveryPointTime":"\/Date(1550773609103)\/"
      }
   }
}

How does this Script work

The script first uses "ConvertFrom-Json" cmdlet to parses the object to valid PowerShell object. It then uses the VMMap property of the Recovery Plan Context object to get various pieces of the information.

The script also uses a foreach look to factor for multiple Vms in the Recovery Plan. It will output an array of the VMs with the required information, which can then be consumed in your logic to perform actual functions on the VMs. E.g. You can then connect to the VM and install some extensions on it.

How to Invoke This Runbook/Script

This script can be directly invoked from Azure Site Recovery (ASR)'s Recovery Plan or can be invoked from a master Runbook. It is a best practice to invoke this Runbook from a master Runbook. A sample for this Runbook is provided and discussed here: Script Sample - Azure Automation - Runbook for ASR Recovery Plan

Location of the Script

You can find this script in GitHub at this location: ASR-Get-VMSInfoFromASRContext.ps1

1. Recovery Plan Name
2. Failover Type
3. Failover DIrection
4. Subscription Name
5. Resource Group Name
6. VM Name

The Azure Automation Runbook should be able to receive this information and parse it.

Best Practice and this Script Sample

As a best practice, you should leverage the main Runbook to perform operational functions. E.g. If the automation that you want to trigger on Failover, requires connection to the company's internal network, then you would want it to run on a Hybrid worker (which will be connected to your internal network). As such, I recommend that the parsing of the recovery plan context should be done in another Runbook, which should be invoked by the main Runbook.

This sample PowerShell Runbook accepts the Recovery Plan context and passes it to the second Runbook. That Runbook may or may not run on a Hybrid worker.

Sample Recovery Plan Context

A sample Recovery Plan Context looks like below. Please note that this is formatted appropriately here. In the Runbook it will look flattened i.e. instead of multiple lines, the context will be contiguous and will be presented in a single long line. Its data type is Object when coming from ASR Recovery Plan.

{
   "RecoveryPlanName":"RecoveryPlan-Test",
   "FailoverType":"Test",
   "FailoverDirection":"PrimaryToSecondary",
   "GroupId":"Group1",
   "VmMap":{
      "aaaaaa-1111-1111-1111-aaaaaaaaaaa":{
         "SubscriptionId":"bbbbbbbb-2222-2222-2222-bbbbbbbbbbb",
         "ResourceGroupName":"rg-dr-test-asr",
         "CloudServiceName":null,
         "RoleName":"VMName-test",
         "RecoveryPointId":"cccccccccc-3333-3333-3333-cccccccccccc",
         "RecoveryPointTime":"\/Date(1550773609103)\/"
      }
   }
}

Location of the Script

You can find this script in GitHub at this location: ASR-RunbookForRecoveryPlans.ps1

Runbook for Parsing the Recovery Plan Context

The Runbook for parsing the Recovery Plan Context is discussed here: Script Sample - Get VM's Information from ASR Context. The Runbook/script in that link is the one that is being invoked by the Runbook discussed in this blog post.

In this post, we will see how to make quick changes to this custom role leveraging Azure PowerShell script.

Making the updates

Just like any other Azure PowerShell script, you will connect to Azure and select the right subscription by using the following cmdlets.

Add-AzureRmAccount
Select-AzureRmSubscription -SubscriptionName $SubscriptionName

Then you will fetch the current custom role by using the below cmdlet.

$role = Get-AzureRmRoleDefinition $roleName

Optionally, if you want to inspect the current role, then you can use below cmdlet to generate JSON file for the current custom role.

Get-AzureRMRoleDefinition -Name $roleName | ConvertTo-Json

Then you can make updates to the "$role" object. One such sample could be as outlined below which adds the action to allow stopping of the VMs and then updates the description of the custom role.

$role.Actions.Add("Microsoft.Compute/virtualMachines/deallocate/action")
$role.Description = "Can monitor, Start, Stop and restart virtual machines."

Finally, the role is updated back to the Azure portal by using the below cmdlet.

Set-AzureRmRoleDefinition -Role $role

Location of the Complete Script

You can find this script in GitHub at this location: Update-CustomRoleSample.ps1

In the last script sample, we saw how to check remote computers if these services are set correctly or not. That sample can be found here: Script Sample - Check Azure Site Recovery (ASR) Pre-requisites.

In this sample, we will see how to set these services correctly on the remote computers.

How it works

This script will query the remote computers and set the service status for the 3 ASR Required services. These 3 services are:

1. Volume Shadow Copy(VSS)
2. COM+ System Application(COMSysApp)
3. Microsoft Distributed Transaction Coordinator Service (MSDTC)

The script will set the startup type of the services to "automatic" and will then "start" the service. It uses the below cmdlets to perform these actions.

Set-Service -name 'msdtc' -ComputerName $Computer -StartupType Automatic
Start-Sleep -s 5
Get-Service -Name 'msdtc' -ComputerName $computer| Start-Service

The cmdlets to configure other two services is similar.

Location of the Script

You can find this script in GitHub at this location: Set-ASRPrerequisiteServivces.ps1 on GitHub

If you have a large environment then you can do this in an automated fashion by leveraging this script sample. You can learn more about these services related requirements by checking the "Checking Services" section in here: Troubleshooting Azure Site Recovery (ASR) - Data Replication Initiation Issues - Part 2

How it works

This script will query the remote computers and check for service status for the 3 ASR Required services, and export to CSV. These 3 services are:

1. Volume Shadow Copy(VSS)
2. COM+ System Application(COMSysApp)
3. Microsoft Distributed Transaction Coordinator Service (MSDTC)

The script uses WMI to fetch the data from the remote computer(s) and checks if these services are in the "running" state and whether or not the startup type is set to "automatic". If both conditions are not met then the state is reported as "Set Incorrectly". If both conditions are met then the result is reported as "Set Correctly"

Location of the Script

You can find this script in GitHub at this location: Check-ASRPrerequisiteServices.ps1 on GitHub

Next Script - Setting these services automatically

Next, we will check the script sample to set these services appropriately. This sample can be found here: Script Sample - Set Azure Site Recovery (ASR) Pre-requisite Services

The script sample in this blog performs exactly this parsing for you. You will need to tweak the script to include the tags as per your environment.

Note: The EA portal is only available to the EA customer. If you are not an EA customer, then if you try to login, you will get the following error:Invalid User: The account provided is not a valid user of the Microsoft Azure Enterprise Portal. Please sign in with a valid account. If you believe you have received this message in error, please contact Support.

Sample Input EA Azure Billing CSV

The tags section in the CSV will look similar to below JSON format key-value pairs under the column for "Tags":

{  "ApplicationOwner": "aman@domain.com",  "BusinessUnit": "Infrastructure",  "ApplicationType": "Test",  "Department": "Infrastructure"}

Now as a requirement, you will want to split this to separate columns for the Tags for "Application Owner", "Business Unit", "ApplicationType" and "Department".

How does the script parse this JSON

The script parses the JSON by splitting it into multiple values and then saving the new object as an output CSV. Right now these tag values are hard coded into the script and needs to be tweaked as per your requirements. Look out for the names of the variables and output member names with same name as that of the Tags mentioned above. These are the ones that you will need to alter.

The script also takes into factor if the tag was added incorrectly with space in the name instead of pascal casing (i.e. name with no space between two words and first letter capital for each word).

Location of the Script

The script is located in the GitHub along with samples for Input and Output CSV files at this location: GitHub location for Format Billing CSV Script

Note: An alternative to using the billing related csv file is to leverage PowerBI and build dashboards by consuming the EA related billing APIs which are available out of the box. These are in preview at the time of the writing of this blog. Parsing tags is still an issue with these. We will cover these in one of the future blog post.

How it works

This is the most simple but very useful script sample. It exports by simply using the below cmdlet. You can use the below cmdlet alone as well instead of using the whole script, if you are already logged into Azure and have the right subscription selected.

(Get-AzureRmOperationalInsightsSavedSearch -ResourceGroupName $ResourceGroupName -WorkspaceName $WorkspaceName).Value.Properties | ConvertTo-Json

Note that it uses "ConvertTo-Json" cmdlet to export the Saved Searches to JSON output. You can later reuse this JSON to import these saved searches into a different environment as well.

Location of the Script

You can find this script in GitHub at this location: Export Log Analytics Saved Searches

There are two ways to export all the Runbooks all at once.

First method - Using Azure Automation PowerShell ISE Add-On

You can install the Azure Automation PowerShell ISE Add-on and connect to your Azure Automation Account. Then you can download all the Runbooks locally and copy the same. This Add-On can be downloaded from GitHub or PowerShell gallery as per the instructions provided here: Azure Automation PowerShell ISE Add-On

Second method - Using this script sample

This script sample requires you to update the input variables for your environment i.e. related to your subscription, Azure Automation account name etc. Then it connects to the Azure using below cmdlet.

Add-AzureRmAccount
Select-AzureRmSubscription -SubscriptionName $SubscriptionName

Then it exports the Azure Automation Runbooks using below cmdlets. Here it first fetches all the Runbooks and then uses "Export-AzureRmAutomationRunbook" cmdlet to export these to a local folder.

$AllRunbooks = Get-AzureRmAutomationRunbook -AutomationAccountName $AutomationAccountName -ResourceGroupName $ResourceGroupName
$AllRunbooks | Export-AzureRmAutomationRunbook -OutputFolder $OutputFolder

Finally, it exports the variables in the environment by using the below cmdlets. Here it first fetches the variables and then uses "Export-Csv" cmdlet to export the variables to a csv file.

$variables = Get-AzureRmAutomationVariable -AutomationAccountName $AutomationAccountName -ResourceGroupName $ResourceGroupName
$variablesFilePath = $OutputFolder + "\variables.csv"
$variables | Export-Csv -Path $variablesFilePath -NoTypeInformation

Location of the script

This script is located in the Github here: Export Azure Automation Runbooks and Variables

Update [03-27-2018]: This book is available for FREE for a limited time only (on 27, 28 and 29th March, 2018 till midnight Pacific Time). Download Now!!!

This is a quick and practical approach to learning ARM Templates for Azure. It covers only the most essential topics that you will need 95% of the time while working with ARM Templates. The goal is to have you working faster and quicker on ARM Templates without compromising any of the best practices.

The book assumes that you do not have prior knowledge of ARM Templates. If you have no development background then this book is for you. It starts by building the fundamentals on which ARM Templates are built. The more practical approach means less theory and more focus on the practical aspects that can help you start working and delivering on ARM Templates.

You can view the book on Amazon.com here: https://www.amazon.com/dp/B07C8LSBSN

You can also search for the book in your local Amazon marketplace.

Note: Windows images on Azure do not have Special Administrative Console (SAC) enabled by default. SAC is supported on server versions of Windows but is not available on client versions (e.g. Windows 10, Windows 8 or Windows 7).

How to Enable SAC on Windows VM

On a Linux VM, the serial access will be available but for Windows VM you need to perform additional steps in the VM to enable this. The steps to enable are:

• Login to your VM via Remote Desktop

• From an Administrative command prompt run the following commands

  bcdedit /ems {current} on

  bcdedit /emssettings EMSPORT:1 EMSBAUDRATE:115200

• Reboot the system for the SAC console to be enabled

How to Access

Currently, this option (in Preview at the time of writing this blog) is available only via the portal. To access the Serial Console go to:

1. The Virtual Machine (VM) for which you want to access the Serial Console
2. Ensure that the VM is up and running
3. Navigate to the "Serial console (Preview)" option under "Support + Troubleshooting" section in VM settings

If the VM is not up and running then you will view the below error.

The serial console connection to the VM encountered an error: 'Bad Request'
 (400) - The VM is in a stopped deallocated state.  Please start the VM and retry the serial console connection.

When the VM is starting you will see the below message, which you are connected to the Serial Console.

Once connected you can run commands like ch -? for information on using channels and simply type ? for general help.

Note: The console automatically will disconnect after a period of inactivity.

The access is being opened to more and more subscriptions and regions. If you don't see the access today, you just need to wait and it should be coming soon.

This series talks about various aspects of security as related to different aspects of Azure.

This blog is an Index of various blogs in the series "Demystifying Azure Security":

1. Understanding RBAC Roles
2. Custom RBAC Roles
3. Just In Time VM access
4. Azure Policies - Basics
5. Azure Policies - Assigning a Policy
6. Creating a Custom Policy - Part 1 - Viewing Definition of an existing Policy
7. Creating a Custom Policy - Part 2 - Understanding the Policy Structure
8. Creating a Custom Policy - Part 3 - Defining your Custom Policy
9. Azure Policies - Initiative Definitions
10. Azure SQL Database - Transparent Data Encryption
11. Azure SQL Database - Auditing & Threat Detection
12. Azure SQL Database - Set Server Firewall
13. Azure SQL Database - Firewall Rule for Virtual Networks
14. Azure SQL Database and Azure Storage - Service Endpoints on Virtual Network
15. Azure SQL Database - Dynamic Data Masking

This series talks about various aspects of working with Azure Site Recovery or ASR.

This blog is an Index of various blogs in the series:

• Troubleshooting Azure Site Recovery (ASR) - Data Replication Not Working
• Troubleshooting Azure Site Recovery (ASR) - Data Replication Initiation Issues
• Azure Site Recovery (ASR) - New feature added to target Resource Groups
• Suspending and Resuming Azure Site Recovery (ASR) Replication on a single or multiple servers
• ASR Setup for VMs running in VMWare without VMware level User Access
• Getting Started - Azure Site Recovery (ASR) In New Azure Portal

Dynamic Data Masking is a feature of Azure SQL Databases, that allows you to hide the sensitive data. E.g. your database contains information regarding the Credit Cards of your customers. When exposing the database you want to ensure that the credit cards are not exposed. They should automatically be presented in the format "xxxx-xxxx-xxxx-1234" i.e. only exposing the last 4 digits.

Feature Basics

This feature can be accessed by navigating to your database and then clicking on the "Dynamic Data Masking" option under settings. By default, there are no masks applied. Click on "+ Add mask" to add a new mask.

Note that whatever masks you apply are not applied to the administrators. Additionally, you can provide the SQL users who should be excluded from masking.

Azure SQL Database will also automatically try to recommend the fields that should be masked.

Adding Masking Rules

When adding Masking Rules you provide below information:

1. Name for the mask is auto-populated (based on your selections)
2. Schema
3. Table in that schema
4. Column in the table, where mask should be applied
5. Masking Criteria

Note that the Masking criteria vary based on the type of the column. E.g. If a column does not have numerical value then the masking criteria for "Number (random number range)" will show as disabled.

The different masking criteria that you can apply are:

1. Default value (0, xxxx, 01-01-1900)
2. Credit Card value (xxxx-xxxx-xxxx-1234)
3. Email (aXXX@XXXX.com)
4. Number (random number range)
5. Custom string (prefix [padding] suffix)

Azure Service Endpoints allow access to SQL or Storage services over the network, without going out of the network.

To configure this feature, you can navigate to your Virtual Network and then under the settings, select the "Service endpoints". Click on "+Add" to add a Service Endpoint.

In the popup, select the provider for which you want to configure the Service Endpoint.

Service Endpoints on the Virtual Networks are available for:

1. Microsoft.Sql provider
2. Microsoft.Storage provider

Also, select the subnet on which you want to configure the Service Endpoint and then hit "Add".

It will take some time (approximately 15 minutes) to configure the Service Endpoints at the backend. Once configured, you will see the configured endpoints in the portal as shown below.

Note that even after you configure service endpoint for SQL you will need to allow access at the SQL Server level as well. Service Endpoint ensures that the communication will happen at the network level. The Firewall configuration for the network is needed to allow that communication via Firewall on the SQL Server. This is explained in detail here: Azure SQL Database - Firewall Rule for Virtual Networks

Overall, this is a very powerful feature that is easy to configure and provides you with lots of flexibility.

Before going through this blog, please make sure you have covered these:

• Set Server Firewall for Azure SQL Databases
• Service Endpoints for Azure SQL on Virtual Network. Note that this is pre-requisite for the firewall rules to be configured on the Virtual Network level.

Setting up the Firewall Rule for Virtual Networks, at the SQL Server level, enables you to allow access to a subnet in a Virtual Network in Azure on all the SQL Databases on the SQL Server. The firewall rules are always set at the server level, hence any rule you put will allow access on all the databases on the SQL Server.

Setting up the Rules is very simple. You navigate to the firewall settings for the SQL Database/SQL Server (as discussed in the previous blog). Then you focus on the lower section on the blade for Virtual Network Firewall rules as shown below.

Note that you can:

1. Add existing virtual network
2. Create a new virtual network (and provide access)

As a best practice, you should plan the virtual network and subnets before the configurations on the SQL Server Firewall.

When you click on "Add existing virtual network" you are presented with the below wizard. Here you select:

1. The name for the rule. This could be any descriptive name for the rule.
2. The Subscription where the virtual network exists
3. Virtual network where you want to allow the access
4. Subnet name within the Virtual network where you want to allow the access

Below, is the screenshot of the rule with values populated. If the Service Endpoint is not enabled for the "Microsoft.Sql" provider then you will view a message for the same and the wizard will attempt to enable the same.

Thats all there is to it. Just hit Ok and then hit Save to apply the rule.

Azure SQL Databases have a powerful layer of security at the SQL Server level. This layer is provided by the SQL Server Firewall. Azure provides you granular control to configure this firewall and to manage who gets access to your Azure SQL Database. By default, everything is blocked by the firewall. If you want to get access to Azure SQL Database then you will have to configure the Firewall at the SQL Server level. Only the IP addresses you configure have access to the SQL Databases on the Server.

Another key point to understand is that once you configure a rule then because that rule is applied at the server level, it is applied to all the SQL Databases on that Server. So it is important to ensure that you segregate your databases on different SQL Servers if you don't want to share the access to those databases.

Accessing the Firewall Settings

You can access the firewall settings by navigating to your Azure SQL Database. Then at the top of the blade, you will find the option for "Set server firewall". Click on this button to access the firewall settings.

Another way to access the settings is on Azure SQL Servers. Navigate to the related Azure SQL Server for your database. Under the settings, find the option for "Firewalls and virtual networks". Clicking on this will also take you to the same firewall settings as the settings are set at the server level in both ways.

Firewall Rule Settings

Let us look at the firewall rule settings in more details.

1. First, you have the option to enable or disable the access to Azure services as the toggle for "Allow access to Azure services". This is set to On by default when creating the Database and SQL server. You can disable it here. This option allows Azure services to provide monitoring data and recommend changes at the database level.
2. Second, you have the option to configure a Rule. This is where you configure which IP Addresses have access to the SQL Server.
3. One commonly used Rule is to open the access for Client IP. This is the IP address of the machine from where you are connected to the Azure Portal. This is provided for the common scenario where you want to connect and access the SQL database via SSMS (i.e. SQL Server Management Studio) or by any other way from your development box. There is a button to simply add the rule for allowing Client IP by clicking on "+Add client IP". The client IP is also authomatically displayed under the "Allow access to Azure services" section
4. Lastly, there is an option to allow access to the SQL databases from a particular Subnet in a Virtual Network in Azure, without having to manually configure their IP addresses. This feature requires Service Endpoints to be configured at the Virtual Network level. We will discuss this feature in a later blog.