Friday, May 30, 2014

Setting the Hostname in a SysPreped AMI

When you create an Windows AMI (Amazon Machine Image) it is configured to generate a random server name.  Often this name does not meet your needs.  Maybe your company has a specific naming convention (e.g US-NYC-1234) or you just want to use a descriptive name (e.g. WEB01).  Whatever the reason, let's look at how to set the name when you launch the machine.

In this post we will use PowerShell to read the name from a Tag on the instance.  When done, you set the hostname in launch wizard by simply filling in the Name tag.  See the image below.  Our script will read this tag and rename the server when it boots for the first time.



It is important to automate the name change.  As your cloud adoption matures, you quickly realize that you cannot have an admin log in and rename the server when it's launched.  First, it takes too long.  Second, you want servers to launch automatically, for example, in response to an auto-scaling event.

So how can you set the name?  You will find a ComputerName element in the SysPrep2008.xml file that ships with the EC2 Config Service (or in the unattended.xml file if you're not using the EC2 Config Service.)  The computer name is in the specialize section.  In the snippet below, you can see the default value of "*".  The star means that windows should generate a random name.

<component language="neutral" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" versionScope="nonSxS" publicKeyToken="31bf3856ad364e35" processorArchitecture="amd64" name="Microsoft-Windows-Shell-Setup">
    <ComputerName>*</ComputerName>
    <CopyProfile>true</CopyProfile>
    <RegisteredOrganization>Amazon</RegisteredOrganization>
    <TimeZone>Eastern Standard Time</TimeZone>
</component>

If you want to change the name you can simply hard-code whatever you want here.  Of course, if you hard-code if before you run SysPrep, every machine you create from the AMI will have the same name.  That's not what we want.  So the trick is to set the name when the machine first boots and before specialize runs.

Let's quickly review how SysPrep works.  When you run SysPrep, it wipes any identifying information form the machine (e.g. Name, SIDs, etc.)  This is known as the generalize phase.  After the generalize phase you shutdown the machine and take the image.

When a SysPreped image first boots, it runs windows setup (WinDeploy.exe).  This is known as the specialize phase.  If you have ever bought a new home computer, you have experienced the setup wizard that allows you to configure your timezone, etc.  In the cloud you cannot answer questions so you have to supply an unattended.xml file with the answers to all the questions.  

We need to inject our script into the specialize phase before setup runs.  Our script will get the machine name from the EC2 API and modify the unattended.xml file.  Here is a sample script to do just that.  The script has three parts.

  • The first part uses the meta-data service to discover the identity of the instance and the region the machine is running in.
  • The second part of the script uses the EC2 API to get the name tag from for the instance.  Note that I have not included any credentials.  I assume that the instance is in a role that allows access to the Get-EC2Tag API call.
  • The third part of the script modifies the unattended.xml file.  This is the same file shown earlier.  The script simply finds the ComputerName node and replaces the * with the correct name.


Write-Host "Discovering instance identity from meta-data web service"
$InstanceId = (Invoke-RestMethod 'http://169.254.169.254/latest/meta-data/instance-id').ToString()
$AvailabilityZone = (Invoke-RestMethod 'http://169.254.169.254/latest/meta-data/placement/availability-zone').ToString()
$Region = $AvailabilityZone.Substring(0,$AvailabilityZone.Length-1)

Write-Host "Getting Tags for the instance"
$Tags = Get-EC2Tag -Filters @{Name='resource-id';Value=$InstanceId} -Region $Region
$InstanceName = ($Tags | Where-Object {$_.Key -eq 'Name'}).Value
Write-Host "`tFound Instance Name: $InstanceName"
Write-Host "`tFound Instance Owner: $InstanceOwner"

If($InstanceName -ne $null) {
 Write-Host "Setting the machine name to $InstanceName"
 $AnswerFilePath = "C:\Windows\Panther\unattend.xml"
$AnswerFile = [xml](Get-Content -Path $AnswerFilePath) $ns = New-Object System.Xml.XmlNamespaceManager($AnswerFile.NameTable) $ns.AddNamespace("ns", $AnswerFile.DocumentElement.NamespaceURI) $ComputerName = $AnswerFile.SelectSingleNode('/ns:unattend/ns:settings[@pass="specialize"]/ns:component[@name="Microsoft-Windows-Shell-Setup"]/ns:ComputerName', $ns) $ComputerName.InnerText = $InstanceName $AnswerFile.Save($AnswerFilePath) }

So how do we get this script to run before setup? That's the tricky part.  Let's dig a bit deeper.  I said earlier that when a SysPreped image first boots it will run WinDeploy.exe.  To be more specific, it will run whatever it finds in the HKLM:\System\Setup registry key.  SysPrep will put c:\Windows\System32\oobe\windeploy.exe in the registry key before shutdown.

So we need to change that registry key after SysPrep runs, but before the system shuts down.  To do that we need to pass the /quit flag rather than /shutdown.  I'm writing about AWS, so I assume you are calling SysPrep from the EC2Config service.  If you are, you need to edit the switches element of the  BundleConfig.xml file in the EC2Config folder.  The switches element is about midway down the file.  See the example below.  Just remove /shutdown and replace it with /quit.

<Sysprep version="6.0">
      <PreSysprepRunCmd>C:\Program Files\Amazon\Ec2ConfigService\Scripts\BeforeSysprep.cmd</PreSysprepRunCmd>
      <ExePath>C:\windows\system32\sysprep\sysprep.exe</ExePath>
      <AnswerFilePath>sysprep2008.xml</AnswerFilePath>
      <Switches>/oobe /shutdown /generalize<syspr/Switches>
</Sysprep>

Alright, we are almost there.  Now you can run SysPrep and it will give you a chance to make changes before shutting down.  You want to replace the HKLM:\System\Setup registry key with the script we created above.  Don't forget to add a line to call WinDeploy.exe at the end of the script.

With all that done (it's not as bad it sounds) you can shutdown and take an image.  It will take a few tries to get all this working correctly.  I recommend that you log the output of the script using Start-Transcript.  If the server fails to boot you can attach the volume to another instance and read the log.

34 comments:

  1. Thanks for sharing this, much appreciated.

    ReplyDelete
  2. Good article.

    However, I'm currently stuck at trying to run Sysprep with /quit. Even when I manually run sysprep.exe /oobe /quit /general /unattend:file.xml I am kicked from the RDP session before sysprep completes.

    Perhaps the network connection is reprovisioned? Any ideas?

    ReplyDelete
    Replies
    1. Did you happen to figure this out?

      Delete
  3. I was able to reproduce the error described in the comments above. I'm not sure what changed and didn't have time to investigate. As a workaround, I created a batch file that will run sysprep and the modify the registry. It will still kick you out after sysprep runs but the batch file will finish the work and then shutdown. Just download the zip file from the link below and unzip it on the C: drive. Then run the sysprep.bat file. I hope that helps.

    https://dl.dropboxusercontent.com/u/13738331/Blog/SysPrep.zip

    ReplyDelete
    Replies
    1. Brian,
      The Dropbox link to your SysPrep batch file is broken. Do you have an updated link, or are you able to paste a copy of the batch file contents in a comment here?
      Thanks!

      Delete
    2. So I have been at AWS for almost three years now. I guess it's time to drink the coolaid. https://s3.amazonaws.com/blog.brianbeach.com/downloads/SysPrep.zip

      Delete
  4. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
  5. I believe you get the IMAGE_STATE_UNDEPLOYABLE message if sysprep has been run more than three times. It should complete despite the warning.

    ReplyDelete
    Replies
    1. Yes, sorry... It worked, just had to wait some time.
      Do you know why if I change Ec2SetPassword to Disabled in your config.xml, after I launch an instance with the new AMI, I can't logon with the old password?

      Delete
    2. Is it because of "ScramblePassword" command on sysprep2008.xml?

      Delete
  6. You should either hard code a password in the sysprep file (bad security) or enable Ec2SetPassword (good security). If you do neither of these sysprep generates a random password and you are locked out. If you read the fine print in the EC2 config service it explains this. ScramblePassword command should always be in the sysprep. It will check if the it is configured to run in the config.xml file before scrabling the password.

    ReplyDelete
  7. Brian,
    I wanted to use your information to name and then join a domain. The domain joining portion that would be placed into sysprep2008.xml is as a follows:

    *
    true
    Amazon
    Eastern Standard Time



    "domain name"
    "Password"
    "Username that can join machines to domain"

    "domain name"
    "PathWithinAD_YouWAntComputersPLaced
    False



    I know your original article did not have joining domains in mind but "IF YOU DID", how would you go about it? Your post publishers removes special characters.

    ReplyDelete
  8. Hi Brian,

    We are trying your steps on a windows beanstalk environment and this method doesn't seem to be stable (could be a beanstalk thing). I made a baked ami using your sysprep scripts and many times it fails on the invoke-webrequest to 'http://169.254.169.254" (network errors and not credential errors). Ever seen that before?

    ReplyDelete
    Replies
    1. You can eliminate that error by adding the -UseBasicParsing flag. For example, to return the local instance id:

      $InstanceID = (Invoke-WebRequest -UseBasicParsing '169.254.169.254/latest/meta-data/instance-id').Content

      Delete
  9. The Batch folder was a nice touch. Thank you.
    All is fine with the AMI creation - set the name tag upon instance launch for the computer name.

    Windeploy.ps1 - log gives the following:
    PS>TerminatingError(Get-EC2Tag): "No credentials specified or obtained from persisted/shell defaults."

    Running the PS locally, I get more details:
    + CategoryInfo: InvalidOperation: (Amazon.PowerShe...GetEC2TagCmdlet:GetEC2TagCmdlet) [Get-EC2Tag], InvalidOperationException

    + FullyQualifiedErrorId: InvalidOperationException,Amazon.PowerShell.Cmdlets.EC2.GetEC2TagCmdlet

    I can verify that the script is getting the InstanceID and Region.

    Thoughts?

    ReplyDelete
    Replies
    1. It sounds like your instance doesn't have the credentials to make those Get-EC2Tag API calls to AWS. Try ensuring that the the instance has an IAM profile associated:

      http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html

      Delete
  10. Spires,

    Sounds like you don't have credentials configured. I typically have the instance added to an EC2 Instance Profile Role. Alternatively you could add credentials using Set-AWSCredentials.

    ReplyDelete
    Replies
    1. Thank you, Travis and Brian. I appreciate your response.
      IAM configured and I'm able to set the computer name upon launch of the instance using tags.

      The name change seems to not be accessible across the network. Is there a step to update the DNS with the new computer name?

      Delete
    2. Does this make sense?

      Before, when the instance assigned its own name based on the IP, I had no issue networking two instances together (such as sharing a network drive \\{randomized-computer-name}\netdrive.) Now, I assign the computer name and it's not recognized on the network by name - it is, however, recognizable by IP. Something isn't registering. Thoughts?

      Delete
    3. Spires, The name you are setting is local to the instance. It does not update DNS. The best solution is to join to instance to AD (and as a result DNS). If you do not have AD in your VPC you may need to add entries to the host file. Note that two machines in the same subnet should find one another using NetBIOS broadcasts. But outside the subnet you need a service like DNS or WINS to enable name resolution. .

      Delete
    4. Hi
      I didn't have any network access during OOBE,before WinDeploy runs. So I couldn't lookup instance tags before generalize phase runs. Were you able to lookup tags and update unattend.xml before generalize?
      Some stuff I found useful for debugging:
      * net user admin Password01 /add
      * net localgroup administrators admin /add
      * VNC server (see any error messages during OOBE)
      * Windows System Image Manager (WSIM) from ADK to validate sysprep answer files

      Delete
    5. Hey Brian,

      I am unable to get to that zip link you provided above. Is it still active?

      Delete
    6. Hi Brian,
      Thank you for the Tutorial.
      It helps too much.
      I really did the same steps but the problem that i have.
      Sysprep can't get the instance metadata.
      It keeps trying but it failed with the errors below:
      1)
      Failed to fetch instance metadata
      http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key with exception The remote server returned an error: (404) Not Found.
      2)
      Check and re-enable Ec2WindowsActivate plugin if last activation was against old kms configuration
      3)
      Ec2SetPassword: Password Error: Getting public openssh-key failed with exception
      System.Net.WebException: The remote server returned an error: (404) Not Found.
      at System.Net.HttpWebRequest.GetResponse()
      at Ec2ConfigLibrary.Model.InstanceMetaDataHndlr.GetMetaData(String metadataUrl, Boolean log)
      at Ec2Config.LegacyConfiguration.Services.SetPassword.SetPasswordService.GetMetaWithRetries()

      Can you please provide me any changes that i have to do.

      Delete
    7. My script is not asking for the openssh-key so this error is likely coming from somewhere else. Where did you see this error?

      Do you have a key pair assigned to this instance to encrypt the randoom admin password or do you have a known password.

      Delete
    8. Brian,
      here is the err that i found on
      c:\windows\panther\setuperr.log file

      2016-09-27 18:19:35, Error CSI 00000001 (F) 80220005 [Error,Facility=FACILITY_STATE_MANAGEMENT,Code=5] #57# from CWcmScalarInstanceCore::PutCurrentValue(options = 0, value = { type: 8204 (0x0000200c), bytes ( 38 (0x00000026) ): 6400650076002d007700650062002d0069006e007300740061006e00630065002d0032000000 })
      [gle=0x80004005]
      2016-09-27 18:19:35, Error [setup.exe] SMI data results dump: Source = Name: Microsoft-Windows-Shell-Setup, Language: neutral, ProcessorArchitecture: amd64, PublicKeyToken: 31bf3856ad364e35, VersionScope: nonSxS, /settings/ComputerName
      2016-09-27 18:19:35, Error [setup.exe] SMI data results dump: Description = Value is invalid.
      2016-09-27 18:19:35, Error [0x060432] IBS The provided unattend file is not valid; hrResult = 0x80220005
      2016-09-27 18:19:35, Error [0x060565] IBS Callback_Unattend_InitEngine:The provided unattend file [C:\Windows\Panther\unattend.xml] is not a valid unattended Setup answer file; hr = 0x1, hrSearched = 0x1, hrDeserialized = 0x0, hrImplicitCtx = 0x0, hrValidated = 0x1, hrResult = 0x80220005
      2016-09-27 18:19:35, Error [0x0600c2] IBS Callback_Unattend_InitEngine:An error occurred while finding/loading the unattend file; hr = 0x1, hrResult = 0x80220005[gle=0x00000490]


      Thanks in advance!

      Delete
  11. Worked at first try with the zip file here. Magnificent contribution, thanks a lot.

    ReplyDelete
  12. Worked at first try here with the zip file (Win2012 R2 64 bits in EC2). Magnificent contribution.

    ReplyDelete
  13. Brian,
    I know this is not 100% what you are talking about but is it possible to use cloud formation and not make the changes to the registry and use sysprep? and if so how would you do that?

    ReplyDelete
  14. Alas, as of late it appears that all Windows instance classes lack access to the metadata provider during the OOBE phase (something we'd seen only with the R3 class before.) Without that, it's tough to do much in the way of meaningful customization - pity, as this approach has been very useful to us so far. C'est la vie!

    ReplyDelete
  15. thank you Brian for the quick answer.
    Now i made some changes and everything is working fine but the problem is:
    I am using Terraform to build instances, when i spin up 7 instances 2 of them work fine and they have the hostname changed but the other they have the error message below:
    "Windows could not parse or process unattend answer file [C:\Windows\Panther\unattend.xml]. The answer file is invalid"
    i checked the unattend file in these instance and i found that the computerName is changed on them but it didn't work.
    I don't know what is the issue or id there is another way to change the hostname.


    Thank you in advance and if there is any other way to change the hostname on AWS windows instances as i need that to join these instances to the domain.


    ReplyDelete
  16. Brian,
    Thank you for the article.
    It helped so much.
    Everything works fine in your article and the only thing that i really have to test again is the comment on unattend.xml file generated by AWS.

    the first thing i added on the windeploy script is to delete that comment and keep just the *.
    The second thing and the most important is to keep the hostname under the standard: length of hostname is maximum of 15 characters and doesn't contain:
    { | } ~ [ \ ] ^ ' : ; < = > ? @ ! " # $ % ` ( ) + / . , * &, or contain any spaces.
    you can check the link below for more information.
    And thank you again Brian.

    https://technet.microsoft.com/en-us/library/ff715676.aspx

    ReplyDelete