xp_cmdshell: a call to 'CreateProcess' failed with error code: '5'
Problem:
SQL Server 2005 error: xp_cmdshell: a call to 'CreateProcess' failed with error code: '5'. We get above error when trying to execute XP_CMDSHELL even though it has been enabled using Surface Area Configuration tool and the login has SYSADMIN privileges.
Solution:
First of all look at the error message and number generated, 5 relates to access denied. You have to identify whether you get this error message at all the times or only during specific query execution that uses XP_CMDSHELL. Also it is better to check the privileges for the SQL Server service account.
Further check with the Windows server local security policy & domain security policy are not blocking the C:\WINDOWS\SYSTEM32\CMD.EXE process. Ensure the relevant login uses this statement has Read & Execute permission associated to this process. If you are using a Local System account for SQL Server services with the local security policies enabled on a domain, then ensure to start the SQL Server services with a Network Service account where the Administrator login will be used to perform such command-line operations in SQL Server.