Nothing much to do on my side and application team could not identified the problem. So i have decided to lodge a case with Microsoft PSS to find the root cause. The purpose of this article is to share some step that you should take if you would like to report issue to PSS on Web Server issue.
1. Go to https://partner.microsoft.com/global/supportsecurity/40032273 and call the toll free number according to your country.
2. Fill up incident template which send by PSS and a SR number will be given
3. Collect IIS hang dump. To do this follow below instruction
a)Download and install the Microsoft Debuggers
Download and install the MS Debugging Tool from http://msdl.microsoft.com/download/symbols/debuggers/dbg_x86_6.11.1.404.msi
NOTE: You do not have to install them on your server, but can install on a workstation and then XCOPY the directory to your server. They will work on your server after you have done this without rebooting the server. This debug tool work fine when running on IIS 6.
b) Run it when the high CPU issue occurs.
Run AutoDump Plus from a command line (cmd.exe). In the debuggers directory (Default is c:\Program Files\Debugging Tools For Windows) run the command (changing the path where you would like the files created in):
cscript.exe adplus.vbs -hang -iis -o c:\outdirhang –quiet
Please ignore any warning and collect the hang dump for 3 times with 5 minutes interval between each time.
4. Send event log.
a)Go to Run and type “eventvwr”
b)Right click the Application (Security, System) and click Save Log File As…The Save as type is .evt.
5. Send IIS log
a) Go to IIS manager MMC.
b) Right click the problematic web site and click Properties.
c) Under Web Site tab, click Properties and find the “Log file directory”
6. Compress the event viewer, IIS Log and Dump file as the size is quite huge. Microsoft PSS will send you the workspace to upload the file.
7. Once they receive the log and dump file, they will begin to analyse it.
There are two Severity when report a case, by default
a) Severity A - Support after 6pm
b) Severity B - Support from 9am to 6pm.
You should raise the severity if require. Once the severity is raise from B to A, you need to pay for the support.
Finding from PSS:-
a) Application coding issue especially on string operation issue which causing the CPU to high spike.
Now the ball is pass back to Application team to resolve the issue on the coding side.
On my side, i just perform fine tune on the application pool to Recycle every 1 hour.
At this moment, with 400+ people access to the 2vcpu and 5GB Memory Virtual Machine running on Hyper V platform, the user experience is still good. The Average CPU utilization is 60%.
Not bad when you convert 4 Quad Core (16 cores) CPU and 32GB of RAM into 2 vCPU with 5 GB RAM.
Cheer for Hyper V!
In my next article, i will share some performance counter which i use to monitor.
Stay Tuned.
