vSphere 5.0x ESXi hosts crash when using Trend Micro Deep Security

The environment in question is running ESXi 5.0; using Deep Security(a vShield Endpoint API product by trend micro)… not very sure how common this implementation is.

Blades were crashing intermittently every few days with this resultant PSOD :

The backtrace points to a known symptom seen after installing Trend Micro Deep Security (reference kb.vmware.com/kb/2012584 )

The KB above mentions that we should expect to see dvifilter-dsa errors in the vmkernel log;

Expected entries were found in the log:
2012-08-07T23:25:31.107Z cpu16:771421)dvfilter-dsa: tb_trace_write_formatted:105: alloc_guest:214 guest alloc guid: 420449f6-9a18-8fc5-b116-a32a4c45afe4, domid: 3999995

VMware points to trend micro KB 1060125 for the solution. In a nutshell you have to disable the timer setting causing the problem.

 

1. From the ESXi console, execute this command to find out the value that is configured for the Filter Driver heap memory size:

% esxcfg-module -g dvfilter-dsa

If the value for the “DSAFILTER_HEAP_MAX_SIZE” is adjusted from its default value then the outcome will be similar to:
dvfilter-dsa enabled = 1 options = ‘DSAFILTER_HEAP_MAX_SIZE=134217728’

2. Use this command to disable timer and preserve the configured value for the DSAFILTER_HEAP_MAX_SIZE:
% esxcfg-module -s “DSAFILTER_HEAP_MAX_SIZE= 134217728 DSAFILTER_MOD_TIMER_ENABLED=0” dvfilter-dsa
Note:
Set the DSAFILTER_HEAP_MAX_SIZE to the value that was observed after running the “esxcfg-module -g dvfilter-dsa” command.

If the value for the “DSAFILTER_HEAP_MAX_SIZE” is not changed from its default value then the outcome will be similar to:
dvfilter-dsa enabled = 1 options = ‘ ‘
In this case you can use the following command to disable timers:
% esxcfg-module -s “DSAFILTER_MOD_TIMER_ENABLED=0” dvfilter-dsa

 

3. Verify if the settings were successfully applied by executing this command:
% esxcfg-module -g dvfilter-dsa
4. Reboot the ESXi server for the changes to take effect.

Note:
The setting will not take effect until the driver is reloaded. Reloading will require a reboot (best option) of ESXi or unloading/loading of the driver.

This workaround is not needed if the environment is running Trend Micro Deep Security 7.5 SP4, 8.0 SP1 and later.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s