HP t730 Thin Client and Broadcom BCM5719 NICs

Posted 2020-08-10 13:17 by Traesk. 35 views.

I recently purchased a HP t730 Thin Client to use as a pfSense server. I was unable to get BCM5719 working with this PC and thought I'd share my experiences.

Using pfSense v2.4.5, VMware ESXi v6.7 or Windows 10 the BCM5719 NICs seemed to work fine initially, but after a while all network ports became unresponsive.
Sometimes the NIC froze directly at boot but one time it took as long as 24 hours before it stopped working.
I did not find any logs in Windows 10, but the errors in pfSense and VMware were as follows:

pfSense (seen both on connected monitor and when connected via SSH):
bge0: watchdog timeout -- resetting
bge3: APE lock 5 request failed! request = 0x8414[0x1000], status = 0x8434[0xffff]

ESXi (seen in vmkernel logs):
WARNING: ntg3: Ntg3PhyRead:369: vmnic0:PHY reg (25) read timeout

I also noticed some dropped packets during the time the NICs were responsive, not sure if that issue is related.

Dell TMGR6 BCM5719 was the first card I bought. The following did not make any difference:
  • Updating to latest firmware from Dell
  • Crossflashing to HP 331i (3372) firmware (found here) using this tool (thank you!)
  • Removing the PCIe-riser in the t730 and connecting directly to the system board. It seems this is no normal PCIe slot and the riser is required - card was not found when directly connected
  • Updating the t730 to the latest BIOS (v00.01.15 Rev.A)
  • Changing various BIOS settings on the t730 (ex. PCI SERR# Generation and PCI VGA Palette Snooping)
  • Installing Windows 10 and installing every driver available from HP
  • Disabling MSI/MSIX (hw.pci.enable_msix=0, hw.pci.enable_msi=0) and TSO (ifconfig bge0 -tso) in pfsense
  • Disabling MSI (esxcli system module parameters set -m ntg3 -p intrMode=0) in ESXi
Unfortunately I thought it was an issue with the Dell card, so I bought another BCM5719: HP 331T HSTNS-BN82 649871-001.
The HP card had exactly the same issues as the Dell card.
I did try a firmware update for the HP card before I concluded that the issue must be the combination of t730 and BCM5719.

I ended up buying a Dell Intel I350-T4 9YD6K. It adds 30 seconds or so to POST, but other than that it's worked flawlessly for about two months now.

If you're in need of SR-IOV, you might want to do some further research before buying this card though.
Dell initially removed support for SR-IOV in their editions of I350-T4. More info on Intel's forum.
But in the release notes for Intel NIC and NDCs Firmware Family Version 15.0.1 (2013-12-13) it states:
Enhancements:
1. Added SR-IOV support on 1G (I350 based adapters)

This firmware is newer than the thread on Intel's forums.
However, I'm not able to test whether it's properly supported now or not. I barely know what it's for or whether it actually works with the t730. Just thought I'd mention it in case it's of interest to anyone.

Another issue with the t730 is that VMware ESXi v7.0 does not seem to work.
When doing a clean install, the installation is successful, but upon reboot BIOS will give an error that no boot device is found.
When I installed v6.7 and upgraded to v7.0 it did boot, but after a couple of minutes it rebooted and BIOS was not able to boot again.
When playing around with some other things I did not get GPT disk with legacy boot partition to boot on the t730. The issue with ESXi might be that the t730 doesn't understand how to boot it, but I've not investigated this any further.

Comments
No comments yet.