Windows Event ID 4978 – Security: IPsec Main Mode Negotiation Failed

Start by examining the specific failure details recorded in the Security log:

1. Open Event Viewer → Windows Logs → Security
2. Filter for Event ID 4978 using the filter option
3. Double-click the most recent 4978 event to view details
4. Note the Source Address, Destination Address, and Failure Point fields
5. Check the Authentication Method and Error Code values

Use PowerShell to extract detailed information:

Get-WinEvent -FilterHashtable @{LogName='Security'; Id=4978} -MaxEvents 10 | Format-List TimeCreated, Message

For more detailed analysis with specific properties:

Get-WinEvent -FilterHashtable @{LogName='Security'; Id=4978} | ForEach-Object { [xml]$xml = $_.ToXml(); $xml.Event.EventData.Data }

Check local IPsec policies and compare with remote endpoint requirements:

1. Open Windows Defender Firewall with Advanced Security
2. Navigate to Connection Security Rules to review active policies
3. Right-click problematic rules and select Properties
4. Verify Authentication tab settings match remote requirements
5. Check Advanced tab for protocol and port configurations

Use PowerShell to examine IPsec policies:

Get-NetIPsecRule | Where-Object {$_.Enabled -eq 'True'} | Format-Table DisplayName, Profile, Direction

Check Main Mode cryptographic settings:

Get-NetIPsecMainModeCryptoSet | Format-List DisplayName, Proposal

Verify authentication methods:

Get-NetIPsecAuthProposal | Format-List DisplayName, AuthenticationMethod

Verify that IPsec traffic can reach the remote endpoint:

1. Test basic connectivity to the remote IP address:

Test-NetConnection -ComputerName [RemoteIP] -Port 500 -InformationLevel Detailed

2. Check NAT-T connectivity on port 4500:

Test-NetConnection -ComputerName [RemoteIP] -Port 4500

3. Verify Windows Firewall rules allow IPsec traffic:

Get-NetFirewallRule -DisplayName '*IPsec*' | Where-Object {$_.Enabled -eq 'True'}

4. Check for blocking rules that might interfere:

Get-NetFirewallRule | Where-Object {$_.Action -eq 'Block' -and $_.Enabled -eq 'True'} | Format-Table DisplayName, Direction

5. Use netsh to verify IPsec monitor status:

netsh ipsec dynamic show all

When using certificate-based authentication, verify certificate validity and trust chains:

1. Open Certificate Manager (certmgr.msc) and check the Personal store
2. Verify the IPsec certificate is present and valid
3. Check certificate expiration dates and ensure they're within validity period
4. Validate the certificate chain in Trusted Root Certification Authorities

Use PowerShell to examine certificates:

Get-ChildItem -Path Cert:\LocalMachine\My | Where-Object {$_.EnhancedKeyUsageList -match 'IP security IKE intermediate'}

Check certificate validity and expiration:

Get-ChildItem -Path Cert:\LocalMachine\My | Select-Object Subject, NotAfter, Thumbprint | Format-Table

For pre-shared key authentication, verify the key configuration:

1. Open Local Security Policy (secpol.msc)
2. Navigate to IP Security Policies on Local Computer
3. Right-click the active policy and select Properties
4. Check authentication methods in the policy rules

Enable comprehensive IPsec logging for detailed troubleshooting:

1. Enable IPsec audit logging in Group Policy or Local Security Policy:

Navigate to Computer Configuration → Windows Settings → Security Settings → Advanced Audit Policy Configuration → System Audit Policies → Logon/Logoff

2. Enable Audit IPsec Main Mode for both Success and Failure
3. Configure advanced IPsec logging using netsh:

netsh ipsec dynamic set config ipsecloginterval=60

4. Enable IKE logging for detailed negotiation analysis:

netsh wfp set options keywords=IKEV1+IKEV2

5. Use Windows Performance Toolkit for packet-level analysis:

netsh trace start capture=yes provider=Microsoft-Windows-WFP tracefile=ipsec_trace.etl

6. Reproduce the issue, then stop tracing:

netsh trace stop

Analyze the trace file using Network Monitor or Wireshark for detailed packet inspection.