Blazor Server 'Error: server timeout elapsed without receiving a message from the server.' even when sending my own keep alive message

I've been trying to resolve server timeout disconnects in my Balzor Server app for many months now. I've tried lots of different things, but nothing has worked thus far. I'm just about done with the application, but i can't fix these disconnects.

The client application thinks it hasn't received a message from the server and "Attempting to reconnect to the server: 1 of 8" start appears every 30 seconds or so. It successfully reconnects every time, but I need these reconnect messages to stop.

Here is an example of the SignalR configuration that I've tried unsuccessfully.

     Blazor.start({         reconnectionOptions: {             maxRetries: 100,             retryIntervalMilliseconds: 2000         },         configureSignalR: function (builder) {             let c = builder.build();             c.serverTimeoutInMilliseconds = 30000;             c.keepAliveIntervalInMilliseconds = 10000;             builder.build = () => {                 return c;             };         }     });

By adjusting the time durations in this code I could change the frequency of the disconnects, but they still happen.

I decided to send my own "FakeKeepAlive" message to the client.

    private async Task StartFakeKeepAlive(CancellationToken ct)    {        try        {            using (var timer = new PeriodicTimer(TimeSpan.FromMilliseconds(Utils.GlobalKeepAliveIntervalInMilliseconds)))            {                while (!ct.IsCancellationRequested)                {                    await timer.WaitForNextTickAsync(ct);                    now = DateTime.Now;                    Debug.WriteLine($"FakeKeepAlive {now.ToString("h:mm:ss.FFF")} ");                    await JS.InvokeVoidAsync("console.log", "CSDKeepAlive: At ", DateTime.Now.ToString("h:mm:ss.FFF"));                    await InvokeAsync(StateHasChanged);                }            }        }        catch (Exception ex)        {            logger.Warn(ex, "StartFakeKeepAlive() exception");            throw;        }    }

Note that I tried InvokeAsync(StateHasChanged) and JS Interop to send a console.log message to the client and both of these.

This helped in that it delayed the start of the disconnects, but after about 5 minutes the disconnects reappeared.

Here is the Chrome Console output:

[2024-04-15T20:21:24.628Z] Information: Normalizing '_blazor' to 'https://localhost:5001/_blazor'. blazor.server.js:1
[2024-04-15T20:21:24.673Z] Information: WebSocket connected to wss://localhost:5001/_blazor?id=fulG9OiaSbyn6AO1yCwIwQ. blazor.server.js:1
CSDKeepAlive: At 4:22:01.339 blazor.server.js:1
CSDKeepAlive: At 4:22:31.333 blazor.server.js:1
CSDKeepAlive: At 4:23:01.331 blazor.server.js:1
CSDKeepAlive: At 4:23:31.326 blazor.server.js:1
CSDKeepAlive: At 4:24:01.331 blazor.server.js:1
CSDKeepAlive: At 4:24:31.328 blazor.server.js:1
CSDKeepAlive: At 4:25:01.331 blazor.server.js:1
CSDKeepAlive: At 4:25:31.327 blazor.server.js:1
CSDKeepAlive: At 4:26:01.329 blazor.server.js:1
CSDKeepAlive: At 4:26:31.327 blazor.server.js:1
CSDKeepAlive: At 4:27:01.329 blazor.server.js:1
CSDKeepAlive: At 4:27:31.339 blazor.server.js:1
CSDKeepAlive: At 4:28:01.327 blazor.server.js:1
[2024-04-15T20:28:31.332Z] Error: Connection disconnected with error 'Error: Server timeout elapsed without receiving a message from the server.'.
log @ blazor.server.js:1
_stopConnection @ blazor.server.js:1
features.reconnect.transport.onclose @ blazor.server.js:1
_close @ blazor.server.js:1
stop @ blazor.server.js:1
_stopInternal @ blazor.server.js:1
await in _stopInternal (async)
stop @ blazor.server.js:1
serverTimeout @ blazor.server.js:1
(anonymous) @ blazor.server.js:1
setTimeout (async) _resetTimeoutPeriod @ blazor.server.js:1
_processIncomingData @ blazor.server.js:1
Xt.connection.onreceive @ blazor.server.js:1
s.onmessage @ blazor.server.js:1
blazor.server.js:1
[2024-04-15T20:28:34.341Z] Information: Normalizing '_blazor' to 'https://localhost:5001/_blazor'. blazor.server.js:1
[2024-04-15T20:28:34.354Z] Information: WebSocket connected to wss://localhost:5001/_blazor?id=VbjkvE9CAzGk89_vdVcqng. blazor.server.js:1
[2024-04-15T20:29:07.836Z] Error: Connection disconnected with error 'Error: Server timeout elapsed without receiving a message from the server.'.
{Details omitted, but same as above}
[2024-04-15T20:29:10.843Z] Information: Normalizing '_blazor' to 'https://localhost:5001/_blazor'. blazor.server.js:1
[2024-04-15T20:29:10.850Z] Information: WebSocket connected to wss://localhost:5001/_blazor?id=5MLFdZ6qPPwEdrn5djWidg.

Here is the relevant code from program.cs. Note the commented out code is an attempt to set the keep alive interval here. The above examples use the default ServerTimeout and KeepAliveIntervals.

    //builder.Services.AddServerSideBlazor().AddHubOptions(option =>    //    {    //        option.KeepAliveInterval = TimeSpan.FromMilliseconds(Utils.GlobalKeepAliveIntervalInMilliseconds);    //    });    builder.Services.AddServerSideBlazor().AddCircuitOptions(option =>        {            option.DetailedErrors = true;        });

Here is the the entire packages section from my .csproj file.

<ItemGroup><PackageReference Include="Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore" Version="8.0.3" /><PackageReference Include="Microsoft.AspNetCore.Identity.EntityFrameworkCore" Version="8.0.3" /><PackageReference Include="Microsoft.AspNetCore.Identity.UI" Version="8.0.3" /><PackageReference Include="Microsoft.EntityFrameworkCore.Relational" Version="8.0.3" /><PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="8.0.3" /><PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="8.0.3"><PrivateAssets>all</PrivateAssets><IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets></PackageReference><PackageReference Include="MudBlazor" Version="6.15.0" /><PackageReference Include="NLog" Version="5.2.8" /><PackageReference Include="NLog.Web.AspNetCore" Version="5.3.8" /><PackageReference Include="Radzen.Blazor" Version="4.24.5" /><PackageReference Include="Versus_NAPI_Core" Version="4.0.0-20230918.7" /></ItemGroup>enter image description here

Here are the rest of the dependencies:

Screenshot of the other Visual Studio Dependencies

I've been chipping away at this for weeks. I believe I've read every Stack Overflow post that might have something to do with this and all of the Microsoft documentation. My experience is inconsistent with everything I've read, so I'm wondering if there is something more deep under the hood that I'm missing.

I can't discern any reason why the disconnects only start after 5 minutes and why my 'FakeKeepAlive" messages don't get though after the connection is reset.

For what its worth, I started the application in .NET 7.0 but upgraded to .NET 8.0. This issue occurred before I upgraded.