Aspire NET 9 Blazor App: Increasing Polly Retry for HttpClientFactory while using a Http Message Handler

I'm trying to increase my retry duration from the standard 10 seconds in my Blazor application but the resilience handler doesn't seem to be working. I've tried both of the setups below but my Http request is still attempting to retry after 10 seconds. I do have a message handler attached to my client. Does the piping order matter?

My Program.cs file looks like this:

builder.Services.AddHttpClient("api-client", o =>{    o.BaseAddress = new("https+http://myapi");}).AddHttpMessageHandler<KoHttpClientDelegate>().AddStandardResilienceHandler(o =>{    o.CircuitBreaker.SamplingDuration = TimeSpan.FromSeconds(240);    o.TotalRequestTimeout.Timeout = TimeSpan.FromSeconds(240);    o.AttemptTimeout.Timeout = TimeSpan.FromSeconds(120);    o.Retry.MaxDelay = TimeSpan.FromSeconds(120);});

I also tried this:

builder.Services.AddHttpClient("api-client", o =>{    o.BaseAddress = new("https+http://myapi");}).AddHttpMessageHandler<KoHttpClientDelegate>().AddStandardResilienceHandler().Configure(o =>{    o.CircuitBreaker.SamplingDuration = TimeSpan.FromSeconds(240);    o.TotalRequestTimeout.Timeout = TimeSpan.FromSeconds(240);    o.AttemptTimeout.Timeout = TimeSpan.FromSeconds(120);    o.Retry.MaxDelay = TimeSpan.FromSeconds(120);});

I'm using the Polly.Core library.

UPDATEIf I remove the Http Message Handler from the client builder the retry policy works.