Blazor WASM app with both JWT token and Azure AD authentication

I have a Blazor web assembly app. I am trying to let the user either log in with their azure ad credentials or with their credentials stored in a database.

I can get them both to work but not at the same time. I need to implement a custom authentication state provider to save the authentication credentials for the JWT so they are authorized on each page / API call.

Having this custom authentication state provider cases a conflict when using

builder.Services.AddMsalAuthentication<RemoteAuthenticationState, CustomUserAccount>(options =>    {        builder.Configuration.Bind("AzureAd", options.ProviderOptions.Authentication);        //   options.ProviderOptions.DefaultAccessTokenScopes.Add("https://graph.microsoft.com/User.Read");        options.ProviderOptions.DefaultAccessTokenScopes.Add("api://2913a4cb-b93e-4107-bd31-1f963e967f30/API.Access");        options.UserOptions.RoleClaim = "roles";        options.ProviderOptions.Cache.CacheLocation = "localStorage";        options.ProviderOptions.LoginMode = "redirect";       }).AddAccountClaimsPrincipalFactory<RemoteAuthenticationState, CustomUserAccount, CustomAccountFactory>();`

to set up the Azure AD login.

How do I enable it so the user can log in with either and be authorised to use the APIs/view the pages?