Quantcast
Channel: Active questions tagged blazor - Stack Overflow
Viewing all articles
Browse latest Browse all 4839

How can I pass "streaming" (i.e., Server-Sent Events) from a Blazor service back to the calling .RAZOR page?

$
0
0

I have a blazor application that I am using to have a conversation with MistralAI.

When prompted, MistralAI will use Server-Sent Events to "stream" data back.

On my .RAZOR page, after a users enters their query, I have this:

<div><EditForm Model="this" OnValidSubmit="@QueryAI"><input type="text" /></EditForm></div>@code {    StreamingQueryResponse streamingResponse = await service.getStreamingResponseAsync(conversation);    String word = streamingResponse.choices[0].delta.content;}

I have a service, registered with the application that calls MistralAI:

public async Task<StreamingQueryResponse> getStreamingResponseAsync(Conversation conversation){    try    {        string requestString = JsonSerializer.Serialize(conversation);        StringContent content = new StringContent(requestString, Encoding.UTF8, "application/json");        using HttpResponseMessage httpResponse = await opService.StreamingPostAsync("http://my.endpoint/", content);        httpResponse.EnsureSuccessStatusCode();        using Stream responseStream = await httpResponse.Content.ReadAsStreamAsync();        bool go = true;        do        {            var buffer = new byte[1024];            var bytesRead = await responseStream.ReadAsync(buffer, 0, buffer.Length);            if (bytesRead == 0) break;            string response = Encoding.UTF8.GetString(buffer, 0, bytesRead);            try            {As                 var jsonString = JsonSerializer.Deserialize<StreamingQueryResponse>(response);                if (jsonString.choices != null)                {                    // At this point, I am getting all of the tokens (words) from the LLM.                    // I just need to figure out how to send those to the .RAZOR page without "return"-ing.                    // Console.WriteLine(jsonString.choices[0].delta.content);                    go = (jsonString.choices[0].finish_reason == null);                    return jsonString;                }            }            catch (Exception ex)            {                Console.WriteLine(ex.Message);            }        } while (go);    }    catch (Exception ex)    {        Console.WriteLine(ex.Message);    }    return null;}

The opService:

public async Task<HttpResponseMessage> StreamingPostAsync(string uri, HttpContent content = null){    HttpRequestMessage request = new HttpRequestMessage    {        RequestUri = new Uri(uri, UriKind.Absolute),        Method = HttpMethod.Post,        Content = content    };    request.Headers.Add("Accept","application/json");    HttpClient client = _clientFactory.CreateClient(uri);    HttpResponseMessage httpResponse = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);    return httpResponse;}

As my comment in the code snippet above states, when I put a Console.WriteLine() within the "if" statement of my service, I see the stream coming back from MistralAI. However, I cannot figure out how to get that stream back to the .RAZOR page so I can display the "live" chat.

When I use return I, obviously, just get the first word back, and that's it.

How do I go about doing this?


Viewing all articles
Browse latest Browse all 4839

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>