I've scraped quite a few websites and normally this hasn't been a big issue (those that use http requests with form data or json APIs). Now I stumbled upon one which has me absolutely stuck. After hours of analyzing it with chrome developer tools I determined that it:
- uses websocktes
- uses blazor (apparently a microsoft technology I've never heard of) for communication via those websockets
The problem is that all the messages that this blazor thing sends or receives via websockets are in binary format and it's entirely unclear what is going on. The only other thing I managed to find out regarding this that apparently something like SignalR is being used here.
Is it possible to scrape this website (I need to login and parse some data, although already the login happens via blazor) using python, how do I start understanding what those binary messages are?
Or is this a revolution and such sites are not scrapable?
Not sure how to proceed at all