But - I have found that every single model available is lacking in creating a great UX, even when you are aggressively using playwright in your agent loop. IMO this comes down to a fundamental limitation of how LLMs work. If an app has a button that is too small, or white text on white background, or inconsistent layouts, or non deterministic ux state due to network calls (looking at you spotify).... the agent will not notice or care. A playwright driving agent can click on a button wether its 1px or 1000px wide.

But ultimately, a good ux is about designing for the constraints that we as humans have to live with as mere mortals with all our imperfections. This is not easy to do, and requires a lot of craftmanship by the developers as well as buy in from the suites and middle managers.

In your windows vista example, the issue is almost surely the latter of my previous sentance - middle manager buy in. No one at microsoft leadership gives a shit about the user experience its all kpi seeking nonsense. The user having a good experience is not measured in their spreadsheet.

When you think about it, pretty much every single product we use helps address a problem related to our fundamental human flaws. Im typing this on an expensive ergonomic keyboard because my wrists hurt, youtube premium exists because I dont want to waste my precious time on stuff I dont want to see, we use operating systems with GUIs because our mortal minds cant easily comprehend thousands of lines of text in seconds.

When I use a horrible website I have a very intuitive, almost instinctive reaction similar to how I react to physical pain. An LLM will never experience this, so it will never create a ux as good as a human can on its own.