Sketch2Code

Sketch2Code was launched by Microsoft AI Lab in 2018 and is an open-source project aimed at helping developers easily convert hand drawn sketches into HTML code. Using Sketch2Code, developers can draw wireframe sketches and export them as source code. Sketch2Code uses artificial intelligence and automation to develop code from wireframe sketches, supporting basic web visual elements such as tags, text fields, text paragraphs, images, and buttons.
When it was released in 2018, the model behind this service had already been trained on millions of images and could detect objects of various types. With this information, the tool can generate HTML code fragments for different elements in the design, and then infer the layout of the design based on the position of the recognized elements, and generate the final HTML code accordingly.
In the demonstration of GPT 4, there is also a demo demonstration that converts sketches into web pages. It is likely that Microsoft has already collaborated and iterated with OpenAI on this technology, and the latest trained image data may far exceed 2018.