You can turn asphalt into a lawn or cover up an extra object in the photo, if it does not contradict physics.
Researchers from MIT and IBM have developed a Paint Studio tool that allows you to add, edit and delete objects in photos in one motion. The system works with artifacts, but produces a plausible result.
According to MIT Professor Antonio Torralba, the tool has a “strong personality” and will not perform actions if it considers that they are not suitable for the image.
All drawing applications follow user instructions, but ours can decide not to draw anything if a person tries to put an object in an impossible position. This is a drawing tool with a strong personality and it opens a window that will allow you to understand how generative adversarial neural networks represent the visual world.
Antonio TorralbaMIT professor
Demonstration of the application posted on a separate site . The tool currently works only with already loaded images of poor quality, and you can draw with a limited number of “brushes”: grass, trees, bricks, domes, sky, clouds and doors.
Most often, the result is realistic, and in contradiction with the real world, the algorithm does not change anything in the image. In some cases, the neural network leaves artifacts in the pictures – the algorithm is not yet perfect.
To create a tool used neural network, consisting of two parts. One generates examples of realistic photos, while the other is trying to find the difference between the generated image and the real one. Thus, the model is improved and learns to produce images of a certain level of realism.