Augmented reality makes for fascinating technology, but it can be difficult to come up with real use cases that could save the user time (just imagine the man-hours spent removing backgrounds by online stores).
AR Cut & Paste is a project that combines background removal with augmented reality images, and posting directly in Photoshop.
The background removal uses a remote service to perform the background removal (Written about in BASNet: Boundary-Aware Salient Object Detection with code also available).
The pasting into Photoshop utilizes the OpenCV computer vision library (utilizing the SIFT scale-invariant feature detection) through Cyril’s ScreenPoint library to recognize the boundaries Photoshop window and then pastes into photoshop.
The app itself is developed in React Native.
You can watch the prototype in action in the video below (clicking on the tweet will open the thread with more details on the project.:
4/10 – Cut & paste your surroundings to Photoshop
— Cyril Diagne (@cyrildiagne) May 3, 2020
Code: https://t.co/cVddH3u3ik
Book: @HOLOmagazine
Garment: SS17 by @thekarentopacio
Type: Sainte Colombe by @MinetYoann @ProductionType
Technical Insights: ?#ML #AR #AI #AIUX #Adobe #Photoshop pic.twitter.com/LkTBe0t0rF
The Links:
- AR Cut & Paste code in React Native
- BasNet Http-Service Code for background removal
- Screenpoint server code for detecting the camera’s position
- The original BasNet code
- The BASNet: Boundary Aware Salient Object Detection Paper at the University of Aberta
You’ll need to set up the BasNet-Http Service and Screenpoint server to use the app.
A nice example of an AR app with a practical use case.