Image resize by solving a sparse linear system
Contents
Part 1: project profile
Project name
Content-aware Image Resize.
Project short description
This project presents a methodology for a two stages content aware image resizing. The first stage computes the saliency matrix which will be used in the second stage to relocate all pixels in a smaller image size.
Dates
Start date: 2008/01/01
End date: 2008/09/06
People involved
Project head(s)
- Pierluigi Taddei
- Vincenzo Caglioti
Students currently working on the project
- Alessio C. Bonsignore
- Stefano Anelli
Part 2: project description
State of the art
This project is based on the article Non-homogeneous Content-driven Video-retargeting, Lior Wolf, Moshe Guttmann, Daniel Cohen-Or The School of Computer Science, Tel-Aviv University, 2007, in which the author describes content aware video resize using a linear system to compute the mapping function.
The problem
The normal way of resizing a picture doesn’t take in account it’s content and differs from the resizing of a web page where each object is modified on the basis of its content. So, there is the need for an image resizing that is content aware.
The Idea
The idea behind content aware resizing is to treat in a different way the important regions of the image respect to those regions with less information. To achieve this goal it’s possible to analyze the source image building a saliency matrix based on gradient magnitude, features detection and eventually face detection (not included in the project). After that a sparce linear system is set up using several different constraits which force pixels to keep a proper position in a smaller size image. The system solution gives the new position of each pixel that better fit the image keeping as much information as possible from the source.
The Project
All these algorithms were implemented in Matlab and tested on several pictures (source code is available here: liquidResize source code ).
The project comprehend two versions of the liquidResize() function. One for both width and height resize, and the other only for width. Moreover, the first modifies the saliency matrix to take into account the features detected. The second insted does the same thing using more constraits in the linear system.
Report
The report about this project is available here: Report
Conclusion
From results emerge that several image can be successfully resized as the one in the image below, but many situations are still troublesome. Straight lines for example will easily be bent. Moreover, if there's too much information in the image it will be resized uniformly just like a classic image resized techinque.
The main issue is the computational time. To get a 50% width resize of a small 300x225 image (as the kitty example) you need almost two minutes. And to resize both width and height you have to double that time.
