Here's an objective quality metric, which gives good results for watermarked images quality assessment: Komparator.
FYI, the paper to cite when using Komparator is:
D. Barba, P. Le Callet, A robust quality metric for color image quality
assessment, Proceedings of the IEEE International Conference on Image
Processing, pp.437-440, 2003.
You'll find here:
- The C++ sources (along with a README file, explaining how to compile).
- Results data in an excel spreadsheet.
- The PPM images used in the results file.
Here's a few comments on the results, as well as some piece of advice on the *best* way to use Komparator.
In the "OverviewKomparator.xls" file, you'll find the subjective
quality scores (provided by observers during subjective experiments)
The mean score is written in red font. These scores are given for
several watermarking algorithms (noted A1 to A10) you will also find
there the objective quality scores (provided by Komparator).
Here's the way we proceed:
We compute the mean subjective quality score, as well as the mean
objective quality score for each embedding algorithm (for the 5
selected input images).
The correlation coefficient computed between the subjective and objective scores proves the metric's reliability.
The 9 first watermarking algorithms comes from Peter Meerwald's web page, the 10th was proposed in "A robust image watermarking technique based on quantization noise visibility thresholds, Signal Processing, In Press, Florent Autrusseau and Patrick Le Callet".
Komparator performs better when comparing algorithms altogether.
All details are given in "Evaluation of standard watermarking techniques,
Electronic Imaging 2007, Security, Steganography, and Watermarking of
Multimedia Contents IX, E. Marini, F. Autrusseau, P. LeCallet, P.
Campisi."
As you'll see on the spreadsheets' plots, Observers scores are given in
plot #1, and objective scores are in plot #2. The lowest is the
objective score, the best is the visual quality.
To summarize, if you want to assess your embedding algborithm in terms
of invisibility, you'll have to launch Komparator on the 5 selected
images (for every tested algorithm), get the average objective score,
and put this in plot #2.
If anything is unclear, don't hesitate to tell me
Any feedback on your results would be very appreciated, and would help a lot improving the metric.
Florent Autrusseau.