Image recognition: Generated tags

[jarrar.hussain]

Hi Guys,

I am using ImageMagick from last few years, but never use it intensively just resizing and some cropping.

But now I am working on a project where I have a big challenge to analysis more than 1M images per 48hrs and for that I would like to use ImageMagic 6.7.7-10 on Ubuntu 14.04.

The challenge is to identify the following:
- object category (sofa, bedroom, bathroom, kitchen, etc)
- object color & shape
- Colors
- scenic & non scenic images

Sample image URLS:
https://d3sk3dy0x5b87a.cloudfront.net/i ... 9d7a4f.jpg
https://d3sk3dy0x5b87a.cloudfront.net/i ... 184d53.jpg
https://d3sk3dy0x5b87a.cloudfront.net/i ... f7c9f5.jpg
https://d3sk3dy0x5b87a.cloudfront.net/i ... ceea68.jpg
https://d3sk3dy0x5b87a.cloudfront.net/i ... 27ade2.jpg
https://d3sk3dy0x5b87a.cloudfront.net/i ... 17226e.jpg

Any help would be much appreciated, thanks for your time and looking forward for reply.

Best regards,

Jarrar Hussain.

[snibgo]

1 million images per 48 hours? That is 6 images per second. You can't do much in that time. Okay, multiple processors etc, maybe 1 image/second, but that's not much time.

I wouldn't use IM 6.7.7. Don't start a new project with old software. IM was confused in those days about greyscale images. Use the current v6 or v7. My own preference would be to use v6, but in a way that is compatible with v7.

Basic analysis is simple enough: images like https://d3sk3dy0x5b87a.cloudfront.net/i ... f7c9f5.jpg, when chopped into 4 pieces, have pretty much the same characteristics in each piece. But if you need to classify the image as either "cloth" or "painted wall", that would be harder.

Object recognition is non-trivial. Wikipedia has a brief overview https://en.wikipedia.org/wiki/Outline_o ... ecognition, and you can follow the trail from there. In essence, most methods simplify the image and find matches.

I'm curious. If you can say, what is the purpose behind the work?

[fmw42]

Your task seems extremely daunting to me. Imagemagick does not have good tools for object recognition. It is an image processing tools. It has some tools to do color analysis (histograms, and mean and std statistics), template matching (but which is not scale or rotation independent), some texture measures (-feature and image moments), perceptual hash and connected-components labeling. But as for recognizing objects. That is really tough. You would have to script a lot of these processes and scripting is not very efficient. I do not think there is any really good tool that does exactly what you want to do. Also Imagemagick is not especially optimized for speed. It is more focused on good speed and high quality. The number of images to process in that short of time may be impossible for any tool depending upon your image sizes, unless you do massively parallel processing (as snibgo mentions below). I agree with snibgo that you should work with the most current version of IM. See viewtopic.php?f=4&t=21269 regarding issues being changed during the release around the one you have.

[snibgo]

About 20 years ago, I had a similar image processing throughput problem. The solution then was to use a hundred PC motherboards networked together. These days, perhaps a hundred modules like the Raspberry Pi could be used. Then each module would have 16 seconds to process each image, which is far more reasonable. The cost for the hardware wouldn't be much.

The controlling software that queues the work, partitions it among the processors and collates the results, is critically important.

[jarrar.hussain]

Hi Guys,

Thanks for your quick response

Actually I am satisfy with the performance & quality of ImagicMagic and with my current requirement & wants to continue.

Target of this project:
I get product data as csv/xml/json -feed for our commerce platform including images. I import that data into our system and images as well.

Using ImageMagic I do the following: resize, optimize, extracting meta-data like page-gemoatry, quality, resolution, etc) but here I want to extract some more meta data specially:

• - Identify unique colors (but not hundreds of shades) at max 10 colors
  - distinguish between scenic & non scenic images
  - identify image if it contain one or more object(s) like:
  
  
  - identify image if it contains only product material or zero object like:
  
  
  
  
  
  
  - object category (sofa, bedroom, bathroom, kitchen, etc)

Let me know if you need more sample images or details.

Thanks & Regards
Jarrar

Email: jhussain@x24factory.com
Skype: jarrar.hussain

[snibgo]

- Identify unique colors (but not hundreds of shades) at max 10 colors

That's easy: "-colors 10".

- distinguish between scenic & non scenic images

Define "scenic". You can test for blue sky, or blue sky with clouds.

- identify image if it contain one or more object(s) like:

If you want to identify objects like "sofa", "table and chairs", that's harder.

- identify image if it contains only product material or zero object like:

The first two examples are fairly easy: statistics like colour, tone and texture are pretty constant across the entire image. The others are harder.

- object category (sofa, bedroom, bathroom, kitchen, etc)

Difficult. Not impossible, but not easy. For example, you could identify planes in the image. If there is a ceiling, we are inside. If there is a large horizontal surface near the floor, it's probably a bed. Etc.

If you have only a desktop-type computer to work with, forget about identifying objects in 1 million photos in 2 days. I reckon 200 days is nearer the mark.

[jarrar.hussain]

Hi Snibgo,

Thanks for your reply. That's great, please give me Linux based command line samples for the following:

- Identify unique colors (but not hundreds of shades) at max 10 colours
- identify image if it contains only product material or zero object like:


- distinguish between scenic & non scenic images
yes your right, but this is extreme senior, what I want to identify is that the image is with some background colour like white, blue, etc.. some kind of indicators.

Regarding performance, I have no issue and I am not running this amount of images on my desktop for that I have a BIG machine

Thanks
Jarrar

[fmw42]

see http://www.imagemagick.org/Usage/quantize/#extract

Code: Select all

convert image +dither -colors 10 -unique colors txt:-

To see if the image is mostly one color, look for a low standard deviation in each channel. You will have to determine what threshold values are low enough from testing a number of images in each category.

Code: Select all

convert image -format "%[fx:standard_deviation.r],%[fx:standard_deviation.g],%[fx:standard_deviation.b]" info:

[jarrar.hussain]

Hi,

Code: Select all

convert image +dither -colors 10 -unique colors txt:-

Here I get always 10 colours except black&white images ... what I want to ignore here is hundreds of shades... and 10 colours only for real colour-rich images and not for basic images with few colours

Code: Select all

convert image -format "%[fx:standard_deviation.r],%[fx:standard_deviation.g],%[fx:standard_deviation.b]" info:

How do I interpret these values?
0.223343,0.13436,0.129902

Thanks,
Jarrar

[fmw42]

I assume if you have a colorful image, the 10 unique colors will be quite different. If you do not have a colorful image, you will get similar colors or shades of gray for grayscale images. You can throw out grayscale images by checking the colorspace and/or type from the verbose information or from string formats. See http://www.imagemagick.org/script/escape.php. You can decide if the colors are different or similar by looking at the standard deviations of these 10 colors

Code: Select all

convert image +dither -colors 10 -unique colors -format "%[fx:standard_deviation.r],%[fx:standard_deviation.g],%[fx:standard_deviation.b]" info:

The standard deviations are for red, green and blue channels and are in the range 0 to 1. You must decide what is small enough for each channel to consider a constant color image.