python - Finding squares in Image

I need to find the squares in an image using OpenCV (no problem in matlab or any other, generally what i expect are some ideas).

Consider the test image below :

I need to find those coloured squares in above image accurately (not the white long strips).

What I have done :

• 
  

  I applied the common method (which comes with OpenCV samples), ie find contours in all color planes, approximate it and check for number of elements=4. It works to some extend that, few squares are detected, especially the dark ones.

  
  


• 
  

  Next step i did was prediction. ie this arrangement is fixed. So, if some are obtained, I can predict remaining ones. It also worked to some further extend. But accuracy was very bad.

  
  
  

But I feel prediction is not a good method here and it doesn't always provide accurate answers as given by the first step.

What I need :

1) Is there any other better methods to detect these squares more accurately? Or multiple methods?

One important point is that, time is not a problem here. Algorithm can be slow, it doesn't matter. But accuracy is the major criteria.

Sometimes, images can be much more blurry.

And one of the major problem I faced is that some squares have almost similar color as that of background (check column 3 first and second squares).

Looking for ideas, thanks in advance

UPDATE :

Below is the maximum accurate result I got :

Of course, the result image is resized a little bit.

UPDATE 2 :

I have given a much more better solution in my answer below: https://dsp.stackexchange.com/a/7526/818