Thanks to EvilSOB's confirmation, I dropped idea of converting screen every frame.
What I did is scanned image with colors on load only once, and stored cell color information into the array for further uses, and that suits my needs.

Rendering every full frame indeed impossible... maybe pixel shader can help somehow.