SDR# has an original algorithm that compensates gain and phase imbalances between the IQ channels. Without this compensation strong signals are mirrored with respect to the center of the spectrum. All quadrature sampling receivers have their imaging problems. Solutions that were developed in this regard scale from manual adjustment of phase and gain parameters, to much complex mathematical models that find out these parameters by analyzing IQ signals and do some magic. The algorithm I developed is somewhere between these two extremes. Here’s a brief description of how it works:
The algorithm tries to figure out parameters that maximize an objective function (image
rejection utility function, actually). The function simply sums the distances between every FFT bin and its corresponding image. We build that function such that it takes two parameters, phase and gain errors (our target). The return value is a scalar. The nearer the parameters to optimality the higher the return value. The maximum point of this function corresponds to a perfectly balanced signal. Have a look on this plot:

For an ideal transmitted IQ signal:


The output of QSD/ADC stages subject to imbalances is defined by these equations:


We first initialize the alpha and phi parameters with 0. The algorithm then moves by random steps and evaluates the utility function until it reaches (or approaches) the maximum. Only moves that improve the problem are retained. We iterate over many buffers improving the estimation every time. This is a multi-dimensional concave optimization problem. This approach is computationally effective, and often simple to implement. Sometimes, it’s the only way to solve problems when no known/valid analytic model exists.
Here’s a signal from FUNcube Dongle without IQ correction:

The same signal with IQ correction activated. Nice image rejection, isn’t it? 

The source code can be found here.
Notes:
- This algorithm supposes that the phase and gain errors are constant over the spectrum, which is the case on most sound cards, but not all.
- I have a few ideas to extend this algorithm to cover phase and gain errors that are dependent on frequency.
Update:
This algorithm and associated code are disclosed for reference only. For commercial usage, contact me at info *at* sdrsharp.com. In either case I’ll be pleased to help you.