The invention provides for automatically identifying the location of a displayed video window based upon a characterization of selected portions of the image for realness based upon a distribution of luminance values for the selected portions. The image is then searched mathematically for a large rectangle of realness, and if found, a similar operation is performed in a smaller rectangle around each of the edges of the large rectangle, in turn, zooming in to a resolution of one pixel, thus identifying the position of the edge. This process can be repeated as often as necessary in order to maintain a fix on the edges of the video window.