A method and system for automatically detecting villi texture in in-vivo images. In-vivo images may be received from an in vivo imaging capsule, and at least one image may be selected for analysis. Textural details may be extracted, for example using morphological top hat transforms. The image may be divided into segments or patches. The patches may be pruned using a set of threshold conditions, which are calculated based on grayscale or color statistics of the patches. The patches may be classified as containing villi texture or not based for example on rotation-invariant textural features computed in the Fourier domain. A sum, score or rating for the image may be calculated based on the classification of each patch. The final score of the image may be averaged over a number of, for example, consecutive images, and the score may be used for detecting entry into the small bowel and segmenting the image stream to anatomical regions.