A mixed matrix membrane is provided which comprises a continuous phase organic polymer and small pore alumina containing molecular sieves dispersed therein. The molecular sieves have a silica-to-alumina molar ratio of less than 1.0, more preferably, less than 0.3, and most preferably less than 0.1. In some cases, the molecular sieves have no appreciable amounts of silica. Exemplary compositions include aluminophosphates (AIPO) and silicoaluminophosphates (SAPO). When these molecular sieves are properly interspersed with a continuous phase polymer, the membrane will exhibit a mixed matrix membrane effect, i.e., a selectivity increase of at least 10% relative to a neat membrane containing no molecular sieves. The molecular sieves have pores with a largest minor crystallographic free diameter of 4.0 Angstroms or less. The molecular sieves may be selected from the group having IZA structure types including AEI, CHA, ERI, LEV, AFX, AFT, and GIS. Examples of preferred molecular sieves include: AIPO-18, SAPO-18, AIPO-34, SAPO-34, SAPO-44, SAPO-47, AIPO-17, SAPO-17, CVX-7, SAPO-35, SAPO-56, ALPO-52, and SAPO-43. Finally, methods for making and using such mixed matrix membranes to separate gases from a mixture containing two or more gases are also disclosed.