A system and method can support a distributed transaction lock in a transactional middleware machine environment. The system can use a global transaction identifier to locate a hash bucket in a transaction hash table, wherein the transaction hash table contains a plurality of hash buckets. Then, the system can lock said hash bucket in the transaction hash table, and allow only one process to access one or more global transaction entry structures in the transaction table before said hash bucket is unlocked, wherein said one or more global transaction entry structures are associated with the locked hash bucket in the transaction hash table.