A method and apparatus for coding video data in a scalable or three-dimensional video coding system are disclosed. In one embodiment, context for the input data associated with an inter-layer or inter-view coding syntax element of a current block in an enhancement-layer (EL) or a dependent view is formed or the input data is parsed to identify the context based on information associated with one or more neighboring blocks, information associated with the current block, or both. In another embodiment, encoding or decoding is applied to the input data associated with the current block in the EL using inter-layer texture prediction based on the BL block in the BL, wherein a flag indicating whether there is any transform coefficient coded for the current block is signaled.