A single-use coring probe for collecting a frozen aliquot from a frozen biological sample includes a hollow coring bit and an ejector adapted. The ejector is operable to eject a frozen sample core from the bit as it moves from a retracted position to an extended position. Use of the ejector converts the probe to a disabled configuration to discourage reuse of the coring probe to obtain another sample. The probe may include a locking mechanism adapted to prevent re-use of the single-use coring probe by locking the ejector in the extended position. A hand-held coring device can be used to take frozen sample cores from frozen samples. A tissue container is suitable for holding a frozen tissue sample in frozen storage and also for holding the sample while the sample is sectioned and/or a full-depth frozen sample core is extracted from the frozen tissue.