A building panel assembly is provided for a modular building. The panel assembly is configured to provide a structural wall, roof, floor of a building comprising a plurality of panels. Each panel comprises an outer layer, having a first flange extending from a first distal end and a second flange at an opposing end. Each panel comprises an intermediate layer, which may include insulating material, sandwiched between the inner and outer layers. The assembly further comprises a plurality of framing anchors in the form of brackets configured to mount building panels together. The building panels are configured to provide structural support to the building, such that additional framing is not required, the building panels forming a self-supporting structure when connected together.