公开(公告)号：US09306863B2

公开(公告)日：2016-04-05

申请号：US14099291

申请日：2013-12-06

Applicant: Intel Corporation

Inventor： Mark S. Birrittella

IPC: H04L12/935 ,  H04L29/06 ,  H04L12/805

CPC classification number: H04L47/36 ,  H04L49/30 ,  H04L69/22

Abstract: Method, apparatus, and systems for Link Transfer, bit error detection and link retry using flit bundles asynchronous to link Fabric Packets. A first type of packet comprising a Fabric Packet is generated and its data content is divided into multiple data units called “flits.” The flits are then bundled into a second type of packet comprising Link Transfer Packets (LTPs). The LTPs are then sent over single link segments in a fabric comprising many point-to-point links. Each LTP includes a CRC that is used to ensure that data transmitted over each link segment is error free, and comprises a unit of retransmission. The size of the fabric packets may vary, and they may be larger or smaller than an LTP. The transfer scheme enabled flits from multiple fabric packets to be bundled into a single LTP. Upon receipt at a fabric endpoint, the flits from the LTPs are extracted and reassembled to regenerate the Fabric Packets.

Abstract translation: 链路传输的方法，设备和系统，使用与链路Fabric数据包异步的flit bundle的位错误检测和链接重试。 生成包括Fabric Packet的第一类型的分组，并将其数据内容划分成称为“flits”的多个数据单元。然后，将这些数据包捆绑成包括链路传输分组（LTP）的第二类型的分组。 然后，LTP通过包括许多点对点链路的结构中的单个链路段发送。 每个LTP包括CRC，用于确保通过每个链路段发送的数据是无错误的，并且包括重传单元。 织物分组的大小可以变化，并且它们可以大于或小于LTP。 传输方案使来自多个Fabric数据包的传输方案捆绑成一个LTP。 在结构端点接收时，提取并重新组合来自LTP的闪烁以重新生成Fabric数据包。

公开(公告)号：US09946676B2

公开(公告)日：2018-04-17

申请号：US14669975

申请日：2015-03-26

Applicant: Intel Corporation

Inventor： Mahesh Wagh ,  Zuoguo Wu ,  Venkatraman Iyer ,  Gerald S. Pasdast ,  Mark S. Birrittella ,  Ishwar Agarwal ,  Lip Khoon Teh ,  Su Wei Lim ,  Anoop Kumar Upadhyay

IPC: G06F17/50 ,  G06F13/40 ,  G06F13/36

CPC classification number: G06F13/4022 ,  G06F13/36 ,  G06F13/4068

Abstract: A system-on-a-chip, such as a logical PHY, may be divided into hard IP blocks with fixed routing, and soft IP blocks with flexible routing. Each hard IP block may provide a fixed number of lanes. Using p hard IP blocks, where each block provides n data lanes, h=n*p total hard IP data lanes are provided. Where the system design calls for k total data lanes, it is possible that k≠h, so that ┌k/n┐ hard IP blocks provide h=n*p available hard IP data lanes. In that case, h−k lanes may be disabled. In cases where lane reversals occur, such as between hard IP and soft IP, bowtie routing may be avoided by the use of a multiplexer-like programmable switch within the soft IP.

公开(公告)号：US09887804B2

公开(公告)日：2018-02-06

申请号：US15075590

申请日：2016-03-21

Applicant: Intel Corporation

Inventor： Mark S. Birrittella

IPC: H03M13/00 ,  G06F11/10 ,  H04L1/00 ,  H03M13/09 ,  H04L1/18 ,  H04L1/08 ,  H04L1/16

CPC classification number: H04L1/0045 ,  H03M13/09 ,  H04L1/0058 ,  H04L1/0061 ,  H04L1/0071 ,  H04L1/08 ,  H04L1/1607 ,  H04L1/1657 ,  H04L1/1874

Abstract: Method, apparatus, and systems for detecting lane errors and removing errant lanes in multi-lane links. Data comprising link packets is split into a plurality of bitstreams and transmitted over respective lanes of a multi-lane link in parallel. The bitstream data is received at multiple receive lanes of a receiver port and processed to reassemble link packets and to calculate a CRC over the data received on each lane. The link packets include a transmitted CRC that is compared to a received CRC to detect link packet errors. Upon detection of a link packet error, per-lane or per transfer group CRC values are stored, and a retry request is issued to retransmit the bad packet. In conjunction with receipt of the retransmitted packet, per-lane or per transfer group CRC values are recalculated over the received data and compared with the stored per-lane or per transfer group CRC values to detect the lane causing the link packet error.

公开(公告)号：US20150222533A1

公开(公告)日：2015-08-06

申请号：US14173782

申请日：2014-02-05

Applicant: INTEL CORPORATION

Inventor： Mark S. Birrittella ,  Thomas D. Lovett ,  Todd Rimmer

IPC: H04L12/721 ,  H04L12/947 ,  H04L12/741

CPC classification number: H04L45/745 ,  H04L5/0055 ,  H04L45/66 ,  H04L45/74 ,  H04L47/17 ,  H04L47/39 ,  H04L49/25 ,  H04L61/3075 ,  H04Q2213/13215

Abstract: Method, apparatus, and systems for reliably transferring Ethernet packet data over a link layer and facilitating fabric-to-Ethernet and Ethernet-to-fabric gateway operations at matching wire speed and packet data rate. Ethernet header and payload data is extracted from Ethernet frames received at the gateway and encapsulated in fabric packets to be forwarded to a fabric endpoint hosting an entity to which the Ethernet packet is addressed. The fabric packets are divided into flits, which are bundled in groups to form link packets that are transferred over the fabric at the Link layer using a reliable transmission scheme employing implicit ACKnowledgements. At the endpoint, the fabric packet is regenerated, and the Ethernet packet data is de-encapsulated. The Ethernet frames received from and transmitted to an Ethernet network are encoded using 64b/66b encoding, having an overhead-to-data bit ratio of 1:32. Meanwhile, the link packets have the same ratio, including one overhead bit per flit and a 14-bit CRC plus a 2-bit credit return field or sideband used for credit-based flow control.

Abstract translation: 用于通过链路层可靠地传输以太网分组数据的方法，装置和系统，并且以匹配的线速度和分组数据速率促进以太网和以太网到网络的网关操作。 从网关接收的以太网帧提取以太网报头和有效载荷数据，并将其封装在结构数据包中，以转发到托管以太网数据包所针对的实体的结构端点。 结构分组被划分成flits，它们被分组成组，以形成通过使用隐式确认的可靠传输方案在链路层上通过结构传送的链路分组。 在端点，重新生成Fabric数据包，并对以太网数据包进行解封装。 从以太网网络接收并发送到以太网的以太网帧使用64b / 66b编码进行编码，具有1:32的开销数据比特率。 同时，链路分组具有相同的比率，包括一个开销比特每个flit和一个14比特CRC加上一个2比特的信用回报字段或用于基于信用的流量控制的边带。

公开(公告)号：US10372647B2

公开(公告)日：2019-08-06

申请号：US14977773

申请日：2015-12-22

Applicant: INTEL CORPORATION

Inventor： Thomas D. Lovett ,  Michael A. Parker ,  Mark S. Birrittella

IPC: G06F13/00 ,  G06F13/40 ,  G06F1/12 ,  G06F13/364 ,  G06F13/42

Abstract: Methods and apparatus for implementing time synchronization across exascale fabrics. A master clock node is coupled to a plurality of slave nodes via a fabric comprising a plurality of fabric switches and a plurality of fabric links, wherein each slave node is connected to the master clock node via a respective clock tree path that traverses at least one fabric switch. The fabric switches are configured to selectively forward master clock time data internally along paths with fixed latencies that bypass the switches' buffers and switch circuitry, which enables the entire clock tree paths to also have fixed latencies. The fixed latency of the clock tree path is determined for each slave node. The local clocks of the slave nodes are then synchronized with the master clock by using master clock time data received by each slave node and the fixed latency of the clock tree path from the master clock node to the slave node that is determined. Techniques for determining a clock rate mismatch between the master clock and a local clock is also provided.

公开(公告)号：US10230665B2

公开(公告)日：2019-03-12

申请号：US14136293

申请日：2013-12-20

Applicant: INTEL CORPORATION

Inventor： Thomas D. Lovett ,  Albert Cheng ,  Mark S. Birrittella ,  James Kunz ,  Todd Rimmer

IPC: H04L12/933 ,  H04L12/911 ,  H04L1/00 ,  H04L12/851 ,  H04L12/805 ,  H04L12/863 ,  H04L12/865 ,  H04L1/12 ,  H04L25/14

Abstract: Methods, apparatus, and systems for implementing hierarchical and lossless packet preemption and interleaving to reduce latency jitter in flow-controller packet-based networks. Fabric packets are divided into a plurality of data units, with data units for different fabric packets buffered in separate buffers. Data units are pulled from the buffers and added to a transmit stream in which groups of data units are interleaved. Upon receipt by a receiver, the groups of data units are separated out and buffered in separate buffers under which data units for the same fabric packets are grouped together. In one aspect, each buffer is associated with a respective virtual lane (VL), and the fabric packets are effectively transferred over fabric links using virtual lanes. VLs may have different levels of priority under which data units for fabric packets in higher-priority VLs may preempt fabric packets in lower-priority VLs. By transferring data units rather than entire packets, transmission of a packet can be temporarily paused in favor of a higher-priority packet. Multiple levels of preemption and interleaving in a nested manner are supported.

公开(公告)号：US09819452B2

公开(公告)日：2017-11-14

申请号：US15184277

申请日：2016-06-16

Applicant: Intel Corporation

Inventor： Mark S. Birrittella

IPC: H04L1/18 ,  H04L1/16 ,  H04L1/08 ,  H04L29/08

CPC classification number: H04L1/1657 ,  G06F9/00 ,  H04L1/08 ,  H04L1/1867 ,  H04L1/1874 ,  H04L67/104

Abstract: Methods, apparatus, and systems for implementing a link layer retry protocol utilizing implicit ACKnowledgements (ACKs). Peer link interfaces are configured to facilitate confirmed error-free delivery of link-layer packets through use of implicit ACKs, while also providing retransmission of packets for which errors are detected and guaranteeing the link control data is either successfully received or data transfer over the link is prevented. In conjunction with transmitting packets, reliable packets are copied into sequential slots in a replay buffer. Each link interface tracks the slot at which each reliable packet is buffered, and in response to detection of an error, a retry request is sent to the transmit-side to retransmit the errant packet. The previously buffered copy of the errant packet is retrieved from the replay buffer and retransmitted. Through use of a link roundtrip detection mechanism, absence of a retry request by the time a replay buffer has returned to the slot of a reliable packet (plus a predetermined number of additional transfer cycles, if applicable) provides an implicit ACK that the packet was received without error.

公开(公告)号：US09628382B2

公开(公告)日：2017-04-18

申请号：US14173782

申请日：2014-02-05

Applicant: INTEL CORPORATION

Inventor： Mark S. Birrittella ,  Thomas D. Lovett ,  Todd Rimmer

IPC: H04L12/721 ,  H04L12/741 ,  H04L12/947

CPC classification number: H04L45/745 ,  H04L5/0055 ,  H04L45/66 ,  H04L45/74 ,  H04L47/17 ,  H04L47/39 ,  H04L49/25 ,  H04L61/3075 ,  H04Q2213/13215

Abstract: Method, apparatus, and systems for reliably transferring Ethernet packet data over a link layer and facilitating fabric-to-Ethernet and Ethernet-to-fabric gateway operations at matching wire speed and packet data rate. Ethernet header and payload data is extracted from Ethernet frames received at the gateway and encapsulated in fabric packets to be forwarded to a fabric endpoint hosting an entity to which the Ethernet packet is addressed. The fabric packets are divided into flits, which are bundled in groups to form link packets that are transferred over the fabric at the Link layer using a reliable transmission scheme employing implicit ACKnowledgements. At the endpoint, the fabric packet is regenerated, and the Ethernet packet data is de-encapsulated. The Ethernet frames received from and transmitted to an Ethernet network are encoded using 64b/66b encoding, having an overhead-to-data bit ratio of 1:32. Meanwhile, the link packets have the same ratio, including one overhead bit per flit and a 14-bit CRC plus a 2-bit credit return field or sideband used for credit-based flow control.

公开(公告)号：US10305802B2

公开(公告)日：2019-05-28

申请号：US15396497

申请日：2016-12-31

Applicant: Intel Corporation

Inventor： Mark S. Birrittella ,  Thomas D. Lovett ,  Todd M. Rimmer

IPC: H04L12/741 ,  H04L12/721 ,  H04L12/947 ,  H04L5/00 ,  H04L12/801 ,  H04L29/12

Abstract: Method, apparatus, and systems for reliably transferring Ethernet packet data over a link layer and facilitating fabric-to-Ethernet and Ethernet-to-fabric gateway operations at matching wire speed and packet data rate. Ethernet header and payload data is extracted from Ethernet frames received at the gateway and encapsulated in fabric packets to be forwarded to a fabric endpoint hosting an entity to which the Ethernet packet is addressed. The fabric packets are divided into flits, which are bundled in groups to form link packets that are transferred over the fabric at the Link layer using a reliable transmission scheme employing implicit ACKnowledgements. At the endpoint, the fabric packet is regenerated, and the Ethernet packet data is de-encapsulated. The Ethernet frames received from and transmitted to an Ethernet network are encoded using 64b/66b encoding, having an overhead-to-data bit ratio of 1:32. Meanwhile, the link packets have the same ratio, including one overhead bit per flit and a 14-bit CRC plus a 2-bit credit return field or sideband used for credit-based flow control.

公开(公告)号：US20160283429A1

公开(公告)日：2016-09-29

申请号：US14669975

申请日：2015-03-26

Applicant: Intel Corporation

Inventor： Mahesh Wagh ,  Zuoguo Wu ,  Venkatraman Iyer ,  Gerald S. Pasdast ,  Mark S. Birrittella ,  Ishwar Agarwal ,  Lip Khoon Teh ,  Su Wei Lim ,  Anoop Kumar Upadhyay

IPC: G06F13/40 ,  G06F13/36 ,  G01R31/317

CPC classification number: G06F13/4022 ,  G06F13/36 ,  G06F13/4068

Abstract: A system-on-a-chip, such as a logical PHY, may be divided into hard IP blocks with fixed routing, and soft IP blocks with flexible routing. Each hard IP block may provide a fixed number of lanes. Using p hard IP blocks, where each block provides n data lanes, h=n*p total hard IP data lanes are provided. Where the system design calls for k total data lanes, it is possible that k≠h, so that [k/n] hard IP blocks provide h=n*p available hard IP data lanes. In that case, h−k lanes may be disabled. In cases where lane reversals occur, such as between hard IP and soft IP, bowtie routing may be avoided by the use of a multiplexer-like programmable switch within the soft IP.

Abstract translation: 诸如逻辑PHY的片上系统可以被划分为具有固定路由的硬IP块和具有灵活路由的软IP块。 每个硬IP块可以提供固定数量的车道。 使用p硬IP块，其中每个块提供n个数据通道，h = n * p提供总硬IP数据通道。 在系统设计要求k个总数据通道的情况下，k≠h可以使得[k / n]硬IP块提供h = n * p可用的硬IP数据通道。 在这种情况下，h-k通道可能被禁用。 在发生通道反转的情况下，例如在硬IP和软IP之间，可以通过使用软IP内的多路复用器可编程开关来避免路由路由。