Abstract:
Distributed system resources may be managed by applying user created policies to the resources. To ensure that valid policies are applied, remote validation for the policies may be implemented. A validation event for a policy may be detected. A remote validation agent may be identified for the policy and a validation request sent to the remote validation agent that includes information for validating the policy. The remote validation agent may return a validation result for the policy. If valid, a policy action that triggered the remote validation event for the policy may be allowed. If invalid, the policy action that triggered the remote validation event for the policy may be denied.
Abstract:
A control node of a multi-tenant stream management service receives a request to initialize a data stream to be comprised of a plurality of data records. The control node determines, based on a partitioning policy, parameters to be used to configure subsystems for ingestion, storage and retrieval of the records. The control node identifies resources to be used for a node of retrieval subsystem The retrieval node is configured to implement programmatic record retrieval interfaces, including respective interfaces to implement non-sequential and sequential access patterns. The control node configures the retrieval node using the selected resources.
Abstract:
Multi-party updates may be performed for distributed systems. An agreement request may be received that proposes updates to a distributed system. An authorization scheme for the agreement request may be determined and approvers for the proposed updates identified according to the authorization scheme. Notifications may be provided to the approvers indicating the proposed updates to the distributed system. Responses from the approvers may be evaluated to determine whether the authorization scheme is satisfied for the proposed updates. If the authorizations scheme is satisfied, then the proposed updates may be performed to the distributed system.
Abstract:
Methods, systems, and computer-readable media for generating message envelopes for heterogeneous events are disclosed. A plurality of events are received. The events are associated with computing resources in a multi-tenant environment. The events comprise an event formatted using a content schema and a different event formatted using a different content schema. A message is generated comprising a message envelope and the event. The message envelope comprises a plurality of field names and one or more field values formatted using an envelope schema. A different message is generated comprising a different message envelope and the different event. The different message envelope comprises the field names and one or more different field values formatted using the envelope schema. The message is filtered based at least in part on the message envelope, and the different message is filtered based at least in part on the different message envelope.
Abstract:
A partitioning policy, comprising an indication of an initial mapping of data records of a stream to a plurality of partitions, is selected to distribute data records of a data stream among a plurality of nodes of a stream management service. Data ingestion nodes and storage nodes are configured according to the initial mapping. In response to a determination that a triggering criterion for dynamically repartitioning the data stream has been met, a modified mapping is generated, and a different set of ingestion and storage nodes are configured. For at least some time during which arriving data records are stored in accordance with the modified mapping, data records stored at the first set of storage nodes in accordance with the initial mapping are retained.
Abstract:
Multiple edits to a hierarchical data structure may be atomically applied. A request to perform modifications with respect to a portion or the entire hierarchical data structure may be received. A copy of the requested portion of the hierarchical data structure may be created separate from the hierarchical data structure. The portion of the hierarchical data structure may remain available for read access. Modifications may be applied to the copy of the portion of the hierarchical data structure. In response to a request to commit the modifications to the portion of the hierarchical data structure, the copy of the portion of the hierarchical data structure may atomically replace the portion of the hierarchical data structure
Abstract:
A programmatic interface is implemented, enabling a client of a stream management service to select a data ingestion policy for a data stream. A client request selecting an at-least-once ingestion policy is received. In accordance with the at-least-once policy, a client may transmit an indication of a data record one or more times to the service until a positive acknowledgement is received. In response to receiving a plurality of transmissions indicating a particular data record, respective positive acknowledgements are sent to the client. Based on a persistence policy selected for the stream, copies of the data record are stored at one or more storage locations in response to one particular transmission of the plurality of transmissions.
Abstract:
A control node of a multi-tenant stream processing service receives a request indicating an operation to be performed on data records of a particular data stream. Based on a stream partitioning policy, the control node determines an initial number of worker nodes to be used. The control node configures a worker node to perform the operation on received records. In response to a determination that the worker node is in an unhealthy state, the control node configures a replacement worker node.
Abstract:
Resource data objects describing resources in a system may be maintained in multiple different hierarchies for applying policies to manage the resources. Lookup requests may access the different hierarchies to determine which policies are applicable to a given resource based on the policies identified in each of the hierarchies. Modifications to hierarchies may be performed in isolation so that the application of policies in other hierarchies is unchanged by modifications to a different hierarchy. Access restrictions may be enforced with respect to hierarchies so that different users may be permitted access to different hierarchies for system resource management.
Abstract:
Methods and apparatus for vibration cancellation are disclosed. Vibration data from one or more vibration detectors associated with a storage device is collected. The vibration data represents vibrations experienced by the storage device. In response to the vibration data from the one or more vibration detectors, one or more movements for respective ones of one or more counter-vibration actuators to at least partially cancel of the vibrations experienced by the storage device is calculated. The one or more counter-vibration actuators perform the one or more movements.