摘要:
In general, the invention is directed to systems and methods of distributing tasks amongst servers or nodes in a cluster in a cloud-based big data environment, including: establishing a high_server_threshold; dividing active servers/nodes into at least three (3) categories of high usage servers, comprising servers on which usage is greater than the high_server_threshold; medium usage servers, comprising servers on which usage is less than the high_server_threshold, but is greater than zero; and low usage servers, comprising servers that are currently not utilized; receiving one or more tasks to be performed; scheduling the tasks by: first requesting that medium usage servers take tasks; if tasks remain that are not scheduled on the medium usage servers, schedule remaining tasks on low usage servers; if any tasks remain that are not scheduled on medium usage servers or low usage servers, scheduling remaining tasks on high usage servers.
摘要:
The present invention is generally directed to systems and methods of providing automatic scaling pure-spot clusters. Such dusters may be dynamically rebalanced for further costs savings. In accordance with some methods of the present invention may include a method of utilizing a cluster in a big data cloud computing environment where instances may include reserved on-demand instances for a set price and on-demand spot instances that may be bid on by a user, the method including: creating one or more stable nodes, comprising spot instances with a bid price above a price for an equivalent on-demand instance; creating one or more volatile nodes, comprising spot instances with a bid price below a price for an equivalent on-demand instance; using one or more of the stable nodes as a master node; and using the volatile nodes as slave nodes.
摘要:
In a method for operating a data storage system, a request is issued that an ownership attribute of a logical arrangement of storage space associated with a first storage system be associated with a second storage system. In response to the request, the ownership attribute is changed to a state of unowned. In response to the request, and in response to the state of unowned, the ownership attribute of the logical arrangement of storage space is changed to a new set of attributes associated with the second storage system to change ownership of the logical arrangement of storage space from the first storage system to the second storage system.
摘要:
A system and method for disk ownership in a network storage system. Each disk has two ownership attributes set to show that a particular file server owns the disk. In a preferred embodiment the first ownership attribute is the serial number of the file server being written to a specific location on each disk and the second ownership attribute is setting a SCSI-3 persistent reservation. In a system utilizing this disk ownership method, multiple file servers can read data from a given disk, but only the file server that owns a particular disk can write data to the disk.
摘要:
A system and method for reliable peer-to-peer communication is provided. The system and method includes a cluster connection manager that acts as a unified resource for opening, closing and maintaining peer-to-peer communication channels between applications executing on storage systems comprising a storage system cluster. The cluster connection manager communicates with cluster connection manager clients to maintain and manage the communication channels. The cluster connection manager may also be operatively interconnected with a plurality of cluster interconnect devices, thereby enabling failover operation in the event that a cluster interconnect device suffers an error condition.
摘要:
A system and method to perform discretization and cardinality reduction of item attributes using collaborative filtering techniques are described. Data input by a user is received over a network, the input data further including a plurality of items and associated item metadata related to events performed by the user. The input data is further processed to obtain a predetermined number of groupings, each grouping having a calculated value based on a distance parameter between corresponding attributes of each item stored within the item metadata. Finally, a similarity parameter is computed between each pair of items within the plurality of items based on associated groupings and recommendations of the items are presented to the user based on the corresponding calculated similarity parameter.
摘要:
A system and method to compute attribute-based similarity between items using collaborative filtering techniques are described. Events input by a user over a network are received, the events further including a plurality of items and associated item metadata. A similarity value is further computed between each pair of items within the plurality of stems based on corresponding attributes of each item stored within the item metadata. Finally, recommendations of the items to the user are presented based on the corresponding calculated similarity value.
摘要:
In general, the invention is directed to systems and methods of distributing tasks amongst servers or nodes in a cluster in a cloud-based big data environment, including: establishing a high_server_threshold; dividing active servers/nodes into at least three (3) categories of high usage servers, comprising servers on which usage is greater than the high_server_threshold; medium usage servers, comprising servers on which usage is less than the high_server_threshold, but is greater than zero; and low usage servers, comprising servers that are currently not utilized; receiving one or more tasks to be performed; scheduling the tasks by: first requesting that medium usage servers take tasks; if tasks remain that are not scheduled on the medium usage servers, schedule remaining tasks on low usage servers; if any tasks remain that are not scheduled on medium usage servers or low usage servers, scheduling remaining tasks on high usage servers.
摘要:
The present invention is generally directed to a distributed computing system comprising a plurality of computational clusters, each computational cluster comprising a plurality of compute optimized instances, each instance comprising local instance data storage and in communication with reserved disk storage, wherein processing hierarchy provides priority to local instance data storage before providing priority to reserved disk storage.
摘要:
A method for managing a plurality of servers is disclosed. Each server of the plurality of servers has access to data stored by other servers. The data is stored to one or more data storage devices. Coordinating information is written for the plurality of servers to a master mailbox record. The coordinating information includes data that each server uses to recover after a failure by a server. The master mailbox record is stored on a selected storage device at a location known to the plurality of servers, and the selected storage device is designated as a lock storage device. A plurality of lock storage devices is chosen so that in the event of failure of a server of the plurality of servers, at least one lock storage device will be available to the remaining servers.