Improved computer technology is disclosed for enabling high performance stream processing on data such as complex, hierarchical data. In an example embodiment, a dynamic field schema specifies a dynamic field format for expressing the incoming data. An incoming data stream is then translated according to the dynamic field schema into an outgoing data stream in the dynamic field format. Stream processing, including field-specific stream processing, can then be performed on the outgoing data stream.
Disclosed herein are methods and systems for hardware-accelerating various data processing operations in a rule-based decision-making system such as a business rules engine, an event stream processor, and a complex event stream processor. Preferably, incoming data streams are checked against a plurality of rule conditions. Among the data processing operations that are hardware-accelerated include rule condition check operations, filtering operations, and path merging operations. The rule condition check operations generate rule condition check results for the processed data streams, wherein the rule condition check results are indicative of any rule conditions which have been satisfied by the data streams. The generation of such results with a low degree of latency provides enterprises with the ability to perform timely decision-making based on the data present in received data streams.
H03M 13/00 - Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
A system is disclosed that comprises a field programmable gate array (FPGA), a network interface, and hardware description code, wherein the hardware description code is compilable into a plurality of bit configuration files for loading onto the FPGA, wherein each bit configuration file defines a pipelined processing operation for a hardware template. The FPGA comprises configurable hardware logic, and the FPGA can be accessible over a network via the network interface for commanding the FPGA to load a bit configuration file from among the bit configuration files onto the FPGA to thereby configure hardware logic on the FPGA to perform the pipelined processing operation defined by the loaded bit configuration file, and wherein the FPGA is configured to (1) receive streaming data and (2) process the streaming data through the configured hardware logic to perform the pipelined processing operation defined by the loaded bit configuration file on the streaming data.
G06F 21/72 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in cryptographic circuits
G06F 21/76 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in application-specific integrated circuits [ASIC] or field-programmable devices, e.g. field-programmable gate arrays [FPGA] or programmable logic devices [PLD]
G06F 21/85 - Protecting input, output or interconnection devices interconnection devices, e.g. bus-connected or in-line devices
G06Q 40/06 - Asset management; Financial planning or analysis
G06F 17/00 - Digital computing or data processing equipment or methods, specially adapted for specific functions
G06F 9/48 - Program initiating; Program switching, e.g. by interrupt
4.
Method and apparatus for hardware-accelerated machine learning
A feature extractor for a convolutional neural network (CNN) is disclosed, wherein the feature extractor is deployed on a member of the group consisting of (1) a reconfigurable logic device, (2) a graphics processing unit (GPU), and (3) a chip multi-processor (CMP). A processing pipeline can be implemented on the member, where the processing pipeline implements a plurality convolution layers for the CNN, wherein each of a plurality of the convolutional layers comprises (1) a convolution stage that convolves first data with second data if activated and (2) a sub-sampling stage that performs a member of the group consisting of (i) a max pooling operation, (ii) an averaging operation, and (iii) a sampling operation on data received thereby if activated. The processing pipeline can be controllable with respect to which of the convolution stages are activated/deactivated and which of the sub-sampling stages are activated/deactivated when processing streaming data through the processing pipeline. The deactivated convolution and sub-sampling stages can remain instantiated within the processing pipeline but act as pass-throughs when deactivated. The processing pipeline performs feature vector extraction on the streaming data using the activated convolution stages and the activated sub-sampling stages.
G06F 21/72 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in cryptographic circuits
G06F 21/76 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in application-specific integrated circuits [ASIC] or field-programmable devices, e.g. field-programmable gate arrays [FPGA] or programmable logic devices [PLD]
G06F 21/85 - Protecting input, output or interconnection devices interconnection devices, e.g. bus-connected or in-line devices
5.
Intelligent data storage and processing using FPGA devices
A system is disclosed that comprises a field programmable gate array (FPGA), a network interface, and a plurality of hardware templates. The FPGA comprises configurable hardware logic, and the hardware templates define a plurality of different pipelined processing operations. The FPGA can be accessible over a network via the network interface for commanding the FPGA to load a hardware template from among the hardware templates onto the FPGA to thereby configure hardware logic on the FPGA to perform the pipelined processing operation defined by the loaded hardware template, and wherein the FPGA is configured to (1) receive streaming data and (2) process the streaming data through the configured hardware logic to perform the pipelined processing operation defined by the loaded hardware template on the streaming data.
G06F 21/72 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in cryptographic circuits
G06F 21/76 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in application-specific integrated circuits [ASIC] or field-programmable devices, e.g. field-programmable gate arrays [FPGA] or programmable logic devices [PLD]
G06F 21/85 - Protecting input, output or interconnection devices interconnection devices, e.g. bus-connected or in-line devices
G06Q 40/06 - Asset management; Financial planning or analysis
G06F 17/00 - Digital computing or data processing equipment or methods, specially adapted for specific functions
G06F 9/48 - Program initiating; Program switching, e.g. by interrupt
Various methods and apparatuses are described for performing high speed format translations of incoming data, where the incoming data is arranged in a delimited data format. As an example, the data in the delimited data format can be translated to a structured format such as a fixed field format using pipelined operations. A reconfigurable logic device can be used in exemplary embodiments as a platform for the format translation.
G06F 16/20 - Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
G06F 16/25 - Integrating or interfacing systems involving database management systems
G06F 16/28 - Databases characterised by their database models, e.g. relational or object models
G16H 10/60 - ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
7.
Method and apparatus for hardware-accelerated machine learning
A multi-functional data processing pipeline for use with machine learning is disclosed. The multi-functional pipeline may comprise a plurality of pipelined data processing engines, the plurality of pipelined data processing engines being configured to perform processing operations, and the pipelined data processing engines can include correlation logic. The multi-functional pipeline can be configured to controllably activate or deactivate each of the pipelined data processing engines in the pipeline in response to control instructions and thereby define a function for the pipeline, each pipeline function being the combined functionality of each activated pipelined data processing engine in the pipeline. In example embodiments, such pipelines can be used to accelerate convolutional layers in machine-learning technology such as convolutional neural networks.
G06F 21/72 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in cryptographic circuits
G06F 21/76 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in application-specific integrated circuits [ASIC] or field-programmable devices, e.g. field-programmable gate arrays [FPGA] or programmable logic devices [PLD]
G06F 21/85 - Protecting input, output or interconnection devices interconnection devices, e.g. bus-connected or in-line devices
8.
Method and system for accelerated stream processing
Disclosed herein are methods and systems for hardware-accelerating various data processing operations in a rule-based decision-making system such as a business rules engine, an event stream processor, and a complex event stream processor. Preferably, incoming data streams are checked against a plurality of rule conditions. Among the data processing operations that are hardware-accelerated include rule condition check operations, filtering operations, and path merging operations. The rule condition check operations generate rule condition check results for the processed data streams, wherein the rule condition check results are indicative of any rule conditions which have been satisfied by the data streams. The generation of such results with a low degree of latency provides enterprises with the ability to perform timely decision-making based on the data present in received data streams.
H03M 13/00 - Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
Methods and systems are disclosed where an FPGA offloads a plurality of processing tasks from a processor. The FPGA can process streaming data received via a network interface, and the FPGA can be controllable in response to control instructions received from the processor. The FPGA comprises resident hardware logic for a plurality of data processing engines that are combinable as a processing pipeline within the FPGA. In response to the control instructions, the FPGA can control which of the data processing engines are activated and which of the data processing engines are deactivated to selectively tap into the streaming data to perform pipelined processing operations on the streaming data via the activated data processing engines. The deactivated data processing engines remain on the FPGA and provide a pass through path for the streaming data whereby the deactivated data processing engines do not perform processing operations on streaming data received thereby.
G06F 21/72 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in cryptographic circuits
G06F 21/76 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in application-specific integrated circuits [ASIC] or field-programmable devices, e.g. field-programmable gate arrays [FPGA] or programmable logic devices [PLD]
G06F 21/85 - Protecting input, output or interconnection devices interconnection devices, e.g. bus-connected or in-line devices
G06Q 40/06 - Asset management; Financial planning or analysis
G06F 17/00 - Digital computing or data processing equipment or methods, specially adapted for specific functions
G06F 9/48 - Program initiating; Program switching, e.g. by interrupt
10.
Method and system for high performance integration, processing and searching of structured and unstructured data
Disclosed herein are methods and systems for integrating an enterprise's structured and unstructured data to provide users and enterprise applications with efficient and intelligent access to that data. In accordance with exemplary embodiments, the generation of feature vectors about unstructured data can be hardware-accelerated by processing streaming unstructured data through a reconfigurable logic device, a graphics processor unit (GPU), or chip multi-processor (CMP) to determine features that can aid clustering of similar data objects.
Disclosed herein are methods and systems for hardware-accelerating various data processing operations in a rule-based decision-making system such as a business rules engine, an event stream processor, and a complex event stream processor. Preferably, incoming data streams are checked against a plurality of rule conditions. Among the data processing operations that are hardware-accelerated include rule condition check operations, filtering operations, and path merging operations. The rule condition check operations generate rule condition check results for the processed data streams, wherein the rule condition check results are indicative of any rule conditions which have been satisfied by the data streams. The generation of such results with a low degree of latency provides enterprises with the ability to perform timely decision-making based on the data present in received data streams.
H03M 13/00 - Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
Various methods and apparatuses are described for performing high speed format translations of incoming data, where the incoming data is arranged in a delimited data format. As an example, the data in the delimited data format can be translated to a mapped variable field format using pipelined operations. A reconfigurable logic device can be used in exemplary embodiments as a platform for the format translation.
Various methods and apparatuses are described for performing high speed translations of data. In an example embodiment, record layout detection can be performed for data. In another example embodiment, data pivoting prior to field-specific data processing can be performed.
G06F 7/00 - Methods or arrangements for processing data by operating upon the order or content of the data handled
G06F 16/25 - Integrating or interfacing systems involving database management systems
G06F 16/28 - Databases characterised by their database models, e.g. relational or object models
G06F 21/71 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information
14.
PIPELINES FOR HARDWARE-ACCELERATED MACHINE LEARNING
A multi-functional data processing pipeline for use with machine learning is disclosed. The multi-functional pipeline may comprise a plurality of pipelined data processing engines, the plurality of pipelined data processing engines being configured to perform processing operations, and the pipelined data processing engines can include correlation logic. The multi-functional pipeline can be configured to controllably activate or deactivate each of the pipelined data processing engines in the pipeline in response to control instructions and thereby define a function for the pipeline, each pipeline function being the combined functionality of each activated pipelined data processing engine in the pipeline. In example embodiments, such pipelines can be used to accelerate convolutional layers in machine-learning technology such as convolutional neural networks.
Methods and systems are disclosed where a plurality of precompiled hardware templates are stored in memory, each of the hardware templates being configured for loading onto a re-configurable logic device such as a FPGA to define a data processing operation to be performed by the re-configurable logic device, each of the data processing operations defined by the precompiled hardware templates having an associated performance characteristic. A processor selects a precompiled hardware template from a plurality of the precompiled hardware templates in the memory for loading onto the re-configurable logic device based at least in part on the associated performance characteristics of the data processing operations defined by the precompiled hardware templates.
G06F 21/72 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in cryptographic circuits
G06F 21/76 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in application-specific integrated circuits [ASIC] or field-programmable devices, e.g. field-programmable gate arrays [FPGA] or programmable logic devices [PLD]
G06F 21/85 - Protecting input, output or interconnection devices interconnection devices, e.g. bus-connected or in-line devices
Various methods and apparatuses are described for performing high speed format translations of incoming data, where the incoming data is arranged in a delimited data format. As an example, the data in the delimited data format can be translated to a fixed field format using pipelined operations. A reconfigurable logic device can be used in exemplary embodiments as a platform for the format translation.
Improved computer technology is disclosed for enabling high performance stream processing on data such as complex, hierarchical data. In an example embodiment, a dynamic field schema specifies a dynamic field format for expressing the incoming data. An incoming data stream is then translated according to the dynamic field schema into an outgoing data stream in the dynamic field format. Stream processing, including field-specific stream processing, can then be performed on the outgoing data stream.
Disclosed herein are methods and systems for hardware-accelerating various data processing operations in a rule-based decision-making system such as a business rules engine, an event stream processor, and a complex event stream processor. Preferably, incoming data streams are checked against a plurality of rule conditions. Among the data processing operations that are hardware-accelerated include rule condition check operations, filtering operations, and path merging operations. The rule condition check operations generate rule condition check results for the processed data streams, wherein the rule condition check results are indicative of any rule conditions which have been satisfied by the data streams. The generation of such results with a low degree of latency provides enterprises with the ability to perform timely decision-making based on the data present in received data streams.
H03M 13/00 - Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
Disclosed herein are methods and systems for integrating an enterprise's structured and unstructured data to provide users and enterprise applications with efficient and intelligent access to that data. In accordance with exemplary embodiments, the generation of classification information about unstructured data can be hardware-accelerated by processing streaming unstructured data through a reconfigurable logic device, a graphics processor unit (GPU), or chip multi-processor (CMP) to generate the classification metadata about the unstructured data.
Methods and systems are disclosed where a plurality of precompiled hardware templates are stored in memory, each of the hardware templates being configured for loading onto a re-configurable logic device such as a FPGA to define a data processing operation to be performed by the re-configurable logic device, each of the data processing operations defined by the precompiled hardware templates having an associated performance characteristic. A processor selects a precompiled hardware template from a plurality of the precompiled hardware templates in the memory for loading onto the re-configurable logic device based at least in part on the associated performance characteristics of the data processing operations defined by the precompiled hardware templates.
G06F 21/72 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in cryptographic circuits
G06F 21/76 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in application-specific integrated circuits [ASIC] or field-programmable devices, e.g. field-programmable gate arrays [FPGA] or programmable logic devices [PLD]
G06F 21/85 - Protecting input, output or interconnection devices interconnection devices, e.g. bus-connected or in-line devices
G06Q 40/06 - Asset management; Financial planning or analysis
G06F 17/00 - Digital computing or data processing equipment or methods, specially adapted for specific functions
G06F 9/48 - Program initiating; Program switching, e.g. by interrupt
G06F 3/06 - Digital input from, or digital output to, record carriers
21.
Method and apparatus for accelerated data translation using record layout detection
Various methods and apparatuses are described for performing high speed translations of data. In an example embodiment, record layout detection can be performed for data. In another example embodiment, data pivoting prior to field-specific data processing can be performed.
G06F 7/00 - Methods or arrangements for processing data by operating upon the order or content of the data handled
G06F 17/30 - Information retrieval; Database structures therefor
G06F 21/71 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information
22.
Method and apparatus for accelerated record layout detection
Various methods and apparatuses are described for performing high speed translations of data. In an example embodiment, record layout detection can be performed for data. In another example embodiment, data pivoting prior to field-specific data processing can be performed.
G06F 7/00 - Methods or arrangements for processing data by operating upon the order or content of the data handled
G06F 17/30 - Information retrieval; Database structures therefor
G06F 21/71 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information
23.
Method and apparatus for record pivoting to accelerate processing of data fields
Various methods and apparatuses are described for performing high speed translations of data. In an example embodiment, record layout detection can be performed for data. In another example embodiment, data pivoting prior to field-specific data processing can be performed.
G06F 12/14 - Protection against unauthorised use of memory
G06F 3/00 - Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
G06F 17/30 - Information retrieval; Database structures therefor
G06F 21/71 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information
24.
METHOD AND APPARATUS FOR ACCELERATED DATA TRANSLATION
Various methods and apparatuses are described for performing high speed translations of data. In an example embodiment, record layout detection can be performed for data. In another example embodiment, data pivoting prior to field-specific data processing can be performed. Data can be streamed through computer systems in any of a number of formats. For example, as described in the cross-referenced patent applications, a delimited data format is a common format used for passing data between data processing system or over networks, particularly with respect to passing record-oriented data.
An encryption technique is disclosed for encrypting a plurality of data blocks of a data segment where the encryption selectively switches between a blockwise independent randomized (BIR) encryption mode and a cipher block chaining (CBC) encryption mode based on a configurable feedback stride. A corresponding decryption technique is also disclosed.
H04L 9/06 - Arrangements for secret or secure communications; Network security protocols the encryption apparatus using shift registers or memories for blockwise coding, e.g. D.E.S. systems
H04L 9/14 - Arrangements for secret or secure communications; Network security protocols using a plurality of keys or algorithms
26.
Method and system for high performance integration, processing and searching of structured and unstructured data using coprocessors
Disclosed herein is a method and system for integrating an enterprise's structured and unstructured data to provide users and enterprise applications with efficient and intelligent access to that data. In accordance with exemplary embodiments, the generation of metadata indexes about unstructured data can be hardware-accelerated by processing streaming unstructured data through a reconfigurable logic device to generate the metadata about the unstructured data for the index.
An integrated circuit for data encryption/decryption and secure key management is disclosed. The integrated circuit may be used in conjunction with other integrated circuits, processors, and software to construct a wide variety of secure data processing, storage, and communication systems. An embodiment of the integrated circuit includes a run-time scalable block cipher circuit, wherein the run-time scalable block cipher circuit is run-time scalable to balance throughput with power consumption.
H04L 9/00 - Arrangements for secret or secure communications; Network security protocols
H04L 9/06 - Arrangements for secret or secure communications; Network security protocols the encryption apparatus using shift registers or memories for blockwise coding, e.g. D.E.S. systems
28.
Intelligent data storage and processing using FPGA devices
A re-configurable logic device such as a field programmable gate array (FPGA) can be used to deploy a data processing pipeline, the pipeline comprising a plurality of pipelined data processing engines, the plurality of pipelined data processing engines being configured to perform processing operations, wherein the pipeline comprises a multi-functional pipeline, and wherein the re-configurable logic device is further configured to controllably activate or deactivate each of the pipelined data processing engines in the pipeline in response to control instructions and thereby define a function for the pipeline, each pipeline function being the combined functionality of each activated pipelined data processing engine in the pipeline.
G06F 21/72 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in cryptographic circuits
G06F 21/76 - Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure computing or processing of information in application-specific integrated circuits [ASIC] or field-programmable devices, e.g. field-programmable gate arrays [FPGA] or programmable logic devices [PLD]
G06F 21/85 - Protecting input, output or interconnection devices interconnection devices, e.g. bus-connected or in-line devices
G06Q 40/06 - Asset management; Financial planning or analysis
G06F 17/00 - Digital computing or data processing equipment or methods, specially adapted for specific functions
G06F 3/06 - Digital input from, or digital output to, record carriers
29.
METHOD AND APPARATUS FOR ACCELERATED FORMAT TRANSLATION OF DATA IN A DELIMITED DATA FORMAT
Various methods and apparatuses are described for performing high speed format translations of incoming data, where the incoming data is arranged in a delimited data format. As an example, the data in the delimited data format can be translated to a fixed field format using pipelined operations. As another example, the data in the delimited data format can be translated to a mapped variable field format using pipelined operations. A reconfigurable logic device can be used in exemplary embodiments as a platform for the format translation.
Various methods and apparatuses are described for performing high speed format translations of incoming data, where the incoming data is arranged in a delimited data format. As an example, the data in the delimited data format can be translated to a fixed field format using pipelined operations. As another example, the data in the delimited data format can be translated to a mapped variable field format using pipelined operations. A reconfigurable logic device can be used in exemplary embodiments as a platform for the format translation.
Various methods and apparatuses are described for performing high speed format translations of incoming data, where the incoming data is arranged in a delimited data format. As an example, the data in the delimited data format can be translated to a fixed field format using pipelined operations. A reconfigurable logic device can be used in exemplary embodiments as a platform for the format translation.
Various methods and apparatuses are described for performing high speed format translations of incoming data, where the incoming data is arranged in a delimited data format. As an example, the data in the delimited data format can be translated to a mapped variable field format using pipelined operations. A reconfigurable logic device can be used in exemplary embodiments as a platform for the format translation.
Various techniques are disclosed for offloading the processing of data packets. For example, incoming data packets can be processed through an offload processor to generate a new stream of outgoing data packets that organize data packets in a manner different than the incoming data packets. Furthermore, in an exemplary embodiment, the offloaded processing can be resident in an intelligent switch, such as an intelligent switch, such as an intelligent switch upstream or downstream from an electronic trading platform.
Various techniques are disclosed for distributing data, particularly real-time data such as financial market data, to data consumers at low latency. Exemplary embodiments include embodiments that employ adaptive data distribution techniques and embodiments that employ a multi-class distribution engine.
G06F 15/16 - Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
35.
Method and system for high throughput blockwise independent encryption/decryption
An encryption technique is disclosed for encrypting a plurality of data blocks of a data segment where the encryption selectively switches between a blockwise independent randomized (BIR) encryption mode and a cipher block chaining (CBC) encryption mode based on a configurable feedback stride. A corresponding decryption technique is also disclosed.
Disclosed herein is a method and apparatus for hardware-accelerating various data quality checking operations. Incoming data streams can be processed with respect to a plurality of data quality check operations using offload engines (e.g., reconfigurable logic such as field programmable gate arrays (FPGAs)). Accelerated data quality checking can be highly advantageous for use in connection with Extract, Transfer, and Load (ETL) systems.
Disclosed herein is a method and system for accelerating the generation of pattern indexes. In exemplary embodiments, regular expression pattern matching can be performed at high speeds on data to determine whether a pattern is present in the data. Pattern indexes can then be built based on the results of such regular expression pattern matching. Reconfigurable logic such a field programmable gate arrays (FPGAs) can be used to hardware accelerate these operations.
Disclosed herein is a method and system for integrating an enterprise's structured and unstructured data to provide users and enterprise applications with efficient and intelligent access to that data. In accordance with exemplary embodiments, the generation of metadata indexes about unstructured data can be hardware-accelerated by processing streaming unstructured data through a reconfigurable logic device to generate the metadata about the unstructured data for the index.
A re-configurable logic device such as a field programmable gate array (FPGA) can be used to deploy a data processing pipeline, the pipeline comprising a plurality of pipelined data processing engines, the plurality of pipelined data processing engines being configured to perform processing operations, wherein the pipeline comprises a multi-functional pipeline, and wherein the re-configurable logic device is further configured to controllably activate or deactivate each of the pipelined data processing engines in the pipeline in response to control instructions and thereby define a function for the pipeline, each pipeline function being the combined functionality of each activated pipelined data processing engine in the pipeline.
A re-configurable logic device such as a field programmable gate array (FPGA) can be used to deploy a data processing pipeline, the pipeline comprising a plurality of pipelined data processing engines, the plurality of pipelined data processing engines including a data reduction engine, the plurality of pipelined data processing engines being configured to perform processing operations, wherein the pipeline comprises a multi-functional pipeline, and wherein the re-configurable logic device is further configured to controllably activate or deactivate each of the pipelined data processing engines in the pipeline in response to control instructions and thereby define a function for the pipeline, each pipeline function being the combined functionality of each activated pipelined data processing engine in the pipeline.
Methods and apparatuses for processing data are disclosed, including methods and apparatuses that leverage a reconfigurable logic device to offload decompression and search operations from a processor to thereby enable high speed data searches within data that has been stored in a compressed format.
Disclosed herein is an improved architecture for regular expression pattern matching. Improvements to pattern matching deterministic finite automatons (DFAs) that are described by the inventors include a pipelining strategy that pushes state-dependent feedback to a final pipeline stage to thereby enhance parallelism and throughput, augmented state transitions that track whether a transition is indicative of a pattern match occurring thereby reducing the number of necessary states for the DFA, augmented state transition that track whether a transition is indicative of a restart to the matching process, compression of the DFA's transition table, alphabet encoding for input symbols to equivalence class identifiers, the use of an indirection table to allow for optimized transition table memory, and enhanced scalability to facilitate the ability of the improved DFA to process multiple input symbols per cycle.
G06F 15/00 - Digital computers in general; Data processing equipment in general
G06F 15/18 - in which a program is changed according to experience gained by the computer itself during a complete run; Learning machines (adaptive control systems G05B 13/00;artificial intelligence G06N)
43.
Method and system for high performance integration, processing and searching of structured and unstructured data using coprocessors
Disclosed herein is a method and system for integrating an enterprise's structured and unstructured data to provide users and enterprise applications with efficient and intelligent access to that data. Queries can be directed toward both an enterprise's structured and unstructured data using standardized database query formats such as SQL commands. A coprocessor can be used to hardware-accelerate data processing tasks (such as full-text searching) on unstructured data as necessary to handle a query. Furthermore, traditional relational database techniques can be used to access structured data stored by a relational database to determine which portions of the enterprise's unstructured data should be delivered to the coprocessor for hardware-accelerated data processing.
Disclosed herein is a method and system for hardware-accelerating various data processing operations in a rule-based decision-making system such as a business rules engine, an event stream processor, and a complex event stream processor. Preferably, incoming data streams are checked against a plurality of rule conditions. Among the data processing operations that are hardware-accelerated include rule condition check operations, filtering operations, and path merging operations. The rule condition check operations generate rule condition check results for the processed data streams, wherein the rule condition check results are indicative of any rule conditions which have been satisfied by the data streams. The generation of such results with a low degree of latency provides enterprises with the ability to perform timely decision-making based on the data present in received data streams.
An integrated circuit for data encryption/decryption and secure key management is disclosed. The integrated circuit may be used in conjunction with other integrated circuits, processors, and software to construct a wide variety of secure data processing, storage, and communication systems. A preferred embodiment of the integrated circuit includes a symmetric block cipher that may be scaled to strike a favorable balance among processing throughput and power consumption. The modular architecture also supports multiple encryption modes and key management functions such as one-way cryptographic hash and random number generator functions that leverage the scalable symmetric block cipher. The integrated circuit may also include a key management processor that can be programmed to support a wide variety of asymmetric key cryptography functions for secure key exchange with remote key storage devices and enterprise key management servers. Internal data and key buffers enable the device to re-key encrypted data without exposing data. The key management functions allow the device to function as a cryptographic domain bridge in a federated security architecture.
H04L 9/06 - Arrangements for secret or secure communications; Network security protocols the encryption apparatus using shift registers or memories for blockwise coding, e.g. D.E.S. systems
46.
Method and system for high performance integration, processing and searching of structured and unstructured data using coprocessors
Disclosed herein is a method and system for integrating an enterprise's structured and unstructured data to provide users and enterprise applications with efficient and intelligent access to that data. Queries can be directed toward both an enterprise's structured and unstructured data using standardized database query formats such as SQL commands. A coprocessor can be used to hardware-accelerate data processing tasks (such as full-text searching) on unstructured data as necessary to handle a query. Furthermore, traditional relational database techniques can be used to access structured data stored by a relational database to determine which portions of the enterprise's unstructured data should be delivered to the coprocessor for hardware-accelerated data processing.
Disclosed herein is a method and system for hardware-accelerating the generation of metadata for a data stream using a coprocessor. Using these techniques, data can be richly indexed, classified, and clustered at high speeds. Reconfigurable logic such a field programmable gate arrays (FPGAs) can be used by the coprocessor for this hardware acceleration. Techniques such as exact matching, approximate matching, and regular expression pattern matching can be employed by the coprocessor to generate desired metadata for the data stream.
A system and method for inspecting a data stream for data segments matching one or more patterns each having a predetermined allowable error, which includes filtering a data stream for a plurality of patterns of symbol combinations with a plurality of parallel filter mechanisms, detecting a plurality of potential pattern piece matches, identifying a plurality of potentially matching patterns, reducing the identified plurality of potentially matching patterns to a set of potentially matching patterns with a reduction stage, providing associated data and the reduced set of potentially matching patterns, each having an associated allowable error, to a verification stage, and verifying presence of a pattern match in the data stream from the plurality of patterns of symbol combinations and associated allowable errors with the verification stage.
49.
Method and apparatus for approximate pattern matching
A system and method for inspecting a data stream for data segments matching one or more patterns each having a predetermined allowable error, which includes filtering a data stream for a plurality of patterns of symbol combinations with a plurality of parallel filter mechanisms, detecting a plurality of potential pattern piece matches, identifying a plurality of potentially matching patterns, reducing the identified plurality of potentially matching patterns to a set of potentially matching patterns with a reduction stage, providing associated data and the reduced set of potentially matching patterns, each having an associated allowable error, to a verification stage, and verifying presence of a pattern match in the data stream from the plurality of patterns of symbol combinations and associated allowable errors with the verification stage.
An encryption technique is disclosed for encrypting a data segment comprising a plurality of data blocks, wherein the security and throughput of the encryption is enhanced by using blockwise independent bit vectors for reversible combination with the data blocks prior to key encryption. Preferably, the blockwise independent bit vectors are derived from a data tag associated with the data segment. Several embodiments are disclosed for generating these blockwise independent bit vectors. In a preferred embodiment, the data tag comprises a logical block address (LBA) for the data segment. Also disclosed herein is a corresponding decryption technique as well as a corresponding symmetrical encryption/decryption technique.
A firmware socket module is deployed on a reconfigurable logic device, wherein the firmware socket module has a communication path between itself and an entry point into a data processing pipeline, wherein the firmware socket module is configured to provide both commands and target data to the entry point in the data processing pipeline via the same communication path, wherein each command defines a data processing operation that is to be performed by the data processing pipeline, and wherein the target data corresponds to the data upon which the data processing pipeline performs its commanded data processing operation. Preferably, the firmware socket module is configured to provide the commands and target data in a predetermined order that is maintained throughout the data processing pipeline. Also, the firmware socket module may be configured to (1) access an external input descriptor pool buffer that defines the order in which commands and target data are to be provided to the data processing pipeline, and (2) transfer the commands and target data from an external memory to the data processing pipeline in accordance with the identified defined order. Results of the processing by the data processing pipeline are also returned to external memory by the firmware socket module, whereupon those results can be subsequently used by software executing on a computer system.
G06F 15/78 - Architectures of general purpose stored program computers comprising a single central processing unit
G06F 9/38 - Concurrent instruction execution, e.g. pipeline, look ahead
G06F 15/80 - Architectures of general purpose stored program computers comprising an array of processing units with common control, e.g. single instruction multiple data processors
52.
Firmware socket module for FPGA-based pipeline processing
A firmware socket module is deployed on a reconfigurable logic device, wherein the firmware socket module is configured to provide both commands and target data to an entry point in a data processing pipeline, wherein each command defines a data processing operation that is to be performed by the data processing pipeline, and wherein the target data corresponds to the data upon which the data processing pipeline performs its commanded data processing operation. Also, the firmware socket module may be configured to (1) access an external input descriptor pool buffer that defines an order in which commands and target data are to be provided to the data processing pipeline, and (2) transfer the commands and target data from an external memory to the data processing pipeline in accordance with the defined order.
Disclosed herein is an improved architecture for regular expression pattern matching. Improvements to pattern matching deterministic finite automatons (DFAs) that are described by the inventors include a pipelining strategy that pushes state-dependent feedback to a final pipeline stage to thereby enhance parallelism and throughput, augmented state transitions that track whether a transition is indicative of a pattern match occurring thereby reducing the number of necessary states for the DFA, augmented state transition that track whether a transition is indicative of a restart to the matching process, compression of the DFA's transition table, alphabet encoding for input symbols to equivalence class identifiers, the use of an indirection table to allow for optimized transition table memory, and enhanced scalability to facilitate the ability of the improved DFA to process multiple input symbols per cycle.
A device for matching an input string to a pattern via a deterministic finite automaton (DFA), the DFA comprising a plurality of states including a current state and a plurality of possible next states, the input string comprising a plurality of input symbols The device comprises at least two parallel pipeline stages; a first one of the pipeline stages being configured to retrieve a plurality of transitions to a possible next state of the DFA from a pre-populated memory, and a second one of the pipeline stages configured to choose, based at least in part upon the DFA's current state, one of said retrieved transitions from which the integrated circuit will determine the next state of the DFA, wherein the second one of the pipeline stages is downstream from the first one of the pipeline stages.
55.
Method and device for high performance regular expression pattern matching
Disclosed herein is an improved architecture for regular expression pattern matching. Improvements to pattern matching deterministic finite automatons (DFAs) that are described by the inventors include a pipelining strategy that pushes state-dependent feedback to a final pipeline stage to thereby enhance parallelism and throughput, augmented state transitions that track whether a transition is indicative of a pattern match occurring thereby reducing the number of necessary states for the DFA, augmented state transition that track whether a transition is indicative of a restart to the matching process, compression of the DFA's transition table, alphabet encoding for input symbols to equivalence class identifiers, the use of an indirection table to allow for optimized transition table memory, and enhanced scalability to facilitate the ability of the improved DFA to process multiple input symbols per cycle.
A method and apparatus use decision logic deployed on a reconfigurable logic device to process a stream of financial information at hardware speeds. The decision logic can be configured to perform data reduction operations on the financial information stream. Examples of such data reductions operations include data processing operations to compute a latest stock price, a minimum stock price, and a maximum stock price.
A data processing system comprising: a processing device; and a computer system having a system bus, wherein the computer system is configured to communicate with the processing device over the system bus to control an operation of the processing device; and wherein the processing device comprises a re- configurable logic device configured to receive and process streaming data through a pipeline deployed on the re-configurable logic device, the pipeline comprising a plurality of pipelined data processing engines, the plurality of pipelined data processing engines being configured to perform different processing operations, wherein the pipeline comprises a multi-functional pipeline, and wherein the re-configurable logic device further comprises a control processor, wherein the control processor is configured to controllably activate or deactivate each of the pipelined data processing engines in the pipeline and thereby define a function for the pipeline, the pipeline function being the combined functionality of each activated pipelined data processing engine in the pipeline.
For a programmable logic device in communication with a mass storage medium, the programmable logic device being configured to process data moving to or from the mass storage medium in accordance with a template loaded thereon, the template defining one or more processing functions, each function having an associated performance characteristic for data processing performed thereby, a method for selecting a template for programming the programmable logic device, the method comprising: selecting a stored template from a plurality of stored templates for loading into the programmable logic device at least partially on the basis of the associated performance characteristics for each function defined by the templates.
A data processing apparatus comprising: a data storage device for storing data in a compressed format; a processor communicating with the data storage device to store data therein and process data retrieved therefrom; and a reconfigurable logic device for offloading a plurality of processing operations from the processor, the reconfigurable logic device comprising a hardware logic template configured as a data processing pipeline, the data processing pipeline comprising a decompression engine and a downstream search engine. The decompression engine and the search engine are configured to operate in a pipelined manner by performing their respective decompression and search operations simultaneously at hardware processing speeds such that the decompression engine performs the decompression operation on compressed data while at the same time the search engine performs the search operation on decompressed data that was previously decompressed by the decompression engine.
G06F 5/00 - Methods or arrangements for data conversion without changing the order or content of the data handled
G06F 9/06 - Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
patents
