Revealed previously this 7 days, Google Reasoning Dataflow support clearly plays against Amazon's streaming-data handling support Kinesis and big information items like Hadoop -- particularly since Reasoning Dataflow is designed on technological innovation that Search engines statements changes the methods behind Hadoop.
But on nearer look, Reasoning Dataflow is better believed of as a way for Search engines Reasoning customers to improve the programs they create -- and the information they down payment -- with statistics elements. A Hadoop killer? Probably not.
Google expenses the support as "the newest phase in our attempt to create information and statistics available to everyone," with an focus on the program you're composing rather than the information you're adjusting.
Significantly, Search engines Reasoning Dataflow is intended to substitute MapReduce, the application at the center of Hadoop and other big information techniques systems. MapReduce was initially designed by Search engines and later open-sourced, but Urs Hölzle, mature vice chairman of technological facilities, announced in the Search engines I/O keynote on Wed that "we [at Google] don't really use MapReduce any longer."
In position of MapReduce, Search engines uses two other tasks, Flume and MillWheel, that obviously affected Dataflow's style. The former allows you handle similar piplines for information techniques, which MapReduce did not offer on its own. The latter is described as "a structure for developing low-latency data-processing programs," and has obviously been in extensive use at Search engines for a while.
Most popular, Reasoning Dataflow is recognized as excellent to MapReduce in the quantity of information that can be prepared effectively. Hölzle stated MapReduce's inadequate efficiency started once the quantity of information achieved the multipetabyte variety. For viewpoint, Facebook or myspace stated this year it had a 100-petabyte Hadoop group, although the organization did not go into details about how much customized adjustment was used or even if MapReduce itself was still in function.
Ovum specialist Tony morrison a2z Baer recognizes Search engines Reasoning Dataflow as "part of an overriding pattern where we are seeing an blast of different frameworks and techniques for taking apart and examining big information. Where once big information techniques was essentially symbolic of MapReduce," he said in an e-mail, "you are now seeing frameworks like Ignite, Surprise, Giraph, and others offering solutions that allow you to choose the strategy that is right for the analytic issue."
Hadoop itself seems to be slanting away from MapReduce in support of more innovative (if demanding) handling methods, such as Apache Ignite. "Many issues do not offer themselves to the two-step procedure of map and decrease," described InfoWorld's Andrew Oliver, "and for those that do, Ignite can do map and decrease much quicker than Hadoop can."
Baer concurs: "From the looks of it, Search engines Reasoning Dataflow seems to have a similarity to Ignite, which also controls storage and prevents the expense of MapReduce."
The individual biggest difference between Hadoop and Search engines Reasoning Dataflow, though, can be found in where and how each is most likely to be implemented. Data tends to be prepared where it rests, and because of this Hadoop has become a information shop as much as a information techniques program. Those eying Search engines Reasoning Dataflow are not likely to move petabytes of information into it from an current Hadoop set up. It's more likely Reasoning Dataflow will be used to improve programs already published for Search engines Reasoning, ones where the information already exists in Google program or is being gathered there. That's not where almost all Hadoop tasks, now or later on, are likely to end up.
But on nearer look, Reasoning Dataflow is better believed of as a way for Search engines Reasoning customers to improve the programs they create -- and the information they down payment -- with statistics elements. A Hadoop killer? Probably not.
Google expenses the support as "the newest phase in our attempt to create information and statistics available to everyone," with an focus on the program you're composing rather than the information you're adjusting.
Significantly, Search engines Reasoning Dataflow is intended to substitute MapReduce, the application at the center of Hadoop and other big information techniques systems. MapReduce was initially designed by Search engines and later open-sourced, but Urs Hölzle, mature vice chairman of technological facilities, announced in the Search engines I/O keynote on Wed that "we [at Google] don't really use MapReduce any longer."
In position of MapReduce, Search engines uses two other tasks, Flume and MillWheel, that obviously affected Dataflow's style. The former allows you handle similar piplines for information techniques, which MapReduce did not offer on its own. The latter is described as "a structure for developing low-latency data-processing programs," and has obviously been in extensive use at Search engines for a while.
Most popular, Reasoning Dataflow is recognized as excellent to MapReduce in the quantity of information that can be prepared effectively. Hölzle stated MapReduce's inadequate efficiency started once the quantity of information achieved the multipetabyte variety. For viewpoint, Facebook or myspace stated this year it had a 100-petabyte Hadoop group, although the organization did not go into details about how much customized adjustment was used or even if MapReduce itself was still in function.
Ovum specialist Tony morrison a2z Baer recognizes Search engines Reasoning Dataflow as "part of an overriding pattern where we are seeing an blast of different frameworks and techniques for taking apart and examining big information. Where once big information techniques was essentially symbolic of MapReduce," he said in an e-mail, "you are now seeing frameworks like Ignite, Surprise, Giraph, and others offering solutions that allow you to choose the strategy that is right for the analytic issue."
Hadoop itself seems to be slanting away from MapReduce in support of more innovative (if demanding) handling methods, such as Apache Ignite. "Many issues do not offer themselves to the two-step procedure of map and decrease," described InfoWorld's Andrew Oliver, "and for those that do, Ignite can do map and decrease much quicker than Hadoop can."
Baer concurs: "From the looks of it, Search engines Reasoning Dataflow seems to have a similarity to Ignite, which also controls storage and prevents the expense of MapReduce."
The individual biggest difference between Hadoop and Search engines Reasoning Dataflow, though, can be found in where and how each is most likely to be implemented. Data tends to be prepared where it rests, and because of this Hadoop has become a information shop as much as a information techniques program. Those eying Search engines Reasoning Dataflow are not likely to move petabytes of information into it from an current Hadoop set up. It's more likely Reasoning Dataflow will be used to improve programs already published for Search engines Reasoning, ones where the information already exists in Google program or is being gathered there. That's not where almost all Hadoop tasks, now or later on, are likely to end up.
No comments:
Post a Comment